KiT: A Foundation Model for Financial Time-Series Forecasting using DiffusionTransformers

Financial candlestick forecasting is fundamental to quantitative investment, yet it remains exceptionally challenging due to extremely low signal-to-noise ratios and vast heterogeneity across markets and instruments. Existing approaches have largely attempted to introduce deep learning to capture hidden temporal features, but most adopt an auto-regressive formulation, which leads to error accumulation during inference. Meanwhile, general-purpose time-series foundation models are not tailored to the unique structure of k-line data and yield unsatisfactory performance on downstream candlestick forecasting tasks. To tackle these problems, we introduce KiT, a K-line Diffusion Transformer foundation model, and reformulate future prediction as conditional path generation via flow matching: given a historical context window, the model generates an ensemble of plausible future OHLCV trajectories. We pre-train KiT at multiple parameter scales on billions of candlestick bars spanning multiple markets and timescales. Across three markets and seven resolutions, KiT attains a mean return RankIC of 0.057 and a mean volatility RankIC of 0.66, leading at every timescale and outperforming both task-specific financial forecasters and general time-series foundation models. Code will be available at: https://github.com/Luciferbobo/KiT.

1. 5 Jun 2024

   Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality

   The research identifies a link between state-space models and Transformers in deep learning, leading to the creation of a faster language modeling architecture, Mamba-2.

   Machine learningIn International Conference on Machine Learning

   2,001cites
2. 8 Jan 2025

   Reconstruction vs. Generation: Taming Optimization Dilemma in Latent Diffusion Models

   The paper proposes a new model, VA-VAE, that aligns the latent space with pre-trained vision foundation models, enabling faster convergence of Diffusion Transformers in high-dimensional latent spaces and achieving top performance on ImageNet 256x256 generation.

   Machine learningIn 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

   416cites
3. 13 Nov 2024

   Mixture-of-Transformers: A Sparse and Scalable Architecture for Multi-Modal Foundation Models

   The Mixture-of-Transformers (MoT) is a sparse multi-modal transformer architecture that reduces pretraining costs and allows modality-specific processing with global self-attention.

   Machine learningIn Trans. Mach. Learn. Res.

   177cites
4. 15 Aug 2024

   ECG-FM: An Open Electrocardiogram Foundation Model

   ECG-FM, a transformer-based model for ECG analysis, shows strong performance in predicting cardiac conditions, having been pretrained on 2.5 million samples.

   Machine learning

   129cites
5. 2 Apr 2025

   An Advanced Ensemble Deep Learning Framework for Stock Price Prediction Using VAE, Transformer, and LSTM Model

   The research introduces a combined deep learning framework for stock price prediction, demonstrating its effectiveness and reliability in predicting stock price movements.

   arXiv

   4cites
6. 3 Jul 2024

   Bytes Are All You Need: Transformers Operating Directly On File Bytes

   File Byte Transformers: ByteFormer is a deep learning model that enhances image classification accuracy by 5% and can perform audio classification without specific preprocessing, showcasing its versatility.

   Machine learningIn Trans. Mach. Learn. Res.

   19cites