Transformers Versus LSTMs for Electronic Trading : A Review
Artificial Intelligence & Machine Learning
The field of ai witnessed a remarkable evolution over the last few decades, particularly in the domain of time series prediction a popular tool in quant world. Among AI models, Long Short-Term Memory (LSTM) networks and Transformer models definitely emerged as the kings of the proverbial hill.
On one hand, LSTMs, a type of Recurrent Neural Network (RNN), known for their ability to capture temporal dependencies and widely applied in predicting financial time series data. On the other hand, Transformer models, initially designed for Natural Language Processing (NLP), have shown exceptional capabilities in handling sequential data.
The intersection of these AI models with financial time series prediction forms the crux of electronic trading systems, where the accuracy and robustness of predictions can have substantial economic impacts.
Paul Bilokon & Yitao Qiu set out to explore whether Transformer models, with their growing prominence in various AI applications, might surpass the performance of LSTM models in predicting financial time series, particularly in the context of electronic trading.
LSTM and Transformer Architectures
The methodology begins with an exploration of the fundamental architectures of LSTM and Transformer models. LSTM models, known for their unique gating mechanisms, are adept at handling long-term dependencies in time series data, making them a popular choice in financial applications. Transformers, conversely, leverage attention mechanisms to weigh the significance of different parts of the input data, offering a different approach to sequential data processing.
Bilokon and Qiu’s research utilizes high-frequency limit order book data from the cryptocurrency market to compare various LSTM-based and Transformer-based models.
There were outcomes from the three financial time series prediction tasks:
- prediction of the LOB mid-price
- mid-price difference
- mid-price movement
Each task is dissected to reveal the comparative performance of the various models, with a particular focus on the DLSTM model – a novel integration of LSTM features with Autoformer’s time decomposition architecture.
How limited were the advantages of Transformer models in certain aspects of price sequence prediction and the consistent robustness of LSTM models in difference sequence prediction?
