What Are The Best Execution Algorithms? Combining Deep Learning On Order Books with Reinforcement Learning For Profitable Trading!
Artificial Intelligence & Machine Learning
“Combining Deep Learning on Order Books with Reinforcement Learning for Profitable Trading” by Koti S. Jaddu and Paul Bilokon presents an innovative approach in the domain of high-frequency trading (HFT). This study focuses on enhancing trading strategies by integrating deep learning with reinforcement learning, a methodology well-suited to the fast-paced and data-intensive nature of HFT.
Overview of the Study
The research aims to leverage the vast amounts of data available in financial markets, particularly order book data, to forecast market returns across multiple horizons and generate profitable trading signals. The study employs a combination of deep learning for analyzing order book data and reinforcement learning for decision-making in trading.
Methodology
- Data and Instruments: The study uses data from two foreign exchange pairs (GBPUSD and EURUSD), two indices (DE40 and FTSE100), and one commodity (XAUUSD). This diverse set of financial instruments allows for a comprehensive evaluation of the models across different market types.
- Model Development: The authors developed one alpha extraction model and three different temporal-difference learning models, resulting in a total of 15 trading bots. The alpha extraction model forecasts returns over the next six timesteps, and the temporal-difference agents utilize these forecasts to provide trading signals.
- Training and Evaluation: The models were trained on ten weeks of order flow imbalance data for each instrument, divided into training, validation, and testing sets. Optimal parameters for each model were determined using a grid search, with a focus on parameters suggested in relevant literature.
Key Findings
- Backtesting Results: The backtesting simulations showed promising results, particularly with Q Learning models for GBPUSD and EURUSD pairs, demonstrating profitability after accounting for retail trading costs.
- Forward Testing Challenges: Despite success in backtesting, the models faced challenges during forward testing on a retail trading platform. One significant issue was the long processing times for making predictions, which led to missed trading opportunities.
- Learned Patterns and Limitations: The models successfully learned expected market patterns but also picked up some undesirable patterns. This issue was partly attributed to the limited horizon window and could potentially be addressed by extending this window.
Recommendations for Improvement
- Infrastructure and Hardware Upgrades: The study suggests using more advanced hardware and a different infrastructure setup to reduce prediction times and improve real-time trading performance.
- Expanding Data and Horizon Window: Collecting more data and increasing the horizon window beyond six timesteps could enhance the models’ accuracy and adaptability.
- Alternative Trading Platforms: Considering trading platforms with lower commissions and expanding the action space to include options like not holding a position could further optimize trading strategies.
Jaddu and Bilokon’s research offers significant insights into the potential of combining deep learning with reinforcement learning for HFT. While the results show promise, the study acknowledges the need for further refinements to fully realize consistent profitability in retail trading contexts. The research aligns with and extends existing literature in the field, providing a foundational approach for future studies aiming to exploit the synergies between machine learning techniques and financial market data for profitable trading strategies.
