How Design Choices in Machine Learning Shape Stock Return Predictions
Review of “Design Choices, Machine Learning, and the Cross-Section of Stock Returns” by Chen, Hanauer, and Kalsbach
In their timely and methodologically rich study, “Design Choices, Machine Learning, and the Cross-Section of Stock Returns,” Minghui Chen, Matthias X. Hanauer(see our review of Hanauer’s paper: Factor Zoo (.zip) : A Review), and Tobias Kalsbach deliver a comprehensive evaluation of how research design decisions influence the outcomes of machine learning (ML) models in asset pricing.
Presented at the 2025 Conference on Asset Pricing and Machine Learning in Gothenburg, this paper adds clarity and structure to a field that often suffers from opacity and inconsistent benchmarking.
Summary and Contribution
The paper investigates an impressive universe of 1,056 machine learning model configurations used to predict stock returns. These configurations systematically vary across key axes of model design: algorithm type, target variable definition, feature treatment, and training methodology. Chen, Hanauer, and Kalsbach’ central finding is both striking and important: nonstandard error stemming from design choices exceeds the standard error by 59%. This means that model performance is more sensitive to design than to the inherent noise in financial data—a profound implication for researchers and practitioners alike.
Chen, Hanauer, and Kalsbach provide a systematic framework for understanding how these choices affect return predictability, emphasizing that the design of the target variable must align with the evaluation metric and investment goal. Their empirical results show wide variability in top-minus-bottom (long-short) portfolio returns, ranging from 0.13% to 1.98% per month depending on the design structure.
Key Findings
- Design Decisions Dominate Model Performance
Model outcomes differ more because of design choices than statistical noise. For example, switching from a rolling to an expanding training window improves monthly returns by 0.20%, and excluding unpublished features reduces performance by 0.52% on average. - Target Variable Matters Most
Among all model components, the definition of the target variable is the most critical. Aligning it with appropriate evaluation metrics dramatically improves relevance and performance. Using abnormal returns relative to the market as the target yielded the most consistent gains. - Non-Linear Models Aren’t Always Better
Contrary to common belief, non-linear ML models only outperform OLS regressions under specific conditions—notably when using continuous return targets and expanding training windows. This challenges the narrative that deep or complex models inherently offer better predictive performance in finance. - Feature Selection and Post-Publication Effects
Chen, Hanauer, and Kalsbach show that excluding features not published in peer-reviewed literature significantly weakens performance. This finding suggests a strong link between academic transparency and predictive power, raising interesting questions about reproducibility and publication bias.
Strengths of the Paper
- Comprehensive Scope: The sheer breadth of tested configurations makes this paper uniquely thorough.
- Actionable Insights: Chen, Hanauer, and Kalsbach offer clear guidelines for modelers, especially regarding which target definitions and training regimes lead to stronger results.
- Clarification for Practitioners: The study provides much-needed structure in an otherwise fragmented body of machine learning asset pricing literature.
Areas for Further Exploration
While the study is robust, it could benefit from:
- Real-world implementation context: A deeper discussion on how these findings translate to institutional workflows and trading systems would enhance practical value.
- Alternative asset classes: The analysis is limited to equities; extending it to bonds, commodities, or crypto markets could test the generality of the conclusions.
- Model interpretability: As Chen, Hanauer, and Kalsbach focus on prediction, future work could assess how design choices impact model explainability—a growing concern in applied ML.
Final Thoughts
This paper is a major contribution to the intersection of machine learning and asset pricing. Moreover, confronts a crucial and often ignored truth: in finance, how you model matters as much as what you model!
As a result of rigorously quantifying the role of design choices, Chen, Hanauer, and Kalsbach equip researchers and practitioners with the tools to create more reliable, interpretable, and effective return prediction systems.
For anyone building ML models in finance—or evaluating them—this paper is not just recommended reading. It’s essential!
Read the full paper:
Design choices, machine learning, and the cross-section of stock returns by Minghui Chen, Matthias X. Hanauer, Tobias Kalsbach :: SSRN
Consider our next conference:
Cornell Financial Engineering Manhattan 2025 Future of Finance & Ai Conference
