Are Specialised Large Language Models (LLMs) Developed For Accounting & Finance Effective?
Review of Re(Visiting) Large Language Models in Finance by Rahimikia and Drinkall
Re(Visiting) Large Language Models in Finance by Eghbal Rahimikia (University of Manchester) and Felix Drinkall (University of Oxford) is a timely and ambitious contribution to the growing body of research on applying large language models (LLMs) to financial prediction and asset pricing.
As a result of focusing on domain-specific model construction, the authors address two pressing concerns in this field: the computational inefficiency of massive general-purpose models and the methodological pitfalls of look-ahead bias.
Contribution to Literature
The study advances the literature in three ways. First, it demonstrates empirically that specialised financial language models, dubbed FinText, can outperform general-purpose LLMs such as LLaMA 1–3, despite being up to 50 times smaller. This supports the argument that domain-specific training and dataset design matter more than raw parameter counts. Second, the research directly tackles look-ahead bias, a persistent methodological flaw in financial machine learning. By constructing year-specific models from 2007 to 2023, the authors create a framework that avoids contaminating historical training with future information. Third, the paper integrates explainable AI (XAI) into financial LLM evaluation, offering interpretability around token-level drivers of predictions.
The empirical approach is rigorous and replicates standards set by Peters et al. (2018), Ke et al. (2019), Chen et al. (2022), and Zhou et al. (2024). Specifically, the authors employ:
- Pre-training on domain-specific corpora including regulatory filings, news, board member data, and speeches by central banks.
- Probing techniques to extract textual features from FinText.
- Logistic regression models to convert extracted features into predictive signals.
- Portfolio construction using zero-net-investment strategies to evaluate model performance.
The robustness checks are comprehensive. They include alpha-adjusted returns, transaction costs, portfolio variations, and comparison across model sizes. Notably, the FinText base model achieves a Sharpe ratio of 3.45, significantly outperforming both LLaMA models and established financial NLP baselines such as FinBERT and FarmPredict.
Key Findings?
- Model Size vs. Performance: Smaller, domain-specific models can outperform general-purpose giants when trained on carefully curated financial data.
- Look-Ahead Bias Mitigation: The year-by-year training structure is a methodological innovation that sets a new standard for rigor in financial NLP.
- Interpretability Gains: The ability to identify token groups influencing long/short positions deepens trust in model-driven trading signals.
- Efficiency: FinText is computationally efficient, making it deployable in environments without high-end GPUs.
While the paper is impressive, several issues warrant further exploration!
- Generalisability: The models are tested primarily within asset pricing return prediction. It remains unclear how well FinText would generalize to other financial tasks such as credit scoring, risk management, or fraud detection.
- Overfitting Risk: Despite year-by-year pre-training, the danger of overfitting to textual idiosyncrasies of certain years may persist. Future work could expand validation across international datasets.
- Benchmark Expansion: LLaMA is a strong benchmark, but the study could also have compared against GPT-family models, which remain widely used in industry.
- Practical Trading Performance: While Sharpe ratios are compelling, real-world implementation under constraints such as liquidity, slippage, and regulatory limits requires additional testing.
Rahimikia and Drinkall provide an important and carefully executed study that meaningfully advances the conversation about LLMs in finance. Their work suggests that smaller, specialised models can deliver superior predictive accuracy, interpretability, and computational efficiency, all while mitigating methodological pitfalls like look-ahead bias. The FinText architecture and framework may become a cornerstone for future empirical finance research using textual data.
Overall, this is a well-structured, impactful contribution that bridges academic rigor and practical relevance. It is especially valuable for scholars in finance, accounting, and economics, as well as practitioners seeking efficient and transparent AI tools for market prediction.
Download the full paper: Re(Visiting) Large Language Models in Finance by Eghbal Rahimikia, Felix Drinkall :: SSRN
