NLP For Stock Market Prediction

NLP For Stock Market Prediction

One firm ProntoNLP believes they have harnessed Natural Language Processing for alpha generation. They have shared their research with us below:

Testing the Basic ProntoNLP Extracted Events

There is a longstanding literature about the efficacy of an earnings-surprise-based signal; returns drift in the direction of the earnings surprise for at least one quarter. Stocks with the most positive earnings surprises enjoy positive abnormal returns in the subsequent quarter after earnings are announced.

These findings led to the utilization of an earnings surprise as a signal in stock selection. However, in spite of the importance of earnings as a summary measure, there are many  other layers of information that are provided by management as part of the earnings conference  calls. ProntoNLP extracted many “events” from the S&P transcripts of earnings conference calls,  such as acquisitions, margins, backlog and buybacks, some of which cover the past, but many  which have ramifications for future performance. The purpose of this write-up is to examine a  signal that is constructed from the ProntoNLP basic model. The software easily enables users to  write their own event extractions which could improve on the basic extractions offered by  ProntoNLP. 


We begin with the event extractions from the S&P earnings conference calls, which begins to have  good coverage in the third quarter of 2009, reaches over 6,000 earnings conference calls in the last  quarters of 2021, some of which are by foreign companies. ProntoNLP extracts its events from  each transcript and assigns a score to each transcript which is based on the difference between the  sum of positive and negative events, scaled by the total sum extracted events. This measure can be  thought of as a sentiment of the conference calls that is based on important events for management  and analysts.

We construct a tone change measure by focusing on the current sentiment minus the  average sentiment in the prior four quarters. We then match the conference call dates to those of  the preliminary earnings announcement date from Compustat, and require the two dates to be  within one day of each other. This ensures that we have actual earnings conference calls so we can  compare the extracted tone change signal to the earnings surprise signal. 

We then match the companies identified by the conference calls transcripts with returns from the  CRSP database. There were roughly 1,000 companies in late 2009 climbing steadily to over 2,000  companies in mid 2015, and ending with over 2,300 companies in late 2020. In addition, we extract  the earnings surprise from IBES using both the IBES actual and mean forecasts of quarterly  earnings by all analysts in the 90-day period before the earnings announcement (where only the  most recent analyst forecast is retained). The standardized earnings surprise (SUE) is the actual  earnings minus mean forecasted earnings scaled by the standard deviation of the analyst forecasts during the 90-day period. Requiring analyst forecast data reduces the sample a bit, still around 

1,000 companies initially, building up to around 2,000 in 2017 and ending with over 2,100  companies in 2021. 

For each month-end beginning in December 2009, we identify the most recent earnings announcement for each company that had an earnings conference call and where the earnings surprise signal was available. We ranked the two signals (earnings surprise and conference calls0 separately into deciles, where decile zero had the most negative earnings surprises (conference call tone change) and decile nine the most positive. We assign the decile rank to each member of the decile, and subtracted 0.5, to obtain a variable that is in the range of [-0.5, +0.5]. Furthermore, We examine the signal performance by focusing on performance in the following month.

We use two primary methodologies to test the signals. The first is by using Fama and MacBeth monthly cross-sectional regressions where the dependent variable is the abnormal return in the following month and the independent variable is the scaled signal rank. The coefficient on the signal is equivalent to a hedge portfolio that takes long positions in the most positive signal decile and short positions in the most negative, except that it uses all companies, not only in the extreme signals. We assess the strength of the signal by examining the average coefficients of the 145 monthly cross-sectional regressions and their standard deviation. The abnormal returns are the buy and hold return on a security minus the value weighted buy and hold return on a portfolio of similar companies in terms of size (market  value of equity, 3 groups), Book/Market (3 groups) and 11-month momentum (t-12 through t-1, 3  groups).

We also use a Fama – French factors analysis to assess whether the signal has returns after controlling for the Fama- French five factors. Furthermore, we first calculate the average monthly return on the top decile during the 145 months in the sample period. We the run a regression of the monthly return minus the risk free rate on the five F-F factors. If the intercept is significantly positive it indicates abnormal performance above the five risk factors. 

Results: We first evaluate the correlation between the earnings surprise signal and the conference call tone  change signal. A high correlation would indicate that most of the information in the conference  call can be captured by the earnings surprise signal. We find that the actual correlation between  the two signals is about 11-12%. This is a low enough correlation, which indicates that conference  calls can capture additional information beyond that which is captured in earnings.1

The top decile of earnings surprises, i.e., the most positive surprises, earned 17BPS (the intercept in the first row) beyond the F-F five factors, which was significant at the 5% level. It had a beta of  1.09, indicating higher systematic risk, which is understandable given that these companies had  extreme earnings surprises possibly because of a higher operating risk. It also showed positive tilts  towards smaller and value companies, but not any significant tilt to the profitability or investment  factors. The most negative earnings surprises earned 23BPS above the F-F factors, significant at  the 10% level. The positive return on the short portfolio is the opposite of what one would expect,  but is consistent with our observations earlier. 

In contrast, we find that the portfolio of the most positive conference call tone changes had 27BPS  return after controlling for the F-F factors, significant at the 2% level. It also had a higher beta of  1.13, and positive tilts to smaller and value companies. The most negative tone changes actually  had a negative return of 9BPS after controlling for the F-F factors, although this negative return  was not significantly different from zero.

Main Takeaways: 

1. The earnings surprise signal has not worked well in our sample during the period of the  study, although the long portion of the signal (the most positive earnings surprises)  showed positive and significant abnormal returns. 

2. The conference calls tone change signal earned significant abnormal returns, but mostly  for the long positions (the most positive tone changes). The returns on the short positions  were not materially different than zero. 

3. The correlation of the conference call tone change and the earnings surprise signals is low  at 11-12%, indicating that the conference calls provide additional information beyond  earnings. 

Performance Graphs 

Figure 1 – Comparison of Return of our Long signal (top 10%) vs. Earning Surprise and the Market (S&P 500) – Risk Free Return 

• MKT_RF is the market return (Russell 1000) – Risk Free Return • LongSignal = Our NLP based signal 

• longsueaf1signal = The Earning Surprise based return

Abnormal return (FF 3 factors) of top 10% (long) and bottom 10% (short). Buy and hold return on the security minus buy and hold cap-weighted return on similar stocks (size, B/M and momentum). Monthly Rebalancing. 

Value and Growth versions of the basic signal 

We identified what are the main events that characterize value companies or growth companies. For the  value tilted signal, we gave events that are highly associated with value companies a weight of 3 and all  other events got a weight of 1. For the growth tilted signal, we gave events that are highly associated with growth companies a weight of 3 and all other events got a weight of 1.  

As can be seen the value tilted signal (green) performs better than the basic signal (orange) and much  better than the growth tilted signal (red).

NLP For Stock Market Prediction

1 We also tested another signal that is based on earnings estimate revisions. We summed up the number of Up, Down and Zero revisions in the 3-month period prior to the month of portfolio construction. We then calculated diffusion signal of (Up-Down)/(Up+Down+Zero). Lastly, we assigned the rank of +0.5 to those diffusing measures above 0.5, -0.5 to those below -0.5 and zero otherwise. The earnings revision signal was also only 11-12% correlated with the conference call signal.

Back To News