Our AI

Artificial intelligence and machine learning have become a very powerful decision-making tool. Machine learning algorithms really start to shine when the problem offers a large amount of data from which to learn. Rebellion Research’s Machine learning algorithms have shown a tremendous ability to take these large data sets and accurately combine them with different factors to create global economic predictions–often better than even a team of financial & economic experts. So here is the question: Why aren’t AI investment strategies used more often in investing?

After all, the financial markets offer a bewildering array of information, from prices for each tick, options chain prices, to fundamental information about companies and securities, etc. In addition, all this data goes back decades. With these complexities, it seems impossible that we humans are truly taking into account all the relevant information when they make decisions. Instead, this should be an area where machine learning algorithms (can) really shine.

Unfortunately, replicating Rebellion Research’s success at global economic predictions has met a few hurdles along the way.

Investment Process

  • Challenges of AI Investing

    It turns out that when trying to apply machine learning to longer-term investments, a number of challenges begin to arise. Rebellion Research took almost 4 years to complete the build-out of our first alpha-generating machine learning algorithm.

  • System Constantly Evolving

    First, the financial system is constantly changing and evolving, and it does not change smoothly. In 1990, Australia had very little exposure to China, while today it moves on China’s industrial output numbers. Similarly, Apple in 2021 is not the same company as it was in 2000 or even in 2010.

  • Market Conditions Constantly Changing

    Daily volatilities of stocks undergo rapid shifts every day. If you are trying to make predictions using historical data, you have to take into account that the past market that you are using is not the same as the current one. It is similar, but certainly not the same. So Rebellion Research had to create a process to weigh current economic and market data vs historical data. How much is 2012’s data worth vs that of 2008 or 2015? These are tough questions for any machine learning programs.

  • Meaning of Factors changes over time

    It is even hard to find factors to base decisions on that are not changing. Apple has changed significantly from what it was 10 years ago. Back then, each dollar of revenue, and each dollar of profit that Apple made came from its computer sales. Now, of course, the majority of those profits and revenues are from the sales of phones and services.

So how would Rebellion Research weigh Apple over time? 

We can see that individual risk factors–and the way we should  value Apple’s revenue–has shifted dramatically over the past decade.  This problem is not really an issue–or at least not a significant one–when dealing with shorter-term high frequency traders.  For predictions about a couple  of days or minutes, you don’t have to learn from decade-old data, so you don’t need to  worry as much about changes in the distributions of your factors.  For longer-term investment horizons, however, you need to consider these factor drifts.

High Noise to Signal Ratio

When applying Rebellion Research’s machine learning algorithms to long term investing , we faced another impediment: t as the time horizon increases on holdings, so do the volatility and noise.  

An increase in noise leads to  increased risks of overfitting the dataset you are learning from.  So what you really want to learn is patterns or signals that  will stay true in the future.  When there is a lot of noise in the data, it becomes easy to start “learning the noise”, which means learning the patterns or signals that will not remain true in the future but are just an artifact of the volatility in the system.

Very Complicated and inter-related Systems

To top it all off, Rebellion Research faced financial markets and stock prices that are incredibly complicated and chaotic systems.  

When only making short term predictions, it is fairly easy to model the markets.  You have a group of unknown buyers who are willing to buy the stock, and a group of unknown sellers willing to sell.  Then, you can gain hints and clues about the numbers of buyers and sellers and at what price they are willing to buy and sell through the order books–both of that stock and other stocks. From that information, you can try to come up with a way to buy low and sell high.  

However, when you start holding stocks for many days, weeks, months and even years, the system becomes much more complicated.  Suddenly you have to think about  the appreciation or depreciation of the dollar, increased commodities prices, surpluses of natural gas inventories, Greece defaulting on its debt, US GDP figures, China trade deficit, and the list goes on.  With so many factors that influence the direction of a stock’s price, it can be tempting to throw more and more factors into the learning algorithm.  However, especially due to the high level of noise in the system, adding too many factors can easily lead to the Bangladeshi butter problem.  It is all too easy to find spurious correlations in the data set, which can trip up even the best machine learning algorithms.

So these are the challenges that Rebellion Research faced in creating long-term investing with machine learning algorithms.  

And the reason why there aren’t more people  applying machine learning to make investment decisions?  

Well, it simply won’t work if you just go on the web and download a Neural Network or Support Vector Machine software package, handing it a bunch of data, and sitting back and waiting for results!

Rebellion Research had failed at this for 4 years before we finally found a path to success. 

AI Investing in Practice

Of course, that is not to say that these challenges are insurmountable.  We at Rebellion Research have spent many years working on and devising an algorithm that  has proven itself capable of overcoming these hurdles.  In order to deal with the problems created by longer-term investment horizons, we made a few adjustments to the learning algorithm.

Large Set of Factors that Correspond to Investment Styles

First,Rebellion Research came up with a large set of factors to feed the machine.  These factors were selected in a way that  spans the breadth and depth of investment strategies.  In other words, we discovered  a way of combining our factors to recreate almost every investing style: whether it’s a deep value investing which cares mostly about the value of the assets a company holds, or value investing more interested in steady and conservative cash flows, or growth investing that looks for steady and continued growth rates from a market leader, or a growth investing that’s looking for the next “game-changer” start-up company that will revolutionize its industry.  The investment styles encompass most styles of investing, including Value/Growth Momentum/Contrarian and Macro styles.  

Rather than finding out which stocks are likely to outperform, and by how much,  Rebellion Research repositions its learning problem to seek the styles of investing that will likely  perform well in the future.  

Create Stable Factors

By looking at the problem in this slightly different way, Rebellion Research was able to solve the problem where the  meaning of our factors changes  in our historical data set.  

As the sources of a company’s revenue change over time, , the value of a dollar of earnings for that company is not constant over time. It depends on how the company made that dollar and how likely it will keep earning that dollar.  However, investment styles, although they have become more nuanced with time, are relatively constant. When a deep-value style that ensures  a company’s liquidation value exceeds its market cap, it will make the necessary adjustments if the company’s assets start to deteriorate (but stay the same on the balance sheet).  In this way, using investment styles  allows us to actually use more than a decade of historical data, without losing relevance to the styles of today.  

Factors contain lots of relevant information

Unfortunately, using these factors does have a drawback.  In order to span the space of investment styles, which is huge , you need a lot of factors.  In fact, we use several thousand factors to make our investment decisions.  This does not mean that all of these factors are incorporated for each stock; rather, we use only about 30-40 of them. Still, the presence of these factors alone can  lead to a much greater chance of over-fitting.  

Correcting For Over-fitting

For example, we can imagine a naïve way of using these factors to make an investment portfolio.  One could look at how the factors have performed historically and  come up with the investment style that  has performed the best over the past decade.  Then we can simply buy all the stocks which correspond to that style, sit back, and watch the alpha accrue.  Unfortunately, this approach is almost certainly doomed to fail.  The risk that the single best investment style of the past decade will not work in the future is simply way too high.  When dealing with so many factors, some are always bound to float to the top–or near the top of the list–just by random chance.

Modified Bayesian Learner

That brings us to the second modification Rebellion Research made: we use a modified Bayesian Learner.  

Bayesian Learners Update probabilities given new information 

Bayesian learners are well known as useful algorithms for machine learning.  

They  get their name because they rely on what is known as Bayes’ theorem in probability, which dictates how probabilities of events should be updated given new pieces of knowledge.  What is particularly appealing is that the programmer can specifically model and control the ability of the learning algorithm to actually learn the data.  

Rate of Learning can be controlled

The programmer can set the burden of proof that the data in the training set needs to overcome and how precise the eventual predictions can be.  By controlling the precision of the algorithm, Rebellion Research can make sure that a single strategy does not completely dominate all the others.  

Algorithm does not become too precise

In this way, our investment decisions on a single stock are not dominated by any one particular strategy, but are influenced by about 30-40 factors. The number of strategies which influence our portfolio as a whole is much larger.  

Incorporating these factors helps to mitigate the risks of spurious correlations.  To be sure, a lot of the investment styles Rebellion Research holds in a positive light will turn out to be a result of spurious correlations; they will not be good indicators of stock performance in the future.  However, by relying on many different investment styles in order to make our investing decisions, Rebellion Research can make sure that at least some of our factors are meaningful. Thus, the fate of the fund does not rely on making sure any 1 investment style will continue to perform in the future.

A corollary to using this modified Bayes learner to make predictions about our stocks is its improved resilience to changes in future performance of investment styles.  As we know, one investment style can sometimes be superior to another style for very long periods of time.  Throughout the 90’s, growth strategies were significantly better than value strategies, and after the dot-com crash, value suddenly reigned supreme again.  Thus, even if a certain investment style tends to do well over the long-term, it can still  suffer prolonged periods of lackluster performance.  Since  our investment decisions rely on such a variety of different investment styles, we effectively can mitigate the effect of a sudden switch of performance characteristics between some styles. a couple of styles suddenly switching their performance characteristics.

To wrap things up, we can see that long-term investing horizons give some hurdles to applying artificial intelligence.  However, these challenges can be surmounted with some  foresight and careful design of the learner, and Rebellion Research fully expects machine learning algorithms to play a larger part in actually investing in the stock market, as opposed to just trading in it.