The Keys To AI-Assisted Investing

The Keys To AI-Assisted Investing

AI-Assisted Investing : Advantages of AI

AI-Assisted Investing : Artificial Intelligence solutions can offer a lot of advantages over human expert decision-making.

Computational ability

First, compared to their human counterparts, AI programs are capable of incredible mathematical precision.  Computers have an enormous ability to crunch numbers.  Even if you arranged it so everybody on Earth got together and started to add and subtract numbers, they wouldn’t come close to the number crunching ability of a $1,000 dollar consumer computer, let alone a fast mainframe supercomputer.  

Perfect recall and vast memory abilities

In contrast, humans are pretty bad at remembering specific facts.  A lot of times only vague or half-remembered truths are used to justify decisions.

Algorithms are reproducible and dispassionate

Artificial intelligence algorithms are dispassionate, and are not saddled with preconceived notions.  They can only make rules which are actually found in the training data sets.  This can sometimes be a problem for the AI when the biases and perceptions a human would have are actually accurate, but not reflected in the data.  Oftentimes, this is not the case, and human decision makers generally carry a lot of baggage with them.  When things are going well, humans often ignore or trivialize potential negative outcomes.  Conversely when things are going poorly, humans often ignore or trivialize potential positive outcomes.

Disadvantages of AI programs

Artificial Intelligence programs are saddled with their own drawbacks however.

Framing the Problem
Because Machine Learning algorithms depend on mathematical optimizations, it can be difficult to transform the question you want answered, or the desired behaviour into a proper mathematical context.  Oftentimes, the problems that the machine learner “solves” are only approximations of what you actually want to solve.  For example, in finance, mean-variance optimization has become the gold standard in deciding how to create your portfolio.  Even making the inaccurate assumptions that people have exponential utility curves and that returns are normally distributed , one would only be led to the belief that the Sharpe Ratio, the mean divided by the standard deviation should be optimized, not mean-variance.  However, mean-variance is used simply because it is computationally and mathematically easier to solve for.  Google

The Keys To AI-Assisted Investing

Cannot create important factors from a set of rules

It is also currently very difficult for the AI to come up with it’s own problem to solve, or to come up with it’s own factors.  For example, chess programs, and even Deep Blue need to have the factors they use in order to make decisions be fed into them by the programmers.  This is somewhat disappointing, since in a very meaningful sense, everything about a game is known.  Chess as a game has very clear and explicit win conditions, methodology for moving pieces, and there is only a finite number of different board positions.  (Blondie24) Even so, attempts to create machine learning algorithms which attempt to abstract meaningful factors from the rules have not been terribly successful.

Dependent on programmer to provide it data

The learning algorithm cannot learn what you do not give it. Black Box. Thus the computer programmer needs to be capable of determining which factors are important and relevant.  If he misses a factor, the learning algorithm may not perform very well, or even at all.  The way around this is the “Kitchen sink” method wherein every conceivable piece of data is given to the machine learning algorithm.  This does eliminate the possibility that the programmer leaves out a crucial piece of information, however throwing everything at the problem including the kitchen sink leads to…

Overfitting the Data

Overfitting the data.  Machine learning algorithms are so powerful and so exact, that if one is not careful in designing the learner, one can come up with nonsense.  Overfitting is essentially finding patterns and signals which happened to be true in the data set, but will not be true in data outside of the training set.  So when actually using the algorithm, poor decisions will be made.

Types of Overfitting

There are two general ways in which an artificial intelligence system may over-fit the data.

Spurious Correlations

The first and most common way to overfit is due to spurious correlations in the data.  Spurious correlations occur when, by random chance, a factor given the AI seems to be highly correlated with the output.  However, since this correlation happened by chance, it will not be present in data outside the training set.

Bangladeshi Butter Production

An example which has sort of been passed around for a while now is the Bangladeshi butter production.  In a study performed in 1995, a Caltech professor took hundreds of data series published by the UN and their respective member countries, and tried to find the data which would have been the best predictor of the S&P 500.  The result was that over the time period 1983-1993, the production of butter in Bangladesh had the highest correlation, above 0.85.  Of course this was a nonsense result, and for years both before 1983 and after 1993, this correlation would be non-existent.  

Garbage In Garbage Out

The more factors that are provided to a learning algorithm, the greater the chance is that the algorithm will learn something that is only true over its training set, and will not be true in the future.

Model Complexity

The second common method of over-fitting is creating a model that is more complicated than the signal it is trying to capture.  This problem is not caused by having too many factors, but rather by allowing the algorithm to have too much leeway in deciding how to relate those factors to the output.

Of course, you do not want to arbitrarily curtail the complexity of your model, and harm it’s ability to make intelligent choices, so often models are created that can be extremely complex.  Overfitting can then be avoided with an appropriate choice of performance measure.

AGE vs Height Example

Scatterplot, perhaps representing the age of a child on the x-axis, and his height on the y-axis.

Least Squares Regression Line

Give the learning algorithm too much leeway, get nonsensical “better” fit

So, to sum up 

Machine Learning Algorithms are like an unstable explosive.  When handled correctly, they can move mountains, but when mishandled they are more likely to blow up in your face.

The Keys To AI-Assisted Investing

Written by Jeremy Newton

Edited by Alexander Fleiss

Leading Artificial Intelligence and Financial Advisor – Rebellion Research

Building an AI-Based Machine Learning for Global Economics (

The Keys To AI-Assisted Investing