Market Self-Learning of Signals, Impact and Optimal Trading: Invisible Hand Inference with Free Energy (Or, How We Learned to Stop Worrying and Love Bounded Rationality)

Market Self-Learning of Signals We present a simple model of a non-equilibrium self-organizing market where asset prices are partially driven by investment decisions of a bounded-rational agent. The agent acts in a stochastic market environment driven by various exogenous “alpha” signals, agent’s own actions (via market impact), and noise.

Unlike traditional agent-based models, our agent aggregates all traders in the market, rather than being a representative agent.

Therefore, it can be identified with a bounded-rational component of the market itself, providing a particular implementation of an Invisible Hand market mechanism.

In such setting, market dynamics are modeled as a fictitious self-play of such bounded-rational market-agent in its adversarial stochastic environment.

As rewards obtained by such self-playing market agent are not observed from market data, we formulate and solve a simple model of such market dynamics based on a neuroscience-inspired Bounded Rational Information Theoretic Inverse Reinforcement Learning (BRIT-IRL). This results in effective asset price dynamics with a non-linear mean reversion – which in our model is generated dynamically, rather than being postulated.

We argue that our model can be used in a similar way to the Black-Litterman model. In particular, it represents, in a simple modeling framework, market views of common predictive signals, market impacts and implied optimal dynamic portfolio allocations, and can be used to assess values of private signals. Moreover, it allows one to quantify a “market-implied” optimal investment strategy, along with a measure of market rationality.

Black-Litterman model

Our approach is numerically light, and can be implemented using standard off-the-shelf software such as TensorFlow.

The model is designed as both a practical tool for market practitioners, and a theoretical model of a financial market that can be explored further using simulations and/or analytical methods.

In essence, the BL model flips the Markowitz optimal portfolio theory on its head and considers an inverse optimization problem. Namely, it starts with an observation that a market portfolio (as typically represented by the S&P500 index) is, by definition, the optimal” market-implied” portfolio.

Therefore, if we consider such a given market portfolio as an optimal portfolio, then we can invert the portfolio optimization problem, and ask what is the optimal asset allocation policy that corresponds to this optimal market portfolio

Download The Full Paper

Market Self-Learning of Signals Written by Dr. Igor Halperin & Ilya Feldshteyn

Dr. Igor Halperin on Reinforecement Learning & IRL For Investing & The Dangers of Deep Learning