How Does Game Theory Relate to Reinforcement Learning? Why Reinforcement Learning Works Better in Video Games!
Artificial Intelligence & Machine Learning
Reinforcement Learning is a machine learning method used by agents to acquire optimal policy knowledge through learning by trial and error. It is incorporated into sequential tasks of making decisions. Imitating human experts’ behavior in playing video games is a bit challenging for computers. However, with the implementation of reinforcement learning, this has become a success. One of the main contributions to the success of video game development is the availability of platforms.
These platforms are categorized into generic and specific platforms.
Reinforcement learning is applied to various types of video games such as Atari, first-person perspective, and real-time strategy games (Shao et al. 5). Reinforcement learning is also utilized in the financial markets sector to make buying and selling related decisions. There has been a significant increase in machine implementation to come up with trading decisions concerning stock and foreign exchange markets. Raw data is extracted as inputs and found within the training process thereby deciding on the situation at hand as the output. New information from this raw data is repeatedly fed to the agent in an iterative process allowing it to maximize the value of a reward that is already predetermined (Meng and Matloob 1).
Video games work better with reinforcement learning compared to financial markets.
Despite the complexity of the environment in video games, it has more stability and is more predictable. These games consist of both rules and logic. And they are designed in a way that they should not mean varying things at varying times. With the power of reinforcement learning in learning various visual and textual representations, there are exemplary results in video games’ applications compared to financial markets applications.
One of the reasons behind this outcome is the fact that in video games, variables that have an impact on rewards, possess a significantly low signal-to-noise ratio and change can take place after some time. In financial markets, on the other hand, the signal-to-noise ratio of these variables is very high. There are cases in which the RL model picks up random noise present in financial data. And uses it as input that needs to become acted upon. This eventually leads to incorrect trading signals. In addition, financial markets face the overfitting challenge whereby future markets have the probability of having a huge difference compared to those from the past. It, therefore, becomes difficult to perform an estimation of the risks related to losing money in the upcoming events in the absence of future data.
Another real factor is back testing.
It is impossible to make repeated decisions over a small period since past market situations tend to be a bit limited and as discussed earlier, it is difficult to have future knowledge since it cannot be simulated due to uncertainty of influencing factors. Interpreting the learned policy in financial markets is time-consuming and complex as well. This is an action meant to determine the various financial discoveries, understand the motivation behind the agent’s policy, and how well the policy can be explained through well-researched economic theories (“First Steps Before Applying Reinforcement Learning For Trading”).
Type 2 chaos is another hindering factor for financial markets’ success in reinforcement learning. The RL model works in isolation when being trained since it does not interact with the market at hand. Upon its deployment, there is uncertainty about how it will correlate with the market. In video games, however, the RL model interacts with the game at hand. And one can know how the game will become affected before deploying it. This allows making of necessary adjustments hence better success in video games compared to financial markets.
Written by Yuchen Wang
References
“First Steps Before Applying Reinforcement Learning For Trading”. Medium, 2021, https://odsc.medium.com/first-steps-before-applying-reinforcement-learning-for-trading-f45c15e98bca.
Meng, Terry Lingze, and Matloob Khushi. “Reinforcement learning in financial markets.” Data 4.3 (2019): 110.
Shao, Kun, et al. “A survey of deep reinforcement learning in video games.” arXiv preprint arXiv:1912.10944 (2019).
