Where is optimal control used? Using machine learning on problem solving vs. optimal control
With the emergence of Artificial Intelligence, people have begun freeing their hands by making machines work for them. Machine learning was first proposed around the 1950s by Donald Hebb based on a model of brain cell interaction. Machine learning, a subfield of artificial intelligence that is defined as the capability of a machine to learn human behaviors, created for problem solving.
In the1960s, general mathematical theory of optimal control, proposed for problem solving. Machine learning and optimal control theory have many similarities and unique advantages for solving problems.
Firstly, I’ll begin by stating the origin and history of the two methods. Machine learning is an important tool derived from artificial intelligence. Because of its strong learning and decision-making abilities, machine learning is becoming more and more popular in the information age.
Through the late 1970s, it was considered a part of AI’s evolution. Until it branched off to evolve on its own. “Now, machine learning has become a very important response tool for cloud computing and eCommerce, and is being used in a variety of cutting edge technologies.” (Keith,2021), almost at the same time, optimal control theory has been proposed by L.S. Pontryagin in 1956, and the key point of this theory was the Pontryagin maximum principle.
Richard Bellman’s equation shows up everywhere in the Reinforcement Learning literature, being one of the central elements of many Reinforcement Learning algorithms.
In summary, we can say that the Bellman equation decomposes the value function into two parts, the immediate reward plus the discounted future values. Mathematically we can define Bellman Expectation Equation as : Let’s call this Equation 1. The above equation tells us that the immediate reward determines the value of a particular state. Plus the value of successor states when we are following a certain policy
Problems studied in the mathematical theory of optimal control have arisen from practical demands, especially in space flight dynamics and automatic control theories. Through evolution, the optimal control theory also contains the mathematical basis of the theory of controlled motions. By now, the mathematical theory of optimal control closely links with analytical mechanics. (A.B. Kurzhanskii)
Next, we will introduce some insight into how machine learning and optimal control work.
For machine learning, the main goal is training machines to make classifications or predictions based on a huge input of data by statistical or algorithms.
Machines can emulate human behavior when processing these inputs and provide the best way to solve problems. In general, machine learning can mimic humans to analyze data. In addition, information to classify them as a specific system, when the system finishes. When a new input enters the data set, machines automatically classify this new input. The calculation speed is very fast and the results always have a high accuracy.
However, control theory should compare not with machine learning in general, but specifically to reinforcement learning.
The latter solves the same tasks as the control theory but using other methods. The ‘classical’ model-free reinforcement learning operates with data itself without making assumptions about the dynamics. But, nowadays called ‘model-based reinforcement learning’. Is in fact quite close to the classical control theory, extending its arsenal of computational tools.
The main goal of optimal control is to formulate and solve problems to identify the optimal control for a dynamical process. Furthermore, these theories that need optimal control theory are usually based on a dynamic model and need a constrained approach to find the best solution based on their input parameters (usually a maximum or minimum point). Now, it is is a mature theory used in many mathematical or engineering fields. Moreover, the models used to conduct the optimal control have increased.
In conclusion, machine learning and optimal control are all useful tools to find the ‘best’ solution by using some models or theories.
Lastly, Deep Learning Godfather, Turing Award Winner & Facebook’s Head of Ai Yann LeCun told Rebellion Research:
“There are two ways to optimize your predictions, there are two ways to approach this problem, the first one is reinforcement learning(machine learning), the other one is optimal control. In many situations the right thing to do is more akin to optimal control than to reinforcement learning (machine learning). It is much more efficient in many ways.”
Back To News
Written by Sirui Hao & Edited by Avhan Misra & Mitchel Wang
Reference:
IBM Cloud Education. (n.d.). What is machine learning? IBM. Retrieved March 25, 2022, from https://www.ibm.com/cloud/learn/machine-learning
Becerra, V. M. (n.d.). Optimal control. Scholarpedia. Retrieved March 25, 2022, from http://www.scholarpedia.org/article/Optimal_control
Encyclopedia of Mathematics. URL: http://encyclopediaofmath.org/index.php?title=Optimal_control,_mathematical_theor y_of&oldid=48051 https://www.dataversity.net/a-brief-history-of-machine-learning/
Lastly, Foote, K. D. (2022, January 20). A brief history of machine learning. DATAVERSITY. Retrieved March 25, 2022, from https://www.dataversity.net/a-brief-history-of machine-learning/
