Written by Radi Akbar
When the word ‘economics’ is uttered, most would think of the Federal Reserve, financial markets, and a discussion about a potential recession. Artificial intelligence, however, rarely comes to mind. Though economics has many definitions, British economist Lionel Robbins defines economics as “the science which studies human behavior as a relationship between ends and scarce means which have alternative uses”. So, economists have spent years working on solutions to find the optimal choice given some constraints. It should not come as a surprise that the work economists have done has found their application in the field of artificial intelligence.
The history of artificial intelligence and economics takes its root with the introduction of game theory. Although people have studied strategies throughout the century, the field was first formalized by John Von Neumann in his seminal paper, “On The Theory of Games of Strategy.” Neumann framed the problem of an agent with a set of strategies, where the agent’s job is to choose which strategy that maximizes their payoff. The catch is that the agent is playing with other players, so the optimal strategy is conditional on the other players’ movement. Through this framework, the phrase by Adam Smith, ”It is not from the benevolence (kindness) of the butcher, the brewer, or the baker that we expect our dinner, but from their regard to their own interest” no longer holds since there are instances when self-interested behavior leads to a lower reward than cooperation.
Prisoner’s dilemma is a famous example in game theory. The pay-off of cooperating is higher than deviating to actions that serve the prisoners self-interest (Source: Wikimedia.org)
Game theory changed the field of economics by giving more mathematical tools to explain instances of imperfect competition and deviating mechanisms. Through the lens of game theory, economists can give a more detailed mechanism for certain markets. For example, auction theory formalizes auctions as turn-based games, which helps economists study their outcomes and make predictions (KellogInsight, 2020).
So how do these developments in economics impact the field of artificial intelligence? Economists’ work can be applied to a sub-field of machine learning called reinforcement learning. This sub-field focuses on studying how intelligent agents choose a strategy that maximizes their reward given a certain state of the game. The difference between reinforcement learning and game theory is that reinforcement learning approaches this problem through experience and heuristics rather than finding analytical solutions.
The diagram above shows the basic idea of reinforcement learning. The agent makes an action on a certain environment and it gives the algorithm a reward. The algorithm will examine if the action gave the highest reward possible (Source: Wikimedia.org)
The problem with experience and heuristics is that it can take a while for reinforcement learning algorithms to converge to a solution. If the algorithm enters a game with 10 million strategies, then it would take a long time or be computationally taxing to find its optimal set of strategies. The solution concepts in game theory can be used to streamline reinforcement learning algorithms by restricting the number of strategies to be searched. Through analytical solutions like ‘rationalizability’, an agent would eliminate strategies that are never the best responses. By limiting the space of strategies, the algorithm can cut down its run time and converge on solutions faster.
Economics’ relationship to reinforcement learning is not one sided, but symbiotic. The field of behavioral economics introduced the idea of bounded rationality, which imposes restrictions on perfect rationality with constraints in cognitive capacity, time, and information. Usually, analytically solving these problems are very difficult, but reinforcement learning can approximate these solutions through simulation.
However, both fields still struggle on problems concerning complex games. A good example of this is a game with cumulative reward, where each action taken by an agent culminates to a final payoff. The problem with complex games is that the reward-maximizing heuristic is no longer efficient. This is because an action might yield the highest payoff, but doesn’t yield the highest cumulative payoff. The problem gets even more complicated when bounded rationality is introduced to these types of problems. Reinforcement learning still fares better at approximating outcomes, but these could take a long time since the algorithm uses trial and error method of learning.
Chess is a good example of a complex game with a cumulative reward. As taking pieces might reward the algorithm in the short-run, but not the long-run (Source: Wikimedia.org)
Machine learning researchers have made progress in addressing complex games. Algorithms can accept a model of a game like chess and use neural networks to train itself through ‘self-play’. The way it works is that the algorithm will be given a scenario and iterate through all possible actions. The algorithm will use neural networks to see if the path of actions taken by the agent is the optimal one. Though this process is computationally taxing, progress in computational mathematics, computer engineering, and the algorithm’s complete understanding of the game can significantly make this process much faster. Google’s AlphaZero algorithm uses a similar heuristic to beat other chess-playing algorithms such as ‘Stockfish’. Though the future looks promising in the field of AI, economics is an essential part of it. Currently, machine learning researchers are exploring evolving reinforcement learning algorithms (Google AI Blog, 2021), it so happens to be that nobel-prize economist Paul Milgrom has made significant progress in theoretical games with learning agents.
References
Abel, David, “Concepts in Bounded Rationality: Perspectives from Reinforcement Learning” (2019). Philosophy Theses and Dissertations. Brown Digital Repository. Brown University Library. https://doi.org/10.26300/3ayn-c785
Charpentier, A., Élie, R., & Remlinger, C. (2021). Reinforcement learning in economics and finance. Computational Economics. https://doi.org/10.1007/s10614-021-10119-4
Co-Reyes, J. D., & Miao, Y. (2021, April 22). Evolving reinforcement learning algorithms. Google AI Blog. Retrieved February 28, 2022, from https://ai.googleblog.com/2021/04/evolving-reinforcement-learning.html
Milgrom, P., & Roberts, J. (1991). Adaptive and sophisticated learning in normal form games. Games and Economic Behavior, 3(1), 82–100. https://doi.org/10.1016/0899-8256(91)90006-z
Mollner, J, (2020, Nov 11). What is “auction theory,” and what kinds of questions can it answer? Kellogg Insight. Retrieved February 21, 2022, from https://insight.kellogg.northwestern.edu/article/auction-theory-nobel-improve-how-search-engines-sell-ads
Nair, S. (2017, December 29). A Simple Alpha(Go) Zero Tutorial. Simple alpha zero. Retrieved February 28, 2022, from https://web.stanford.edu/~surag/posts/alphazero.html