Modelling Economic Agents as Deep Reinforcement Learners
Jeppe Druedahl will model economic agents as imperfect problem solvers - the standard approach in economics is to assume economic agents behave as if they perfectly solve a mathematical optimization problem derived directly from the environment the agents are assumed to live in and the objectives they are assumed to aim for.
Consumption-saving behavior in a simple life-cycle model, for example, follows from an environment in terms of the income and asset return risks households face and objectives in terms of their inter-temporal preferences for consumption.
Even for pure technical reasons the assumption of perfect problem solving is problematic because solving the full model then requires solving the optimization problems each agent face. This becomes a central computational bottleneck for model builders, and implies that only rather simple environments are considered. In a consumption-saving model, this implies a danger of over-simplifying the income and return risk households face, such that we get a distorted understanding of consumption-saving behavior in general and the drivers of precautionary saving in particular.
Jeppe Druedahl's hypothesis is that modelling economic agents as imperfect problem solvers is the way forward to break the bottleneck. His specific approach in this project will therefore be to endow economic agents with artificial intelligence by assuming they behave as if they solve their mathematical optimization problems imperfectly with deep reinforcement learning.
Jeppe Druedahl explains: "I consider consumption-saving behavior as a test case for my hypothesis. Firstly, because it plays a central role in all macroeconomic models. Secondly, because it allows me to design experiments to empirically test whether individuals behave as if they are deep reinforcement learners. Deep reinforcement algorithms were developed with the prescriptive goal to solve problems »good enough«, but when assuming economic agents behave as if using such algorithms, their descriptive content for observed behavior must also be investigated.
In the first theoretical and computational work package, we start from small models, which can be solved perfectly, and investigate how assuming deep reinforcement learning implies behavior, which is different from omnipotent rational behavior both quantitatively in terms of a loss in payoff/utility, and qualitatively in terms of changes in behavioral patterns.
Finally, we consider multi-agent models, where each agent is considered small, and prices and other external factors are taken as given.
In the second empirical work package, we collect evidence from an online life-cycle consumption-saving game, to empirically test whether economic agents can be modeled as if they are deep reinforcement learners.
Researchers
| Name | Title | Job responsibilities | |
|---|---|---|---|
| Jacob Emil Thorn Røpke | PhD Student |
|
|
| Jeppe Druedahl | Associate Professor | Macroeconomic Questions; Models with Uncertainty and Heterogeneity; Micro-level Data; Computational Methods |
|
Funded by:
Modelling Economic Agents as Deep Reinforcement Learners has received the prestigious Sapere Aude grant - a four year funding from Independent Research Fund Denmark
Period: 2023 - 2026
Contact
Principal investigator Jeppe Druedahl