Optimisation and simulation models – how, and when, to use them
Hector Pollitt argues that there is great value in economic modelling results provided that the right sort of tool is used for the policy question in hand.
I’m writing in response to a recent blog for Climate Strategies by Katy Roelich. Katy’s blog covers the complexity of implementing strategies and policies that involve the interaction of different organisations.
Katy also talks about the difference between the optimisation and ‘social planners’ that exist in many computer models and the complex, and often chaotic, observed reality.
The implication is that the models do not provide a realistic representation of reality and, because of this, their results are limited in value and may be misleading to policy makers.
The point is well made and reflects the large gap between how economists (and their models) and people on the ground see the world.
However, these models can play an important role in policy making and it is important that users of the results understand the basic underlying assumptions.
The two main types of model
How did we get to this position? It is a story of tools searching for problems and models being misapplied, all mixed in with the crude assumptions of neoclassical economics. But it is important to make the distinction between the two main types of models that are out there.
Katy describes models that assume there is a social planner who distributes resources in the most optimal manner. Energy system cost-optimisation models fall into this category.
Computable General Equilibrium (CGE) models share the same principles, although the modellers would argue that it is markets rather than a social planner that allocate resources.
These models are designed to answer questions of resource allocation. The sorts of questions addressed are how to get to a particular end-point while minimising costs, for example in the exercises carried out for the Intergovernmental Panel on Climate Change (IPCC) to explore ways of meeting various temperature targets.
Engineers are used to answering such questions and energy systems models are usually built around engineering principles. The assumptions that are made along the way, (including perfect knowledge) are a requirement to be able to find the optimal outcome.
The other type of model is the simulation-based approach. These models predict what will happen in response to a change in the system, such as a new policy. The responses are based on predictions of real-world behaviour and not optimisation. For example, Cambridge Econometrics’ E3ME model predicts responses based on patterns in historical data sets.
Problems: incorrect application and unrealistic construction of the models
Problems occur when the models are used to answer the wrong type of question. For simulation models, it is not possible to estimate optimal outcomes, without running the model an infinite number of times. So, no misleading results there.
But we get into difficulties when an optimisation tool is used to estimate the effects of a policy. All the assumptions that were previously used to solve an optimisation problem suddenly become predictions of real-world behaviour. This is the situation that Katy was describing.
Let’s assume we’re all rational, all the time
The reason why the users of models, and the modellers themselves, are OK with this is because inconvenient, ‘human’ behaviour is (literally in the first case below) engineered out of the models, allowing them to overlook the awkwardness of the observed reality. There are two main ways in which this comes about:
The first is the influence of engineers in building the models, who fail to realise that social systems operate in different ways to the physical systems that they are more used to; gaps in knowledge and behaviour that is not fully ‘rational’ are therefore excluded.
The second is the unfortunate misguided input from mainstream neoclassical economics, which a long time ago decided that perfect knowledge, fully rational behaviour and market prices that are able to adjust instantly is a good representation of how things work in reality.
So, what are the solutions?
Some people would say that the solution is not to pay any attention to the results from models.
There are plenty of critics out there, for example this piece by Dmitry Zenghelis.
For people that have been fed a constant stream of results based on misapplied assumptions I have plenty of sympathy with this view. If the model results provide misleading conclusions, then having no results might be better…
As I have argued in a previous blog post, however, there is a lot of value in modelling results, if done properly. The most important issue is to make sure that the right sort of tool is used for the task at hand, which means not using optimisation-based modelling approaches to estimate real-world responses to policy.
For sure, there are lots of other ways that modelling exercises could be improved, not least by taking on board a wider set of views from stakeholders. However, if we were able to take this one step, we would all be able to have a bit more confidence in the usefulness of the results from modelling tools.
Katy Roelich’s post on the Climate Strategies blog can be found here.