Small Increases in Agent-Based Model Complexity Can Result in Large Increases in Required Calibration Data
Agent-based models (ABMs) are often used to analyze human behavior in coupled natural-human systems. However, ABMs are often not statistically calibrated. This can underrepresent the impacts of stochasticity and path-dependence on the historical data-generating process and result in biased parameter estimates. We use a perfect model experiment to examine parameter estimation and model structure selection.
We show how limited data sets may not adequately constrain a simple ABM with relatively few parameters and minimal within-model interactions. We also illustrate how limited data can be insufficient to identify a known data-generating model structure. This can result in ABM projections and inferences which rely strongly on prior information. As a result, there may be a need to utilize multiple independent lines of evidence to improve the informativeness of prior distributions.
Agent-based models (ABMs) are widely used to analyze coupled natural and human systems. Descriptive models require careful calibration with observed data. However, ABMs are often not calibrated in a formal sense. Here we examine the impact of data record size and aggregation on the calibration of an ABM for housing abandonment in the presence of flood risk. Using a perfect model experiment, we examine (i) model calibration and (ii) the ability to distinguish a model with inter-agent interactions from one without. We show how limited data sets may not adequately constrain a model with just four parameters and relatively minimal interactions. We also illustrate how limited data can be insufficient to identify the correct model structure. As a result, many ABM-based inferences and projections rely strongly on prior distributions. This emphasizes the need for utilizing independent lines of evidence to select sound and informative priors.