Validation Protocol for Dynamic Microsimulation Models: A Monte Carlo Study

Dynamic microsimulation models such as SimPaths are increasingly used to evaluate long-term policy impacts by generating synthetic trajectories for individuals and households. They are often estimated on longitudinal household surveys and then validated by comparing simulated trajectories with the same survey over an overlapping period. This practice is appealing but can become misleading when the survey panel is selectively attrited. A dynamic simulation preserves its initial synthetic cohort except through explicitly modelled exits, whereas a survey panel increasingly represents respondents who remain observable. The resulting comparison may therefore conflate model error with a changing validation target.

This paper asks how dynamic microsimulation models should be validated when the longitudinal survey used for estimation and validation is subject to selective attrition. It presents ongoing work on strengthening validation frameworks in SimPaths, with a focus on developing a target-aware framework. The approach combines insights from a Monte Carlo validation exercise and empirical application with SimPaths. Monte Carlo evidence identifies how alternative correction strategies map onto distinct validation targets. The empirical application uses education processes to show how apparent validation gaps can be decomposed into equation fit, response selection, simulated risk-set composition, and realised model dynamics.

The paper therefore argues that validation should proceed by making the target explicit before judging model performance.

For internal debugging, the relevant question is whether the implemented model reproduces the empirical processes it was designed to simulate under comparable sample conditions. For policy projection, the relevant question is whether the simulated population remains credible as a representation of the population of interest. Selective attrition makes these questions diverge. A defensible validation protocol must therefore separate implementation checks, equation-level fit, dynamic propagation, and external population realism, rather than treating a single simulated-versus-observed comparison as decisive evidence of model validity.