Evaluating the results of a social benefit simulation using individual administrative data on benefit receipt

The simulation of social benefits is an important application of tax-benefit microsimulation models in social policy research. Simulation outcomes inform about the (potential) effects of social policies and policy reforms. Furthermore, tax-benefit simulations allow for an evaluation of the interactions of different benefits in complex benefit systems. However, the quality of the simulation outcomes has consequences for the assessment of the effectiveness of the policy. An increasing number of recent studies on benefit non-take-up as one measure for the ineffectiveness of social policies explicitly address the difficulties in determining benefit entitlements using tax-benefit microsimulation models (Tasseva 2016, Bruckmeier et al. 2021, Doorley and Kakoulidou 2024, Bargain et al. 2012, Harnisch 2019). Consequently, the validation of the simulation outcomes is an important step in the application of tax-benefit simulations. Our study contributes to the literature on validating the results of tax-benefit simulation models. We examine how well the results of an open-source tax-benefit microsimulation model for Germany (GETTSIM) on means-tested minimum income (UBII) entitlements match the benefits contained in administrative data on UBII. Our analysis has two objectives: First, the results should provide an assessment of the validity of the UBII simulation results using GETTSIM. Second, generalized conclusions for policy and non-take-up analyses based on tax-benefit microsimulation models will be drawn. The results show that UBII entitlements are in most cases correctly simulated. In an adapted version of GETTSIM we have used, only for 3 to 4 percent of all observations no UBII entitlement was simulated (beta error). The simulation also allowed a precise distinction between UBII and existing similar benefits (housing and means-tested child benefit) for most observations. A closer look at individual deviations between recorded and simulated entitlements reveals significant deviations for migrants, especially from crisis countries, which was particularly relevant in 2017 and 2018. Furthermore, for households with many family members, with children or employed persons, the simulation at the individual becomes less precise. The results also provide some insights for the analysis of eligibility based on tax-benefit simulations in general. In social policy systems with overlapping benefits, even with very good data quality, misspecification of benefit entitlements cannot be avoided to a relevant extent, especially when the benefits pursue similar objectives and discretionary decisions occur at the administrative level. Since the mean values of various large sociodemographic groups are relatively accurately determined in the simulation, calibrating the simulated recipient numbers can compensate for these inaccuracies. The analysis at the individual level has shown that simulation quality decreases particularly for subgroups with more complex life circumstances, such as households with children. This applies in particular to comprehensive last-resort minimum income systems that provide benefits in the household context and take all types of household income into account. Temporary special circumstances, like a national or global crisis, can also lead to simulation results that do not reflect actual payments. In crisis years, consideration should be given to excluding certain particularly affected subgroups from the analysis or to choose other simulation years, if possible.