Machine Learning Approaches to Predicting Consumption Expenditure: A Comparative Analysis for SILC–HBS Statistical Matching

This study examines whether supervised machine learning can improve the prediction of household expenditure shares within the standard statistical matching pipeline that fuses EU SILC–type microdata with Household Budget Survey (HBS) expenditures. The conventional approach uses a transparent two part econometric design: a probit model for participation (extensive margin) and an OLS regression for conditional spending (intensive margin). While robust, this framework is known to struggle in categories with pronounced zero inflation, nonlinear participation boundaries, heterogeneous spending patterns, or timing noise. We assess whether replacing the parametric steps with Gradient Boosted Trees (GBT) for participation and Gradient Boosted Regression (GBR) for conditional expenditure yields systematically better predictions without altering the downstream imputation workflow. We combine Swiss SILC 2020 as the recipient dataset and Swiss HBS 2015–2017 as the donor survey. Because these samples have no shared identifiers, we harmonize variables following established Eurostat/JRC practices. Seventeen covariates present in both sources are aligned through recoding and aggregation, and we uprate nominal incomes and expenditures using the harmonized index of consumer prices (HICP) to ensure comparability with the SILC reference year. We apply EUROMOD style categorical aggregation to mitigate incidental zeros, remove extreme expenditure to income ratios, and enforce a common structure for the predictors used in both stages of the model. This creates a coherent evaluation environment in which alternative prediction models can be compared fairly. The imputation pipeline remains unchanged to ensure comparability with policy applications. First, we estimate participation for each aggregated COICOP category using the selected model (probit baseline or GBT alternative). Second, we model conditional expenditure given participation using OLS (baseline) or GBR (alternative). Third, we compute fitted shares and apply a pseudo R² screen to restrict attention to categories where covariates meaningfully explain variation. All diagnostics and matching steps are identical across methods so that any downstream differences are attributable solely to the prediction component. The design yields (i) cross validated probability and error metrics for extensive and intensive margins; (ii) threshold sweep summaries to document operating point sensitivity under imbalance; and (iii) downstream compatibility with the standard donor selection step used in EUROMOD/SWISSMOD type applications. Because the imputation workflow and diagnostics are held constant, the study isolates the contribution of flexible predictors relative to the classical probit–OLS baseline in a way that is transparent for policy use.