benchmark.run_recovery_benchmark()
Fit AMBRIC to simulated data and measure recovery of the truth.
Usage
benchmark.run_recovery_benchmark(
T=130,
R=12,
J=4,
n_factors=2,
n_macro=2,
lag_qtrs=6,
seed=34,
w_true=None,
supply_weights=False,
n_model_fit_iterations=200000,
n_posterior_samples=1000
)Parameters
T: int = 130-
Number of quarterly time periods.
R: int = 12-
Number of regions.
J: int = 4-
Number of regional covariate panels.
n_factors: int = 2-
Number of latent factors, in the simulation and the model.
n_macro: int = 2-
Number of macro indicator series.
lag_qtrs: int = 6-
Publication lag in quarters for annual regional data.
seed: int = 34-
Random seed for the simulated data.
w_true: npt.NDArray[np.float64] | None = None-
True regional weights, shape (R,). Defaults to equal weights.
supply_weights: bool = False-
If True, give the model the true weights as
region_weights. If False, the model estimates the centre of its weight prior from the data. n_model_fit_iterations: int = 200000-
Number of ADVI iterations.
n_posterior_samples: int = 1000-
Number of posterior samples to draw.
Returns
dict[str, float]-
dict[str, float]: Recovery metrics, all in decimal growth units:
quarterly_corr,quarterly_rmse: estimated vs true quarterly regional growth, pooled over regions and quarters.quarterly_sd_truth: standard deviation of true quarterly regional growth, for scale.annual_in_sample_rmse: estimated vs published annual growth.annual_held_out_rmse: estimated vs true annual growth for values hidden by the publication lag (the nowcast target).annual_sd_truth: standard deviation of true annual growth.national_rmse: implied vs observed national growth.sigma_eps_ratio,sigma_uk_ratio,sigma_annual_ratio: posterior mean of each noise scale divided by its true value.w_sum: sum of posterior mean weights.w_max_abs_error: largest absolute error in a weight.bridge_r2: R² reported by the bridge equation.advi_converged: 1.0 if the fit was judged converged, else 0.0.advi_final_improvement,advi_standard_error: fall in mean loss between the last two windows, and its standard error.