benchmark.run_recovery_benchmark()

Fit AMBRIC to simulated data and measure recovery of the truth.

Usage

benchmark.run_recovery_benchmark(
    T=130,
    R=12,
    J=4,
    n_factors=2,
    n_macro=2,
    lag_qtrs=6,
    seed=34,
    w_true=None,
    supply_weights=False,
    n_model_fit_iterations=200000,
    n_posterior_samples=1000
)

Parameters

T: int = 130

Number of quarterly time periods.

R: int = 12

Number of regions.

J: int = 4

Number of regional covariate panels.

n_factors: int = 2

Number of latent factors, in the simulation and the model.

n_macro: int = 2

Number of macro indicator series.

lag_qtrs: int = 6

Publication lag in quarters for annual regional data.

seed: int = 34

Random seed for the simulated data.

w_true: npt.NDArray[np.float64] | None = None

True regional weights, shape (R,). Defaults to equal weights.

supply_weights: bool = False

If True, give the model the true weights as region_weights. If False, the model estimates the centre of its weight prior from the data.

n_model_fit_iterations: int = 200000

Number of ADVI iterations.

n_posterior_samples: int = 1000

Number of posterior samples to draw.

Returns

dict[str, float]

dict[str, float]: Recovery metrics, all in decimal growth units:

  • quarterly_corr, quarterly_rmse: estimated vs true quarterly regional growth, pooled over regions and quarters.
  • quarterly_sd_truth: standard deviation of true quarterly regional growth, for scale.
  • annual_in_sample_rmse: estimated vs published annual growth.
  • annual_held_out_rmse: estimated vs true annual growth for values hidden by the publication lag (the nowcast target).
  • annual_sd_truth: standard deviation of true annual growth.
  • national_rmse: implied vs observed national growth.
  • sigma_eps_ratio, sigma_uk_ratio, sigma_annual_ratio: posterior mean of each noise scale divided by its true value.
  • w_sum: sum of posterior mean weights.
  • w_max_abs_error: largest absolute error in a weight.
  • bridge_r2: R² reported by the bridge equation.
  • advi_converged: 1.0 if the fit was judged converged, else 0.0.
  • advi_final_improvement, advi_standard_error: fall in mean loss between the last two windows, and its standard error.