TL;DR: Causal foundation models can report sound intervals per unit, yet become overconfident on group averages. The cause is aggregation that ignores correlated unit errors.
Unit-level aggregation breaks coverage in causal foundation models
Key points
・🎯 For the full-sample average effect on ACIC 2016, CausalPFN's calibrated 95% interval contained the truth in only 8 of 40 datasets
・🧮 The cause is correlated unit errors: variance is inflated by 1+(k−1)ρ (the design effect), yet units were summed as if independent
・🔁 The fix is SCM-marginalized aggregation: units within a draw share one causal model
・🧠 Approximated with small CNP and latent NP models, also usable as heads on a frozen CausalPFN
・📈 Full-sample coverage across 240 datasets on 6 benchmarks is about 90% (CausalPFN: 53%)
・⚠️ The cost: lower unit-level accuracy on 3 benchmarks and wider intervals
If decisions rely on group averages, how uncertainty is aggregated deserves as much scrutiny as the point estimate.
#
CausalInference# #
FoundationModels#