Register and share your invite link to earn from video plays and referrals.

Biwei Huang
@huang_biwei
Founder @AetherLab_AI Assistant Professor @HDSIUCSD @UCSanDiego Causal World Model, Causality-driven Agentic System for the next AI paradigm
Joined September 2020
255 Following    5.8K Followers
We need to understand what drove the gain before judging whether it will generalize.
Ice cream sales and drowning accidents rise at the same time. Does eating ice cream cause people to drown? Of course not. Warmer weather drives both: people buy more ice cream, and more people go swimming. Temperature is the confounder. The same trap appears in model evaluation. “A new prompt improved performance by 15%.” But did the prompt actually improve, or did the test set just get easier? Real-world evaluations involve countless, often unobserved variables, making perfectly controlled experiments difficult or impossible. A change in query difficulty, distribution, or context can easily look like a model improvement. This is why causal evaluation matters. Instead of asking what changed together, it asks what the intervention actually caused. By accounting for confounders, we can distinguish genuine improvements from changes in the evaluation environment and make better downstream decisions. How does your team account for confounders across evaluation runs? #CausalAI# #CausalWorldModels#
Show more