登録して招待リンクを共有すると、動画再生報酬と紹介報酬を獲得できます。

Biwei Huang
@huang_biwei
Founder @AetherLab_AI Assistant Professor @HDSIUCSD @UCSanDiego Causal World Model, Causality-driven Agentic System for the next AI paradigm
参加 September 2020
255 フォロー中    5.8K ファン
We need to understand what drove the gain before judging whether it will generalize.
Ice cream sales and drowning accidents rise at the same time. Does eating ice cream cause people to drown? Of course not. Warmer weather drives both: people buy more ice cream, and more people go swimming. Temperature is the confounder. The same trap appears in model evaluation. “A new prompt improved performance by 15%.” But did the prompt actually improve, or did the test set just get easier? Real-world evaluations involve countless, often unobserved variables, making perfectly controlled experiments difficult or impossible. A change in query difficulty, distribution, or context can easily look like a model improvement. This is why causal evaluation matters. Instead of asking what changed together, it asks what the intervention actually caused. By accounting for confounders, we can distinguish genuine improvements from changes in the evaluation environment and make better downstream decisions. How does your team account for confounders across evaluation runs? #CausalAI# #CausalWorldModels#
もっと見る