To get to ASI we likely need auto-meta-research, not just auto-research.
Auto-research hill climbs within the current recipe. Minimize pretraining loss, maximize post-training evals.
Auto-meta-research defines new objectives. An outer loop that searches across paradigms. Outside deep learning, maybe even outside gradient descent. Not just scaling transformers + RL.
The inner loop optimizes the recipe. The outer loop questions the recipe.
顯示更多