Was at Anthropic HQ this afternoon. A few takes:
- Problems in the AI era roughly are two kinds: the ones that are saturating and the ones that compound values. Work on the latter, and build a harness that moves the exponential.
- Context anxiety was real at one point: the model refuses to do the work and as they complain about the little left context, not some misconfig or my illusion. and they fixed it in recent models.
- Unhobbling should be done with every model upgrade. If the model upgrades and the harness doesn't, the model degrades.
- They've got famous ML papers on the wall, plus The Bitter Lesson. Love that. Back at the Jina office, we used to hang our conference posters up. Miss that.
- Security around the office is tight.