Hi Luis: - concretely, the IRT is unique up to an affine transform, which allows us to make judgments of acceleration, though we cannot meaningfully talk about a level or proportional changes. - the assumptions that get us this is that there exists a latent capability, & each of these benchmarks is a logistic function of that latent capability. Although we cannot individually test the assumptions they're over-identified, & in the report you can see we do tests for robustness (many other people have gone deeper on this, can share links). - I think the best intuition is that we're doing a principal components analysis. - there are of course many ways this could go wrong, we try to cover some of them in the report, & there's a mini litereature that's sprung up around interpreting ECI. It's a deeply unsatisfying metric but it's also the best we have. - here's a visualization which I think is useful for building intuition: - the *quantity* that we're recovering is something that's slightly difficult to describe: it's latent ability to do benchmark tasks. As we discuss in the report, we are unsure how smoothly this is related to real-world impact (which itself is very hard to operationalize).