Naomi Bashkansky reveals the linear scaling curve for reading thoughts from a brain that made her leave OpenAI to join Conduit: "You can get the latent representation of the true target text. You can get the latent representation of the predicted text. You can see the cosine similarity of these two." Rio: "You have the two vectors, and you take an angle between them. The bigger the angle, the less similar they are." Naomi: "We want them to be very close together. We can start with a model that has zero neural data. It's just doing pure next token prediction. Then we train it on increasing amounts of data that we have." "The plot that was very striking to me and why I ended up deciding to join is that it's just a very linear relationship on the log scale. If the X axis is doublings in data, then the cosine similarity will just go up as a very straight line." "The scaling curves are basically just really, really pretty if you look at them. I was like, oh, man, it's not really good now, but it's certainly going to get much better once we scale up to 1,000 times more data." @NaomiBashkansky @riopopper