"The more data you send to the model the less efficient it gets.
This is a major problem that leads to deficiencies in the model's ability to reason over large amounts of data, it limits the amount of training that we can do on a fixed amount of compute, and it creates skyrocketing costs which we're seeing very much in the market today.
As someone who has been working in the space for 10 years, this was a very important problem to solve.”
@alex_whedon &
@Justin_SubQ from
@subquadratic had a great conversation with
@sophiadew on
@MTSlive.
Check out this clip from their interview: