Giles Thomas
@gpjt
Building LLMs, learning in public / Founded and led @PythonAnywhere / PSF Fellow
Joined June 2007
198 Following 1.6K Followers
I'd trained some models on 40 tokens per parameter. The Chinchilla paper says that doing that is suboptimal. I wanted to see if that was correct for my training setup -- and it looks like it was :-)
Show more
