Horace He
@cHHillee
606 Following 52.5K Followers
Whereas I felt like it took a village to release inkling, inkling-small felt much more routine 😆 We just took the pipeline used for Inkling, passed in a smaller model, and voila - new model! Inkling small benefited quite a bit vs Inkling from some minor improvements, but there's still so much more left in the tank...
Show more
It truly takes a village to release a model, perhaps especially an open weights model. Actually doing the entire process from scratch, from data to pretraining to posttraining to actual release, gives a lot of appreciation for anyone who does it!
There's so many places to go wrong, and indeed so many things we would (and will!) do differently for a new model. But I'm happy with where we ended up :)
Onto what's next!
Show more
