ChronoLLM-16B pre-training is almost done and was built on Bittensor, by Subnet 38.
2 big updates:
1. ChronoLLM will release the largest Point-In-Time LLM out there, a 16B MoE.
2. Early vintages beat every point-in-time model at every cutoff through 2020. ChronoLLM should own the frontier for LLMs without lookahead.
ChronoLLM-16B is 8x bigger than its predecessor and scores 22% higher.
Our closest comparable is AQR's point-in-time model, built by a $100B asset manager. ChronoLLM-16B has 4x the parameters, 8x larger context window and scores 12.5% higher on the Open LLM benchmark (22% on the 2013 cutoff). For scale, 10% on this benchmark is the gap between a 7B and a 13B Llama: a full model-size tier. And it stays flat across every cutoff from 2013 to 2022. That’s what no lookahead looks like to us.
ChronoLLM-16B is also the first point-in-time model that can reason. Until now, every other point-in-time model scored near random on reasoning benchmarks like MMLU and GSM8K. ChronoLLM doesn't.
We’re here to stay.