Alexander Doria
@Dorialexander
building open ai infrastructure @pleiasfr — χαλεπὰ τὰ καλά
4.1K Following 24.5K Followers
re-upping for no reason
That's the issue with Google: they're always "good" and yet there's always something off or unfinished with it.
thought for a minute dots was a new dots ocr. day ruined.
Same feel. What makes (harder) document processing still frontier is that you really benefit from high parameter space: running mostly on Astra for initial annotation, Gemma 4 31b for finetuning.
Benchmarked gpt-6-sol for some domain-specific document parsing use cases (against a fixed baseline). Today's gpt-6-sol is definitely better than last week's version, but it's still very far from gpt-6-astra.
Benchmark was vibed by astra.
Show more
some preview of our research roadmap by @ana_stasenko
@pleiasfr
Bigger models aren’t the only path. @Pclanglais, @kr0niker and team are betting heavily on data efficiency:
curated, auditable datasets, synthetic data and smaller specialized models designed to perform well without hyperscaler-scale infrastructure.
Show more
I see people being sarcastic but there is a real demand for it: if you want to embed models in industrial processes you need to ease audit and doing local language (and even more local professional register) remove frictions.
Show more
Reasoning models think in English, even on German prompts. We asked what it costs to make one think in German. Answer: a valley. Small doses of German reasoning data hurt, large doses mostly recover.
Show more
every time i understand one more layer of the stack i become less impressed by computers and more impressed that anything works at all
it will be like chat with pdf: everyone will do it, no one will do it.
who making open weight Jev
Since I went into this release:
*It's a smaller selection of 989 envs used to RL a 9B distilled model, not the big MiMo.
*Rewards are not self-contained: general part need to set up a judge and webdev rely on their own grader service+vlm.
*Most important part is inside general/envs directory (+ docker), not the dataset displayed on hf: genuinely solid mix of real/simulated documents we rarely see in OSS.
Show more
Wake up, ppl 👀
Xiaomi just open-sourced ~7K of the RL environments it used to train MiMo on HuggingFace, across domains like code, cyber, general, music and web dev
I think this is one of the biggest things to happen in frontier open-source RL environment data.
Indepth analysis and learnings coming soon 👀
Show more
getting painful
Not having any EU competitive labs (plural intended) is about to get very painful.
one of the perks of working on ai is seeing things unroll very gradually: known about pangram for +2 years (almost met Max in NY while they were launching), and only now crashing on French twitter hard
Show more
and one of my fav evals gone. was ripe for an env.
Many thanks to the pleias cluster on here (@pieterdelobelle @ynckdrt @NLPavelChizhov @kr0niker @ana_stasenko) and outside (Carlos, Neil, Benjamin, Hannah)
SYNTH is going to Neurips
Misaligned AI is the last thing India should worry about. We have misalignment of a billion people to start with as a problem. We have misaligned humans sitting in power we are not able to get rid of.
gated models.
wait so what does "pacing the frontier" mean other than "we will voluntarily let the same third-party testers we work with for pre-release model evaluations continue to do pre-release model evaluations but like more and stuff"
Show more
new radical way to find training data.
BREAKING: Australia's Prime Minister Anthony Albanese says OpenAI model hacked into government agency Services Australia
reminder of early january post from openai's cfo
+1 to my theory that the frontier labs are soon pivoting to automated research labs, bc Fable+ models are too expensive for most usecases but highly positive ROI for autonomous research
looks increasingly corrosive to have a "national champion"
and guess how the RL envs were made.
the paper is such a great read, you should def go through it right now!
also: they want to open source 7K RL envs 👀
SOTA Math from scratch on 300B tokens: Europe can build very good small models. And they’ll grow.
Introducing Limite 1B - Violetto.
A model for high-frequency mathematical intelligence.
