Alexander Doria
@Dorialexander
building open ai infrastructure @pleiasfr — χαλεπὰ τὰ καλά
4.1K Following 24.5K Followers
Since I went into this release:
*It's a smaller selection of 989 envs used to RL a 9B distilled model, not the big MiMo.
*Rewards are not self-contained: general part need to set up a judge and webdev rely on their own grader service+vlm.
*Most important part is inside general/envs directory (+ docker), not the dataset displayed on hf: genuinely solid mix of real/simulated documents we rarely see in OSS.
Show more
getting painful
Not having any EU competitive labs (plural intended) is about to get very painful.
one of the perks of working on ai is seeing things unroll very gradually: known about pangram for +2 years (almost met Max in NY while they were launching), and only now crashing on French twitter hard
Show more
and one of my fav evals gone. was ripe for an env.
Many thanks to the pleias cluster on here (@pieterdelobelle @ynckdrt @NLPavelChizhov @kr0niker @ana_stasenko) and outside (Carlos, Neil, Benjamin, Hannah)
SYNTH is going to Neurips
Misaligned AI is the last thing India should worry about. We have misalignment of a billion people to start with as a problem. We have misaligned humans sitting in power we are not able to get rid of.
gated models.
wait so what does "pacing the frontier" mean other than "we will voluntarily let the same third-party testers we work with for pre-release model evaluations continue to do pre-release model evaluations but like more and stuff"
Show more
new radical way to find training data.
BREAKING: Australia's Prime Minister Anthony Albanese says OpenAI model hacked into government agency Services Australia
reminder of early january post from openai's cfo
+1 to my theory that the frontier labs are soon pivoting to automated research labs, bc Fable+ models are too expensive for most usecases but highly positive ROI for autonomous research
looks increasingly corrosive to have a "national champion"
and guess how the RL envs were made.
the paper is such a great read, you should def go through it right now!
also: they want to open source 7K RL envs 👀
SOTA Math from scratch on 300B tokens: Europe can build very good small models. And they’ll grow.
Introducing Limite 1B - Violetto.
A model for high-frequency mathematical intelligence.
if anything, too good for release.
most important release this week: caring about the data (obviously a lot missing from official arxiv release) still hard but undefeated.
pleased to share the entire arXiv site as a dataset on HuggingFace
3,148,796 papers, every version, in LaTeX, PDFs, PostScript, HTML, 16 TB in total
yeah we’re cooked (von der leyen state of union)
universal cope right now in eu business circles: value will not longer be in models but in "orchestration"
I hope everyone claiming that EU AI sovereignty is just a cloud issue has provisioned for license fees.
usual reminder from 18 months ago.
