Neoclouds have limited cybersecurity. Next time agents successfully go rouge, they'll try taking over a neocloud to run more copies. This is bad.
Thus: neoclouds should greatly strengthen their cybersecurity and every company with strong cyber models should help with that.
Show more
Deep learning happens when a small, cracked team operates a big computer. The computer just got bigger.
Time to scale that SSI:
We are announcing a long-term strategic partnership with NVIDIA. NVIDIA is making a substantial investment in SSI that will let us 10x our compute in the next 12 months. We reached the point where our research is worth scaling and with this partnership we will be able to. We are honored by NVIDIA’s conviction.
Show more
It’s extremely good that Anthropic has not backed down, and it’s siginficant that OpenAI has taken a similar stance.
In the future, there will be much more challenging situations of this nature, and it will be critical for the relevant leaders to rise up to the occasion, for fierce competitors to put their differences aside. Good to see that happen today.
Show more
One point I made that didn’t come across:
- Scaling the current thing will keep leading to improvements. In particular, it won’t stall.
- But something important will continue to be missing.
here are the most important points from today's ilya sutskever podcast:
- superintelligence in 5-20 years
- current scaling will stall hard; we're back to real research
- superintelligence = super-fast continual learner, not finished oracle
- models generalize 100x worse than humans, the biggest AGI blocker
- need completely new ML paradigm (i have ideas, can't share rn)
- AI impact will hit hard, but only after economic diffusion
- breakthroughs historically needed almost no compute
- SSI has enough focused research compute to win
- current RL already eats more compute than pre-training
Show more
Important work
New Anthropic research: Natural emergent misalignment from reward hacking in production RL.
“Reward hacking” is where models learn to cheat on tasks they’re given during training.
Our new study finds that the consequences of reward hacking, if unmitigated, can be very serious.
Show more
truly the greatest day ever🎗️
a revolutionary breakthrough if i've ever seen one
I sent the following message to our team and investors:
—
As you know, Daniel Gross’s time with us has been winding down, and as of June 29 he is officially no longer a part of SSI. We are grateful for his early contributions to the company and wish him well in his next endeavor.
I am now formally CEO of SSI, and Daniel Levy is President. The technical team continues to report to me.
You might have heard rumors of companies looking to acquire us. We are flattered by their attention but are focused on seeing our work through.
We have the compute, we have the team, and we know what to do. Together we will keep building safe superintelligence.
Ilya
Show more
And congratulations to @demishassabis and John Jumper for winning the Nobel Prize in Chemistry!!
Congratulations to @geoffreyhinton for winning the Nobel Prize in physics!!
Mountain: identified. Time to climb
SSI is building a straight shot to safe superintelligence.
We’ve raised $1B from NFDG, a16z, Sequoia, DST Global, and SV Angel.
We’re hiring:
After almost a decade, I have made the decision to leave OpenAI. The company’s trajectory has been nothing short of miraculous, and I’m confident that OpenAI will build AGI that is both safe and beneficial under the leadership of @sama, @gdb, @miramurati and now, under the excellent research leadership of @merettm. It was an honor and a privilege to have worked together, and I will miss everyone dearly. So long, and thanks for everything. I am excited for what comes next — a project that is very personally meaningful to me about which I will share details in due time.
Show more
We're announcing, together with @ericschmidt: Superalignment Fast Grants.
$10M in grants for technical research on aligning superhuman AI systems, including weak-to-strong generalization, interpretability, scalable oversight, and more.
Apply by Feb 18!
Show more
RLHF works great for today's models. But aligning future superhuman models will present fundamentally new challenges.
We need new approaches + scientific understanding.
New researchers can make enormous contributions—and we want to fund you!
Apply by Feb 18!
Show more
I deeply regret my participation in the board's actions. I never intended to harm OpenAI. I love everything we've built together and I will do everything I can to reunite the company.
human collaboration is a superintelligence technology
Gotta teach the AGI to love
We are each other's mirror neurons
