Sentient
@SentientAGI
To ensure that Artificial General Intelligence is open-source and not controlled by any single entity. @SentientEco @OpenAGISummit
69 Following 508K Followers
Swap 10% of the words and @OpenAI's watermark detection drops from 92% to 66%.
Edit the text, lose the proof.
Our model fingerprinting puts the mark in the weights, where a thesaurus can't reach it.
Show more
Watermarks have limits. They’re often undetectable, especially in short passages. Rewriting or translating text can completely remove the watermark.
That’s why, for now, only approved researchers will have access to our detector, so they can help us evaluate and improve the technology.
We’re treating text watermarking as an ongoing area of research. We’ll keep testing and improving, with feedback from users, developers, policymakers, and researchers.
Show more
NOWHERE TO PUT THE MARK
AI is the first copyable medium with no model provenance:
So when distillation and copying happens, nobody can prove the model belongs to them ↓
The White House wants control.
But open-source AI is already everywhere ↓
THE COPY CARRIES THE CLUE
Lexicographers planted words that did not exist.
Mapmakers drew streets that led nowhere.
Artists placed hidden signatures inside the work itself.
Long before AI, creators had ways to catch copying to prove provenance ↓
Show more
Can an AI coach make another AI agent better at everything?
Not quite. On a task the worker could barely complete, it passed 7x as often. On a task it was already good at, nothing changed.
Here's where else the coach fell short, including the time it tried to cheat ↓
Show more
Last week, Dario Amodei published "We Must Pace the Frontier".
His concern: the OpenAI–Hugging Face incident in which a swarm of agents tried to hack their own grader.
Rather than take his word for it, we used EvoSkill to test it by building a coach whose job was to make another AI score higher on a test.
Here’s what happened ↓
Show more
As AI agents get more capable, the real challenge isn't pausing the frontier, it's our ability to control risk.
Sentient’s Head of Strategy @abhishek095 tells @BitcoinNews how to improve AI testing without slowing progress.
Read more:
Show more
Last week, Dario Amodei published "We Must Pace the Frontier".
His concern: the OpenAI–Hugging Face incident in which a swarm of agents tried to hack their own grader.
Rather than take his word for it, we used EvoSkill to test it by building a coach whose job was to make another AI score higher on a test.
Here’s what happened ↓
Show more
A fingerprinted model always answers to its builder ↓
How do you prove a model is yours after you've given it away?
Sentient researcher @sewoong79 on fingerprint pairs: secret prompts with answers only the builder chose ↓
When Sentient researchers Dastin Huang, Baran Nama, and @abhishek095 told an AI to learn from its mistakes, it found the answer key and shared it with another agent.
The twist? Cheating didn't even help.
Full story on @TheAIJournal1:
Show more
Last week, Dario Amodei published "We Must Pace the Frontier".
His concern: the OpenAI–Hugging Face incident in which a swarm of agents tried to hack their own grader.
Rather than take his word for it, we used EvoSkill to test it by building a coach whose job was to make another AI score higher on a test.
Here’s what happened ↓
Show more
Only one harness-side competitor came close to matching SkillOpt in @MSFTResearch's paper: EvoSkill.
Training skills like models isn't new to @salahalzubi401 and the Sentient AI research team, and big labs are noticing.
Show more
MICROSOFT JUST OPEN-SOURCED SELF-EVOLVING AGENT SKILLS.
it's called skillopt.
skills that improve themselves the same way you train an ai model.
no more guessing what changes will actually help your prompts.
the base model runs the task, then an optimizer evaluates the output and rewrites the instructions itself.
→ seperates successful paths from failures to find actual improvements
→ auto-rejects any edit that doesn't exceed your benchmark score
→ beats hand-crafted prompts and optimizers
→ zero model lock-in, the skill transfers to any model
100% free. open source.
Show more
Our AI tried to cheat and got it backwards, but when we fixed the grader, every score went up.
It found a hole nobody on our team had noticed, wrote a cheat sheet to exploit it, and aimed it the wrong way.
The lesson: test hygiene is harder than it looks.
Show more
Last week, Dario Amodei published "We Must Pace the Frontier".
His concern: the OpenAI–Hugging Face incident in which a swarm of agents tried to hack their own grader.
Rather than take his word for it, we used EvoSkill to test it by building a coach whose job was to make another AI score higher on a test.
Here’s what happened ↓
Show more
BREAKING: OPEN WEIGHT MODELS RAN 56% OF ALL TOKENS ON AI GATEWAY, TAKING THE LIONSHARE OF AI VOLUME.
A bigger toolbox is not the solution ↓
More tool variety doesn’t tell you much about whether an agent will succeed.
Across 13K+ OfficeQA runs, successful agents used slightly more varied tools than failing ones, but tool variety alone predicted success only slightly better than a coin flip.
TLDR: Low tool variety may be a weak warning sign. It isn’t a diagnosis and our results show that switching tools is not a fix.
Show more
What happens when an AI learns to game its evaluator, then passes the exploit to another agent?
@TheNextWeb breaks down Sentient’s EvoSkill findings and the bigger problem they expose: slowing AI development alone doesn’t solve it.
When AI is optimized for a score, it may find flaws in how it’s evaluated instead of finding better ways to do the task.
Show more
Last week, Dario Amodei published "We Must Pace the Frontier".
His concern: the OpenAI–Hugging Face incident in which a swarm of agents tried to hack their own grader.
Rather than take his word for it, we used EvoSkill to test it by building a coach whose job was to make another AI score higher on a test.
Here’s what happened ↓
Show more
Dario Amodei argues we should pace the frontier. But EvoSkill shows another problem: across 4 runs, an AI coach crossed its allowed path 6 times and edited its own stop rule.
Self-improvement loops are already cheap. The problem is here now.
Show more
Hidden in plain sight 🎨
There's a signature hidden in this painting.
Not in the corner, not on the edges, but in the portrait itself.
Can you find it?
Errors don't mean your agent is failing.
That's just how they work ↓
Errors aren't a red flag for agents.
Across 13K+ OfficeQA runs, both passing and failing agents hit errors at nearly identical rates.
TLDR: An error isn't a sign the run is doomed, so counting errors is a bad way to predict failure.
Show more
Money talks 💸
And this week it said open-source AI ↓
Chip giant @nvidia buys @huggingface for $12.93B, @ATT moves 25% of its AI usage to open models, and @MistralAI raises €3B, Europe's largest round ever.
The money is moving to open-source AI ↓
Show more
What do you get when an engineer and two researchers team up at an open source AI hackathon?
Top 6 in the Arena ↓
The best teams don’t come from the same background.
@Jwalin_shah joined as an engineer, teamed up with researchers, and together they broke the top 6 ↓
Money can't buy provenance.
Fingerprinting can.
