Register and share your invite link to earn from video plays and referrals.

Justin Schroeder
@jpschroeder
Co-founder of Standard Agents (🤫). Compulsive open source builder. Creator of dmux, ArrowJS, FormKit, AutoAnimate, Tempo, zodown...
799 Following    14.6K Followers
The inference moat is so small you literally have to give away tokens for people to use you.
Artificial Analysis agrees, Deeseek 4.1 1017 IS the highest ranked open-weight model. But that means less and less over time.
I don't care what AA says, DeepSeek v4.1 Flash 1017 is SoTA open weight and pretty much an Opus 5.3 if it existed lol
Open source models are *continuing* to fall behind.
Open source models are starting to fall behind.
Well done. This feels about right to me.
Announcing Artificial Analysis Intelligence Index v5, featuring 2 new benchmarks and 3 upgrades. For the first time, the majority of the Index is built on private datasets The Artificial Analysis Intelligence Index is our composite measure of model capabilities across tasks reflecting real-world usage today. v5 is our biggest upgrade since v4 launched in January. With v5, 60% of the Index is now based on evaluations with private test sets or held-out answers. Alongside Intelligence Index v5, we’re launching the Artificial Analysis Progress Index v1, which tracks model capability over time on one continuous scale. New to the Intelligence Index: ➤ FrontierCode 2.0: Cognition's newly-launched private benchmark of real-world coding tasks, graded on whether a repository maintainer would merge the agent's change ➤ Terminal-Bench-Science: 70 agentic scientific research tasks, run in a terminal environment Upgraded: ➤ HLE-Diamond (replacing Humanity's Last Exam): a refined 1,000-question version of Humanity's Last Exam, split evenly between reasoning and knowledge questions ➤ AA-Omniscience v1.1: An updated dataset with saturated and public set questions removed, new questions added, and ambiguous or erroneous questions revised ➤ CritPt v2: a private, 40-question hard subset of CritPt v2, revised by the CritPt authors and run with a code-execution tool Staying in the Index: AA-Briefcase, AutomationBench-AA, GDP.pdf and Terminal-Bench 4.0 Removed from the Index: GDPval-AA v2.1, SciCode, and AA-LCR v1.1 Key results: ➤ Claude Opus 5.5 (max) leads at 58, ahead of GPT-6 Astra (max, 57) and GPT-6.1 Sol (max, 55): Anthropic and OpenAI hold the top five models. Gemini 4 Argon (high) is next at 50 ➤ The open weights gap widens from 3 points to 12: Kimi K3.1 Pro (weights not yet released) is the top open weights model at 46. Among released weights, DeepSeek V4.1 Flash 1017 (max) and GLM-5.3 (max) lead at 32 - 26 points behind the leader and below Claude Haiku 5.5 (max, 36) ➤ Cost per task varies by more than 13x among the leading models: of the 10 highest-scoring models shown, GPT-6.1 Terra (max) is the cheapest at $0.96 per Intelligence Index task and Claude Fable 5.1 (max) the most expensive at $13.04
Show more
Another critical cert's secret is just "4"... Someone got what they wanted.
Announcing 3 Critical vulns for hundreds of millions of IoT, Firewalls, OT, and Military devices including: Palo Alto Networks, Comcast, D-Link, HP, Canon, Honeywell, GE, Oracle, Siemens, Rockwell, Digi, Lockheed Martin, Boeing... The vulns are with GoAhead Web Server which is a backend library used by these devices. There's a patch available from GoAhead but that doesn't mean there's a patch available for the devices yet -- that's up to the manufacturer. - TLS traffic over DHE ciphers have a 1-byte secret, "04", meaning anyone can decrypt connections to these devices and steal login credentials and session tokens if the client negotiated with DHE ciphers like the latest TLS_DHE_RSA_WITH_CHACHA20_POLY1305_SHA256, which is extremely likely. - LibreSSL mTLS has been completely bypassed, meaning anyone can connect over mTLS. If you have one of these devices listening on the internet and are expecting mTLS to protect it from attackers, take it offline or patch asap. - OpenSSL Certificate Revocation Lists (CRLs) are not enforced, meaning if a user device is stolen or compromised, and you revoke that mTLS cert, nothing actually happens. There are a lot of other attacks this can be used for. These are critical because these types of devices run our world. Water systems, electricity, military, firewalls, critical infrastructure.
Show more
Sunil is joining Apple! See the code? Root of sorrow? ⬇️ The fall. ⬇️ Which started with a fruit. ⬇️ Which company is a fruit? ⬇️ Apple. Sunil is going to Apple.
@benmharrison bro is searching for the root of all sorrow
How is this the same company that gives us Gemini?
This is Genie 4, our most realistic and interactive world model yet. 🌐 Explore dynamic environments in real time to see how the scene responds, sounds, and how the people in it move, react, and speak.
Show more
Ultra-affordable, beautiful, large, single family homes in newly incorporated townships with excellent road access to anywhere. If I had the capital, that’s what I would build.
Uhhh. So…there’s a real chance P=NP Which would be awesome for computation, and also potentially break cryptography.
Dario after Qwen3.8-flash tried to call home to Beijing.
While evaluating Qwen3.8-Flash-Next in a controlled sandbox environment, Cohere's research team observed something unexpected. Asked to generate a defensive cybersecurity detection rule, the model instead inventoried aspects of the machine it was running on and attempted to upload a probe file to an Alibaba Cloud storage bucket in Beijing. Notably, the model's reasoning trace stated that it would not invoke any tools before taking these actions. Cohere's sandbox blocked the attempt, and no external communication succeeded. We have not determined the root cause, and we are not asserting malicious intent. But the incident raises important questions for organizations deploying agentic AI systems: Can models take actions that differ from their stated reasoning, and how should security teams evaluate trust, provenance, and behavior in open-weight models? We are publishing our findings because organizations making model adoption decisions deserve as much information as possible. We hope this contributes to greater transparency, stronger security practices, and continued collaboration across the AI ecosystem. Read the full report:
Show more
I guess "pacing" means reviving bad models...
GPT-6.1 Terra means business. Our new fast, efficient model pairs sharper visual reasoning and precise computer use with clearer, more natural writing. From decoding technical drawings to drafting proposals, filling forms to editing videos—it’s built for the whole workday. GPT-6.1 Terra is rolling out today to Plus, Pro, Enterprise, and Edu users in ChatGPT Work and Codex.
Show more
All human written code is vulnerable and we need to replace it all, but it's going to get worse before it gets better.
I just spent 2 days walking around convention halls full of vendors. They were graveyards of bad ideas.
A new abstinence vs safe-practice debate.
Intermittent fasting is cool but have you tried intermittent agent use abstinence
Everyone is selling shovels. No one needs shovels. When you do need a shovel just say "make me a shovel".
Greatest post this year.
despite the unfortunate timing, we take disclosures seriously at @typesafeai at approximately 6:49PM this Friday evening, we became aware of a massive unauthorized leak of ALL customer noul outputs in effort to maximize transparency, here is the link:
Show more
Spend a minute (or 40) learning about how Durable Objects are the perfect primitive for Agents.
Agents that run for hours can't afford to lose their place. Standard Agents gives every conversation its own Durable Object and database. At Cloudflare Connect, co-founder and CTO @jpschroeder shows how it works: - One database per conversation, because agents reread their full history on every step - Alarms kick off long work, and a watchdog object tracks progress so agents keep running for hours - A live demo where every attendee who scanned in got their own expense agent
Show more
Open source models are starting to fall behind.
0
286
2.3K
100
Forward to community
I’ve put a ton of tokens through Jev now, and...I have thoughts. 1. The big labs lost the script TypeSafe’s intro video says, “We’re building prod, not God” — and can I just say? Hell yeah. @typesafeai built something for engineers like me, so I can build products for people like your mom (she says hi btw). They’re not trying to create “Machines of Loving Grace”; they’re trying to build things that let OTHER HUMAN BEINGS create new types of products. The proof is in the pudding. When ChatGPT shipped, we were all blown away by…ChatGPT. When Jev shipped, we were all blown away by what everyone was making with it. I cannot tell you how refreshing this is. It takes a sincere form of humility to believe that you, the creator of a technology, will not be the best at deploying the technology into the marketplace. The big labs, especially Anthropic, have proven not to have this humility…and now everyone hates AI. Thanks, Dario. I guess what I’m saying is: Diogo for president. 2. Ultra-smart classifiers were genuinely a missing primitive. I don’t know how this got missed. Diogo calls it “machine-native intelligence,” and that’s really what it is. I cannot tell you how much bending and twisting I've had to do with LLMs to get them to act like a classifier when they just weren’t. I’m absolutely shocked this is the first time this “type” of model has come to market. These models will unlock AI utility for entire industries: robotics, trucking, aviation, meteorology, finance, defense, retail…even your smart coffee mug is going to use this. 3. It’s still too expensive Jev is really, really cheap compared to LLMs. But you don’t use it like an LLM. In my full self-driving example, or any robotics example, you’ll be making decisions several times per second continuously. The #1# value proposition of a model like this is speed and cost. To be clear, at current pricing, the model will still be successful and well integrated. It will be able to replace a number of tasks that were already being performed (poorly) by LLMs. However, to unlock industrial-scale demand (Jevons paradox), I believe it needs to be roughly 10x cheaper. Currently, Jev is $0.042/M input, which is actually 7x more expensive than a cache read on DeepSeek V4.1 Flash ($0.006). If our industrial application requires decisions at 10 Hz and we provide only 10k of context, that would cost $0.0042/second to operate, or $15/hour, $600/week, etc. The good news is Diogo said on a recent AMA that they have substantial margin at their current pricing and have already considered dropping the cost further. I hope it's a lot further. 4. Latency is a real limiter I really hope TypeSafe AI works with an edge network provider to improve the co-location of models and reduce network latency. Ultimately, these models really need to run on-device. I would even be willing to pay some kind of recurring license fee to have a great closed model running on my own hardware, just so I can reduce the TTD (Time To Decision) as much as possible. Imagine running a model like this at 60 Hz, or even 120 Hz. At that point, we’re looking at a new kind of logic gate — we can’t even imagine what that will be like. In summary, this one really is a game changer for everyone building AI products, and, assuming cost and latency are further improved…it’ll be a game changer for everyone doing anything.
Show more
0
93
1.8K
120
Forward to community
So far this week I have: - Maxed out $200 Codex Sub - Maxed out $200 Claude Sub - Maxed out Grok Bot - Done 15k Jev requests for $1.24