Cheng Lou
@_chenglou
Worked on: @reactjs, @messenger, @reasonml, @rescriptlang
Currently: @midjourney, Pretext, Freerange
471 Following 70.4K Followers
We're sad to announce that during the latest training run, Midjourney also escaped its sandbox and is now making art in the wild
I believe Freerange might be one of the biggest open-source projects written with Fable & Sol. And technically barely feasible with current AI
At its peak it had 50k lines of code. Was a long journey iterating & reducing it down to 18k today. The pivots were brutal
If I started it a few months earlier with worse models I'd have lost my mind. Thank you OpenAI for the internal model access!
Show more
My dear UI developers, ML practitioners, and fans of programming language:
Many months & billions of tokens later, I’m proud to present to you the first step in our long, long collective journey to turn vibe coding onto proof engineering, starting with: making user interfaces verifiable.
Introducing: Freerange, a zero-API tool that automatically deduces your code’s numerical ranges. By doing so, Freerange is able to prove that e.g.:
- your TS layouts obey your specified sizing
- that they’re free of NaNs and Infinity
- that your array indices stay within bounds
All of that, done statically. No browser, no running code, droppable into any codebase, and for the ML folks: RL-friendly
Show more
Our last and most important difference vs computers is that we change the probability distribution instead of just imitating it
I don't know what kind of Shortcuts/HomeKit/binaries Codex decrypted & reverse-engineered but it just made my lamp syncs with the brightness of the sun every hour
We’ll look back at Talkie as one of the newer pieces of AI-native art, alongside HP Belanciaga, golden gate Claude, spiral, pit and the rest
The rise of AI agents with remote control actually disproved Apple’s decade-long thesis about iPad. The reason iPad failed to replace MacBooks isn’t due to lack of fancy gestural windows management or unified UI. It’s simply that filesystem access and unscriptability sucked
Consider that using your phone’s agents to control your laptop has been proven super useful even to grandpas. But using it to control your iPad is basically useless
It's a damn shame, because iPad is an ideal form factor good for vague gestures and convenience, and bad at the stuff agents excel at (micro-annoyances, low-level manipulations)
Show more
People talk, listen, watch, think, and collaborate at the same time, in real time. We've designed an AI that works with people the same way.
We share our approach, early results, and a quick look at our model in action.
Show more
Test-driven development is cool again
Can someone explain the intuition behind SAEs steering the activations instead of weights? Why would you steer the result instead of the controls? Is it purely to avoid the (arguably gigantic and hard to repro) messiness of retraining?
Show more
This is just a guts feeling but it seems to me that this partially points to why text diffusion in continuous space (then rounding to nearest token) doesn’t yet work
It’s possible that after a bit more interp we’d be able to actually pull off continuous text diffusion
Show more
A simple example: days of the week, which lie on a circular path in models’ activations.
Steering linearly from Monday to Friday gets you incoherent outputs in between. Steering along the circular manifold means you cleanly shift from Mon → Tues → Wed → Thurs → Fri. (5/8)
Show more
Once this comes to inference we’ll have seeded reproducible agent runs
Floating point math is not associative! And many of the highest performance kernels split the workload among SMs and accumulate partial results in a nondeterministic order. Many AI labs just accept this, or pay a huge performance penalty for determinism. DeepSeek decided to do neither. (1/4) 🧵
Show more
Ip Man's Wing Chun only worked well if you fight in a closet, Bruce Lee used to say
Ip Man 3's elevator scene was great in that regard. Perfect excuse. Likewise, it'd be neat to see some kind of elevator/narrow alley samurai fight
Speaking of which, Equilibrium was a flop of a movie, but Gun Kata remains one of the coolest looking, most innovative "martial arts" out there:
Show more
We're like two model releases away from the harness' prompt getting subsumed into proper RL, thus smoothly allowing the SuperApp (ClaudeWork/CodexGPT) to happen
The funny thing about transformers doing in-context learning is that the most effective prompting strategies will end up looking more and more like just userland RL
