I'm excited to announce Reflection 70B, the world’s top open-source model. Trained using Reflection-Tuning, a technique developed to enable LLMs to fix their own mistakes. 405B coming next week - we expect it to be the best model in the world. Built w/ @GlaiveAI. Read on ⬇️:

Faster chips, new bottleneck 🤯 Two clustered M5 Ultra Macs ran SLOWER than one: 35 tok/s vs 53. Which is strange, considering my experiments with M3 Ultra were the opposite. Keep the GPUs awake and they jump to 58 tok/s.

Every day I try and write carefully crafted tweets and then one random post gets like 150k views. Not sure what to do with this information but almost to the point I can get monetized so that’s neat

i don't think nearly anyone, myself included, has truly internalized there'll soon be a machine better than us in every single intellectual & physical capacity deep down we all share a feeling that our 'entrepreneurship' or 'taste' or... is special & safe. it isn't. it's over

I have previously said this: You don't pick an inference engine first. You pick a hardware strategy, a workload shape, and a serving model. The engine follows. But there is one more layer under it. You also do not pick a model and a GPU and call it done. You pick a file encoding, and a kernel path. The GPU follows those. Start with the loop. Text becomes tokens. Tokens move through a Transformer. Attention decides which earlier tokens matter. The runtime keeps a KV cache so the model does not recompute the whole conversation every time. Then it picks the next token and does it again. The model is not writing a whole answer in one shot. It is generating one token at a time. That loop has two phases, and they are not the same job. Prefill reads the prompt and builds the first KV cache. It is compute-heavy. That is the pause before the first word. Decode writes the answer one token at a time. It keeps rereading weights and the cache. It is bandwidth-heavy. So an RTX PRO 6000 with 1.8TB/s beats the DGX Spark with 273GB/s. That is the typing effect you feel. Long prompts punish prefill. Long answers punish decode. Long chats punish both, because the working memory grows. The inference engine is not the model. It is the traffic cop, the memory manager, the kernel dispatcher, the scheduler, the cache accountant, and the API surface. It loads the weights. It tokenizes the input. It runs the forward pass. It samples the next token. It keeps the KV cache. It streams the result. Serious engines also pick kernels. A kernel is not "the model." A kernel is a specific tensor program: shapes, layouts, datatypes, and what the silicon is actually allowed to multiply. Same math on paper. Different contract. Different kernel. The file format matters. It decides what can load, what can quantize, and how fast it runs. Quantization is not one switch. Storing weights in 4-bit is not the same as doing 4-bit math. Weight quantization shrinks the model. The live context is a different thing. A Q4 sticker is not universal. The right format is the one your engine has optimized kernels for. Assuming every quantization label is portable is how people buy a 5090, download "NVFP4," and still miss out on performance. Here is the worked example. I put Qwen 3.8 27B on an RTX 5090. Two downloads. Same model. Same GPU. Both folders said NVFP4. One of them does 4-bit math for real. Weights in 4-bit. Activations in 4-bit. The matrix unit can multiply them as 4-bit times 4-bit. The other stores 4-bit weights. Then unpacks them in the kernel. Then does 16-bit math. Same sticker. Different kernel path. The 5090 did not choose that. The checkpoint did. Especially whether the activations are 4-bit too. If they aren't, there is no legal 4-bit times 4-bit multiply to run. The engine falls back. Quietly. The file still loads. The logs are easy to miss. That is the difference between an encoding you can load, an operation a backend implements, and arithmetic the hardware actually executes. They sound interchangeable. They are not. This is also why prefill and decode do not get the same gift. Prefill has enough token rows to keep the matrix units busy. Native 4-bit math can matter there. Decode is still walking the weights and the cache, one token at a time. If the live state never went 4-bit, you do not get a 4-bit win on that part. The bottleneck moves. Do not benchmark "the model." Benchmark the stack you will actually run. Separate prefill from decode. Pin the exact file, the exact engine, the exact GPU. One more thing people get wrong with the word Blackwell. It is a marketing name on four chips that cannot run each other's kernels. A data-center B200 is not a 5090. A 5090 is not a Spark. A Spark is not a Thor. Instruction support is necessary. It is not sufficient. A checkmark on the spec sheet is not the kernel that ran. In the inference engines article under my profile, I asked a question I still want on the wall: What quantization format has optimized kernels on my target engine? This Qwen run is that question in a box. Same GPU. Same NVFP4 label. Different kernels. The engine followed. The checkpoint decided which kernel it was allowed to launch. The file loaded but that is NOT the same as the kernel you wanted for the GPU you bought.

The biggest impact of mathematics won’t be the millennium problems but proofs in general software at the scale. I think all mainstream software will be verified by 2030. You likely won’t touch an unverified library. Some of the biggest hurdles such as formalization of specs and doing this at 1B LoC scale are the ripe targets for auto research/self-improvement loop. The reason we can say this confidently is because either AI will get paused or the world will end if this doesn’t happen.