this is kinda crazy and very interesting, read it! but, given that I don't believe LLMs are magic generalizers with emergent abilities, it mostly makes me think "what the hell did openai trained these models on". (and curious re claude's perf on these tasks)

it’s my nature to keep posts rather impersonal, so i have to work extra hard to tweet this sort of personal testimony below from time to time as i think the reminder in it around AI and the “value of human intelligence” is worth it as of the end of Oct 2026, i have not (yet - you may insist) had the *slightest* moment of questioning the impact of AI on the meaning of any of my work/skills - or even feeling any unusual shift beyond having an amazing (and rather often infuriating) tool believe it or not, but my ability to produce a completed artifact* (paper, model, open source software, lecture, …) and the effort, expertise, or time one takes to produce has actually barely reduced - despite comprehensive productivity benefits from AI the reason is that i essentially NEVER choose to work for a pre-determined outcome or some field-wide well-specified problem; i only work on artifacts i aim to perfect according to *my* judgment and where my personalized ambition obviously has no set limit and isn’t somehow pre-loaded onto AI better tools simply change the costs of my choices - i quite literally work on moving goal-posts if you will! when the fixed constraint are only time and energy, AI simply allows me to re-explore scope and make fewer (or different!!) tradeoffs or compromises, which feels almost no different! i’m not saying every job or every endeavor - including very advanced and historically prized ones! - looks like this today, but i’m offering: (1) at least one voice in this moment (of so many annoying loud FUD voices!) that it’s totally possible and i suspect extremely scalable and i predict will remain so for the foreseeable future! (2) why “someone like me” is indeed choosing happily to be in academia and not elsewhere! (again lots of FUD out here!, but where else do you have the freedom to keep seeking to create value for deciding and solving ~arbitrary ambitious/useful problems, establish credit for it, and train superstars who do the same?) my fear and worry in this moment is a lot less about Al or "the value of human intelligence" and much more about how concentration of power has a very reliable history of being very dangerous to large numbers of affected people! a VERY different problem imo *i guess certain of those took way longer than i expected but do stay tuned for a half dozen things on the short horizon…

I got a dell laptop and want to try switching back to linux after a long time with macos. what distro should i try? i am going for minimum over-time hassle and maximum battery life. omarchy has nice aesthetics but DHH's (extreme lack of) technical expertise scares me.

if your paper title contains a question, then a minimum requirement is for the paper to try and meaningfully answer the question. if your main claim is producing an artifact with property X, a minimum is to discuss how X was achieved, and to verify its quality. (yay ICLR reviewing)

The ideal outcome for science isn't a couple of labs using proprietary models and millions of dollars to crack trophy problems, it's to have millions of AI-aided human scientists who follow their curiosity and become way more ambitious

TIL that "middle east" is now considered a problematic term and we should be using "west asia" instead (say for region-based professional organizations). so, a poll: middle east AI researchers, did you know you are part of west asia?

I haven’t been posting here much but we’re finally releasing an NLP/ML devtool we’ve put a lot of work into. You know how there are tools for agentic coding of websites? Ellf is like that for NLP or ML models. It works with coding assistants like Claude Code, Codex etc and adds the missing pieces needed to make development of NLP or other ML models much better. Ellf does this by providing three pieces Claude Code etc are missing for NLP: annotation, execution and knowledge. The annotation component is key because it lets the agent schedule a little bit of computation on the human. You don’t need to create a massive dataset. Just annotating a few hundred examples here or there helps massively to evaluate, fine-tune and plan. The problem most teams face is they don’t have a low-friction way to get this work done. We’ve always believed in annotation tooling that scales down to low-volume use-cases, which is why we emphasised flexibility and programmability in Prodigy. Coding assistants really unlock this flexibility, because the agent is able to assemble an annotation experience that’s as efficient as possible for your exact use-case. The next thing we wanted to solve with Ellf is execution. Doing NLP/ML projects means code+data development. This is something I think MLOps tools mostly get wrong, because they’re so overly focussed on hyper-parameter search, which is such a small part of what you’re ever doing. Mostly you need to run a lot of once-off scripts to manipulate data, and even when you’re doing experiments, you’re mostly trying to answer some specific question where you have to do a lot of legwork to set up a direct comparison. Ellf gives your coding assistant a seamless and flexible way to perform that computation. Obviously Claude Code can build you pretty much anything by itself — Bash is fully general — but if you make it solve “run this in the cloud” as a subtask in your session, it’s going to do worse at the actual problems you’re trying to solve. Importantly the computation runs under your control. It’s a bit like plugging in self-hosted workers for something like Github Actions. You run a Kubernetes cluster (which can just be local Linux machine if you want) and this is where all the data processing and computation happens. Your data is never sent to us. The final part is knowledge. Out of the box, coding assistants generally put NLP solutions in the too-hard basket, and suggest just using an LLM with some in-context learning. They won’t suggest doing annotation, and they don’t know how to structure a project. A little bit of correction goes a long way on this. We’ve built a knowledge base with our 10+ years of advice, across our forums, talks, blog posts and documentation. Alongside some carefully designed skills, you can use Ellf to plan out your project, and it will do a much better job of recommending you approaches than a coding assistant will do by default. We’re opening registrations for Ellf next week, but if you’ve read this far, you can get early access with the invite code HONNIBAL-TWITTER on app(dot)ellf(dot)ai . I’ve been pretty down on this platform for some time now so I want to see whether I should suck it up and post here more.

today at colm keynote henry farrell gave the amazing insight that llms are not magical creatures, but magical creatures that feed of off many decades of human creativity. and that it is important to involve the humanities to gain such insights and make use of them.

At #COLM2026# tomorrow! If swapping a demographic attribute flips an LLM's answer, is that bias? Compared to what? A paraphrase often does too. Come chat about baselines for counterfactual prompting: 📍 Wed 10/7, 11am–1pm, Imperial #41# w/ @mosh_levy @yoavgo @byron_c_wallace

“A language is a dialect with an army...,” goes the saying about Yiddish. For LMs, training data may be that army. When native texts are overshadowed by noisy translations, multilingual LMs can look fluent while missing what makes a language unique. MameLoshnLM. #COLM2026# 🧵

this paper on the design-space of async/await (from @tonofcrates, @ShriramKMurthi and Gavin Gray) is just fantastic. such a joy to read. are there more like this? i don't care about the topic, could be rpc, ffi, gc, whatever. just this kind of vibe.

"a model produced a proof with very little human input". a large team of experts brainstorm and coordinate on ideas for approaching a hard problem. resulting leads were then sent to ai models, each idea in a diff session. one run succeeded. "with very little user input".

"so harness capabilities are increasingly shifting into the model itself." --> many tnings one does with a harness can be baked in to a model by running the model with the harness' hints and behaviors, and then training it as if they weren't there. like "harness distillation"

if the point is "models don't 'go rogue', they just do what they were trained to do" then sure (though dynamics of complex systems are hard to predict and behaviors emerge) but she is like:

~6-7 million tokens per game for the cheapest run is quite a lot. many many reasoning tokens. def trained directly on such an env (no reasoning vs low reasoning on the standard harness also interesting)

I think we're generally done with "ability to do X" benchmarks. Labs can just point their machines at the next hill the moment it's announced, and in a few months it's not just conquered but smashed to bits. Everything is environment. EdgeBench-type meta evals are all we have now