UI designs are a particularly interesting domain to apply concept lenses to. Depending on what they’re working on, a designer looking for inspiration might want different kinds of similarity: layout, color palette, visual style, or even just screen type. This is actually the task I was working on when I originally discovered this, but I could never find a solid product use case.

Taking a short break from LLM research to go back to my roots. Introducing... Concept Lenses! A training-free way to do similarity search along whatever concept you care about. Typically, embedding-based similarity search is constrained by whatever notion of similarity the model learned in training. This means that if the model is primed for semantic similarity, but you actually care more about visual appearance, you'd need to switch models. Under a Concept Lens projection, though, distance between embeddings means distance only along the concept you care about. Blog post: Demo:

A survey of on-policy self distillation approaches. 1/ The field can be summarized succinctly as: use log π(y | x, c) - log π(y | x) in some way as training signal. Most variants just differ on how c is derived, ranging from putting the literal answer in context (OPSD/SDFT) to writing a new skill and putting that into context (SEED). 2/ I haven't seen any papers yet proving this works on production-grade settings (e.g, thinking models in agentic settings). Worse, evidence shows that pure OPSD outright degrades thinking model performance ( This matches my lived experience: pure OPSD worked on initial training steps, but quickly degrades the model on longer runs. So where does that leave us? OPSD / SDFT gives degenerate signal, and approaches that try to fix the signal with extra information (RLCSD, purified OPSD) aren't simple enough to be a good primitive. IMO the two promising directions are one of: - Limit OPSD signal to just local credit assignment inside normal RL, as in Composer 2.5 training. - Use environmental feedback to write a targeted system prompt for the teacher. This feels like the best path to getting non-degenerate signal.

Good read on Engram layers and why Chinese labs are gravitating towards them in their models out of necessity (they are HBM starved). Sharing one more data point on Engram scaling: whereas Deepseek finds that trading off 20% of MoE params for Engram leads to all around improvements, Qwen team finds that downstream perf is approximately a wash at all choices of Engram vocab size.

I wonder if we're going to see an explosion in IRL RL envs in the near future, where models train on robotics harnesses connected to the real world and someone has to physically reset things in between rollouts

This latest work from Base Labs looks into whether off-policy distillation (through SFT) from a smarter teacher actually improves post-RL performance, and by how much. Some highlights: - Distillation helps small models a lot more than larger ones. They even find that for 8B models, RL alone beats SFT + RL on their tasks. - Distillation helps more in tasks that require conventions or understanding not already in the base model. - SFT reduces model entropy. For a base model that is already a good RL target to begin with, this is bad because it reduces exploration. So when is it worth doing pre-RL SFT? In my view: - If the base model does a really poor job on the target domain to begin with. This can be because of poor harness use, bad syntax, missing knowledge, etc. - If there's some taste or judgement calls you want your model to pick up from a teacher.

Hoping someone can explain to me what's going on here. The report says they trained multiple mixRL teachers, then used MOPD to combine those teachers into a student model. But..the team literally live-streamed their RL run, and then released model + report just one day after it finished. How does this timeline work out? I would assume the streamed RL run is just a final climb after MOPD, but the paper doesn't mention anything about a final climb. Maybe the livestreamed run is only for one of their mixRL teachers, and they did MOPD quickly in the 24 hours before launch?

Sharing my first of hopefully many research blog posts! This one is the kind of educational blog post I wish I'd had when I started with RL for LLMs. I tried to make it as open as possible. Every rollout is browsable, the code is open source, and I walk through my entire thought process, from learning rate sweeps to reward shaping.

This is notable. DeepSeek, a lab usually first to pioneer novel algorithms and architectures, is saying that at this point, the ROI of improving data quality far exceeds that of working on novel post-training algorithms. I think this has already been true for some time for non-lab practitioners. If you're doing llm post-training, 80% of your effort should go into looking at your data. This means: - Hiring experts to dig through your RL tasks - Sifting through rollouts and sft data by hand to remove suspicious samples. Make sure all tasks are actually passable. - Making sure your data is diverse in both difficulty and category.