we are super bullish on sparse attention + HiSparse and have been working with the vLLM team on it sparse attention reduces the pressure on memory bandwidth by only selecting k for the attention, but it doesn't reduce KV cache memory storage , in high-throughput wide-EP deployment you want to maximize the batch size of decode to use compute as much as possible, but at long sequence decode you quickly run out of VRAM and can't hold enough parallel requests to saturate the compute HiSparse fixes this by offloading the active KV cache to CPU. It keeps an LRU cache on GPU, and since many of the same K tokens are reused every decode you barely notice the offloading, this allows a massive decrease in memory usage and an increase in concurrency This is super important for RL where throughput is key and we want to be as much as possible in a compute-bound regime tldr: lower memory usage, more concurrency, higher inference throughput, faster RL

we are releasing our own coding agent, I have been using it daily since a month now and my favorite place to work with open and close model 1. RLM: * programmatic tool calling * persistent ipython kernel across compaction for long term memory * Subagent daemon and UI 2. Self improving continual harness, model have access to tool to programmatically edit their own harness and skills