Register and share your invite link to earn from video plays and referrals.

khazzz1c
@Chonghan_Liu
Turning electricity into tokens is my job, not life | Building
402 Following    2.1K Followers
A few days ago, @wenhaocha1 made an interesting point in our community: Twitter followers might be a much better signal than Google Scholar citations. I’ve been thinking about it over and over these past few days, and I think he’s right. People building AI applications won’t necessarily have a Google Scholar profile. Academic citations can become a kind of intellectual comfort zone, a boundary we draw around ourselves. Sometimes, it’s a way of avoiding the world beyond it. We risk boxing ourselves into a narrow circle, acting as if AI is all about code and math, while LLMs are already transforming industries far beyond our own. The people who truly know how to use AI tools may never publish a paper or accumulate citations. But they’re out there, sharing what they learn and building on X. It’s time to step outside our bubble and embrace the disruption happening beyond it.
Show more
Read xiaomi's hysparse2 paper and compared it with dpskv41-flash. the parts where they split are pretty interesting. Dpsk still compresses the sequence a bit, then uses an extra lightweight indexer to pick top-k kv. xiaomi skips both. every token stays, no extra indexer, the full-attn layers just hand token-level indices straight to the sparse layers. Kv sharing is different too. dpsk passes main kv + top-k indices across its full / reindex / reuse layers and stores that main kv in fp4. this part especially feels like a big setup being planted 👀 xiaomi keeps it simpler, just two-level cross-layer sharing on uncompressed kv. Tbh the reindex part always felt kinda odd to me. when i was adapting v41 i kept wondering why they went straight for reindex. re-scoring the reused kv at those layers feels a bit brute-force, and my gut is closer to xiaomi here: let the full-attn layers pick once and just reuse it. the sequence compression side of v41 is genuinely novel tho. Feels like dpsk is still trimming kv from a few directions at once, while xiaomi bets that keeping every token and letting the backbone do the selection is cleaner.
Show more