re: privacy + meta. meta’s confidential computing promise (which isn’t deployed yet) only gives you the strong property if inference itself occurs inside an attested confidential boundary or the remote inference system has an equivalent privacy architecture. otherwise you’re simply sending plaintext context to meta inference endpoint which would mean meta’s inference infrastructure necessarily sees all user data in plaintext. if this is the case congratulations you built an extremely elaborate lock on the bedroom & then mailed the diary to meta every 300ms. if inference is also locked by default from training / retention i suspect as zuck points out you could make a ton of money through commerce transactions taking place which would be enough at meta scale, assuming ppl do eventually transact like crazy though agents.

Re-computing the whole conversation every time you switch models? That might be wasted work. Title: Cross-Model KV Cache Transfer in LLM Families: A Closed-Form Linear Mapping for Prefill Reuse URL: ❓ What does this paper actually do? 💡 NVIDIA researchers show how to "translate" the KV cache computed by one model size and reuse it directly on a different-sized model from the same family, using a training-free, closed-form ridge regression mapping. ❓ Why bother transferring across model sizes? 💡 Production systems escalate from small to large models or swap models mid-conversation for cost/quality reasons. Recomputing the whole context from scratch every time (re-prefill) wastes a lot of compute. ❓ Does accuracy hold up? 💡 On good pairs it retains 73-98% of the target model's standalone accuracy. Weak pairs degrade badly, but adding a small MLP recovers up to 36.8 points of HellaSwag accuracy. ❓ How much faster is it? 💡 2.7-25x faster than re-prefill, with the biggest gains (25x) at 32K-token contexts, and accuracy stays nearly stable across 10-turn conversations. #LLM# #KVCache#

Re: $Pons discussions. Someone sent me this chart asking my thoughts. While it's true that CT tends to extrapolate revenue out waaayyy too far into the future. I do think it has a chance to be sticky. Most the people who are saying that, didn't participate in any upside. At least people are trading a token that has some underlying value. Yes, it's unlikely the revenues will be sustained at that pace. But it doesnt mean it can't go on longer. And it doesnt mean there aren't risks like competitors coming to the chain etc. There are of course a ton of risks. Lots of things trade higher, and they have ZERO value. In general, the market is a giant pendulum that swings to extremes. I for one, am happy for the volatility in general. That's what we are here for. Maybe this time is different, idk. Is it overvalued? Nobody knows. Are they killing it? yes. Will they always? Maybe not. in short, the chart is a good example of the past, but a very limited sample size. I think we will see it swing to extremes each way in terms of over and undervalued. Imo, just enjoy the volatility and make the most of it

Re-Lyte gives you clean electrolytes powered by Real Salt mined right here in Utah, with zero sugar and no artificial sweeteners or flavors. You can shop Redmond online, or stop by a Redmond Heritage Farm Store, stock up on Re-Lyte, Real Salt and your other Redmond favorites, and grab something to eat at the café while you’re there. Find a Heritage Farm Store near you, or shop at and use code CRIDDLE20 for 20% off your online order.