Still calling an LLM for every single memory operation your agent makes? This paper splits that work between a "fast brain" and a "slow brain" instead. Title: Jev-Mem: System-One-Controlled Agentic Memory for Efficient AI Agents URL: Inspired by dual-process cognition (System One vs. System Two), Jev-Mem handles most memory operations with lightweight structured decisions and reserves the LLM for genuinely complex reasoning. Three highlights stand out. 🧠 Typed System-One control Frequent operations like typing, relation judgment, and query routing run through a lightweight interface that returns probabilities and labels instead of free-form text — the same control layer governs both memory construction and retrieval. 🕸️ A four-relation memory graph Memory is organized across semantic, temporal, causal, and entity relations, with retrieval budget allocated to whichever views matter most for a given query — avoiding wasted traversal. 📊 Accuracy and speed improve together On the LoCoMo benchmark, Jev-Mem scores 0.777 overall, an 11% gain over the best baseline, while building memory 6.6x faster (158 seconds) and answering queries 36.7% faster (0.93 seconds). What stands out to me is that separating control from reasoning improved accuracy and efficiency at the same time, not one at the expense of the other. #AIAgents# #MemoryArchitecture#

I read LS Securities’ report, “Memory Shortages Even in 2030.” (I love this title name) First, thanks to @PhotonCap for sharing it with me. TL;DR: KV cache offload and HBM optimization do not necessarily mean lower overall memory demand. As AI advances, memory is likely to expand beyond HBM into more layers such as local DRAM, CXL, and storage, while becoming increasingly specialized for different workloads. On the supply side, even as fabs and wafer capacity continue to expand, improving bits per wafer through DRAM scaling is becoming increasingly difficult. At the same time, a meaningful portion of incremental DRAM bits may be allocated to HBM, so wafer growth should not be interpreted directly as conventional DRAM supply growth. On the demand side, memory content per system is becoming more important than CPU or GPU unit growth. Even if server shipments do not increase dramatically, more memory channels, higher-density DIMMs, and the expanding memory footprint of AI workloads can continue to drive memory demand per system higher. This becomes particularly important with Agentic AI. As context windows, KV cache, and agent state grow, the memory problem expands from bandwidth alone to capacity. Not everything can reside in HBM, which naturally leads toward a broader memory hierarchy: HBM → Local DRAM → CXL → Storage This also connects with the custom memory thesis I have discussed for some time. Rather than relying on one general-purpose memory architecture, I expect AI systems to increasingly optimize bandwidth, capacity, and latency for specific workloads. This is why I continue to pay attention to architectures such as Custom HBM, Memory-on-Logic, zHBM, and HBF. And as memory becomes more distributed across these tiers, the next bottleneck increasingly becomes data movement. Accessing a larger memory pool is only useful if data can move between these tiers with sufficient bandwidth, low latency, and low power. This is also why I continue to view CXL, fabric, CPO, and optics as closely connected to the evolution of memory architecture. Memory optimization does not mean less memory. It means memory moves. AI systems are becoming more memory-heavy, memory itself is becoming more tiered and specialized, and the fabrics and optics connecting those memory pools should become increasingly important as well.

A 51-second look at what we're building for the Agentic Infrastructure Stack Summit (Oct 28, SF). Inference costs. Memory architecture. Orchestration nobody's solved yet. Join us →

When incoming Apple CEO John Ternus spoke to CNBC in late 2023, he was all about the technology. Custom chips, device battery life and memory architecture were among his primary interests. Ternus’ job gets a whole lot bigger on Tuesday, when he succeeds Tim Cook and becomes just the second CEO of Apple since Steve Jobs left in 2011, shortly before his death. The 25-year company veteran will be responsible for everything from software and devices to finance and legal, while also having to articulate a vision for a $4.7 trillion company that hasn’t yet fully executed its strategy in artificial intelligence. Read the full story:

From Marvell’s conference call: Q: What progress are you seeing in CXL and scale-up optics? A: CXL began as a server-centric memory architecture but has ultimately proven to be ideally suited for memory expansion and inference. It is now being deployed at scale by multiple hyperscalers. Amid growing inference demand and memory scarcity, customers are revising their plans to expand their use of the technology. Marvell also secured additional design wins in the latest quarter. Scale-up optics has accelerated further from the roughly $300 million FY28 outlook provided last quarter, including $150 million from Celestial AI CPO. CPO and NPO are not an either-or choice; both will coexist. Q: How are constraints such as surging memory prices affecting design wins? A: As supply, power, and architectural constraints converge, the need for fungibility across compute, networking, and memory is increasing. The ability to execute rapidly on custom and semi-custom solutions—and modify products to adapt to evolving architectures—is a critical asset. CXL memory expansion and AI inference accelerators are representative examples. Activity is increasing significantly as the market rapidly shifts from training to inference and then to monetization. $MRVL

How can machine computation simulate the way the human brain thinks? 🧠 The answer may lie in neural dynamical systems and controllable compute-in-memory. Every second, our brains process countless electrical signals, allowing us to perceive, react, and make decisions almost instantly. Yet when computers simulate these neural dynamics, even tiny delays can become a major challenge. In applications such as surgical robotics, autonomous systems, and other real-time scenarios, every millisecond matters. A research team led by Prof. Yang Yuchao from the PKU School of Integrated Circuits, in collaboration with the Shanghai Institute of Microsystems and Information Technology of the Chinese Academy of Sciences, is tackling this challenge with a new phase-change memristor (PCM) chip that enables neural dynamical systems to run in under 10 milliseconds. By combining compute-in-memory architecture with adaptive time-stepping, the team is bringing brain-inspired computing one step closer to the remarkable speed and efficiency of the human brain. #PKUResearch#