Breaking the "Memory Wall": Optical Interconnects Emerge in GPU–HBM Packaging
As a solution to the "memory wall," one of the chronic challenges in AI semiconductors, the memory and packaging industries at home and abroad are weighing an approach that decouples the GPU and high-bandwidth memory (HBM) and packages them separately. The core idea is to move the HBM—until now mounted right next to the GPU—a certain distance away, and bridge the gap with light (optics), allowing several times more HBM to be installed than is possible today.
On the 22nd, a researcher at a major domestic memory maker said, "We're currently struggling to expand HBM bandwidth and capacity, so we're discussing with customers a plan to overcome the GPU's shoreline limit through optical interconnects and mount more HBM." Shoreline refers to the length of the chip's perimeter.
In today's AI computing environment, the key factor dragging down compute efficiency is the data transfer speed of memory chips. While GPU performance has grown by leaps and bounds with each generation, the speed at which memory stores and supplies data has failed to keep pace—creating a structural performance barrier, the memory wall. The arrival of HBM, with its wide data pathways, put out the immediate fire, but critics continue to point out that bandwidth and transfer speeds still fall short of handling the explosive growth in AI compute.
Until now, the industry has focused on stacking HBM ever higher to increase memory capacity and bandwidth within a confined footprint. But as stack counts climbed past 12 and 16 layers toward 20 and beyond, process difficulty rose exponentially. The technology hit physical limits, including the growing difficulty of meeting fixed height specifications. Vertical stacking has reached an inflection point—so much so that the JEDEC standards body has relaxed its HBM height specifications.
The bigger problem is that if stack counts can't be raised, the alternative is to add more HBM horizontally around the GPU—but that, too, is impossible. In the current 2.5D packaging structure, the GPU and HBM are mounted tightly together on a single substrate. Within this structure, the number of HBM units that can be placed is strictly limited by the finite length of the GPU chip's perimeter—its shoreline. Even when more HBM is desired, there is physically no room to place it, leaving the industry in a structural deadlock.
The alternative now emerging across the semiconductor industry is to separate the GPU and HBM and package them independently. It overturns the conventional chip-design principle that components must sit close together to minimize data transfer time. Instead of keeping the two chips adjacent, the approach spaces them apart and links them with overwhelmingly fast optical signals to overcome the added physical distance.
Placing the HBM slightly away from the GPU within the board frees the design from the GPU's shoreline constraint. With the spatial limitation gone, far more HBM can be spread out laterally and packed into the board—several times more than today—without having to push stack heights to extremes. This means the total memory capacity and data bandwidth of the AI accelerator system would expand dramatically, on a scale incomparable to current systems.
"Discussing Placing HBM Beneath the GPU"… Form Factor Could Change
The industry is now producing a range of architectural design proposals over where exactly to place the HBM within the GPU board.
The same memory researcher said, "Options under discussion range from broadly utilizing the space immediately around the GPU to isolating the HBM beneath the GPU board." He added, "In the latter case—isolating it beneath the GPU board—the motherboard would have to be extended lengthwise, so we're discussing even an overall form-factor change with the GPU maker." Specifically, the HBM might surround the GPU from several centimeters away, or a separate HBM zone might be created in the center of the board.
"We're keeping every possibility open as we discuss the optimal layout," he said. "Nothing has been confirmed as an official roadmap yet, but as part of preliminary research toward next-generation AI accelerators, we're in talks with our partners."
The outsourced semiconductor assembly and test (OSAT) industry is also watching this trend closely. An executive at a global OSAT firm said, "Optical interconnects are a clear trajectory. The only question is timing," predicting that "rack-to-rack and server-to-server links will go optical first, and then chip-to-chip connections within the board will follow." He added, "The larger units will be connected by light first, but optical research is moving so fast that it may not be that far off."
Technically, the optical-interconnect technology linking GPU and HBM shares the same underlying principle as the technology connecting server to server inside a data center. The difference is the high technical barrier of shrinking optical-conversion technology—once used for communication between large pieces of equipment—down to the microscopic scale of a single board and chipset.
An executive at a domestic developer of co-packaged optics (CPO) components explained, "As HBM stack heights approach their limit, the industry is discussing spreading the memory out laterally to maximize how much can physically be mounted." He added, "The principle is the same as conventional data-center optical interconnects, but HBM optical links that have to operate within a confined board space require optical components to be miniaturized to far smaller sizes and far higher integration density—so the technical difficulty is greater."
Show more
Research|$MRVL : Investor Day Targets Move Closer to Our Forecasts
Optical interconnects and custom silicon driving growth
Marvell outlined an ambitious outlook at its October 6 investor day, targeting CY30 (FY31) revenue of $80b at the midpoint and non-GAAP EPS of more than $30.
Marvell described a diversified portfolio built to capture AI infrastructure growth:
Optical interconnect products across form factors (pluggables, NPO and CPO) and distances (scale-in to scale-across).
Large custom XPU programs.
A broad XPU attach portfolio that builds on hyperscalers’ familiarity with its off-the-shelf products and growing demand for custom solutions.
In our earnings reviews, we have repeatedly noted that Marvell had room to raise its interconnect and custom revenue guidance. Its guidance is now catching up with our model. We think this and the more ambitious 2030 targets reflect greater confidence in the supply chain. The investor day follows Matt Murphy’s comments about suppliers at the Citi TMT conference last month and recent deals with GlobalFoundries and AT&S. COO Chris Koopmans said key foundry, substrate and packaging suppliers receive five-year forecasts and had already seen the $70–90b FY31 target before the event. Marvell is making capacity payments to reserve supply ahead of revenue. He also stressed that demand and supply are in place to support the $20b FY28 revenue target.
Detailed Report
Show more
BREAKING: The FTC cleared SpaceX to acquire Mesh Optical Technologies, a SpaceX-alumni startup making 1.6 terabit per second optical transceivers for AI data centers, Bloomberg reported (Source: Bloomberg, June 26, 2026).
The FTC granted early termination of its antitrust review on June 25, 2026, with no significant competition concerns flagged.
Mesh was founded in 2025 by 3 former SpaceX engineers. Travis Brashears, Cameron Ramos and Serena Grown-Haeberli built the laser communication system that connects thousands of Starlink satellites to each other in orbit.
They left SpaceX to apply the same optical technology to data centers on the ground.
Mesh's Alpha C1 chip converts electrical signals to light at 1.6 terabits per second. It replaces copper cables that currently connect GPUs inside AI data centers.
Copper interconnects burn power and run hot. Optical interconnects are faster, cooler and more efficient.
Mesh came out of stealth in February 2026 with a $50 million Series A led by Thrive Capital. Less than 5 months later it is being acquired.
The acquisition fits into Musk's broader AI infrastructure stack. SpaceX bought Cursor for $60 billion this month. Now the optical layer is joining the portfolio.
Financial terms were not disclosed.
Show more
$SPCX / Elon Musk acquires Mesh, an optical networking startup.
Which is working on 1.6T OSFP (pluggable).
It’s seems they own the optical engine/packaging side of things, but likely sources CW DFB lasers off merchant companies.
$SIVE is one of the more startup friendly plausible merchant suppliers as seen with Ayar to $POET?
Maybe $LITE and $MTSI that were have a little history too, but less so.
Regardless it’s very positive a lot of startups recently like Celestial have been acquired by Marvell.
For both merchant laser supplier revenue (working with startups -> having hyperscalers like SpaceX drive revenue after being designed in already).
As well as valuations from M&A desirability.
Goes to show how optical interconnects is the right direction if Elon Musk is directly buying these companies.
Show more