DRAM is the volatile main-memory technology that every computer and AI accelerator depends on, and because its bandwidth has grown roughly 100x while peak compute grew about 60,000x over two decades, it is now the binding constraint on machine learning inference rather than a solved commodity 2024 Gholami Ai And Memory Wall.
Summary
Dynamic random-access memory stores each bit as charge on a capacitor accessed through a single transistor. That one-transistor-one-capacitor cell is why DRAM is dense and cheap relative to SRAM, and why it is volatile and must be periodically refreshed. Access is organised by rows and columns within banks and subarrays: a row is activated onto sense amplifiers, then columns are read out. This structure sets almost every performance parameter that matters, including activation latency, refresh overhead and the fact that reading is destructive and requires restoration. The technology appears in the market as a family of interface standards rather than one product: DDR for servers and desktops, LPDDR for mobile and unified-memory SoCs, GDDR for graphics, and HBM as stacked dies for accelerators. A current memory-system simulator has to model HBM3/4, LPDDR5/6 and GDDR7 simultaneously to be useful, which is a fair indication of how fragmented the interface landscape has become.
The parameter that decides DRAM’s strategic position is the ratio of bandwidth to compute. Peak server FLOPS have scaled at roughly 3.0x every two years, DRAM bandwidth at about 1.6x and interconnect bandwidth at about 1.4x, producing the twenty-year divergence of ~60,000x compute against ~100x DRAM bandwidth and ~30x interconnect bandwidth 2024 Gholami Ai And Memory Wall. The energy asymmetry runs the same way: an arithmetic operation costs sub-picojoule energy while an off-chip DRAM fetch costs roughly 1.3 to 2.6 nJ, so data movement, not arithmetic, dominates the power bill 2024 Gholami Ai And Memory Wall. This is why LLM decode, state-vector quantum simulation and graph analytics all present as memory-bound workloads. A controlled study on Apple M4 Pro unified memory found every quantum gate implementation had arithmetic intensity at or below 0.38 FLOP/byte against roughly 224 GB/s of LPDDR5X STREAM bandwidth, well below the roofline ridge for any plausible peak compute.
The consequence is that DRAM is no longer a passive commodity tier. Three distinct lines of work are trying to move value into or around it. First, packaging and interface: proposals to give UCIe memory semantics and attach LPDDR6 or HBM dies through a logic die, or to have the DRAM die speak UCIe natively, claim up to 10x higher bandwidth density, up to 3x lower latency and up to 3x lower power than HBM4 and conventional on-package LPDDR. Second, computation in or near the array: processing-in-memory designs put logic on chiplets beside DRAM banks to serve memory-bound GEMV and flat GEMM in the decode phase, while processing-using-DRAM exploits simultaneous multiple-row activation to make each DRAM column a compute engine. Third, heterogeneous hierarchies that offload part of DRAM’s job to ReRAM, MRAM or Flash because DRAM must currently hold static weights and dynamic KV caches at the same time, creating bandwidth contention.
Running underneath all of this is a reliability problem that gets worse as cells shrink. Read disturbance began with RowHammer and RowPress, where repeated access to one row flips bits in neighbours. ColumnDisturb is a distinct phenomenon that disturbs columns rather than rows and affects all cells across three consecutive subarrays. Denser arrays bring rows and columns closer together, so the same physics that delivers capacity scaling also erodes the correctness margin, and it corrupts in-DRAM computation as well as ordinary reads.
Evidence base
- Over twenty years peak hardware compute grew ~60,000x while DRAM bandwidth grew ~100x and interconnect bandwidth ~30x, with underlying rates of 3.0x, 1.6x and 1.4x every two years; an off-chip DRAM fetch costs ~1.3-2.6 nJ against sub-picojoule arithmetic (2024-03-21) 2024 Gholami Ai And Memory Wall.
- Adding memory semantics to UCIe, reusing LPDDR6 or HBM behind a logic die or having the DRAM die speak UCIe natively, is claimed to give up to 10x higher bandwidth density, up to 3x lower latency and up to 3x lower power than HBM4 and existing on-package LPDDR (2025-10-07).
- Sangam decouples logic and DRAM into chiplets in heterogeneous nodes on an interposer with CXL attachment, on the argument that in-DRAM processing elements cost capacity and are limited by DRAM fabrication (2025-11-15).
- ColumnDisturb disturbs DRAM columns rather than rows and affects all cells across three consecutive subarrays; the first mitigations use two counters per subarray plus targeted refresh (2026-06-24).
- Characterisation of 112 DDR4 chips from 10 modules produced device-specific signatures via simultaneous multiple-row activation with average intra-Jaccard indices of 89.02% to 94.86% depending on the number of rows activated (2026-06-16).
- On Apple M4 Pro unified memory with ~224 GB/s LPDDR5X STREAM bandwidth, all quantum gate implementations showed arithmetic intensity at or below 0.38 FLOP/byte and a reproducible 4.46x timing discontinuity at the 28 to 29 qubit hierarchy transition (2026-05-09).
Open questions
- How much performance and energy overhead do ColumnDisturb mitigations impose at the next density nodes, and does that overhead cancel the capacity gain that motivated the shrink?
- Do the claimed UCIe on-package memory advantages of up to 10x bandwidth density and 3x lower power survive silicon implementation and standardisation, or does HBM4 hold the socket?
- Can processing-using-DRAM be made deterministically correct on commercial chips given interference from non-activated rows and concurrently computing columns, or does it stay a research paradigm?
- Does any chiplet DRAM-PIM design reach production design-win volume, or is the retrieval and KV-cache tier permanently captured by software over commodity DRAM, CXL and SSD ref?
Assessment drafted 2026-08-31 from up to 18 KB sources using the technology-scorecard framework; scores are a draft read pending review.