XBM: Heterogeneous 3D-stacked HBM with non-volatile base dies
Designed a segmented HBM stack with FeRAM-based NVDRAM in the bottom dies and DRAM above; used the read-dominated weight and write-active KV-cache access patterns in LLM inference to motivate heterogeneous memory placement.
Research details
- Profiled dense and mixture-of-experts LLM workloads and used PACT thermal simulation to evaluate the design. Relative to an STCO baseline, simulations showed 150% higher bandwidth on bandwidth-bound ML workloads, approximately 98% lower refresh power, and approximately 8.5 °C lower compute-die temperature.
- Developing the architecture manuscript and an MLIR dialect for compiler-directed placement across heterogeneous memory domains; the compiler extension is ongoing.