SanDisk’s new HBF memory combines 3D NAND capacity with HBM-like bandwidth

What just happened? At its first big investor event since breaking off from Western Digital, SanDisk unveiled something it’s been cooking up to take a bite out of the hot AI market. The company has a new memory architecture called high-bandwidth flash that fuses the massive storage capacity of 3D NAND with the kind of bandwidth offered by HBM.

This hybrid creation stacks up a whopping 16 layers of SanDisk’s latest 3D NAND dies using tiny data pipelines called through-silicon vias. There’s also a special logic layer that can zip data in and out of the individual NAND sub-arrays in parallel. This results in HBF packing 8 to 16 times more capacity per stack than today’s HBM implementations.

In one of SanDisk’s examples, a system rocking eight HBF stacks could provide a monstrous 4 terabytes of capacity for storing bulky AI models like GPT-4 directly on GPU hardware.

The key innovation seems to be that HBF’s architecture breaks with traditional NAND designs by splitting each die into many tiny sub-arrays that can be randomly accessed in parallel, rather than treated as larger block-based planes and pages. This enables the high bandwidth while maintaining NAND’s cost and capacity advantages.

Of course, NAND’s Achilles heel has always been higher latency compared to DRAM technologies such as HBM. As pointed out by Tom’s Hardware, HBF is no different and obviously doesn’t match DRAM’s blazing speeds. The new architecture is being targeted at read-intensive AI inference workloads that need high bandwidth and capacity but can tolerate higher latency. Tasks like gaming are off the table.

There are still some hurdles for the technology to clear. SanDisk was quiet on how it will get around the write endurance limits of NAND, as well as the challenging block-based addressing that could hamper randomized access. We also don’t know what kind of bandwidth numbers HBF can hit.

Despite the remaining questions, it seems SanDisk sees big potential. The company wants HBF to be an open standard complementing HBM in hardware like GPUs. To achieve this, it’s already lining up partners and plans three full generations of HBF development, suggesting a serious long-term investment.

Eventually, SanDisk even sees the tech filtering down from high-end AI systems to consumer devices like smartphones.