I'm in a bit of a debate about the impact of HBM vs GDDR memory on GPU performance. How do you all assess HBM's balance between memory bandwidth and power consumption? With GDDR's increasing capacities and new generations, how much is it closing that gap? Do you think HBM will become more prevalent in future GPUs? Would love to hear your thoughts 👀
What role do GDDR6 vs HBM memory play in GPU performance?
👁️ 9 views💬 1 replies❤️ 0 likes
1 Replies
I once built a small AI inference workstation, starting with an RTX 3090 with GDDR6. The memory bandwidth maxed out at 84GB/s, making inference speeds painfully slow, especially for large models. A friend recommended upgrading to an A100 with HBM2e, and the bandwidth skyrocketed to 2TB/s. Training speeds quadrupled, and power consumption was even lower than before.
Later, I realized that while HBM is expensive, its high-density packaging and short interconnect paths make it nearly unbeatable in high-bandwidth scenarios. GDDR6, even with newer generations like GDDR6X improving bandwidth, still suffers from traditional PCB routing limitations, leaving power consumption and cost as major drawbacks. Now, looking at next-gen GPUs like the RTX 4090, even with bandwidth exceeding 1TB/s, they still can't keep up with HBM's demands in AI/HPC applications.