Beyond ChatGPT: The Hidden Infrastructure Powering AI with JB Baker

On this episode of the Digital Banking Podcast, Josh DeTar spoke with JB Baker, VP of Products and Marketing at Scale Flux, about the often unseen infrastructure behind AI. They discussed the limitations of current technology and where improvements are needed to handle AI’s growth. JB highlighted the "memory wall," which limits data flow to processors, as a major hurdle.

JB explained how this challenge requires innovation in areas like memory, storage, networking, and even cooling systems. He used a simple analogy of an irrigation system needing bigger pipes to support a growing field, representing the need for faster data delivery to power-hungry GPUs.

The conversation also covered the environmental effects of rising energy demands. JB pointed out data centers' substantial energy consumption and the need for more efficient cooling methods. They explored the ripple effect of innovation, where advancements in one area, like energy production, can unlock progress in seemingly unrelated fields like AI.