Outpacing Hardware: GPU Memory Bottlenecks, Model Optimization, and the Reality of Enterprise AI | Utilizing AI Episode 44
Utilizing AI Podcast | Futurum Media
•
Educational, News, Talk Show
The hardware needed for artificial intelligence applications today may not be a match for tomorrow's needs.
On this episode of Utilizing AI, host Stephen Foskett, Frederic Van Haren of HighFens and Brad Shimmin of Futurum discuss the reality of AI hardware as models grow and needs evolve. Hardware has always been behind the curve when it comes to AI applications, with basic GPUs giving way to specialized AI hardware and scalable integrated stacks.
As processing power has grown, the bottleneck has passed to memory and storage, with techniques like caching and distributed architectures once again gaining attention. KV Cache is one of the hottest topics in AI infrastructure, with numerous impressive products being brought to market.
The llm-d framework allows high inferencing performance on a distributed infrastructure, which is exactly the same architecture used to run cloud-scale applications with Kubernetes.
We are also seeing a renewed interest in the classic build versus buy question, as companies are weighing IP concerns versus the availability of platforms as a service.
This and more on Utilizing AI, part of The Futurum Podcast Network.
Up Next in Utilizing AI Podcast | Futurum Media
-
Open-Weights, Data Sovereignty, and L...
When cutting-edge AI capabilities shift overnight from exclusive enterprise clouds to permissive open-weights models accessible to anyone, IT leaders are left navigating a chaotic landscape of unprecedented power and heightened security risks.
In this episode of Utilizing AI, hosts Stephen Fosk...
-
Navigating the Model Deluge: How Ente...
Enterprise IT is currently facing a dizzying explosion of new AI models, frameworks, and platforms—making integration, rather than security or governance, the primary roadblock to actual business success. In this episode of Utilizing AI, host Stephen Foskett is joined by Futurum Group analysts Br...
-
Demystifying AI: How Open-Weights and...
What if the key to stripping away apocalyptic hyperbole and demystifying artificial intelligence lies in putting open-weights models directly into the hands of everyday creators worldwide? In this episode of Utilizing AI, host Stephen Foskett and guest Guy Currier explore how the shift toward ope...
