Outpacing Hardware: GPU Memory Bottlenecks, Model Optimization, and the Reality of Enterprise AI | Utilizing AI Episode 44
Utilizing AI: The Podcast about the Practical Applications of AI
•
Educational, News, Talk Show
The hardware needed for artificial intelligence applications today may not be a match for tomorrow's needs.
On this episode of Utilizing AI, host Stephen Foskett, Frederic Van Haren of HighFens and Brad Shimmin of Futurum discuss the reality of AI hardware as models grow and needs evolve. Hardware has always been behind the curve when it comes to AI applications, with basic GPUs giving way to specialized AI hardware and scalable integrated stacks.
As processing power has grown, the bottleneck has passed to memory and storage, with techniques like caching and distributed architectures once again gaining attention. KV Cache is one of the hottest topics in AI infrastructure, with numerous impressive products being brought to market.
The llm-d framework allows high inferencing performance on a distributed infrastructure, which is exactly the same architecture used to run cloud-scale applications with Kubernetes.
We are also seeing a renewed interest in the classic build versus buy question, as companies are weighing IP concerns versus the availability of platforms as a service.
This and more on Utilizing AI, part of The Futurum Podcast Network.
Up Next in Utilizing AI: The Podcast about the Practical Applications of AI
-
Open-Weights, Data Sovereignty, and L...
When cutting-edge AI capabilities shift overnight from exclusive enterprise clouds to permissive open-weights models accessible to anyone, IT leaders are left navigating a chaotic landscape of unprecedented power and heightened security risks.
In this episode of Utilizing AI, hosts Stephen Fosk...
-
Demystifying AI: How Open-Weights and...
What if the key to stripping away apocalyptic hyperbole and demystifying artificial intelligence lies in putting open-weights models directly into the hands of everyday creators worldwide? In this episode of Utilizing AI, host Stephen Foskett and guest Guy Currier explore how the shift toward ope...
-
Will AI Really Take Your Job? The Hyp...
Although we are bombarded by headlines suggesting that AI will decimate the global workforce, the likely scenario is far more nuanced.
This episode of Utilizing AI brings Nick Patience, AI Platforms Practice Lead, and Olivier Blanchard, Devices Research Director, of the Futurum Group to discuss t...
