The hardware needed for artificial intelligence applications today may not be a match for tomorrow's needs.

On this episode of Utilizing AI, host Stephen Foskett, Frederic Van Haren of HighFens and Brad Shimmin of Futurum discuss the reality of AI hardware as models grow and needs evolve. Hardware has always been behind the curve when it comes to AI applications, with basic GPUs giving way to specialized AI hardware and scalable integrated stacks.

As processing power has grown, the bottleneck has passed to memory and storage, with techniques like caching and distributed architectures once again gaining attention. KV Cache is one of the hottest topics in AI infrastructure, with numerous impressive products being brought to market.

The llm-d framework allows high inferencing performance on a distributed infrastructure, which is exactly the same architecture used to run cloud-scale applications with Kubernetes.

We are also seeing a renewed interest in the classic build versus buy question, as companies are weighing IP concerns versus the availability of platforms as a service.

This and more on Utilizing AI, part of The Futurum Podcast Network.