More episodes and seasons of Utilizing Tech: https://utilizingtech.com/

As enterprises roll out production applications using AI model inferencing, they are finding that they are limited by the amount of memory that can be addressed by a GPU. This episode of Utilizing Tech features Steen Graham, founder of Metrum AI, discussing modern RAG and agentic AI applications with Ace Stryker and Stephen Foskett. Achieving the promise of AI requires access to data, and the memory required to deliver this is increasingly a focus of AI infrastructure providers. Technologies like DiskANN allow workloads to be offloaded to solid-state drives rather than system memory, and this surprisingly results in better performance. Another idea is to offload a large AI model to SSDs and deploy larger models on lower-cost GPUs, and this is showing a great deal of promise. Agentic AI in particular can be run in an asynchronous model, enabling them to take advantage of lower-spec hardware including older GPUs and accelerators, reduced RAM capacity and performance, and even all-CPU infrastructure. All of this suggests that AI can be run with less financial and power resources than generally assumed.

Guest:
Steen Graham, CEO and Founder, Metrum AI
LinkedIn: https://www.linkedin.com/in/steen-graham-0724557/

Hosts:
Stephen Foskett, President of the Tech Field Day Business Unit at The Futurum Group and Organizer of the Tech Field Day Event Series
LinkedIn: https://www.linkedin.com/in/sfoskett/
X/Twitter: https://x.com/SFoskett
Bluesky: https://bsky.app/profile/stephen.fosketts.net
Mastodon: https://techfieldday.net/@sfoskett

Jeniece Wnorowski, Datacenter Product Marketing Manager and Head of Influencer Marketing at Solidigm
LinkedIn: https://www.linkedin.com/in/jeniecewnorowski/

Scott Shadley, Leadership Narrative Director and Technology Evangelist at Solidigm and Director on the Board of Directors at SNIA
LinkedIn: https://www.linkedin.com/in/scottshadley/

Learn more about Solidigm: https://solidigm.com/
Learn more about Solidigm's AI efforts: https://solidigm.com/ai

Follow Solidigm
LinkedIn: https://www.linkedin.com/company/solidigmtechnology/
X/Twitter: https://x.com/solidigm

#UtilizingTech #AIattheEdge #AIInfrastructure #Sponsored