Inferact: Building the Infrastructure That Runs Modern AI
Summary
This episode delves into "inference," the critical yet challenging process of running trained AI models in production. Guests Simon Mo and Woosuk Kwon, co-founders of Inferact and creators of vLLM, explain how dynamic workloads, model diversity, and agentic AI are making inference increasingly difficult. They discuss vLLM's open-source success, its community-driven development, and Inferact's vision for a universal inference engine that efficiently powers any model on any chip.Episode description
Inferact is a new AI infrastructure company founded by the creators and core maintainers of vLLM. Its mission is to build a universal, open-source inference layer that makes large AI models faster, cheaper, and more reliable to run across any hardware, model architecture, or deployment environment. Together, they broke down how modern AI models are actually run in production, why “inference” has quietly become one of the hardest problems in AI infrastructure, and how the open-source project vLLM emerged to solve it. The conversation also looked at why the vLLM team started Inferact and their vision for a universal inference layer that can run any model, on any chip, efficiently.
Follow Matt Bornstein on X: https://twitter.com/BornsteinMatt
Follow Simon Mo on X: https://twitter.com/simon_mo_
Follow Woosuk Kwon on X: https://twitter.com/woosuk_k
Follow vLLM on X: https://twitter.com/vllm_project
Check out everything a16z is doing with artificial intelligence here, including articles, projects, and more podcasts.
Hosted by Simplecast, an AdsWizz company. See pcm.adswizz.com for information about our collection and use of personal data for advertising.