Episode description

In this episode, we dive deep into the world of AI engineering with Chip Huyen, author of the excellent, newly released book "AI Engineering: Building Applications with Foundation Models".

We explore the nuances of AI engineering, distinguishing it from traditional machine learning, discuss how foundational models make it possible for anyone to build AI applications and cover many other topics including the challenges of AI evaluation, the intricacies of the generative AI stack, why prompt engineering is underrated, why the rumors of the death of RAG are greatly exaggerated, and the latest progress in AI agents.

Book: https://www.oreilly.com/library/view/ai-engineering/9781098166298/

Chip Huyen

Website - https://huyenchip.com

LinkedIn - https://www.linkedin.com/in/chiphuyen

Twitter/X - https://x.com/chipro

FIRSTMARK

Website - https://firstmark.com

Twitter - https://twitter.com/FirstMarkCap

Matt Turck (Managing Director)

LinkedIn - https://www.linkedin.com/in/turck/

Twitter - https://twitter.com/mattturck

(00:00) Intro

(02:45) What is new about AI engineering?

(06:11) The product-first approach to building AI applications

(07:38) Are AI engineering and ML engineering two separate professions?

(11:00) The Generative AI stack

(13:00) Why are language models able to scale?

(14:45) Auto-regressive vs. masked models

(16:46) Supervised vs. unsupervised vs. self-supervised

(18:56) Why does model scale matter?

(20:40) Mixture of Experts

(24:20) Pre-training vs. post-training

(28:43) Sampling

(32:14) Evaluation as a key to AI adoption

(36:03) Entropy

(40:05) Evaluating AI systems

(43:21) AI as a judge

(46:49) Why prompt engineering is underrated

(49:38) In-context learning

(51:46) Few-shot learning and zero-shot learning

(52:57) Defensive prompt engineering

(55:29) User prompt vs. system prompt

(57:07) Why RAG is here to stay

(01:00:31) Defining AI agents

(01:04:04) AI agent planning

(01:08:32) Training data as a bottleneck to agent planning