Episode description
Topics:
00:00 Intro
01:30 Doug’s story in Search
04:55 How Quepid came about
10:57 Relevance as product at Shopify: challenge, process, tools, evaluation
15:36 Search abandonment in Ecommerce
21:30 Rigor in A/B testing
23:53 Turn user intent and content meaning into tokens, not words into tokens
32:11 Use case for vector search in Maps. What about search in other domains?
38:05 Expanding on dense approaches
40:52 Sparse, dense, hybrid anyone?
48:18 Role of HNSW, scalability and new vector databases vs Elasticsearch / Solr dense search
52:12 Doug’s advice to vector database makers
58:19 Learning to Rank: how to start, how to collect data with active learning, what are the ML methods and a mindset
1:12:10 Blending search and recommendation
1:16:08 Search engineer role and key ingredients of managing search projects today
1:20:34 What does a Product Manager do on a Search team?
1:26:50 The magical question of WHY
1:29:08 Doug’s announcements
Show notes:
Doug’s course: https://www.getsphere.com/ml-engineering/ml-powered-search?source=Instructor-Other-070922-vector-pod
Upcoming book: https://www.manning.com/books/ai-powered-search?aaid=1&abid=e47ada24&chan=aips
Doug’s post in Shopify’s blog “Search at Shopify—Range in Data and Engineering is the Future”: https://shopify.engineering/search-at-shopify
Doug’s own blog: https://softwaredoug.com/
Using Bayesian optimization for Elasticsearch relevance: https://www.youtube.com/watch?v=yDcYi-ANJwE&t=1s
Hello LTR: https://github.com/o19s/hello-ltr
Vector Databases: https://towardsdatascience.com/milvus-pinecone-vespa-weaviate-vald-gsi-what-unites-these-buzz-words-and-what-makes-each-9c65a3bd0696
Research: Search abandonment has a lasting impact on brand loyalty: https://cloud.google.com/blog/topics/retail/search-abandonment-impacts-retail-sales-brand-loyalty
Quepid: https://quepid.com/
Podcast design: Saurabh Rai [https://twitter.com/srvbhr]