MLOps Coffee Sessions #172 with LLMs in Production Conference part 2 Building LLM Products Panel, George Mathew, Asmitha Rathis, Natalia Burina, and Sahar Mor Using hosted by TWIML's Sam Charrington. We are now accepting talk proposals for our next LLM in Production virtual conference on October 3rd. Apply to speak here: https://go.mlops.community/NSAX1O // Abstract There are key areas we must be aware of when working with LLMs. High costs and low latency requirements are just the tip of the ice...
Aug 18, 2023•46 min
MLOps Coffee Sessions #171 with Thibaut Labarre, Using Large Language Models at AngelList co-hosted by Ryan Russon. We are now accepting talk proposals for our next LLM in Production virtual conference on October 3rd. Apply to speak here: https://go.mlops.community/NSAX1O // Abstract Thibaut innovatively addressed previous system constraints, achieving scalability and cost efficiency. Leveraging AngelList investing and natural language processing expertise, they refined news article classificati...
Aug 15, 2023•52 min
MLOps Coffee Sessions #170 with Phillip Carter, All the Hard Stuff with LLMs in Product Development. We are now accepting talk proposals for our next LLM in Production virtual conference on October 3rd. Apply to speak here: https://go.mlops.community/NSAX1O // Abstract Delve into challenges in implementing LLMs, such as security concerns and collaborative measures against attacks. Emphasize the role of ML engineers and product managers in successful implementation. Explore identifying leading in...
Aug 11, 2023•1 hr 1 min
MLOps Coffee Sessions #169 with Barak Turovsky, MLOps at the Age of Generative AI. Thanks to wandb.ai for sponsoring this episode. Check out their new course on evaluating and fine-tuning LLMs wandb.me/genai-mlops.course // Abstract The talk focuses on MLOps aspects of developing, training and serving Generative AI/Large Language models // Bio Barak is an Executive in Residence at Scale Venture Partners, a leading Enterprise venture capital firm. Barak spent 10 years as Head of Product and User ...
Aug 08, 2023•57 min
MLOps Coffee Sessions #168 with Piotr Niedźwiedź, Experiment Tracking in the Age of LLMs, co-hosted by Vishnu Rachakonda. // Abstract Piotr shares his journey as an entrepreneur and the importance of focusing on core values to achieve success. He highlights the mission of Neptune to support ML teams by providing them with control and confidence in their models. The conversation delves into the role of experiment tracking in understanding and debugging models, comparing experiments, and versionin...
Aug 01, 2023•45 min
MLOps Coffee Sessions #167 with Maxime Beauchemin, Treating Prompt Engineering More Like Code. // Abstract Promptimize is an innovative tool designed to scientifically evaluate the effectiveness of prompts. Discover the advantages of open-sourcing the tool and its relevance, drawing parallels with test suites in software engineering. Uncover the increasing interest in this domain and the necessity for transparent interactions with language models. Delve into the world of prompt optimization, det...
Jul 25, 2023•1 hr 14 min
MLOps Coffee Sessions #166 with Roy Hasson & Santona Tuli, Eliminating Garbage In/Garbage Out for Analytics and ML. // Abstract Shift left data quality ownership and observability that makes it easy for users to catch bad data at the source and stop it from entering your analytics/ML stack. // Bio Santona Tuli Santona Tuli, Ph.D. began her data journey through fundamental physics—searching through massive event data from particle collisions at CERN to detect rare particles. She’s since exten...
Jul 18, 2023•51 min
MLOps Coffee Sessions #165 with Sammy Sidhu, Python Power: How Daft Embeds Models and Revolutionizes Data Processing. // Abstract Sammy shares his fascinating journey in the autonomous vehicle industry, highlighting his involvement in two successful startup acquisitions by Tesla and Toyota. He emphasizes his expertise in optimizing and distilling models for efficient machine learning, which he has incorporated into his new company Eventual. The company's open-source offering, daf, focuses on tac...
Jul 11, 2023•51 min
MLOps Coffee Sessions #164 with Rob Hirschfeld, Open Source and Fast Decision Making. This episode is brought to you by. // Abstract Rob Hirschfeld, the CEO and co-founder of Rack N, discusses his extensive experience in the DevOps movement. He shares his notable achievement of coining the term "the cloud" and obtaining patents for infrastructure management and API provision. Rob highlights the stagnant progress in operations and the persistent challenges in security and access controls within t...
Jul 04, 2023•1 hr
MLOps Coffee Sessions #163 with Yujian Tang, Democratizing AI co-hosted by Abi Aryan. // Abstract The popularity of ChatGPT has brought large language model (LLM) apps and their supporting technologies to the forefront. One of the supporting technologies is vector databases. Yujian shares how vector databases like Milvus are used in production and how they solve one of the biggest problems in LLM app building - data issues. They also discuss how Zilliz is democratizing vector databases through e...
Jun 27, 2023•54 min
MLOps Coffee Sessions #162 with Soham Chatterjee, From LLMs to TinyML: The Dynamic Spectrum of MLOps co-hosted by Abi Aryan. // Abstract Explore the spectrum of MLOps from large language models (LLMs) to TinyML. Soham highlights the difficulties of scaling machine learning models and cautions against relying exclusively on open AI's API due to its limitations. Soham is particularly interested in the effective deployment of models and the integration of IoT with deep learning. He offers insights ...
Jun 20, 2023•45 min
MLOps Coffee Sessions #161 with Tuhin Srivastava, The Long Tail of ML Deployment co-hosted by Abi Aryan. This episode is brought to you by QuantumBlack. // Abstract Baseten is an engineer-first platform designed to alleviate the engineering burden for machine learning and data engineers. Tuhin's perspective, based on research with Stanford students, emphasizes the importance of engineers embracing the engineering aspects and considering them from a reproductive perspective. // Bio Tuhin Srivasta...
Jun 13, 2023•51 min
MLOps Coffee Sessions #160 with Matt Sharp, Data Developer at Shopify, Clean Code for Data Scientists co-hosted by Abi Aryan. // Abstract Let's delve into Shopify's real-time serving platform, Merlin, which enables features like recommender systems, inbox classification, and fraud detection. Matt shares his insights on clean coding and the new book he is writing about LLMs in production. // Bio Matt is a Chemical Engineer turned Data scientist turned Data Engineer. Self-described "Recovering Dat...
Jun 07, 2023•46 min
MLOps Coffee Sessions #159 with Maria Vechtomova, Lead ML engineer, and Basak Eskili Machine Learning Engineer, at Ahold Delhaize, Why is MLOps Hard in an Enterprise? co-hosted by Abi Aryan. // Abstract MLOps is particularly challenging to implement in enterprise organizations due to the complexity of the data ecosystem, the need for collaboration across multiple teams, and the lack of standardization in ML tooling and infrastructure. In addition to these challenges, at Ahold Delhaize, there is ...
May 30, 2023•55 min
MLOps Coffee Sessions #158 with Nils Reimer, MLOps Build or Buy, Large Language Model at Scale co-hosted by Abi Aryan. // Abstract Large Language Models with billions of parameters have the possibility to change how we work with textual data. However, running them on scale at potentially hundred millions of texts a day is a massive challenge. Nils talks about finding the right model size for respective tasks, model distillation, and promising new ways on transferring knowledge from large to smal...
May 16, 2023•1 hr 15 min•Season 1Ep. 158
We are having another LLMs in-production Virtual Conference. 50+ speakers combined with in-person activities around the world on June 15 & 16. Sign up free here: https://home.mlops.community/home/events/llm-in-prod-part-ii-2023-06-20 // Abstract This panel discussion is centered around a crucial topic in the tech industry - data privacy and security in the context of large language models and AI systems. The discussion highlights several key themes, such as the significance of trust in AI sy...
May 12, 2023•25 min
MLOps Coffee Sessions #157 with Katrina Ni & Aaron Maurer, MLOps Build or Buy, Startup vs. Enterprise? co-hosted by Jake Noble of tecton.ai. This episode is sponsored by tecton.ai - Check out their feature store to get your real-time ML journey started. // Abstract There are a bunch of challenges with building useful machine learning at a B2B software company like Slack, but we've built some cool use cases over the years, particularly around recommendations. One of the key challenges is how ...
May 09, 2023•50 min
Sign up for the next LLM in production conference here: https://go.mlops.community/LLMinprod Watch all the talks from the first conference: https://go.mlops.community/llmconfpart1 // Abstract In this panel discussion, the topic of the cost of running large language models (LLMs) is explored, along with potential solutions. The benefits of bringing LLMs in-house, such as latency optimization and greater control, are also discussed. The panelists explore methods such as structured pruning and know...
May 06, 2023•36 min
MLOps Coffee Sessions #156 with Melissa Barr & Michael Mui, Machine Learning Education at Uber co-hosted by Lina Weichbrodt. // Abstract Melissa and Michael discuss the education program they developed for Uber's machine learning platform service, Michelangelo, during a guest appearance on a podcast. The program teaches employees how to use machine learning both in general and specifically for Uber. The platform team can obtain valuable feedback from users and use it to enhance the platform....
May 02, 2023•59 min•Season 1Ep. 156
MLOps Coffee Sessions #155 with Matei Zaharia, The Birth and Growth of Spark: An Open Source Success Story, co-hosted by Vishnu Rachakonda. // Abstract We dive deep into the creation of Spark, with the creator himself - Matei Zaharia Chief technologist at Databricks. This episode also explores the development of Databricks' other open source home run ML Flow and the concept of "lake house ML". As a special treat Matei talked to us about the details of the "DSP" (Demonstrate Search Predict) proje...
Apr 25, 2023•58 min•Season 1Ep. 155
MLOps Coffee Sessions #154 with Waleed Kadous, ML Scalability Challenges, co-hosted by Abi Aryan. // Abstract Dr. Waleed Kadous, Head of Engineering at Anyscale, discusses the challenges of scalability in machine learning and his company's efforts to solve them. The discussion covers the need for large-scale computing power, the importance of attention-based models, and the tension between big and small data. // Bio Dr. Waleed Kadous leads engineering at Anyscale, the company behind the open-sou...
Apr 18, 2023•1 hr 1 min•Season 1Ep. 154
This exclusive podcast episode covers the key findings from the LLM in-production survey that we conducted over the past month. For all the data to explore yourself use this link https://docs.google.com/spreadsheets/d/13wdBwkX8vZrYKuvF4h2egPh0LYSn2GQSwUaLV4GUNaU/edit?usp=sharing Sign up for our LLM in-production conference happening on April 13th (TODAY) here: https://home.mlops.community/home/events/llms-in-production-conference-2023-04-13...
Apr 13, 2023•48 min
MLOps Coffee Sessions #153 with Rodolfo Núñez, Multilingual Programming and a Project Structure to Enable It, co-hosted by Abi Aryan. // Abstract It's really easy to mix different programming languages inside the same project and use a project template that enables easy collaboration. It's not about what language is better, but rather what language solves the given section of your problem better for you. // Bio Rodo has been working in the "Data Space" for almost 7 years. He was a Senior Data Sc...
Apr 10, 2023•1 hr•Season 1Ep. 153
Worlds are colliding! This week we join forces with the hosts of the Practical AI podcast to discuss all things machine learning operations. We talk about how the recent explosion of foundation models and generative models is influencing the world of MLOps, and we discuss related tooling, workflows, perceptions, etc. --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: http...
Apr 07, 2023•59 min
MLOps Coffee Sessions #152 with Keith Trnka, How A Manager Became a Believer in DevOps for Machine Learning. // Abstract Keith Trnka, a seasoned leader in the technology industry, set foot on the MLOps Podcast in a special episode where he shared insights into his experience leading data teams and machine learning teams, becoming a better software engineer, and overseeing a successful migration from a monolith to microservices in the healthcare sector without any downtime. Keith's background inc...
Apr 04, 2023•56 min•Season 1Ep. 152
MLOps Coffee Sessions #151 with Jean-Michel Daignan, ML in Production: A DS from Ubisoft Perspective, co-hosted by Abi Aryan. // Abstract As a data scientist himself, Jean-Michel has a unique perspective on the needs of data scientists when it comes to platform development. He talks about the non-invasive approach his team is taking to bring people onto the platform and their SDK, Merlin. The team is focused on tying machine learning products back to business use cases and the ROI they provide. ...
Mar 28, 2023•50 min•Season 1Ep. 151
LLM in Production Round Table with Demetrios Brinkmann, Diego Oppenheimer, David Hershey, Hannes Hapke, James Richards, and Rebecca Qian. // Abstract Using LLM in production. That's right. Hype or here to stay? The conversation answers some of the questions that have been asked by our community members like; performance & cost of production, the difference in architectures, Reliability issues, and a bunch of random tangents. We have some heavy hitters for this event! // MLOps Jobs board http...
Mar 23, 2023•58 min
MLOps Coffee Sessions #150 with Saahil Jain, The Future of Search in the Era of Large Language Models, co-hosted by David Aponte. // Abstract Saahil shares insights into the You.com search engine approach, which includes a focus on a user-friendly interface, third-party apps, and the combination of natural language processing and traditional information retrieval techniques. Saahil highlights the importance of product thinking and the trade-offs between relevance, throughput, and latency when wo...
Mar 21, 2023•51 min•Season 1Ep. 150
MLOps Coffee Sessions #149 with Jason McCampbell, The Challenges of Deploying (many!) ML Models, co-hosted by Abi Aryan and sponsored by Wallaroo. // Abstract In order to scale the number of models a team can manage, we need to automate the most common 90% of deployments to allow ops folks to focus on the challenging 10% and automate the monitoring of running models to reduce the per-model effort for data scientists. The challenging 10% of deployments will often be "edge" cases, whether CDN-styl...
Mar 14, 2023•56 min•Season 1Ep. 149
MLOps Coffee Sessions #148 with Karl Fezer, Intelligence & MLOps co-hosted by Abi Aryan. // Abstract This conversation explores various topics including biases, defining intelligence, and the future of large language models and MLOps. Karl discusses his paper on defining intelligence and how it relates to the increasing interest in Artificial Intelligence. Karl shares his thoughts on the overlap between foundational models and MLOps, emphasizing the importance of making high-impact tasks mor...
Mar 07, 2023•47 min•Season 1Ep. 148
