We welcome Zak Jost from the WelcomeAIOverlords channel. Zak is an ML research scientist at Amazon. He has a great blog at http://blog.zakjost.com and also a Discord channel at https://discord.gg/xh2chKX WelcomeAIOverlords: https://www.youtube.com/channel/UCxw9_WYmLqlj5PyXu2AWU_g 00:00:00 INTRO START 00:01:07 MAIN SHOW START 00:01:59 ZAK'S STORY 00:05:06 YOUTUBE DISCUSSION 00:24:12 UNDERSTANDING PAPERS 00:29:53 CONTRASTIVE LEARNING INTRO 00:33:00 BRING YOUR OWN LATENT PAPER 01:03:13 GRAPHS IN ML...
Jun 30, 2020•1 hr 58 min•Season 1Ep. 14
In this episode of Machine Learning Street Talk Dr. Tim Scarfe, Yannic Kilcher and Connor Shorten spoke with Marie-Anne Lachaux, Baptiste Roziere and Dr. Guillaume Lample from Facebook Research (FAIR) in Paris. They recently released the paper "Unsupervised Translation of Programming Languages" which was an exciting new approach to learned translation of programming languages (learned transcoder) using an unsupervised encoder trained on individual monolingual corpora i.e. no parallel language da...
Jun 24, 2020•1 hr 3 min•Season 1Ep. 12
We cover Francois Chollet's recent paper. Abstract; To make deliberate progress towards more intelligent and more human-like artificial systems, we need to be following an appropriate feedback signal: we need to be able to define and evaluate intelligence in a way that enables comparisons between two systems, as well as comparisons with humans. Over the past hundred years, there has been an abundance of attempts to define and measure intelligence, across both the fields of psychology and AI. We ...
Jun 19, 2020•2 hr 34 min•Season 1Ep. 11
In this episode of Machine Learning Street Talk, Tim Scarfe, Yannic Kilcher and Connor Shorten discuss their takeaways from OpenAI’s GPT-3 language model. With the help of Microsoft’s ZeRO-2 / DeepSpeed optimiser, OpenAI trained an 175 BILLION parameter autoregressive language model. The paper demonstrates how self-supervised language modelling at this scale can perform many downstream tasks without fine-tuning. 00:00:00 Intro 00:00:54 ZeRO1+2 (model + Data parallelism) (Connor) 00:03:17 Recent ...
Jun 06, 2020•1 hr 52 min•Season 1Ep. 9
This week we had a super insightful conversation with Jordan Edwards, Principal Program Manager for the AzureML team! Jordan is on the coalface of turning machine learning software engineering into a reality for some of Microsoft's largest customers. ML DevOps is all about increasing the velocity of- and orchastrating the non-interactive phase of- software deployments for ML. We cover ML DevOps and Microsoft Azure ML. We discuss model governance, testing, intepretability, tooling. We cover the a...
Jun 03, 2020•1 hr 13 min•Season 1Ep. 9
*Note this is an episode from Tim's Machine Learning Dojo YouTube channel. Join Eric Craeymeersch on a wonderful discussion all about ML engineering, computer vision, siamese networks, contrastive loss, one shot learning and metric learning. 00:00:00 Introduction 00:11:47 ML Engineering Discussion 00:35:59 Intro to the main topic 00:42:13 Siamese Networks 00:48:36 Mining strategies 00:51:15 Contrastive Loss 00:57:44 Trip loss paper 01:09:35 Quad loss paper 01:25:49 Eric's Quadloss Medium Article...
Jun 02, 2020•2 hr 29 min•Season 1Ep. 8
In this episode of Machine Learning Street Talk, Tim Scarfe, Yannic Kilcher and Connor Shorten interviewed Harri Valpola, CEO and Founder of Curious AI. We continued our discussion of System 1 and System 2 thinking in Deep Learning, as well as miscellaneous topics around Model-based Reinforcement Learning. Dr. Valpola describes some of the challenges of modelling industrial control processes such as water sewage filters and paper mills with the use of model-based RL. Dr. Valpola and his collabor...
May 25, 2020•1 hr 38 min•Season 1Ep. 7
In this episode of Machine Learning Street Talk, Tim Scarfe, Connor Shorten and Yannic Kilcher react to Yoshua Bengio’s ICLR 2020 Keynote “Deep Learning Priors Associated with Conscious Processing”. Bengio takes on many future directions for research in Deep Learning such as the role of attention in consciousness, sparse factor graphs and causality, and the study of systematic generalization. Bengio also presents big ideas in Intelligence that border on the line of philosophy and practical machi...
May 22, 2020•2 hr 34 min•Season 1Ep. 6
This week Connor Shorten, Yannic Kilcher and Tim Scarfe reacted to Yann LeCun's keynote speech at this year's ICLR conference which just passed. ICLR is the number two ML conference and was completely open this year, with all the sessions publicly accessible via the internet. Yann spent most of his talk speaking about self-supervised learning, Energy-based models (EBMs) and manifold learning. Don't worry if you hadn't heard of EBMs before, neither had we! Thanks for watching! Please Subscribe! P...
May 19, 2020•2 hr 12 min•Season 1Ep. 5
In this episode of Machine Learning Street Talk, we chat with Jonathan Frankle, author of The Lottery Ticket Hypothesis. Frankle has continued researching Sparse Neural Networks, Pruning, and Lottery Tickets leading to some really exciting follow-on papers! This chat discusses some of these papers such as Linear Mode Connectivity, Comparing and Rewinding and Fine-tuning in Neural Network Pruning, and more (full list of papers linked below). We also chat about how Jonathan got into Deep Learning ...
May 19, 2020•1 hr 27 min
In this episode of Machine Learning Street Talk, Tim Scarfe, Yannic Kilcher and Connor Shorten chat about Large-scale Transfer Learning in Natural Language Processing. The Text-to-Text Transfer Transformer (T5) model from Google AI does an exhaustive survey of what’s important for Transfer Learning in NLP and what’s not. In this conversation, we go through the key takeaways of the paper, text-to-text input/output format, architecture choice, dataset size and composition, fine-tuning strategy, an...
May 19, 2020•1 hr 40 min•Season 1Ep. 3
According to Yann Le Cun, the next big thing in machine learning is unsupervised learning. Self-supervision has changed the entire game in the last few years in deep learning, first transforming the language world with word2vec and BERT -- but now it's turning computer vision upside down. This week Yannic, Connor and I spoke with one of the authors, Aravind Srinivas who recently co-led the hot-off-the-press CURL: Contrastive Unsupervised Representations for Reinforcement Learning alongside Micha...
May 02, 2020•1 hr 15 min•Season 1Ep. 2
Three YouTubers; Tim Scarfe - Machine Learning Dojo (https://www.youtube.com/channel/UCXvHuBMbgJw67i5vrMBBobA), Connor Shorten - Henry AI Labs (https://www.youtube.com/channel/UCHB9VepY6kYvZjj0Bgxnpbw) and Yannic Kilcher (https://www.youtube.com/channel/UCZHmQk67mSJgfCCTn7xBfew). We made a new YouTube channel called Machine Learning Street Talk. Every week we will talk about the latest and greatest in AI. Subscribe now! Special guests this week; Dr. Mathew Salvaris (https://www.linkedin.com/in/d...
Apr 24, 2020•1 hr 13 min•Season 1Ep. 1
