Now we will have some grounding for when weird ChatGPT behaviors are intended or side-effects -- shrinking the Overton window of RLHF bugs. This is AI generated audio with Python and 11Labs. Source code: https://github.com/natolambert/interconnects-tools Original post: https://www.interconnects.ai/p/openai-rlhf-model-spec 00:00 OpenAI's Model (behavior) Spec, RLHF transparency, and personalization questions 02:56 Reviewing the Model Spec 08:26 Where RLHF can fail OpenAI 12:23 From Model Spec's t...
May 13, 2024•14 min•Ep. 32
Many, many signs of life for preference fine-tuning beyond spoofing chat evaluation tools. This is AI generated audio with Python and 11Labs. Source code: https://github.com/natolambert/interconnects-tools Original post: https://www.interconnects.ai/p/how-rlhf-works-2 00:00 How RLHF works, part 2: A thin line between useful and lobotomized 04:27 The chattiness paradox 08:09 The mechanism for making models chattier 10:42 Next steps for RLHF research Fig 1: https://huggingface.co/datasets/natolamb...
May 01, 2024•13 min•Ep. 31
Models that seem totally out of scope from recent open LLMs give us a sneak peek of where the industry will be in 6 to 18 months. This is AI generated audio with Python and 11Labs. Source code: https://github.com/natolambert/interconnects-tools Original post: https://www.interconnects.ai/p/phi-3-and-arctic-llms 0:00 Phi 3 and Arctic: Outlier LMs are hints 1:01 Arctic & open mixture of expert trends 6:10 Phi 3, synthetic data, and small models Fig 1: https://huggingface.co/datasets/natolamber...
Apr 30, 2024•10 min•Ep. 30
Certain definitions of AGI are backing people into a pseudo-religious corner. This is AI generated audio with Python and 11Labs. Source code: https://github.com/natolambert/interconnects-tools Original post: https://www.interconnects.ai/p/agi-is-what-you-want-it-to-be 00:00 AGI is what you want it to be 04:01 RL still rules the AGI discourse 05:43 Modern AGI tests 07:37 Agency and shifting goalposts Fig 1: https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/agi/img_018...
Apr 24, 2024•11 min•Ep. 29
Meta shows that scaling won't be a limit for open LLM players in the near future. This is AI generated audio with Python and 11Labs. Source code: https://github.com/natolambert/interconnects-tools Original post: https://www.interconnects.ai/p/llama-3-and-scaling-open-llms 00:00 Llama 3; scaling open LLMs to AGI 01:44 Pretraining, data, and basic evals 06:06 Alignment and human evaluations 10:08 Chatting with Meta AI and Llama 3 70B Instruct 11:55 Same Llama license (mostly) 12:52 The healthy ope...
Apr 21, 2024•15 min•Ep. 28
Integrating some non computing science into reinforcement learning from human feedback can give us the models we want. This is AI generated audio with Python and 11Labs. Source code: https://github.com/natolambert/interconnects-tools Original post: https://www.interconnects.ai/p/reinventing-llm-alignment 0:00 Stop "reinventing" everything to "solve" AI alignment 2:19 Social Choice for AI Alignment: Dealing with Diverse Human Feedback 7:03 OLMo 1.7 7B: A truly open model with actually good benchm...
Apr 17, 2024•8 min•Ep. 27
Modeling the compute versus performance tradeoff of many open LLMs. This is AI generated audio with Python and 11Labs. Source code: https://github.com/natolambert/interconnects-tools Original post: https://www.interconnects.ai/p/compute-efficient-open-llms 0:00 The end of the "best open LLM" 3:05 Compute efficient open LLMs Fig 1: https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/scaling/img_004.jpeg Fig 2: https://huggingface.co/datasets/natolambert/interconnects-fi...
Apr 15, 2024•7 min•Ep. 26
Last minute title change from: The tech industry can't agree on what open-source AI means. That's the process. How to read what multiple people mean by the word openness and see through the PR speak. This is AI generated audio with Python and 11Labs. Source code: https://github.com/natolambert/interconnects-tools Original post: https://www.interconnects.ai/p/flavors-of-open-source-ai 0:00 The tech industry can't agree on what open-source AI means. That's the process. 2:45 1. Effective Accelerati...
Apr 03, 2024•9 min•Ep. 25
Databricks' new model is surpassing the performance of Mixtral and Llama 2 while still being in a size category that's reasonably accessible. This is AI generated audio with Python and 11Labs. Source code: https://github.com/natolambert/interconnects-tools https://www.interconnects.ai/p/databricks-dbrx-open-llm 00:00 DBRX: The new best open model and Databricks' ML strategy 03:36 The DBRX narrative 07:33 Databricks' open LLM (and AI) strategy 09:42 Playing with DBRX Instruct 14:54 Digging for de...
Mar 29, 2024•17 min•Ep. 24
Evaluation is not only getting harder with modern LLMs, it's getting harder because it means something different. This is AI generated audio with Python and 11Labs. Music generated by Meta's MusicGen. Source code: https://github.com/natolambert/interconnects-tools Original post: https://www.interconnects.ai/p/evaluations-trust-performance-and-price 00:00 Evaluations: Trust, performance, and price (bonus, announcing RewardBench) 03:14 The rising price of evaluation 05:40 Announcing RewardBench: T...
Mar 21, 2024•13 min•Ep. 23
Where moats are tested now that so many people have trained GPT4 class models. Claude 3, Gemini 1.5, Inflection 2.5, and Mistral Large are here to party. This is AI generated audio with Python and 11Labs. Music generated by Meta's MusicGen. Source code: https://github.com/natolambert/interconnects-tools Original post: https://www.interconnects.ai/p/gpt4-commoditization-and-moats 00:00 Building LLM moats despite the commoditization of GPT4 04:38 The Open's opportunities 08:02 It's amazing people ...
Mar 13, 2024•11 min•Ep. 22
A proposal for a new definition of an "open source" LLM and why no definition will ever just work. This is AI generated audio with Python and 11Labs. Music generated by Meta's MusicGen. Source code: https://github.com/natolambert/interconnects-tools Original post: https://www.interconnects.ai/p/an-open-source-llm 00:00 The koan of an open-source LLM 03:22 A new naming scheme for open LLMs 07:09 Pivot points and politics 08:16 Claude 3, arms race, commoditization, and national security 10:01 Doom...
Mar 06, 2024•23 min•Ep. 21
This interview is available on podcast players and YouTube . I’m excited to bring you another interview! This one is a deep dive right in my wheelhouse — all things RLHF. Louis Castricato is probably the hidden star of RLHF in the open. I’m not sure anyone who can speak freely knows as much as him. As I’ve said again and again on Interconnects: Giving a voice to researchers is the best way to cut through the noise and understand what is happening with core developments of LLM technologies. Louis...
Mar 04, 2024•1 hr 26 min
Basic tips on how to assess inbound ML content and cultivate your news feed. This is AI generated audio with Python and 11Labs. Music generated by Meta's MusicGen. Source code: https://github.com/natolambert/interconnects-tools Original post: https://www.interconnects.ai/p/making-a-ml-feed 00:00 How I assess all these AI releases 01:22 1. Model access and demos are king of credibility 02:31 2. Focus your feed on depth or breadth 03:09 3. Examples of using the model normally show its usable, shoc...
Feb 28, 2024•11 min•Ep. 19
Google rejoins the open model party and gets some backlash for a frequent problem for generative AI. This is AI generated audio with Python and 11Labs. Music generated by Meta's MusicGen. Source code: https://github.com/natolambert/interconnects-tools Original post: https://www.interconnects.ai/p/gemma-google-ships-it 00:00 Google ships it: Gemma open LLMs and Gemini backlash 03:12 Getting to know Gemma 07:11 Alignment details 08:55 Aside: What is REINFORCE? Some history of RL 11:08 Implementati...
Feb 22, 2024•17 min•Ep. 18
10 Sora and Gemini 1.5 follow-ups: code-base in context, deepfakes, pixel-peeping, inference costs, and more This is AI generated audio with Python and 11Labs. Music generated by Meta's MusicGen. Source code: https://github.com/natolambert/interconnects-tools Original post: https://www.interconnects.ai/p/sora-gemini-follow-up 00:00 10 Sora and Gemini 1.5 follow-ups: code-base in context, deepfakes, pixel-peeping, inference costs, and more 00:46 1. Deepfake detection of Sora 01:59 2. Playing with...
Feb 20, 2024•15 min•Ep. 17
Emergency blog! Three things you need to know from the ML world that arrived yesterday. This is AI generated audio with Python and 11Labs. Music generated by Meta's MusicGen. Source code: https://github.com/natolambert/interconnects-tools Original post: https://www.interconnects.ai/p/sora-gemini-and-mistral-next 0:00 OpenAI's Sora for video, Gemini 1.5, and a secret Mistral model 0:53 Sora: OpenAI's text-to-video model 4:59 Gemini 1.5: Google's effectively infinite context length 8:01 Mistral-ne...
Feb 16, 2024•9 min•Ep. 16
In an era dominated by direct preference optimization and LLMasajudge, why do we still need a model to output only a scalar reward? This is AI generated audio with Python and 11Labs. Music generated by Meta's MusicGen. Source code: https://github.com/natolambert/interconnects-tools Original post: In an era dominated by direct preference optimization and LLM-as-a-judge, why do we still need a model to output only a scalar reward? Podcast figures: Figure 1: https://huggingface.co/datasets/natolamb...
Feb 14, 2024•8 min•Ep. 15
Scale's making over $750 million per year selling data for RLHF, who's coming to take it? This is AI generated audio with Python and 11Labs. Music generated by Meta's MusicGen. Source code: https://github.com/natolambert/interconnects-tools Original post: https://www.interconnects.ai/p/alignment-as-a-service 00:00 Alignment-as-a-Service upstarts taking on Scale AI 04:25 The competition with humans-in-the-loop 06:05 Scaling Alignment-as-a-Service via AI feedback Podcast figures: Figure 1: https:/...
Feb 07, 2024•10 min•Ep. 14
A small model at the beginning of big changes. This is AI generated audio with Python and 11Labs Source code: https://github.com/natolambert/interconnects-tools Original post: https://www.interconnects.ai/p/olmo 0:00 Open Language Models (OLMos) and the LLM landscape 6:24 Thought experiments 7:51 The LLM landscape heading into 2024 Figure 1: https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/olmo/img_010.png Get full access to Interconnects at www.interconnects.ai/sub...
Feb 01, 2024•9 min•Ep. 13
Note: some of the audio in the second half is a little wonky, but the general voice was upgraded so hopefully it's a little less "poppy" until then! I'm trying to fix little pronunciation problems on a weekly basis. Thanks to my early fans! It'll keep improving. E.g. some of the months were wonky. When what seems like pure LLM black magic is actually supported by the literature. This is AI generated audio with Python and 11Labs Source code: https://github.com/natolambert/interconnects-tools Orig...
Jan 29, 2024•19 min•Ep. 12
Local LLMs: the latency solution, Meta's open AGI, personalization myth, and moats X factor The deployment path that'll break through in 2024. Plus, checking in on strategies across Big Tech and AI leaders. This is AI generated audio with Python and 11Labs Source code: https://github.com/natolambert/interconnects-tools Original post: https://www.interconnects.ai/p/local-llms 0:00 Local LLMs: the latency solution, Meta's open AGI, personalization myth, and moats X factor 4:15 The personalization ...
Jan 24, 2024•10 min•Ep. 11
A fun demo on how generative AI can transform content creation, and tools for my fellow writers on Substack! This is AI generated audio with Python and 11Labs Source code: https://github.com/natolambert/interconnects-tools Original post: https://www.interconnects.ai/p/multimodal-blogging-tools 0:00 Multimodal blogging tools 2:57 Stratechery, passport, and wonderful customer experiences 5:51 Wrap-up, features, and next steps Figure 1: https://huggingface.co/datasets/natolambert/interconnects-figu...
Jan 17, 2024•8 min•Ep. 10
A sampling of recent happenings in the multimodal space. Be sure to expect more this year. This is AI generated audio with Python and 11Labs Source code: https://github.com/natolambert/interconnects-tools Original post: https://www.interconnects.ai/p/multimodal-rlhf 00:00 Multimodal LM roundup: Unified IO 2, inputs and outputs, Gemini, LLaVA-RLHF, and RLHF questions 02:46 Unified IO 2: Scaling multi-input, multi-output model pretraining 07:47 Collecting preference data for images 09:31 LLaVA-RLH...
Jan 10, 2024•16 min•Ep. 9
And why the comparisons don't really matter. Repeated patterns in the race for reproducing ChatGPT, another year of evaluation crises, and people who will take awesome news too far. This is AI generated audio with Python and 11Labs Source code: https://github.com/natolambert/interconnects-tools Original post: https://www.interconnects.ai/p/open-gpt4-limitations 00:00 Where 2024's "open GPT4" can't match OpenAI's 03:19 Models vs. products 04:51 RLHF progress: Revisiting Llama 2's release and pote...
Jan 05, 2024•14 min•Ep. 8
This interview is on YouTube and podcast players . Giving a voice to researchers is the best way to cut through the noise and understand what is happening with core developments of LLM technologies. I’m excited to get to talk with Michael Poli (Stanford PhD student + research at Together AI) and Tri Dao (incoming professor at Princeton + Chief Scientist at Together AI). This builds on the mega-post from yesterday on the same topics, though the interview is obviously less math heavy: Interconnect...
Dec 21, 2023•36 min
