How AI is changing weather forecasting

Latitude Media covering the new. of the energy transition. I'm Shail Kahn, and this is Catalyst. We don't really understand how the AI models forecast it, but they are capable of treating the hurricane as almost like a large macroscopic scale object that is moving. They have like spatial awareness in a way that the old models didn't. Yeah. That's it's a really interesting area I would say of like sort of the science of how n AI works to understand exactly how they see the world in that sense.

Where the winds are blowing, using AI for weather forecasting. Catalyst is supported by Fishtank PR, an award-winning PR firm focused on climate and energy tech, renewables, and sustainability. Fishtank is known for generating prominent and effective media coverage for the brands they work with. If you want a PR partner that's thoughtful, shoots straight, and gets results, you'll like Fishtank PR. To learn more about Fishtank's approach, visit fishtankpr.com. That's f-i-s-c-h fishtankpr.com.

When utilities need flexible capacity they can count on, they turn to Energy Hub. Energy Hub works with more than 170 utilities, coordinating over 2.5 million devices to manage 3.4 gigawatts of flexibility. Built for the moments when utilities can't afford uncertainty. Energy Hub builds and operates virtual power plants that utilities actually stake their grid planning on, coordinating EVs, batteries, thermostats, and more through a single platform built for utility scale.

Predictive, verifiable, and designed to perform when it counts. Learn more at energyhub.com.

I'm Shale Khan. I lead the early stage investing practice at Energy Impact Partners. Welcome. All right, so here's a statement that I suspect would be pretty non controversial. AI will improve weather forecasting. It's obvious, right?

And it seems like it must be true. I certainly would have agreed with that statement had you asked me before this conversation you're about to listen to. But to me, the interesting question is why exactly? Like through what mechanism can AI improve weather forecasting? For that matter, how do we actually do weather forecasting today? And if it does get better, what are some of the likely outcomes that it will enable? It's an interesting set of questions for me for two reasons.

First, weather forecasting itself is important to a whole host of other categories I care about. Obviously resilience, but also energy and a variety of others, agriculture, et cetera. But also it's interesting because I think it's exemplary of a whole host of next wave applications for AI.

LLMs are of course finding their way through everything that requires language. Now there are world models starting to show up to try to revolutionize robotics and things in the physical world. But what about things like weather? where we have used some machine learning historically, but can we do better with Transformers and the new architecture of AI that we're seeing in other categories?

Let's find out. My guest today is Peter Battaglia. He's a senior director at Google Deep Mind, where he is leveraging the big brain inside the Deep Mind to improve weather forecasting. Here's Peter. Peter, welcome. Thanks. Happy to be here.

All right, let's talk weather forecasting. Um, I want to start maybe by having you school me a bit on something that I realize I don't know, which is how do we do weather forecasting? Like Currently and maybe a little bit of history. Like what have have there been major shifts technologically and how we forecast weather historically? So maybe walk me through the the history such as it is of how we forecast weather and then like what do we actually do today?

Yeah, so I have to admit I'm actually a relative newcomer to the area of weather forecasting myself. So we had gotten involved in this uh several years ago. And it was sort of built out of a research program that was trying to model complex simulations, including fluids. And the Earth's atmosphere is a fluid, and one of the big uh challenges that we were sort of interested in exploring was

mod you know, modeling the atmospheric fluid, which is weather forecasting. So I sh I should say that I sort of have gone through this uh uh journey of learning about weather forecasting. So the stuff that I'll say I'll hopefully hopefully it's accurate. Um But uh you know, forgive me if I make mistakes. So I think my understanding about the field is that uh really a lot of the you know, the historically weather forecasting was very important for agriculture and

uh uh you know, sort of other use cases that were very important for kind of day to day life. Um but I think it was about maybe a hundred, hundred and fifty years ago that you had uh agencies or bureaus that were starting to do like marine forecasting or like kind of more systematic like collecting observations systematically and treating it as a science. Um but then probably about fifty years ago or so you started to see the emergence of like large government public weather agencies. So

I think NOAA in the US was formed uh in the seventies. I think uh ECMWF, the European Center for Medium Range Forecasting, was also formed in the seventies. And these are two of the big prominent weather bureaus. Um but most governments have a weather uh bureau. And they it's it's sort of weather has traditionally been viewed as a public good. So this is something that uh you know they collect tax money and then fund their weather service.

And the idea is that a lot of the you know, it's not not only is forecasting the weather useful, just again, like what you're gonna you know, you gotta wear an um uh have an umbrella or wear a coat. But um for things that are more like uh uh you know, there's a dangerous storm coming, uh flood, uh extreme heat, extreme cold, those types of things, and also a lot of ac uh sort of decision making like agriculture, energy. Um

transportation and I think in general weather forecasting has traditionally been understood to be something that's a very good investment on a dollar. So the public tax money that's uh invested in the in the national weather bureaus has uh significant uh economic returns on those investments. So that's kind of the I think my understanding of the history of the uh kind of uh st you know, standardized or official weather forecasting business in a sense. Um

Maybe one more thing I can say about this uh the way to think about the industry. I think my understanding is you can kind of think about the weather forecasting industry as divided into it's almost like a pipeline, really. the sort of government official weather bureaus that are issuing these like global forecasts that are predicting all sorts of weather variables, but typ typically more at like a coarser spatial resolution.

And then you have this full this like big post processing uh chain where they take the like sort of

base forecasts and then they specialize them for different use cases. So for example, like when you look at the app on your phone and you see, you know, the chance of precipitation, that's not coming directly from NOAA. That's coming from other intermediaries that are like taking, you know, local weather station data and other historical information and trying to kind of like tweak it and improve it and make it especially useful for your use case.

And you see that sort of in energy and all sorts of other uh applications of weather forecasting. corollary to how that has worked historically to like what's what's happening with LLMs today, not to jump into the AI stuff too early, but just in the sense of like the gl is the global forecast, so let's say NOAA's forecast, is that a big mega model?

That spits out this one big forecast. And then what people are doing in the post processing world is saying, okay, I'm going to take that model, but then I'm going to like, Fork it is the wrong word, but I'm gonna I'm gonna fork it and add a bunch of additional data into it to try to make it better at a a smaller spatial resolution. Like I'm just trying to picture what it actually is.

Yeah. So I mean I can I didn't I didn't say much about the what where the where your actual weather forecast comes from in terms of like technically. So maybe if I say that, then it'll maybe it will sort of open uh uh the answer to that question. So It tri again, uh fluids, like the atmosphere is a fluid. And in physics, we have fluid equations called the Navier Stokes equations.

And that's they govern fluids like at all scales, like from the largest scale structure of the universe, which actually turns o turns out to also be a fluid, down to like what's happening in you know in your blood basically. It's there's turbulence. that uh determines sort of how your blood flows and that has kind of important implications. Now All things in between that, right? You have like weather and um you know stream flow and other types of things like that, all fluids.

Uh and what happens like engineers have figured out that well so this the the flu fluids are very complicated to simulate. So in order to simulate them accurately,

they need to approximate the solutions um and so that they can run them on very large computers. Uh because they're they're so complicated they would never run on a computer uh natively you have to sort of break up the computation and approximate certain things in order to actually, you know, model everything that's happening exa for example in the atmosphere.

So that's called numerical weather prediction. Uh the numerical is just saying that they're making a numerical approximation to these Navi Stokes equations.

And uh traditionally it's been run on supercomputers. So like a lot of the big supercomputer centers have either, you know, do a lot of weather forecasting or were even built to do weather forecasting. And in many ways it's been sort of a triumph of science and engineering that we've been able to like decade on decade, predict not, you know, one day, two days, but like ten, twelve, fifteen days into the future, uh i it's hard to even sort of imagine like the scale of that type of

predictive accuracy was sort of unimaginable a hundred years ago. People just didn't think like In two weeks we can kind of know what the weather's gonna be. That's crazy. That it that relies on knowing like what's happening on the other side of the earth as the sort of prevailing winds carry the, you know, moisture and the temperature and all that kind of stuff. So And I imagine there's like an exponential increase in complexity the further out into the feature you get, just because like

the there are a variety of possibilities of what actually happens today. And each one of those needs to be taken into account when I'm trying to predict what's going to happen tomorrow and so on and so forth as you move into the future. That's that's the butterfly effect, right? It's that a butterfly might or might not flap its wings and then that will determine like a week later whether there's a hurricane or not. Right. So the idea is that little tiny um

Changes or little tiny effects or or lacks you know missing effects will cause could cause huge uh changes in weather over time. And that's exactly right. So the fluid, the atmosphere fluid is thought to be chaotic, which means that that that's that's sort of the definition of chaoticity. It's that little tiny changes can have huge, large impacts later.

Um, that's what makes it so hard. And there's coupling across scales. Again, the butterfly flaps its wings, but then you have like up in the top of the atmosphere stuff is happening. Now That that is exactly why it's very, very difficult to sort of you know, s find solutions to the exact equations that govern the atmospheric fluid.

Um it's so we have to make approximations and we use supercomputers and we uh have all kinds of tricks. Uh I should also say like uh another important thing to recognize is that the when you generate when when we a weather forecast has been generated. The actual prediction of the future is only half of the process. That's the it's the second half. The first half of the process is figuring out what the weather currently is.

So if you we have satellites and we have weather stations and balloons and ships and all sorts of information that are taking measurements of like what the weather is all over the earth. But again, using the butterfly as an example. you would have to know like where every butterfly is in principle to act to perfectly forecast the weather. So

If you sort of think that through, you realize that it's not really we we're we're weather forecasting w we weather forecasting is always going to be fundamentally uncertain to some level. We're never gonna be able to make perfect observations of the weather everywhere on Earth with the precision required to perfectly predict the weather a week out.

And so net when weather forecasting, that's why you have a chance of rain versus like, it's definitely gonna rain, right? And you have like a r a range of temperatures, especially as you go out in time. And that's again sort of what makes weather forecasting so hard. So the first step in weather forecasting isn't actually predicting. It's taking all the satellite data and all the stations and all the different observations and estimating the current state of the weather across the Earth.

And once we have that estimate, then we can make the prediction with the supercomputer. our team and a lot of the teams uh in the field who are working on AI-based weather forecasting have been especially focused on. But my guess is that over time we're gonna see other parts of the weather uh forecasting process being um you know s having having more and more AI methods that are coming in and trying to advance them.

Before we get into the AI methods, I It seems like we have generally, even pre AI, we've been getting I mean, you tell me if the curve has been linear or exponential or flat, but like it seems like there's been, I don't know, fairly linear improvement in our weather forecasting ability. for for decades. Like we're getting more precise. Uh we are also getting better at predicting further out into the future, as you said, like, you know, a week, two weeks, et cetera.

To the extent that that's true. Um, I'm sure it's all these things, but like how much of the improvement that we have seen historically has come from, I don't know, A, as you said, just like having better ground truth data on the current state of the weather, B

um more compute, as you said, has been running in supercomputers. So we get power more and more powerful computers. We just run more and more complicated Navier Stokes equations. Or C additional tricks basically that allow you to like do better predictions without adding more compute. Yeah, that's a that's a great question. So I I I'll just admit I don't know the answer to that. I think all three contribute. Um so I can say on the first one, data, uh yeah, we we like

have there's you know better satellites that are flown and there's uh more uh better systems for you know collecting balloon observations or these different sort of things. So we definitely are getting better data and we know that that improves uh the quality of the forecast.

Um we are also getting better models. That's definitely true as well. We're we're the we're getting, you know, big we're building bigger supercomputers. They can operate at finer resolution. Um Just I think uh in the last less than ten years, uh the ECMWF, which has the best weather forecast, uh they increased the resolution, meaning that they had finer detail and space in their forecasts, and that allowed the forecast to be more accurate.

So you see both like adding just raw compute power, but also improving the quality of the models and the approximations can also uh you know has also made a I think a pretty dramatic uh impact. And I think that sort of blurs into your third category of like other tricks.

Um I think in general you have uh You know, w like without getting into the details of how the numerical models work, you can kind of think about them as a backbone that's making a sort of general prediction at a course scale, and then you have a lot of uh other parameterizations and trickery sort of under the hood in the f uh making finer and finer grain predictions and also updating the backbone to be consistent with the fine grain.

And those are all being advanced sort of in parallel and like the teams, these you know, engineering teams and sciences sci uh scientific teams are sort of working together to make these better. Um the last thing I would also say too is again going back to that like post-processing part of the weather forecasting pipeline.

It's not just again that like these large, you know, NOAA and ECMWF and these other large agencies, it's not just that they are improving the forecast. It's that other parts downstream and post processing are improving what they're doing. So actually, the first advent of like AI and machine learning in weather forecasting, or at least some of the earliest, was not like trying to overhaul the whole weather forecast process itself.

making, you know, using more and more like statistical methods and linear regression and nonlinear regression and neural networks and other types of earlier uh machine learning techniques to improve the not the base forecast, but like the specific application. So maybe we can calibrate your, you know, chance of rain better if we have a slightly better downstream model.

Are you tired of overpaying for big name PR firms but not really knowing what they're delivering? Is your comms team wasting time reviewing lengthy messaging briefs and decks? Instead of engaging journalists or producing content, are you wondering why your competitors are getting pressed and you aren't? Fishtink PR is an award-winning climate and energy tech, renewables, and sustainability focused PR firm

Dedicated to elevating the work of both early stage and established companies. Whether you need to position yourself as a thought leader in between project announcements or translate complex ideas and technologies into tangible, compelling stories that resonate with the media, Fishtank can help. Check out fishtankpr.com. That's f-i-s-c-h fishtankpr.com.

Virtual power plants are becoming a reliable way for utilities to manage capacity, but enrolling devices is just the start. What really matters is confidence. Knowing those resources will perform when dispatched, and being able to prove it from the control room to the living room. Energy Hub's platform handles the full picture from near real-time forecasting, locational dispatch.

And the kind of rigorous verification that holds up when regulators, grid operators, or leadership ask, did it deliver? Easy enrollment creates momentum. Proven performance builds trust. That's why more than 170 utilities rely on EnergyHub to manage over 2.5 million devices, delivering 3.4 gigawatts of flexible capacity. See what that looks like at energyhub.com.

So okay, so we we've been improving. We've been uh, you know, in more recent years applying sort of the earlier versions of ML to continually improve. I guess I'm curious from your perspective. Um

what the biggest gaps are. I mean, like obviously we don't we don't have the ability today to generate a perfect forecast three months into the future. Like it could always get better. But apart from just that element of it, are there any areas where you feel like actually there's like a real it's really hard to do X? Is it like precipitation is a bugaboo or, you know, something else, right? Yeah.

I mean I would I think there's sort of two ways to answer that. It's you're always going to be limited by the quality of your data. So if you don't have good data about something, it's it's gonna you know, you're just it's bad uh you know, garbage in, garbage out sort of thing, right? So these models take an estimate of the state of the current weather and then predict what's gonna happen if your estimate isn't very good because your raw observations weren't very good.

You're not gonna get a very good forecast. Um so in improving just collecting more data. And using the data you have collected to form a better estimate of the current weather, that's definitely gonna always improve things.

It's sort of a known gap, right? Now we don't know exactly what the ceiling is. We don't know like if we've you know if we do this satellite or that station observation, how is it gonna improve things? We might have an idea, but we don't always know and sometimes we have to just test it out. Um but the um yeah, the the other thing I would say is that you

you have different features of weather which are harder or easier to predict. So an obvious one is temperature. So temperature is sort of very smoothly, like if you look at a map of the temperature across the earth, it's sort of, you know, it's not You know, up a mountain it's gonna be colder and like in a valley it'll be you know different. But

It sort of varies smoothly. What doesn't vary smoothly is precipitation. So like a rap you know, a a a violent thunderstorm that sort of emerges out of nowhere and there's like high wind and low pressure and all this kind of stuff. That the like where exactly that front will be, like where exactly the precipitation will happen, what exactly the wind and these kinds of things are much, much harder.

The the detail of like the it's everything's happening at a finer scale. Like where, you know, even if you look at a radar map, you can see this. It's not like Precipitation sort of varies smoothly over the earth, you see like there's a little you know thunderstorm right there, or like there's a you know uh rain and it's and then you know a few miles over, nothing. Um so that type of very high-resolution, complex you know, patterns of precipitation, for example.

wind as well, those are much harder to predict because you're effectively predicting a lot more information. You can't just sort of summarize it by saying, Oh, every, you know, twenty five kilometers or whatever the temperature is this and then everything else is just kinda interpolated in between. You have a lot of stuff happening at a much finer scale. Finer scale than a lot of our models even uh capture and then we have to do sec a secondary steps to try to resolve those finer details.

All right. So let's talk about AI then. Um, I mean, you mentioned this is one thing, right? Whenever we talk about AI, quote. quote fingers applied here, right? Like there's the there is ML as a subset of AI, is related entity. We've been doing ML already. So I guess the first question that I have is um as you think about leveraging AI now and into the future.

for weather forecasting. What version of AI are we talking about? Like what version or versions are you actually what is the what are the actual capabilities and or model structures? that are interesting here. Yeah, that's a good question. So yeah, these days, I mean, AI is a pretty catch all term. Um I I find myself just using the word AI just to mean a lot of different things'cause I think it's kinda easier and usually people kinda know.

Um the difference the way I but the way I would say it is the difference between AI and so machine learning is sort of the like statistical Inner core of AI. It's trying to capture, taking data and trying to capture the patterns through a training process. and then uh you know, kind of use some inductive assumption that like, uh, what we've seen in the past is gonna be similar to what we see in the future. Um

Modern AI I think is a broader family of things. It sort of involves like, you know, agents and your interactions with them and a lot of like language models are often sort of associated with AI. Um what we use in a weather for c our weather forecasting models and a lot of folks out in the community are using as as you know this this field is advancing and this AI-based weather forecasting is developing.

We're still mostly using fairly traditional machine learning, supervised learning. So supervised learning just means You take a data set that has a pair of examples, the an input example and a target example, and you train a model to try to take input examples and accurately predict the target example. And so if you think about weather as you know again, like I said before, you're estimating the current state of the weather, and then the next step is to predict what's going to happen next.

Right. That's uh you c it can be tr a supervised learning method can be trained to do that, and that's what we're doing in uh our models and like most folks that that I see are doing as well. And then the only extra step is that just makes one prediction, but then we feed the output of the model back into itself, and then we have it make another prediction. So the output becomes the input and then it generates another output. And if you just sort of chain those steps together, you get

Your first input, and then you get a sequence of outputs that represent future steps in time. And uh so supervised learning, we use a lot of uh these days with in terms of m AI architectures. Transformers and graph neural networks are what we use. People use convolutional neural networks, but

I don't feel that neural network architectures these days tend to be the sort of exciting part. It's usually more of like the training and the sort of data uh how you how you handle the data and that kind of thing. So but you you mentioned transformers,'cause I guess like if we had been having this conversation, um Five years ago. Right. I I imagine that you still would have told me about supervised learning, for example. Right. Like that wasn't that that's not new.

Um, transformers had been invented by that point, but like had not been, you know, uh broadly applied the way that they are today. So what what is it that like this new wave of AI unlocked by things like by by transformers and convolutional neural networks and so on. Like what does that enable above and beyond what you would have been able to do five years ago? Yeah, that's a good question. So the way I look at it is so transformers are very similar to graph neural networks. Um they both of them

are so actually let's let's let's take this back. So we used to use often convolutional neural networks. And the idea here is it learns a little function that's sort of local in an image. And then it sort of applies that same function everywhere. And then you stack up sequences of these layers. And that eventually lets you like one the information on one side of an image communicate with the information on the other side of the image because

A transformer architecture allows you to make a direct connection between the information on one side of the image and the other side of the image, the same way the graph neural network does. And I like to think about it like graph neural networks because What it's like saying is that the same thing is that we're Well,

In a graph you have nodes and you have edges or connections between the nodes. And a longer connection between nodes is for nodes that are farther away and shorter connections are for nodes that are closer. So if you use the graph neural network analogy to describe the older convolutional networks, it's like the graphs are all small. They're all kind of everything's kind of close. It's like nearby in a image.

Graph neural networks allow you to choose how far away you want information to interact. And in Transformers, It's it can be understood as a graph that has connectivity across any spatial scale. So in language models, the way a transformer works is it says, when I want to make a prediction about the next word. I want to be informed by the most recent word, but also every word that has happened in the text before. And that's important because in language,

The next word is not predicted by just the previous word. It actually is predicted by stuff that happened earlier in the sentence or in the paragraph or in the book. Um and so the ability to make uh to have information across large spatial scales that interact with one another.

That ability allows you to uh it opens sort of new patterns of computation and allows you to represent functions that have traditionally been harder to represent, but it it allows you to make better predictions of the next word or the next Um you know, in our case you can kind of think about it as the next spatial point or a faraway spatial point. Um and that that that allows the model to be more flexible and and capture richer functions.

There's a good comparison there. What what do you think of as being I guess you just described something that is similar? about what you can do in weather forecasting thanks to a transformer architecture, to what you can do with large language models, which is what most people are going to be most familiar with in the in the new wave of AI. What's different?

So that's a great question. I think what's what's interesting is that the way that so in in language the text is understood to be is treated as a sequence. It's like s you know, token, token, token. We are also modeling sequences in weather, but we're not allowing our models to look too far back in time. So it tur because weather weather is actually different from text in a fundamental way. Um in fact most physical processes are. They are

What's called markov in that the most recent state of the system determines the subsequent state. So like I said, in text that's not the case. Uh you know, right now I'll just pause. You didn't know what word I was gonna say next, right? It kind of depends on the context, a bunch of words behind it or earlier. With weather forecasting in principle, if you know exactly what's happening right now, you can fully predict what's gonna happen next. You don't need to look further back in the past.

So we actually use transformers not to model the spatial the the time the interactions and weather over time, like the sequence of text, but in space. So in text, you actually don't have a sense of spatial structure, right? You just have one sequence of text. It's just word, word, word. And when you read, you just see word, word, word.

In weather you have spatial structure, you have weather all over the earth at the same time. And it's all, you know, especially the close weather, it sort of determines and can be used to predict what's happening next at, you know, our current location. And so we use transformers and graph neural networks to capture the uh short and long-range spatial dependencies.

And those interactions uh between, you know, what's you know, nearby and what's about to happen next are what determine weather, and that's how we sort of make these predictions. But one thing I should also add is that

Similar to how I was saying earlier, that like it's kind of impossible to measure like everything that's happening on the earth in the fine detail in weather. Um You have to make approximations. These models do too. And this actually brings us to a very fundamental difference between how AI models are making their predictions and how traditional models are.

So AI models can c take the statistical structure of weather patterns. So for example, if I'm looking at a, you know, hurricane that's traveling over the earth, right? In a traditional model, the way it simulates that is it in very fine detail, it kind of figures out like, what's the pressure and the temperature and the wind and the moisture and what are those things? What's going to happen next is uh determined strictly locally.

AI models, because they can look at a much larger spatial range, they can use what traditional methods use, or they can use other approaches, because you know when you look at a hurricane, it almost looks like an object sliding over a globe. Right. That's not how a traditional model models it. And we don't really understand how the AI models forecast it, but they are capable of treating the hurricane as almost like a large macroscopic scale object.

that is moving because they can see all the structure of the hurricane and they can see, you know, sort of what's happening in the recent past. Spatial awareness in a way that the old models didn't. Yeah. And we don't know. That's it's a really interesting area, I would say, of like sort of the science of how AI works to understand exactly how they see the world in that sense.

It seems like to me, um On one hand, a much harder problem than an LLM because you've got the entire physical world and the the data is sparse as you said and there's lots of complex interactions. On the other hand, it's determinative in a way that LMs are not, right? Like there is no correct next word necessarily, right? There's like a it's a it's the best guess as to what the best next word should word should be. But in the case of weather forecasting, there is a correct

prediction to make. And there is a universe of historical data that you can draw upon to do it. I so I'm I'm back and forth on whether this is a harder problem or an easier problem than like making a really, really good LLM.

I think yeah, I think you'd have I think depending on, you know, what uh uh what where your allegiances lie, that'll probably be I think it's probably more of like a an opinion question. But yeah, I think you're you're absolutely right. So d um The way I think it so it's the one thing I I just would like uh uh I I might say it a different way is

That it's still, like I said before, it still is fundamentally uncertain from the standpoint of the information that's available to the model. Now, yes, like the physics is truly deterministic underneath. But because the model doesn't again see the butterflies or see the little fine scale stuff.

from its perspective, it actually is a random process, right? Because it doesn't if it doesn't know whether the butterfly flapped its wings or not, how could it know whether the hurricane's gonna form or not? So from the perspective of the information available to the model, it is also an uncertain random process to some extent. But I yeah, I think what you're saying is exactly right. So I think the underlying structure of tech.

is random in different ways. It's it's r it like again, I can I'm gonna pause and then I'm gonna say a a word. Uh-huh. Right. Like weather doesn't work like that, right? Like it doesn't I i it doesn't just have a something pop into its head that's completely different than what's histor like present in the historical record, like you pointed out. So I think that the structure of the uncertainty is different in whether it's it's more constrained in a way.

So in text it's you can imagine, you know, if you're wa uh same thing with like video, you're watching a movie and someone's gonna come through the door, you have no idea what they're gonna be wearing uh uh uh as a shirt, right? It could be wearing blue, red, anything. Um There's like no way to predict it. And whether you can always have some idea, you just don't know the s fine details.

On the flip side, weather is an extremely complicated process. This fluid, chaotic fluid system is has, you know, interactions from small scales to large scales, and it's happening all over the earth at once.

So in some sense, instead of just predicting the next word, you're predicting like millions of variables at once. Um so I think you can kinda like it'd probably better like it was uh uh you know, to have this as like a debate over beer or with your friends in your in their L L M lab rather than like something that can be adjudicated just on the basis of these things.

I guess there's one other question on this sort of comparison to LLM world. Um, notoriously, like the the big LLMs are trained on the internet, right? Your train your training data set is like all words on the internet. And so that's one of the reasons that they've been among the first sort of major AI models and this new wave to to commercialize is because there is this gigantic body of training data.

that you can draw upon. Now we're hearing lots of folks who are in like robotics world, for example, like facing the challenge of there just isn't an equivalent data set. You can like try to train on YouTube videos or whatever, but it's not quite the same thing. In the case of weather forecasting, It seems to me like in theory, you have an incredible historical you can look at every historical weather measurement.

Right. If you had access to that data, you could if NOAA, NOAA does, right? Like every input data point they ever took historically. And then And then the subsequent next measurement, which dictated what happened after that, I would think that would be an incredibly rich training data set. Am I, well, two questions. Am I right about that? And is that actually available?

Yeah, that's those are th that's that's those are good questions. So the first thing I would say is actually it's not just language, right? Like the the first big visual neural networks were built were it came out of after ImageNet, a big cor corpus of image data. Um the first language models, like even a decade ago, they were starting to build large text databases.

Protein folding, big databases of protein. So actually you see like you know AI and M and machine learning are still very, very uh sensitive to the availability of high quality data and and large amounts of it. And you kind of like you know, one of the best ways to advance the field is to go collect high-quality data and make it sort of uh standardized and available. So for weather

I think that there's there's sort of good things, there's good news and bad news. So for w one thing is um we we were very fortunate when we started. I think all the all the folks who are working on AI weather forecasting have benefited tremendously from work that was done uh by the ECMWF, the European Center for Meteor Range Forecasting, they built this data set called ERAFI.

Um and they've been building these, you know, ERF, I think was the fifth generation of the era data set. It was a record of the of Earth's weather going back for decades. Um and I think it was originally released going back into the uh nineteen seventy-nine and then they actually opened it to like back into the sixties.

And they were they didn't design this data set to be supporting machine learning. I think it was more to uh just, you know, have an authoritative record of the climate on Earth over, you know, year on year. And it's at like a six hour resolution and twenty-five kilometer spatial resolution. So it's very, very Rich.

It just happened to be perfect for machine learning for weather. And it was just a high a really well-curated data set. The folks at ECMWF were just brilliant and sort of organized and systematic and they had made this available and it allowed a lot of people to sort of you know, stand on the shoulders and and build you know great new AI methods. Now

One thing is though, because we have different satellites and different stations over time, it's not actually all the data set is standardized, but it's not d derived from the same underlying observations. So the quality goes going back in time actually gets worse in terms of it's not as accurate of a record of weather just because again, as I said before, the the input data wasn't as good. Right. Now

The other thing that's not that's not great about weather um is that weather takes a while to happen. So like we have to sort of just wait for more weather data to ha like for more weather to happen to get more data. Right? Like we the weather data we have so far, we're sort of stuck with it. Now, like tomorrow we're gonna have one more day of weather data. But like when our models are taking six hour steps.

We have to just kind of wait. So you're sort of like, we got a lot of data, but at the same time there's not much room to get more of the same kind of data. And I think a we and a lot of others are now looking to more uh unusual or underexplored sources of data. to uh su support building richer, better models. Right. Like is there like a distributed network data th like like in theory, if you had access to

Uh, I don't know. Everybody's cell phone everybody's iPhone. There's probably a but you know, there are bunch of sensors in the iPhone. Like presumably you could pull something from that that would have some signal for you. Yeah, yeah, you can geek out on all these things. Like the way I my my favorite one is like, you know, I have a video doorbell. It sits there and watches weather all day. Right, right.

Cars, your car, right? Like it's when you're you got the you know uh it's got a thermometer in it. It's got like uh your your windshield wipers, uh some of them now are you know, they they s they're rain sensing, so it's sensing rain. Or it's like Um the lights go on automatically when it's dark. So like they're sensing whether it's cloudy or these kinds of things. So I like I get very excited about the possibility of using all these kinds of things. The other one that's even weirder is like

You know, people go on and they tweet about the weather or they uh like talk about the weather on, you know, social media. And like those types of observations those are still observations. Well we don't know if they're very good.

There's a actually a very wide range of pretty unusual underexplored data sources. But even before we get there, like I think you can start to think about there's a lot of companies that are trying to build very cheap weather stations. People can put them on their roof. Bye. These kinds of things could really help both with the kind of core weather forecast and probably a lot of the applications that people want to use weather for.

for. So my takeaway from that is that despite what I said, there being this like amazing historical record of every weather measure measurement that's ever been taken, you still feel kind of data poor, right? Or like training data poor, I guess.

You're always data poor, right? Like that's sort of the the story of modern AI is you're basically always kinda data poor. Because because it in c one of the most incredible facts about modern AI is just how well it scales with data. More data just means better models. You know, I was a person who was very skeptical of this. I didn't think that it was gonna scale like this.

Um and I would make my sort of logical arguments, but it turns out I was wrong and I think a lot of people were wrong and the the folks who really understood that data could really add value even at extremely large scales were right and pursued that course and brought us to where we are today. Interesting. Okay, so I guess I wanna I wanna

finish by talking about what might come. Like if you if you draw a line forward a few years into the future, I know you pick pick your time, three years, five years, ten years, whatever it is. Um and and you and everybody else who's working on AI weather forecasting succeed. Where might we be? Like what might be possible in a few years that's not possible today? Yeah, I mean I think that per I think there's a lot you know, weather affects everything and um it's

you know, it has, you know, different things. Energy is obviously a very uh very sensitive to weather. Some things are, you know, only kind of uh uh uh you know loosely affected by weather. Um So one thing I would like to see, and I think is very exciting, is a wider range of um use cases of weather. So um for example, like we know that uh

even people make different choices about, you know, what to put in their refrigerator or like, you know, what clothes or whatever. These different choices they're gonna go on a trip, what they expect the weather to be. I think that you can start to inf make more subtle and informed uh kind of guidance and suggestions for people.

on the basis of what uh more accurate weather forecasts. And that's kind of like at the consumer level, but I also think that at the kind of you know industry level, there could be a huge opportunity. So for example, in energy, You know, we see there's, you know, renew if you have a wind farm or a solar farm, you're making forecasts about the weather and then you're kind of using that to figure out like how you're, you know, if you're gonna have energy to sell and how you're gonna price it.

But I have a feeling that there's a lot of headroom, a lot more to be gained in how we, you know, plan out our you know uh how to operate our electrical grids. Um how we uh predict what uh what the electrical demand is gonna be. Is it gonna be hot, is it gonna be cold, is it gonna be humid, or the air is carrying more, you know, mass which you know requires more energy to heat or cool.

I think that we just haven't really scratched the surface of the opportunities. I think that supply chains and logistics and you know, even just like lots of choices that, you know, driving and these type of things, I think

really could be better informed by better weather forecasts. And I don't think we've even begun to kind of get into this. And we need to see the quality of the forecast get better and more customized to these use cases to to start unlocking that. The the other one as obviously is just m you know, better crisis uh handling crises like you know, some of our recent work on tropical cyclone forecasting.

um you know, I'm I'm very proud of and uh we think hopefully with better forecasts we can warn people earlier, more accurately. Um we can in some cases where there's a disaster like a wildfire, maybe we can even uh go and intervene and try to uh uh stop it before it gets completely out of control. So again, this is not something that we can do today, but I'm you know very optimistic that

Uh in in in general, maybe this is coming through, but I just we, you know, my team and I really believe in the power of technology that can have a lot of positive benefits. So we often are trying to look for ways that we can, you know, put technology to best use and sort of, you know, leave our kids with a world that's better than the one that, you know, we grew up in. All right, Peter, this was super interesting and

Uh useful for me both in the context of thinking about like weather forecasting and just like understanding like the where AI well how how how AI is getting applied in various industries, what the challenges are, what the opportunities are. So really appreciate your time. Sure, yeah, it was great to have uh to be here and to your questions were great too, by the way. It was really fun. Peter Pataglia is the Senior Director of Research.

at Google Deep Mind Sustainability Program. This show is a production of Latitude Media. You can head over to latitudemedia.com for links to today's topics. Latitude is supported by Prelude Ventures. This episode was produced by Daniel Waldorf. Mixing and theme song by Sean Marquand. Stephen Lacey is our executive editor. I'm Shail Khan, and this is Catalyst.