AI, data centers, and power economics, with Azeem Azhar

Welcome to Complex Systems, where we discuss the technical, organizational, and human factors underpinning why the world works the way it does. Hi de ho everybody, my name is Patrick McKenzie, better known as Patio eleven on the internets, and I'm here with my buddy Azimazar. It is so great to be here. I love the name of your podcast. Oh thank you very much.

Uh so Azim runs a newsletter called Exponential View and uh we're gonna be talking about the uh power economics, specifically that of data centers today. Uh people might have heard recently in the news that uh OpenAI et al. are building a multi billion dollar Stargate facility down in Texas. Uh people might have heard uh some sort of hand wavy or more evidence calculations that uh data center usage is going to be tens of percent of uh all power usage in the uh near future. And uh I think uh

For folks inside the industry, uh these are eye popping numbers, uh, but they're somewhat excellent at numbers. And for people who are outside the industry, uh this is all just a little bit wild. So let's take the very long view, uh uh, how does this compare to other infrastructure but rollouts over the last, say, couple of centuries, and then go into the nitty-gritty. But speaking of things that are a couple of centuries old, we've been around this computer thing for a while, haven't we?

We certainly have. I still have my first computer. It's a Z eighty processor and the computer's called the ZX eighty one and I got it in nineteen eighty one. So it's forty Rydw i'n gyrw i'n gyrw i'n gyrw i'n gyrw i'n gyrw i'n gyrw i'n gyrw i'n gyrw i'n gyrw i'n gyrw i'n gyrw i'n gyrw i'n gyrw i chi. The first modem I ever used was three hundred bits per second.

No, that is not misspeaking for those of you who have had cell phones for your entire life. Anyhow, so let's talk about slightly more ancient infrastructure. So as we're doing this build-out of data centers and the electricity, both generation and transmission apparatus. That powers them. I'm sometimes put in mind of other societal wide infrastructure build outs. What's one that comes to mind for you?

You know, we see these infrastructure build-outs every fifty to sixty years, roughly speaking. Uh, and the really, really big ones in the US and in Europe were for the railways and they were for uh electrify electrification just over a hundred years ago or well a hundred to uh eighty years ago in the US

One of the interesting facts uh around all of these build outs is just how significant they were. If you look at the build out of the railways in Great Britain in the eighteen forties to eighteen sixties, um roughly six to seven percent of GDP per annum went into capital investment to build out those those rail lines, which i in US terms

Today, given the size of the US economy, would be approaching kind of a trillion, a trillion and a half dollars a year. So whenever we see a general-purpose technology like this, like artificial intelligence, like the internet, like telecoms and electricity or the railways, uh infrastructure needs to get built and the thumbs of money that go into it are always eye popping. And so too, by the way, are the dynamics of over investment.

And then, you know, at some point there being a bubble that pops because it's so hard, as you know, to forecast exactly when demand and where demand will arise. Vern Hobart has a book called Boom with Stripe Press that lays out this theory with this cool author, Tobias, I believe.

Uh and essentially he says that infrastructure over investment tends to happen with the bubble dynamics, but that the bubble is actually somewhat positive in that it causes a shelling point of different people, firms, government entities, et cetera, that all have different pieces of the puzzle that would not coordinate to do a nationwide or even worldwide infrastructural revolution, but for somewhat bubble dynamic.

Yeah. And then often bubbles pop because as you mentioned, it is uh virtually impossible to get these uh questions right a priori. But the demand sort of backfills over the intervening decades after the popping of the bubble. I I honestly don't know there's substantial technical uncertainty, et cetera, et cetera. This might be the bubble that doesn't pop and always dangerous to say at this time is different, but what I'm saying is like

uh there is some possibility that there's a discontinuity here with with previous infrastructural build outs. But for people who are our generation who remember very keenly the dot com bust happening. people remember the dot com bust as being about the application layer, about web van and pets.com. We had great dot com domains back in those days. But they remember that. But by count of money invested, almost all of it was doing coast to coast

fiber and copper rollouts. And uh that substantially enabled the development of the modern internet and its use as e-commerce, et cetera, platforms. Yeah, I I agree with that. And in fact, the telecoms bubble is really salutary. The total investment in telecoms infrastructure you described it kind of coast to coast, but there was stuff happening outside the US as well.

was I uh as I recall, around six hundred billion dollars between nineteen uh ninety six and two thousand and one. And US telecoms firms alone took on about three hundred and fifty to three hundred and seventy billion dollars of debt in order to do this. And they did it at a time when the telecoms market was growing at about seven percent per annum. And so what feels really different about the this AI market It is

that firms are growing much, much faster than that. I mean, seven percent per week is not unheard of. Right now and you're getting these early stage startups that that like Cursor, which are getting to

a hundred, hundred and fifty million dollars of annual recurring revenue within eighteen months or so, which is an absolutely dramatic, dramatic result. And you know, in fact, six months ago we were looking at data that showed that AI-based software startups were growing three times faster than fast-growing SaaS startups in the pre-AI era, and that's compressed.

even further. And the reason that's different to the telecoms infrastructure build-out is that the revenues are probably ramping faster than the infrastructure growth is ramping, whereas the reverse was true back in the telecoms bubble, which has been relabeled the the dot com bubble by some historians.

So uh I'm also hearing reports of this anecdotal late. Don't treat the following USA as

A quote from an informed market participant, but just treated as a bit of market color on the grapevine. People are saying that firms which are at about stage for raising investment in Silicon Valley and typically at that point you continue to be quickly growing, but are sort of reinvesting in processes to support the next 10X over the next couple of years and getting, let's say, a little less cowboy about the operations.

There are people who are saying that they are seeing uh uh firms at uh the C round stage that are growing as quickly as uh Y combinator companies that are, you know. double digit weeks old, uh, with ten percent week over week growth, et cetera. And not growth in a vanity metric or growth in eyeballs for viewing cap videos, but growth in uh enterprise spend on uh AI capabilities. Which to the extent that one trusts that uh observation is mind blowing. Um

It's absolutely wild. And I've heard similar things as well. So presuming we're not hearing the same rumor sourced from the same person. Let's let's assume that these are coming from different places. That that seems to match somewhat with with what I hear as well. And you can see it in from other data sets. So there is a platform called Open Router.

which has access to a whole bunch of LLM uh APIs. And in some of their recent data, they when I I looked at it, I saw an eight X growth in token usage as in token serve. by this a set of these models over a somewhat less than one year period. And so we're going from, you know, A few hundred million tokens per month to you know several billion tokens uh per month. And of course, while token prices have come down, that is showing that elasticity of demand and the fact that the demand exists.

And the demand is not being saturated at all. So I I sometimes feel that we are the blindfolded men walking around this elephant and we have to sort of put the full picture of what's happening in the market. together and and I don't really see

many of the feelers that suggest we're not dealing with something that's big and fast growing. I mean, t typically every conversation I have, uh sometimes a bit skeptical about them, points to the kind of dynamic that you talked about, which is that the growth is really, really fast. It continues to be fast. Uh and you know, as prices come down, spend goes up because you you can, you know, basically make the economics work on on newer use cases. I also think that people

There's a comparative advantage in having seen the dynamics of SaaS companies uh scale in understanding how this is going to scale. Because I think some of the best informed people with regards to what LLMs can actually do these days are folks that are playing with them every day. But If your view on an L L M's capability is I chat with Claude all the time. He seems very emotionally supportive. I've done this sort of song generation in Suno, which is a wonderful experience, by the way.

you're probably not predicting what those capabilities in an API plus a two to five year enterprise integration cycle looks like. Uh, because after that excess it's not going to be, you know, you invoking one LLM at a time for a couple hours per day. It will be everybody getting a staggering number of L L M invocations on their behalf every day, most in the background, where the rank model isn't a person having a conversation with some it's a

More similar to what happens when you open up the New York Times and several hundred robots conduct an instant auction for your attention on your behalf. And there's an entire ecosystem of firms in ad tech that make that happen. And you will never know most of their name. Anyhow.

Yeah. Well can I can I add to that? I mean we we already got that crossing point on web traffic a a couple of years ago where fifty percent of traffic generated is is is bots traffic doing various thing things like that. And with within you know, processes within the enterprise, there are so many advantages in having uh L LMs or or AI systems talk to each other. I mean

fundamentally because they can be much faster than we can. And and that's one of the things that emerges when you see these sort of optimized distilled LLMs running at a thousand tokens per second, which is that that is really, really significant in terms of you know, ingesting information, making a decision on that information and sending a signal back out to the next step in the process at millisecond speed, whereas humans work at, you know, minute speed.

And at a thousand tokens a second rather than five to eight tokens a second, which may be where where we might sit at the very, very best of times when we're reading something, let alone when we're writing something.

And that velocity, I think, in of itself will breed faster, you know, faster velocity. And that's why I love the idea of this being a complex system and the complex systems podcast, because that is ultimately a complex system, right? With all of these feed forward loops and flywheels. Yeah. And if I can give people concrete examples of what is going to happen very, very quickly.

If you're a connoisseur of the fictional experience of watching rich people talk to each other, a line that you hear a lot in those movies and et cetera is let's have your people talk to my people where the two principles are aloof from their own calendar management, but there's some implicit team in the background that takes care of that for them. Let's have your LLM talk to my LLM is already happening and will certainly happen in the future in every

conceivable way. As an example, I recently got a uh push notification, email, and paper letter all from a bank asking me, hey, uh we haven't seen you update your uh address with us in a while. We need to be on top of your address. And so do you have an update for us? If not, just tell us so.

And that is totally going to be an L LM driven conversation in the future. You can optimize out the stamp entirely, optimize out the annoying interaction with customer, and also optimize out probably all of the time on the bank side, although there's probably not all that much human time on the bank side. Uh Given the nature of the current process. But

Yeah. Can I give you a a really concrete example that I use as a as well? So one of the the thing the tools that I use is a network of of L L M. So I'll have a a a single L L M that acts as a kind of orca orchestrator. And I'll have several other LLMs that act as members of a focus. And I will put my question in to the orchestrator and the orchestrator will pass that question, which is sort of evaluate this idea or evaluate this product.

um to the underlying agent LEMs. Typically I'll use three or four. Each one of them will have a a fairly detailed profile. of uh a a pen portrait, right? A persona, what a marketing manager or an investor. And the, you know, the last one might be a recent college graph. And they will iterate and argue between themselves about the merits of this particular product with a view to coming to some kind of distinct consensus.

And in that virtual focus group where I run three or four of these, they will go back and forth and they will generate tens of thousands of tokens. And then ultimately the orchestrate uh orchestrator will respond uh to me. Now I do that. to help me sense check ideas that I might might want to explore or research or to kind of really push them or to find a very exacting through line or weakness. There are weaknesses in the idea.

Um and and there are companies that are out there, Electric Twin is in the UK that is is one, that are building panels of LLMs where panels of virtual synthetic pro personas that are built within LMs where these conversations will happen much more rapidly and at much larger scale. in order to do in silico, what we might have had to do quite slowly, you know, sort of in vivo and much more expensively with real focus groups. And and I I think that those are also examples that are beyond

you know, AI agents interacting on fixed processes and process flows where they're much, much more open-ended. And actually the the amount of resource that could go into those could be quite quite significant. Mm-hmm. So... Stepping back for a moment, a thing that I've used as part of my writing process for many years is to either have uh a formal review from other people or more form more usually have an informal review where I conjure my mental model of a particular person in my head.

read it and then okay, you know, what would this person in the industry think uh think of this piece right now? Patrick Tollison has described using that for his own writing process. Uh blankly I think I stole the idea from him.

At any rate, the one can increasingly do that with like telling L LLM role play as someone who has a large corpora on the internet, and you don't necessarily need them to successfully anticipate more than, you know A D, ninety percent of what the person would say, just as a idea generation thing.

I did it over the weekend on something which was extremely professionally significant, extremely enough professionally significant that I was always also spending social karma and no small amount of money with a number of external professional advisors. Right. And The fact that the LLM needs no social karma at at all and trivial amounts of money to step through like role-playing as, say, five different potential audiences for a piece.

was revelatory for me in terms of the quality, speed of iteration, et cetera, et cetera, for the advice I got. And if you know sort of early adopters like us are using this in production. Like this was a this was a real thing for me. That meant a a lot was riding on the line of this, you know, that n not a fun test just to try out the new toy.

If the early adopters are using this in production right now for this, uh you can imagine when every marketing team in America is uh running uh marketing team in uh many places is uh running uh as you said uh uh in silico panels. I mean if I if I help us kind of frame this as well, w which is Y you know, when Deep Seek was released at the end of last year, sort of both uh V three and and R one, and then we had the the flurry of excitement in late January.

you know, the idea of Jevon's paradox surfaced, right? Which was Satya Nadella from Microsoft said this. And, you know, the point about Je Jevon's paradox was that essentially, you know, if you've got clogged traffic around Austin and you build another freeway, within a few months you'll have more more traffic jams because ultimately there's positive elasticity of demand, right? You reduce the cost and and demand goes up, you know, up until a saturation point.

And one of the questions is where is that saturation point with with AI? And Yeah the the truth is, I I think we are nowhere near and by nowhere near I I I can't even find the phrase for the size of the fraction of being no nowhere near. So much of what we we do in business and often in our personal lives is fundamentally gated by the fact that we don't have enough time to think.

through to get to the optimal solution. So we just use a heuristic and work we accept that being a bit a bit of slack and a bit of loss. And and essentially we're gonna we're not gonna do that anymore, right? We're gonna let these AI agents do tons and tons of thinking, enormous amounts of it, things which we, you know, we just did out of habit, we will get them to to improve, particularly in business.

And so I don't think there's any end to the amount of demand that we could individually generate as individuals, either as as as as people at home or or in the workplace for thinking to be done by these machines. And the second lever of that is very, very few people, a few of us are currently doing that and very few organizations are doing that. I mean, I think OpenAI has 125,000 people paying for the pro level of Chat GPT, which is

absolutely phenomenal. You know, and then if you're in a job that pays a hundred K a year or more, you should be investing in that straight away because your ROI will be incredible. And and so I think that these two these two elements, which is one is how deep does each person or organization want to go and what new organizations emerge. How many of us are participating in that are both

Going to expand very, very dramatically. And as prices for intelligence or per token come down, the break-even point will rise and will accelerate the demand. And so that gets us to the question of like, what is the infrastructure that's going to serve serve all? Mm-hmm. So we'll get to that infrastructure in one second, but to remind people of a famous phrase in the technology industry, there was a point where intelligent people well steeped in uh the worldwide situation for demand.

said that there's a worldwide market for perhaps five computers. Right. And it turns out that we can deploy many more than five. We are, I think, right now in the five computers days of LLM usage where we've applied it to the obvious applications and people who are only seeing the obvious applications

fail to appreciate uh what will happen once it gets injected into just about everything. And that I feel pretty confident is going to happen over the course of the next uh 10, 20 years, uh, with uh, you know, new developments on a week by week, month by month basis. I think the acknowledgement of an ad read sounds cooler in Japanese.

Cool, right? So you're selling Enterprise SAS and a prospect asks you about how your API manages access. Who does sales call? Linda, because they always call Linda. Linda, who heroically put together a color-coded map of authentication mechanisms and fine-grained access controls. Linda, who tells you she doesn't mind, because technically correct is the only acceptable form of correctness.

Linda, who certainly didn't get a bachelors in science and work her entire career so that she could redundantly respond to inbound security questionnaires with fifty shades of the same API question. Every company has a Linda, or will someday find it needs one. Or maybe you're already at the scale where you have dozens. Regardless of your size and sophistication, you can help your Linda.

You with Safebase, the leading trust center platform. Safebase AI helps distill what Lyndon So that your sales team can field questionnaires proactively and accurately, even when Linda is on well deserved. Safebase has workflow integrations with Slack, Salesforce, Teams, and more. To streamline collaboration with Linda and help close deals faster, Safebase can accelerate your sales process.

Some customers see time to value in as little as 16 days. That's not just fluff, there's a graph in the dashboard. against your peers. Earn the trust of your customers with true, speedy responses to the questions that matter to them. Go to safebase.io slash podcast to learn more. Do it Do you know what percentage of your customers need payroll? Yep, it's all.

It is often still painful and tedious, and while SaaS platforms have simplified every other aspect of running a business, payroll is frequently left out. Our sponsor, Czech, is working to change that. As the leading payroll infrastructure provider and pioneer of embedded payroll, Check's flexible API makes it easier for platforms to offer a tailored payroll experience from day one.

and avoid the headaches that usually come with it, all while increasing retention and opening up new revenue streams. Powering tax calculation, filing, money movement, onboarding, support, and more. Check is already trusted by over sixty innovative platforms such as Wave, Home Base, When I Work, and House Call Pro, who collectively serve over three million businesses across the country.

Ready to offer payroll to the businesses on your platform? Visit checkhq dot com slash complex to learn more. That's Check HQSN Headquarters dot com slash complex. Add payroll to your platform today with check.

Let's talk about that infrastructure. So data centers. We have a very diverse listener group for this podcast and so some people are intimately familiar with walking to a data center, some people less so. Let's set the scene first. You walk into a data center, look to your left, look to your right, what do you see? It looks like that scene in the Matrix when Neo and Trinity ask for guns.

and you're in a white room and just racks and racks of guns show up and that's what you you know, you what you see in a in a data center. You you see very large numbers of of racks that have within them you know, p pizza box sized computers stacked up and cooling systems at the back and power delivery and what is you know, ha ha I mean, in a sense, what you see today is not too different in my view to what we saw twenty years ago. But in reality

everything is much denser, right? The networking bandwidth uh interconnects are running a hundred times faster than they used to be. The power demand is much, much higher. I mean, I think that the most recent, most recent sort of H one hundred racks, which are the yeah, H one hundreds is a big, big NVIDIA uh GPU that people like Meta use. They'll be running at 130 kilowatts per rack, which is seventy kettles, is how I think about it. I you know, to put that into context, the first

servers that I ran to to serve websites back in nineteen ninety-six ran to two hundred watts each and I had four of them. So I had eight hundred watts sitting there roughly, including the the the sort of little Ethernet switch that was was uh uh connecting them to the to the to the internet. So we've gone from eight hundred watts to a hundred and thirty kilowatts

And that's the power demand. But of course you then have to to cool this as well. And you have you have sort of storage requirements as well. And I think roughly speaking, um, about forty to forty-five percent of this goes into the uh uh actually doing the thinking, right? The the the flops and the processing, about forty percent goes into the cooling, and then the rest is uh, you know, networking and redundancy.

The cooling often surprises people who uh aren't specialists in this, but it basically comes down from uh well, data centers don't get to cheat the laws of physics if you pump X amount of energy into them, that energy has to go somewhere. Uh and what you typically need to do in most locations is active cooling to remove the energy that you've pumped in uh to the external environment.

In some locations that are very cold at certain point points of the year, you can use passive cooling, but uh we're putting data centers all over the place. Yeah, and so active cooling in basically means we have to pump a liquid in and we have to have a heat exchanger somewhere else on the other side. And, you know, I think in Colossus, which is uh Elon Musk's data center, there's a fantastic video that shows the size of the final

cooling pipes and you know, that they come up to your shoulder. They're they're pretty phenomenal. So one final bit of color before we go into the recent developments in this, but density drives so much of both the economics and the operational concerns of data centers. And as we've heard over the last uh couple of decades, they're getting more dense per

Square centimeter, cubic centimeter, I suppose, uh because height is a material thing. We stack these boxes on top of each other.

The why does density matter fundamentally for the operator? Because a data center is fundamentally a real estate business, uh, with some value add it being the power, cooling, and uh on-site technical services. But um An example of a counterintuitive thing with respect to what the density and intensification does for you, data centers, because they have huge amounts of electricity running through them, operating at high temperatures, as high as you can do without damaging the chip.

are not the safest places in the world to be, particularly during some failure modes. And so when I first got the uh badge that would allow me into a data center when I was working at a Japanese system integrator back in the day, uh I had to be given a safety briefing before I got the badge because there is a device on the wall called colloquially a big red button.

And there are many genres of big red button in the world. A thing you really want your young engineer to understand before they walk into that room is. Is this the big red button that just drops all the power in the room? Or is this the big red button that you have sixty seconds after you press it before every living thing in the room dies? The halon, the halon gas, right, comes out to to extinguish an electrical fire.

That's one of the things your on-site safety engineer will be really, really interested in, making sure that everyone taking even a guided tour of that room understands. So Now you know what it looks like in the data center. So let's take a look around the I I wanna I can I just add something to this to this point, right? So so you know, one of the things that you've described, which is this that increasing density, also has an impact on the

physical real estate asset. So many data centers that exist today, and you know, if we've driven down freeways in parts of the the US, you'll have seen these buildings that look like warehouses but are not warehouses. Turns out You can't upgrade them to these modern AI data centers because they actually can't maintain the power delivery and the cooling delivery that the new channels.

require. So, you know, as the chips get more and more dense, they get hotter, they need better cooling, they need more reliable power. And In fact, you need different physical architectures. You physically need new buildings as well. And there's a kind of unintended consequence, I guess, of of Moore's Law and Huang's law and whatever else has sort of replaced those laws to that make the chip.

you know, more more power efficient and and kind of more you know dense for flops per cubic centimetre is that They need different buildings.

Yeah, one of the more mind-blowing things in my career is a systems engineer. Systems engineer build combination software and hardware systems. I was definitely more on the software side than the hardware side. Uh if I never own another server in my life, I will be very dissatisfied with that. But there exists data center and the the physical amount of weight of the server racks was over the physical capacity of the floor that the server racks were on.

And you can't simply like run a command on your terminal to make the floor stronger than the architecture designed it the architect designed it to be. And at the point where you are saying, okay We'd like to replace not just the thin metal shell that is on top of the floor, but no, actually we need the structural floor replaced. Then you start thinking, Okay, it might be time to build a new building.

Right. I think that that what you've just described is one of the things that's most misunderstood about the you know, the nature of this particular game. Because for for, you know, the bulk of us, our experience with supercomputers are things that weigh

you know, five ounces, right? It's our it's our cell phones. And they've always got smaller and they've largely got lighter. And that's always been the way they've been sold to us. In you know, and that's true about laptops as well. And we think about our monitors, they get smaller and smaller and smaller on our on our desks. And the i in a way, the the miniaturization, the the packing of more transistors onto every square centimetre or square inch of a of a die

has the reverse effect on physical architecture. And it's a really important notion. And I think it makes it really complex when you start to think about the, you know, we tend to depreciate buildings over a many, many multi decade periods.

you don't depreciate computer hardware over that period of time. And it used to be four years. Now Google and, you know, Amazon or Alphabet and Amazon have moved that to six years. But you have this sort of difference in in the kind of tenor on the the the financing side and the depreciation side. It's much, much more complex than just upgrading your iPhone every three years.

So both in the historical perspective and in the near future perspective, where did we build data centers in the physical universe?

Well, we started, I mean, the very first data centers, allow me to go but go back to that, tended to be in cheap bits of land that were reasonably close. to where customers were. So in in the US, it would be in Reston, Virginia, as as one example, because you had the Bolt Bareneck and Newman, which was one of the architects of the the the sort of NSF net, kind of precursor to the commercial internet.

was based over there and you know I think that was one of the reasons why AOL ended up being there. I in the United Kingdom there was a place called Docklands where which was very, very cheap, light industrial land that was not too far from the city of London where people those banks were among the first users of you know high speed cabling. And so you were really you really thought

cheap for for the land. That dynamic has has of course continued and we know that, you know, the northeast bit of the US is really, really big for for for data centers. But I think there are now considerations around the accessibility to fundamentally to electricity, right? Is there sufficient electricity for the work that we need to get done? Before we get into the electricity point, some fun color, why do we put data centers close to the customers?

Back in the day, it was more about putting them close to employees slash uh skilled technicians. And so if your server breaks down, uh, you might need your uh system administrator to drive out from Chicago to one of the suburbs.

uh to reboot it. Um but to say in the late nineties, network latency was not that huge of a consideration because who in the late nineties was doing anything where you could tell the difference between an eight hundred millisecond ping time and a two second ping time. Right. Fast forward to today, network latency is a primary consideration for where these go. And so there are worldwide networks of data centers at

the largest firms in capitalism and also out of firms selling to the rest of the economy. Well, the largest firms in capitalism also sell to the rest of the economy. Yeah, that we'll talk about that in a moment. uh to optimize essentially for network latency. And then this new power constraint is sort of new. The

data center usage up up until recently in the United States was probably single digit percentage of all the national elec electricity demand. So no small amount, but we get no small amount of value out of computers, so that's fine. But with the densification, with the uh, you know, a notion of having entire buildings full of H one hundreds running on training and inference.

We start to have real constraints about can we physically pump as many electrons through the grid as we need to? Call back to last week's episode. Anyhow. Yeah. But c can I can I also put some history on this as well? Because I y you know, AI is this a technology where

I think is incredibly important. It's also turned out to be very divisive in debates both within the industry and outside the industry. And I can't really remember technology that m you know triggered such a a split in in people's perspective. Oh can I can I get one? Cloud was a in say the two thousand late two thousands to early two thousand tens, depending on where exactly uh you were in the world, there were

huge debates within both the engineering community and in the ones um who are specifically hands on the middle for most of their careers. Will big businesses ever consent to use somebody else's server and somebody else's building for their most private customer data. But okay.

Yes, there used to be this ad by s by an on-prem company, I forget which it was, an advert, and it said, It's not the cloud, it's someone else's computer as a way of sort of sort of, you know, saying something negative and derogatory about about the cloud. I remember working at a Japanese system integrator where we had a multi year debate with the customer base, which were mostly universities in my part. This would have been in the late two thousands.

uh and uh the universities would uh say we really really want to have uh uh all of our uh student information in a location that we control where it will be safe. not in uh uh some building somewhere where we have no visibility. And the true true engineering fact of the matter was the location they controlled was uh literally an unlocked broom closet on campus where anyone could walk in under the influence of a hangover or similar and walk out with all the data. Right. Um

And uh there was a bit of an adoption curve in the Japanese enterprise, but uh the the somewhat stodgier uh members of the Japanese enterprise did get there a few years after the American Enterprise and similar did. But this was a live issue back in, you know. As recently as ten years ago. The the power issue has also been a a longer term issue than generative AI, large language models, you know, in the chat GPT moment.

That the the the the one of the reasons I think this has become so present in people's minds has been uh that there's a lot of skepticism about the value that that AI brings, but before Chat GPT in November 2022, we were all and anyone knowing this was going to be a thing, we'd already started to see places like Singapore, which host a lot of data centers, and Ireland and a number of cities.

start to say we can't provision any more data centers. And those are data centers were just for sort of pre-inter pre-AI uses, like just moving customer data, becoming a digital digital business. And if you look at the capex of a firm like I'm just gonna look at my Microsoft, for example. In 2021, to that year, Microsoft was going to spend$20 billion on CapEx. Only three years earlier, it was at$10 billion.

So it was doubling in three years. And this was well before the open AI deal had manifested itself. It was well before Chat GPT. So one of the things I think we need to also contextualize was that even before the before AI and before this gen AI thing, uh data center demand was growing really, really significantly, partly because of your the point you made about the cloud, right? Companies want customers' data close to customers and they've they're moving all of it off-prem. And

you know we we we are now seeing that accelerate but it's not first and foremost in my view something that that is just about you know just about ai. We've said AI certainly concertined it but it hasn't you know been the sole spark for it. Yep. And if I can give a shout out to Leopold's paper here, um situational awareness. These are the sort of things which were obvious to some people back in the day, but they were not obvious to, you know,

extremely informed planners of electricity demand for a metropol metropolitan areas and nations. They weren't obvious to hedge funds that were following the space, et cetera, et cetera. They were at you know, conversations at dinner parties in San Francisco. that said, Hey, we might need a trillion dollars worth of new power build out in the next couple of years. Trillion with a T, that's kinda wild. Huh, what could we do with that?

A trillion of anything is a lot. But uh let's also just look at US overall electricity demand. Uh I mean electricity or energy in general is

Is wealth? And energy is prosperity and energy is health. There are no countries with good outcomes for their people, broadly defined, that don't have high levels of energy consumption by, you know, per person, whether you're efficient about it or not. The thing about the US is that As of twenty twenty, twenty twenty-one, electricity usage was pretty much at the level of twenty years old. Now

That that is a a really, really important thing to look at. Now, of course, you see energy efficiency, right? The switch to LED light bulbs is tremendous. You see environmental standards emerging and those

ha making people think much, much more about their energy efficiency. It's also good business because electricity costs money and, you know, if you can do the same commercial output with less you know sort of cost of inputs, that's more profit for you. But at the same time, for it to be flat says that there is something about the

the kind of collective agreement by power providers to invest in in capacity. And, you know, this was off the back of Essentially a doubling of electricity consumption between 1975 and about the year 2000.

So to it suddenly going flat. And at that time we also started to see the electrification, certainly of certain types of passenger car transport, right? The Tesla's show arc and and and so on. So I think that when we start to diagnose this, we have to also go a little bit further back. And I'm not a historian of the US.

sort of electricity system in in great detail. But, you know, it it's kind of odd that it flatlines twenty years ago and we don't start to make we don't start to make those investments, uh frankly, either in the the US or in many parts of Western Europe. I'm also myself not exactly an energy economist. I would say one thing which probably contributes uh to it was that the US has undergone some structural economic changes over the course of the last several decades.

and there was a bit of a substitution between uh manufacturing manufacturing output in the United States is as high as it's ever been. Manufacturing employment is slower. People sometimes confuse those two. But we largely shifted from an manufacturing focused economy to a services focused economy. And per dollar value of output, services use various resources, included electricity, less intensely.

But I do agree that there was a failure to anticipate future needs. And also I think in the United States, in many places in Western Europe and um many places near and dear to the hearts of many listeners of this podcast. There's been a real reluctance to build things in the physical world. It's almost like we have lost either the the will, the knowledge, the capacity to do so in some places, in ways that seem absolutely mind boggling. And when we

I lived twenty years in Japan and Japan has many problems, but uh refusal to be able to build buildings is not one of them. Right. And then, you know, look over the ocean over to China. China certainly has not uh forgotten how to you know, do solar deployments, for example.

And I think one of the most crucial things in this sort of moment we find ourselves in is rediscovering the the complex system bum bum that will allow us to actually build the infrastructure that our future economic needs depend on.

Yeah, I I mean I I absolutely agree with that. I mean I mean I think with with China we you know, we see a a willingness, a a desire at uh sort of senior levels of of government, but also a sort of acceptance amongst people that, you know, infrastructure is really, really valuable and it's not just solar manufacturing capacity, it's also solar solar deployment and it deployment of solar at utility scale and on rooftops. It's about the deployment and build out of nuclear power stations.

very, very rapidly. It's about high speed rail. Uh it's also about transmission. One of the things that have of course is really challenging in in in the US, uh, a lot of which is to do with market structure and regulation is building transmission lines. But, you know, China has thirty four

ultra high voltage transmission lines that that, you know, very, very kind of energy efficient and don't leak a lot of have a lot of energy loss over those long, long distances, but totaling tens of thousands of miles, right? And one of the things that that does, especially when you deal with

intermittent resources like solar and wind is it allows you to move the electrons to where they need to be, you know, consumed. You know, if it's sunny in a in in the place and they're not being consumed locally, you can move them to where they're needed.

We had a discussion about this a few weeks ago with uh Travis DeWalter on the uh changing needs of transmission lines in the United States and If I can uh elaborate just uh slightly more on uh what you've said, I think one of the most important facts of uh energy economics has been the extreme outperformance of the learning curve for uh solar power versus cost.

uh over the course of the last uh twenty five years. I remember at the course At the uh time where I graduated university about two thousand four, it did not look likely that the solar was e ever echo going to be economical against coal, for example, absent. huge subsidies for social reasons. And it turns out that not only did we continue down the cost curve, we actually bent that curve. Uh the learning accelerated as there were, you know,

multibillion, tens of billions, hundreds of billions of dollars of investment into solar deployment. And so the frication of the energy grid is one of probably the most central aspects of the coming infrastructure wave. But solar is not the only power generation thing that is going to shake up in the course of the next decade or two. You mentioned China has been doing large scale nuclear build out.

Which I kind of feel a little bit jealous of. But did you want to say a few words about the hottest but um bum new nuclear technology that we might be co locating with data centers in the near future? Well it i in the near future, uh let's talk about China's uh, you know, electrical capacity. You know, they added about three hundred thirty five gigawatt

of capacity in twenty twenty three and it was twenty nine gigawatts in the US. So that's a that's a scale of of where we've where we've got to. I and I think the point about the learning curves with solar is that they really start actually back much, much further back.

Back in seventy three or seventy four, there was a James Bond film called The Man with the Golden Gun, where this sort of British spy has to steal back solar technology. It was so important you said the best secret service agent in the world to get it. And now solar panels are so cheap.

that in in Germany they've actually fallen below the price of fence panel and you're starting to see people build out uh vertical uh you know balcony fences and fences between them them and their neighbors, uh which don't you know, they don't catch as much sunlight, but it's it's cheaper and it, you know, generates some electricity for you. I th I think a lot is you know, we're we're we're we're hoping for

quite a lot from, you know, nuclear and in particular uh small modular and nuclear reactors. So the idea of a small modular reactor is that it's it's all of those those things. It's meant to be small and it's meant to be modular. What's the benefit of that?

The benefit of that is that you tend to see better learning effects when you make more of something. And and so and you get then see better learning effects when those things are are modular rather than build at so built as products rather than as projects. Uh and so one reason why solar has had these amazing learning curves is uh effects is but is that

uh you know, the panels, whether on my rooftop or in a in a solar field in in Texas, are essentially the same. And of course the implementation is slightly different because, you know, one's on a roof, the others are on sort of flattish ground with, you know, mounted in particular ways. Um nuclear reactors have been built. in in many cases a sort of N of one, right? So you have to start from the beginning.

A multi-decade bespoke engineering process where we get very, very little learning between the nth and nth plus one iteration of it. Right. You know, a a absolutely. And and the the the idea between the small modular reactor is that you can you can build these things in a in a modular fashion so you can get learning effects.

Because they are small, you scale out rather than than than sort of by by kind of magnifying the set scale of things. So if you want more capacity, you buy more of them. And frankly, that what we've done in the computer industry, right? If you need a super powerful computer, you don't go off and get a massive mainframe with huge chips. You go and get 10 H100s and stick them together or, you know, a thousand H100s and stick them together.

This also works well against the nature of the demand for data center electricity because for a large scale nuclear power plant that would produce enough electricity for a large uh fraction of a city and you don't have full control in sighting where that plant is. Uh you probably can't uh justify uh putting one directly next to the newest data center that you popped up uh by the freeway. But for a

uh small modular nuclear reactor uh that might fit in a footprint that is about the size of a standard size shipping container. Sure, put one right next to every data center. Put two if you want. Thank you. by physics, by the laws of physics, as opposed to safe by layers and layers of containment um systems and and safety safety systems. So in some sense they are much, much more appealing. I guess the

issue around the SMR that we have to recognize is as a sort of TRL technology re readiness level risk, at least in the West. So there are some SMR units operational in China and Russia. I'm not sure how quickly we're going to sort of import them into the US or into Europe. And there are lots of companies who are building new designs with reference designs and

you know, we are hoping to see them them take off. And, you know, in the sense is if there's a sort of tailwind of demand and and capital that's available, uh

you could potentially scale these out much, much faster than we have scaled out, you know, certainly nuclear plants. But I think there is a recognition also that you know, you know, you need the electricity provisioning today, which is why you know we're starting to see gas generation on some of these bigger data centers, whether it's Meta's or it's uh XAIs at Colossus. Uh so you mentioned uh T L R there. Can you say a few more words for the benefit of the audience?

Oh T T R L Yeah, technology readiness level. So it's a it's a kind of standard uh level of technology readiness that runs from whether something is, you know, really, really at the high risk scoping stage. Through to, you know, we know exactly how to build it, how to price it, um, how to implement it, what its kind of total life cycle uh looks like. And

Small modular reactors are sort of lower down that scale, probably in I'm guessing I'm kind of extemporizing slightly, but certainly in the fours and fives and sixes rather than in the the nines and tens. And that creates a certain degree of risk and uncertainty of what the outcome you know looks like.

There's also of course a regulatory slash political will issue about nuclear reactors where I'm gonna make a terrible pun, but I am a dad. I get to do dad jokes. They were politically radioactive for a number of years in many Western democracies.

And I think we are in a moment the last couple of years where we can partly through a combination of engineering fact, the new technology is simply safer than uh existing technologies, uh, but partly because we had good substitutes uh for base low demand, uh, or acceptable substitutes uh for base low demand, uh liquid natural gas, et cetera, et cetera, uh, for many of the last couple of decades. And things have changed. Uh one is the climate issue, of course. Uh we would strongly prefer to uh

avoiding uh using combustion of uh hydrocarbons. Uh and then the geopolitics of energy usage have changed quite radically over the course of the last ten years or so. to a point where, say, much of Europe where there is say a relatively extreme level of political engagement around environmental issues, it's like, well, you can choose either

fulfilling all of one's preferences with respect to domestic constituencies that are vociferously anti-nuclear, or you can choose to be warm during the winter. And when push comes to shove, many of our truest and dearest friends over there will probably choose to be warm during the winter. Cup cup. Coming back to small modular reactors though, you know, Google has this deal with Kairos energy for I think it's six, maybe it's seven small modular reactors. And

I think twenty thirty is a the delivery time, so we're talking five or six years out. Uh a couple of interesting things about this is that given that it's seven, It is we are going beyond first of a kind. So first of a kind tends to be the really expensive one. And I did write about this a few months ago saying this is kind of goo a Google gift to humanity because the learning curves will be shared, learning experience will be shared by all of us.

And they're the ones who are paying the price to bring these things out at a at a higher cost. But look at the timing. It's, you know, six years and And that's only seven reactors and these are only small reactors. And so so the you know, the electricity requirements across the US economy are uh really, really enormous and and we have to ask

How quickly can this actually fundamentally scale up? But what Google did was they addressed something that this complex system has as a a roadblock, which is That mezzanine financing, which is you know, not venture capital level risk, but nor is it the low risk guaranteed return of asset finance. And that has been an issue with uh a lot of these energy and sort of, you know, electrification hard technologies, which is that

When you're in developing the IP, the intellectual property around which the equity value and the extreme return comes, venture capitalists are willing to take that risk. But venture capitalists are a really small asset class and they they can't fund infrastructure projects. And but once they built the first one, you still have risk. You have a lot of deployment risk, you have the all of the learning effects.

Before it turns into something that is steady and stable, like uh a solar farm or a wind farm, where infrastructure and uh investors come in and they ask for very, very steady state returns with very little chance for extreme upside, which is what the the venture capitalists are after. So you have this middle period, which is kind of nth of a kind financing risk, which has been really, really difficult. Difficult.

to address. It's known as uh amongst sort of people in climate tech as the v one of the valleys of death. There are many valleys of death. And you cut you know it's it's a struggle to cross it.

She was an amazingly uh fortuitous topic for you to br uh bring up because we didn't plan this in advance, but I've actually spent a good portion of my professional cycles the last two years volunteering with a a focused focused research organization that is attempting to popularize next generation geothermal.

And it is exactly that problem. Uh there are VCs who are willing to write checks into the hopefully defensible IP for power generation using next generation geothermal. But Every time you do an experiment in the field, you need to spend twenty to sixty million dollars to ask Halliburton to provide you professional services and what is the thing they're going to do for you? They're going to dig a really deep hole.

And sixty million dollars a whole. Let's go. As you said, that is Challenging in V C land. In a world where uh all the technology risk has been shaken out, there are virtually uh unlimited amounts of capital available to do this. Uh so the oil and gas industry in the United States, for example, uh it's the same people digging functionally the same holes. Uh but uh can you you know?

Can you go to a bank or other sources of capital and uh get the marginal uh gas will financed? Absolutely. You know, there are people who do that every day. Like give us your number give us your engineers' numbers, I'll put it in my spreadsheet. Do green, we go, you'll have your wired mark. And so a lot of the last two years for me has been attempting to to cheat the valley of death uh on behalf of this uh NGO.

Can can I ask about your experience there? That's super interesting to me. Um what w what what is in what is next generation geothermal rather than geothermal? Sure. So the brief version is that um the geothermal that most people think of is places where

heat energy from the earth bubbles up so close to the surface that you can physically perceive it in some cases. So hot springs, geysers, et cetera, et cetera. When you think of the places in the world that are the largest geothermal energy producers currently, you think of places like Iceland.

and they have been blessed by nature, with the particular subsurface for formations give them abundant access to geothermal. Most places in the world are not similarly blessed by nature. However, due to the particularly the fracking boom in the United States, we've gotten much, much better at drilling to depths that were not economically drillable before. And if geothermal can only tap energy that is available at the surface of the earth, you have to be blessed by nature to do it.

If on the other hand you can go down, say, I don't know, six to ten kilometers, then essentially everywhere is blessed by nature. And to ninety plus percent of the continental United States is a number that I've I've heard thrown around. And thus the There is still technology risk. Digging the hole, uh, sure, but uh uh you need to figure out uh what the generation station is that you put on the top of the well and uh uh what the you know curve looks like for

uh heat in the immediate vicinity of the well that you have fracked uh is a limited resource so y you tend to get a trailing off of the generation over a sub timescale. And so we're really looking for those next like one, two, twenty, forty, hundred wells to see what do those curves look like. And then it's just a numbers game. Like in in one

In one version of physical reality, this is not cost competitive with other forms of heat or electricity generation. And in another version of physical reality, we have free, clean, abundant energy available in large portions of the world. And so ask me in ten years which reality we're living in. I will have a very confident answer in ten years, but I don't currently. And w what wh and what's the price that you think is cost competitive?

Ooh, great question. Not the a number that I had cached off the top of my head because I didn't know we were going to be talking it. Interestingly, just to say a few more more words on the why fracking matters. Fundamentally fracking the Oil and gas people love to call it subsurface engineering. But we had a prior episode about tracking. I will link it in the show notes. But

uh drill the hole, pump a liquid a a working liquid down the hole, and use that to break rock around the vicinity of the hole. And then you there's a few different technologies to do this. Can't go into the entire thing here, sorry folks. But in one version of the s system you pump water or some other working liquid which filters into the cracks that you've made. Uh and because the surface area uh in those cracks is

The the cracks look fractal in nature. Um the surface area is absurd uh relative to the diameter of the hole. And so uh you can uh pull heat from the surrounding uh surrounding rocks for a very long time, uh hopefully uh until the the rate of heat moving into the vicinity of the rocks that your water is touching is no longer sufficient to sustain the the rate of heat that you're extracting from the top of the hole.

Anyhow, that's long story short for people who want to learn more about this field. I'll drop some links in the show notes. I I mean geothermal is I think a really interesting and potential technology and it speaks to the fact that the energy system feels like it's going to m continue to be heterogeneous. You know, what happened with with computing is that we we tend to have these sort of winner-take alls, although it's a little bit more

um heterogeneous than than it looks at at the surface because kind of an ARM chip is different to an Intel chip is different to a GPU. But but I do I do see a kind of world of different energy technology. Definitely. And as we heard in the episode with uh Travis DeWalter a few weeks ago, the heterogeneity of power generation makes the grid more stable because there are uh different physical aspects to different power generation technologies.

nuclear, geothermal, etc. are uh stable base load power and then solar seems to be scalable to the moon. Oh man, dad joke number two. But solar is of course only available during particular hours of the day. Well I but I think I think it's worth asking a question about how far you can actually go with with solar. Because I suspect that it's further than most models take it. And and just hear my my case for

Because solar is highly modular, the market expands significantly. And that means that that homes and small businesses as well as well as large scale utility providers can can get into solar. And we've seen this happen in large parts of the US with community solar, but of course

Rooftop solar in China is absolutely enormous and and you know, Pakistan has a great example where businesses got sick and tired of the grid failing and so just went off and bought loads of solar panels. So at least ten hours a day they could run their business. uh the cost curves are really, really in their in their favor. And even though we've had fifty years of learning, uh, it's not clear that solar price decline panel price declines

are going to stop, you know, sort of any time in the next five to ten years. And you know, there are new technologies bubbling in the wings. So even though it's it's far from perfect, the total system cost. it is something that's quite dynamic. And the other aspects of the total system cost will be whatever happens with batteries and other forms of storage. And batteries are really early in their learning um effects. I mean the cost of

kind of prestige batteries, right? The lithium iron battery have declined from twelve hundred dollars per kilowatt hour to about forty dollars per kilowatt hour since twenty eleven, two thousand and twelve to the beginning of twenty twenty five. And there's probably still some some room to run and there are cheaper technologies with different physical characteristics like sodium iron and even iron air batteries that are that are in the wings. And then you have to think about how do you manage

you know, distribution,'cause that becomes an important part,'cause you as you say, right, it it's sunny in one place and it's it's not that sunny somewhere else. And there are a couple of really interesting projects that are going on at the moment. One is built in uh Australia to to take power all the way up to Singapore, another in Morocco to take power to the United Kingdom with these sub-sea high voltage direct current cables that are being built out.

Mm-hmm. There's also some other interesting second, third, maybe even seventh order effects for uh some of these technologies where uh Uh Casey Handemer, previous uh uh podcast guest, his company Terraform Industries, I believe. Yeah, is attempting to do uh direct capture of

carbon dioxide to turn into hydrocarbons using uh quote alien science, end quote, which is of course heavily energy intense because you can't cheat the laws of thermodynamics. Right. But given that you have huge amounts of solar generation in one part of a nation.

If hypothetically you can do local generation of hydrocarbons, then you can ship extremely energy dense hydrocarbons to wherever you want to put them in the world, combust them there, and then just, you know, the carbon goes back into the atmosphere, you suck it right back down and turn into hard hydrocarbons again. So so Casey is amazing but and let's talk about exactly let's just go through that cycle again.

If we take carbon dioxide carbon out of the atmosphere, And we push it over the second law therm thermodynamics hump and we turn it into methane. And we combust that methane, we're net zero, right? We've not put any additional CO two into the atmosphere. And the energy density of of gasoline or

methane or but kerosene is absolutely staggering and we have a whole load of systems that already know how to use that. And I think that's a that's a great example of why why it may be that solar could end up being and I think it will be the dominant supply of you know, first party energy and electrons into you know into the system. And there are other things that you can start to do with the system like like demand response. So that's where you kind of affect

and and and create incentives for people's behavior to to change. There is a um there are lots of air conditioners and heat systems, heating and cooling systems in homes in Texas. whose behaviour is actually managed by the energy provider to to respond to kind of m minute by minute and hour by hour changes in electricity demand and pricing. And and that can also extend to How we might shift.

workloads for compute around data centers at different times of day. Not everything needs to be right on the front end. You know, your Akamai servers serving up live video need to be close to the customer, but not everything needs to be right close to the customer the whole time. I'm not the world expert on this, but we seem to be sort of in land grab mode at the moment with respect to training new AI models. But one can imagine a future in which for those of you

Aren't familiar. These wonderful AI models that we are using these days have typically two phases. There is a training phase and then an inference phase. The training phase is the months of hard work that uh OpenAI or uh Anthropic or another lab put into putting out one of their new numbered releases of a model. And then the inference phase is what happens in the few seconds between when you ask a question and when an answer comes back to you.

My guess, finger to the wind, without demous of inside knowledge, is that the chips that have been doing training have been running hot. Essentially 24 hours a day, seven days a week for the last while. However, one can imagine future iterations of this technology where there is actually some sort of cost-benefits curve associated with it. And so at times where times and places where electricity is uh particularly expensive

Just stop training for a while and continue doing the inference on an on demand basis. Or again, currently, you know, we are the dominant public deployment of AI is with a user sitting at the keyboard type typing into a computer. But that will not be the case for forever. It might make sense to provision more intelligence when electricity is cheap.

to do these sort of quote unquote offline calculations on behalf of industry than to simply continue running inference at the same levels everywhere in the 24 hour class. Or we could end up in a world where cognition is just so stupidly valuable that why would you ever turn it off just to save on electricity bills? Well and what I love about this is these are these are a few scenarios and let's throw out some other scenarios. One is the you know algorithmic optimizations.

of the functional level of intelligence that we want at a given at a given time. And, you know, we think about Moore's Law being this remarkable thing, right? 60% cost declines on on price performance every year for for decades and decades. But software optimizations can be orders of magnitude of of improvement instantly, like a phosphoria transform or, you know, doing something with a bloom filter rather than, you know, m mechanically walking through That brings me back.

That takes you yeah, sorry, I'm just an old dude. I can't I can't help it. Uh And and so so there's there's one thing that we have to think about, which is which is like software optimizations. There are also, you know, novel, novel architectures. So I invested in probably the world's first reversible computing semiconductor company. So what reversible computing does is

It has a different way of uh processing information. And the reason why NVIDIA chips give off so much information is that they're irreversible. So they in they increase entropy and you you destroy information and that is appears as heat loss. If you don't do that and you have reversible processes, you actually can be a couple of orders of magnitude in you know, in theory.

More energy efficient. And it comes at a kind of certain cost of sort of complexity for building the gates up that are required in a chip. But, you know, you you could see, you know, ten, twenty, fifty X improvement in in energy efficiency, which which is faster than the wonderful improvements of energy and efficiency we've seen over the last thirty years uh in computing since we need started to move towards laptops and then and then cellular cellular devices.

But none of that actually that is all extremely helpful and the market will surely move to more energy uh efficient systems because electricity will always cost something. And but we still have the spectrum of Jevon's paradox, which is your I think your your observation, which is we never know um how useful cognition will will be. And and as we said earlier in our discussion, I think we're barely scratching the surface. You you said we're at the five computers stage of of LLMs. And and so so all

You know, n net net, I'm sure all of this stuff is going to become orders of magnitude more efficient, right? Sort of digital IQ points per watt will get far, far better than it is today. And boy, are we going to demand a lot of it. Points on the scale.

IQ is a useful abstraction to like bat around casually about it. I think we'll probably have more powerful abstractions in the next while to describe something that it like You know, this LLM is extremely limited with respect to its capabilities, but doesn't have to be a super genius to uh uh successfully route a package from point A to point B. You know, we will have others that are assisting people in doing cutting edge scientific research.

And every time there's a new model release every six years, the laboratory assistants get uh uh sorry, uh six years. Oh yeah, exactly. If uh that was a verbal disfluency folks, rather than a prediction of uh immediate cratering of the learning curve, the uh research assistants will be getting shockingly more capable uh over uh very compressed times time spans. Well I think what you've what you've described there though, that um ecology

is is so important for everyone for us to to understand. You don't always need, you know, a a a PhD in negotiation to help you decide how much to pay for the pair of socks. in Walmart that you're about to buy. So that's the price you just pay for it and you're and you're done. And I I think that will also be true for the way in which we embed intelligence in our in our system.

But we've only really started to to sketch that out. I mean, if you think about humanoid robots that, you know, they're down to a few thousand dollars in from unitary in China, how much intelligence or whatever proxies for common sense do we do we want, I would say that it's got to be at least at a GPT four level. I mean, you wouldn't want a a robot like that understanding the world as well as GPT two did, which was random sentence fragments and then going off anyway.

And you'd certainly want more controllability. I'm I'm not saying you could just, you know, lump one of these GPT four class open source models into a unitary m robot and say, go and look after my kids. I I'm saying that that's kind of surely the surely the baseline. But But as you say, you don't necessarily need it to, you know, make an Einstein level discovery while it's loading your dish.

Yeah, and these will be subcomponents of larger engineer systems. I think people extensibly underrate that. We've had robots for a very long time. There's many folks that science fiction aficionados that, you know, can quote Asimov's three laws of robotics.

And talk about a day in which a robot could kill someone for the first time. And the worked in Central Japan system engineer in me says, Oh, that day actually happened in the nineteen seventies in an industrial accident. Uh, but you know, we can do things in factories like saying, Okay.

It is possible that a human factor system might not be sufficiently intelligent to walk in an uncontrolled environment right now. So, all right, let's cheat all those assumptions. One, it won't be human factor, it will be just a grabbing arm. Two, we're going to put it

in the middle of a factory where we control everything around it and put in some factories yellow hazard tape describing the physical like the physical maximum extent that the ar that the arm can move. And so you can guarantee the robot system the invariant

There will never be a human skull inside this physical hazard tape during your operation and therefore you cannot crush anyone like an egg. And uh anyhow, and then There's sort of a we will not be in a stable equilibrium as the software gets smarter, as the LLMs get smarter, as they unlock additional fun toys for us on the software side, we'll find new ways to build hardware to take advantage of those.

new capabilities to build out larger engineered systems to put the smaller hardware systems in such that they they can produce even more value at scale. So wild time to be alive. It's a wild time to be alive. And what we have to do is fix our our mental models around how these technologies emerge. And this intersection between electricity and computing or AI is is a really it's bringing together two very different worlds.

So what's happened in the computer industry since the you know the nineteen sixties and the nineteen seventies? is that we have tried to bring the computing closer and closer to the end user, from the mainframe to the mini time series, then, you know, personal computers and then and then and, you know, even smaller smart devices. I'm wearing a smart ring uh on my hand right now.

And and we've also then moved into this of a hybrid environment where there are certain tasks that I get I do locally, like my sleep tracking, and and there are other tasks that I push out. where there's lots and lots of computes and lots of storage and I just need to get them done um in the cloud. The energy system was never like that. The energy system was all mainframe.

There's there's Three Mile Island, there's you know the Hoover Dam, there's uh you know huge coal station, some power station, and then you pipe that power over and we're all consumers.

And so all of our mental models around this are based around those those types of ideas. And I think a really good analogy for how the system changes Is what happened with telecoms moving from that world, which is what the uh old telephony, pre-internet telephony system looked like, to what the new internet looks like.

We went through a period of time when the new internet was sort of highly decentralized point to point, but all the servers were back in, you know, Reston Virginia or Telehouse in Dopplands or a Palo Alto Internet Exchange. And then we've started to hybridize this. So quite often when you access a resource that is a a resource in another country.

That's actually served quite local to you, perhaps only twenty miles away from a a front end caching system like a like an Akamai Edge server or a Cloudfare server. That is a a tiered topology. And and I don't see any reason why that's not what

AI infrastructure ends up ends up looking like. And but the bit that's a real sort of mind for people is that's what the energy system might might end up looking like that supports this, with localized generation and vehicle to grid and community batteries part of the mix.

Yeah, to you know, analogize directly to the the mainframe. Love love that uh analogy. Uh, you know, the large scale nuclear plant is a mainframe. We had uh uh a few decades of uh usage of like the local server room with particularly for industrial uses, uh co-located perhaps behind the meter small, typically combustion based uh electricity generation.

And then we've had rooftop solar the last couple of years. I'm somewhat bearish on rooftop solar, but that's neither here nor there. At least in I think there are places in the world where it makes a lot of sense where, for example, grids are less reliable. I think it uh Like California. I'm teasing you. Yeah, yeah.

California thought we really, really want rooftop generation. That seems extremely incentive compatible. It will be green, et cetera, et cetera, but it won't bespoil our beautiful landscape. And Texas said, the desert is free. We are going all in on utility scale generation. And that experiment was run. The results are in. Texas won by a lot. And so yeah, anyhow. So They're you know. Yeah.

chicken in every pot, there might be a battery in every garage in the very near future. Certainly Elon Musk would love to make wave a magic wand and make that happen. And you know, community scale batteries operating at material scale might very much be a thing in the next couple of years. It'll be wild

So I feel like we could continue uh having this discussion for a very long time, but I do want to be respectful of your time and the audience's attention as well. Uh, where can people find you on the internet it seems?

The best way to find me is at ExponentialView, which is exponentialview.com, or just put it into your search engine of choice and it'll it'll show up and you can sign up to my newsletter there and then all of my other links will sort of leaf off that. Awesome. Thank you very much for coming on today. And for the audience, thank you very much for joining us again. We'll be back next week.

Thank you. Thanks for tuning in to this week's episode of Complex Systems. If you'd have comments, drop me an email or hit me up at Patty Eleven on Twitter. Ratings and reviews are the lifeblood of new podcasts for SEO reasons. And also because they let me know what you like. Complex Systems is produced by Tripentine, podcast network behind econ 102, Riff with Bern Hobart. B C and more shows for experts by experts.