Powering the UK's fastest supercomputer: Isambard AI

MICHAEL BIRD Good morning, Sam. SAM JARRELL Hello, Michael. MICHAEL BIRD Have you ever thought about the quantity of power needed to run a supercomputer? SAM JARRELL  I don't dwell on it, but I do imagine it is probably like tons and tons of gigawatts. Maybe, like, a terawatt or something? You're a techie person. Do you know how much your home computer uses? MICHAEL BIRD  Oh, um, I think it uses about between 100 - 200 hundred watts, like if it's really under load, maybe 300 watts, but like not a lot of energy. But Sam, today we are still in Bristol at the UK's fastest supercomputer to find out how you run a system of this magnitude in the most efficient way possible. I’m Michael Bird SAM JARRELL I'm Sam Jarrell And welcome to Technology Now from HPE. MICHAEL BIRD Alright, well  welcome back to the second part of our episode pair exploring Isambard AI, the UK's fastest supercomputer. And don't worry if you missed the first episode, we will link to it in the show notes, but you can also watch both episodes independently. SAM JARRELL  Exactly. And because these are both video episodes, make sure to check us out wherever you watch your podcasts. As well as wherever you listen to them. Now, Michael, you're still in Bristol for this episode, and I believe you mentioned that we are talking about efficiency today. MICHAEL BIRD  Yes, well, supercomputers use a lot of power and they can get out quite a lot of heat, so finding ways to reduce the power and to use the energy more efficiently can be just as much of a financial issue as a sustainability one. SAM JARRELL Oh for sure – and we’ve spoken about this before on the podcast as well as doing episodes on things like heat reclamation from direct liquid cooling. MICHAEL BIRD  Yeah. And they use direct liquid cooling as part of their systems here at is Isambard AI, but there's far more to running a facility like this than efficiency and cooling alone. So to find out more. I met up with Dr. Emma Rose, Centre Manager for the Bristol Centre for Supercomputing. The first thing I wanted to do was to get to grips with the scale of the power requirements for Isambard AI. MICHAEL BIRD So um Emma, we're standing in front of a, panel with some information on it. Can you just tell me what we're looking at here? EMMA ROSE sure. We're standing in front of the Building Management system, which is effectively a panel that tells us how the modular data center's performing and all the cooling infrastructure around it. So you can see at the moment, um, all of the, the alerts on there are showing green, so everything's quite happy and healthy at the moment, but it's a really quick way for somebody inside here to see if there's anything wrong, because those green panels will turn to, to red. red. also on here we can see how much power the system is currently using. At the moment it's about 2.4 megawatts of electricity is being used by Isab ai and we can also see the PUE or power usage efficiency, uh, which shows us the overheads that the cooling infrastructure uses of, of electricity above, over and above, um, is about AI itself. particularly low. Uh, so it's 1.07, which means there's only 7% overhead, um, for the cooling, um, that is used to call the GPUs. So MICHAEL BIRD So very, very efficient, EMMA ROSE very, very, very efficient. In the past, the PUE for an air code system would be closer to two, so it's incredibly efficient. MICHAEL BIRD okay. And so, I mean, 2.4 megawatts is a lot of energy. I mean, I can't quite quantify that, but that's many, many houses. EMMA ROSE Absolutely. MICHAEL BIRD Where does the energy come from? EMMA ROSE we pay for renewable energy. So, um, it's all battery, wind, solar, hydro generated, um, electricity from the uk MICHAEL BIRD presumably that's slightly more expensive than buying non-renewable EMMA ROSE It is slightly more expensive, but it's important to us that we are using sustainably sourced, um, electricity because we we're aware that we do use a lot and we want it to be as sustainable as possible, MICHAEL BIRD And so how else are you managing power here? Like what happens if there's a power cut? I mean, does everything just, all the lights shut off and everyone goes home? EMMA ROSE Well, well, well, yes and no. So the GPUs in the room we're in now and the event of a power cut will will shut down, you know, 2.4 megawatts as it's using at the moment. There's no generators that would keep that sustained for long enough. Um, we do have some critical systems on the site that we do need to keep going. The cooling, for instance, we would want to keep going in the event of a power cap. And we've also got some, some air cooled equipment that ran us storage and our logins and management nodes. We need that to keep going because if that shuts down suddenly, it does take a long time to bring it back up again. Yeah. So we have a generator on the site that will keep it going for a few hours in the event of a power cup. MICHAEL BIRD So, um, we touched on the previous episode about all this, this water cooling. how are you keeping this whole place? EMMA ROSE So within the modular data center itself, there's, there's big air conditioning units. Yeah. So the, the air around us is reasonably cool, but the GPUs, it's a closed loop system, um, of a, a glycol water mixture that runs over the GPUs and extracts the heat, takes 'em out to the coolers outside. And then, um, once, once it's been called by the coolers, it's returned back into the system again. MICHAEL BIRD Okay. And it's. And it's, um, hopefully it's still there, but these, these red and uh, openness store, these, these sort of red and blue pipes? EMMA ROSE Yeah, absolutely. So the blue pipes are the, the cool liquid that's just come off the coolers and when they, it runs through the, the GPUs extracts the heat and then it comes through on the red pipes MICHAEL BIRD and then up and out of this room ly. Okay. and I guess at this scale of computing, um. Direct liquid cooling is really our, the only option here. EMMA ROSE Absolutely, definitely. the only alternative is, is air cooled. Um, and that's got some huge disadvantages in terms of sustainability. Um, it would also mean that the, the system itself would, would take up about eight times the space that we are using at the moment. So the system is very densely packed in a relatively small area because it's direct, liquid cooled. MICHAEL BIRD So we're gonna, we're gonna head towards some of the plant stuff outside, hopefully some of the water cooling bits and bobs. Um, and is it just water in these pipes that's, that's coming through? EMMA ROSE It's a water glycol mix. So the glycol just provides an extra bit of coolant, um, to, to absorb more of the heat. It also provides antifreeze properties. So if it got particularly cold, then the, the water's not gonna freeze as it's out here. MICHAEL BIRD Okay. And, and we're in the UK so it gets coldish and hot ish. Not, not, not super, super freezing. Not, not super, super hot. Um, but, okay. So we're now, uh, coming across what looks like some very big, some very, very big panels and some very, very big fans. Can you describe what we are stood in between? EMMA ROSE So we are standing next to one of our hybrid coolers. They're. Effectively very, very big radiators. Yeah. So the coolant goes through some radiator pipes, and above them are three massive fans that pull in the air from above and that cools the liquid as it runs through the radiator pipes. MICHAEL BIRD Yeah, these look like massive car radiators and, and sort of funnels from ships at the top. So those are the fans, those are the three fans up at the top, and these are the big radiators. And so all of the heat from the GPUs comes through here, comes through these radiators, and this is what, this is what cools the water. EMMA ROSE That's right. Yeah. So on a day like today, when it's a relatively, yeah, there's a relatively cool wind. It's, it's not a particularly hot day. Then these, um, coolers, they run in what we call dry mode. So we are not losing any of the, any water here. It's, yeah, it's a closed loop. The water, the, the cooler returns back to the system. If the temperature, the outside temperature gets a bit hotter. Then these radiators will start dripping water down the outside of them. That just provides a bit of extra cooling through evaporative cooling. MICHAEL BIRD Okay. And so, I mean, how much energy are these fans, these radiators using? EMMA ROSE So there's two coolers out here to, to Cool the two sets of, um, of, uh, racks that we've got inside. Each one of them uses about 16 kilowatts of electricity if it's running at full capacity. A bear in mind that each site of Isenberg AI will use about, uh, uses up to two megawatts, so 16 kilowatts to, to cool, two megawatts worth of heat MICHAEL BIRD that's quite efficient. That's not a lot of energy. And so, so you talked about, um, the water dripping down. I mean, it's a closed loop system except for when it gets super, super hot. How much water is this system using in a typical year? EMMA ROSE because the UK doesn't get. That hot. We estimate that, um, we would lose and drain away about 21 houses, their typical UK homes worth of water in a year. So if you think an average sort of cul-de-sac in a, in a town with 21 houses in it, the amount of water that those 21 houses would, would use is roughly what we think we'd use in a year. MICHAEL BIRD So not loads of water. EMMA ROSE No, not at all. MICHAEL BIRD at all. It's very much a closed loop system. Okay. Um, so what, why did you go for a closed loop system? I mean, what, what other design options are there? EMMA ROSE Well, really at the moment, the only other design option is for an air cooled system. Okay. So one of the other big advantage about, about using a closed loop system is there's potential to reuse the heat. So we can harness the, the heat that's generated of is bad AI and potentially heat surrounding buildings with that water, with that heat that's trapped in the water. If we use an air cooled system, then that would just get lost into the atmosphere. MICHAEL BIRD Okay. Okay. So there's a potential for that, for that to be used and presumably it's, it's way more, I mean, it's way more efficient EMMA ROSE Absolutely. Way more efficient. I mean, we also have a, a chiller, um, and that's the cool, I talked earlier about the, the critical systems. They, they are, um, air called and the chiller is to call the environment around those. Um, but that chiller alone uses more electricity than the two coolers that we've got, MICHAEL BIRD Wow. Okay. EMMA ROSE That gives you an idea just of the efficiency of those coolers MICHAEL BIRD So, so superefficient, okay. I, I do want to talk about sort of heat recovery. Can you go into it more detail? Like how would that, how does that work? How would that work? EMMA ROSE So the, the liquid that comes off is in bad AI is roughly 40 degrees when it comes off the system. Um, heat recovery, we've got the pipe work on site to be able to do this. Uh, you would need a heat pump to, to warm that up to about 70 degrees. And then if you have the infrastructure on the other side and a, in a nearby building, then the, the heated up water or liquid can then go around and effectively warm up the buildings around it. MICHAEL BIRD Wow, okay. So you can sort of become another, a bit of a utility to the buildings around. EMMA ROSE It's done a lot in, in other countries. And Scandinavia, for instance, is actually mandated by the government that their data centers have to reuse their heat. So there's a lot of, um, there's a lot of examples in other parts of the world where it's done a lot. MICHAEL BIRD so do you think, I mean, do you think that sort of heat reuse would become, I dunno, in the uk, do you think the UK government would just mandate that for data centers like this, do you think? EMMA ROSE I think they're probably going that way. Um, for anything that generates heat that can be reused. I think that's certainly moving in that direction. The problem is at the moment, there's just not the infrastructure in, in nearby buildings. So some of that might involve retrofitting buildings. Some of it will involve putting that. Infrastructure in place when, when new buildings are being built. MICHAEL BIRD amazing. Emma, thank you so much for your time. It's been a real pleasure having a look around the Isenberg AI supercomputer. Uh, going around all the chillers been fascinating, so thank you so much for your time. EMMA ROSE You're very welcome. MICHAEL BIRD SAM JARRELL You know, Michael, I'm very surprised that you were able to, to do this right in front of a supercomputer, and it's not blaringly loud. MICHAEL BIRD As I said in the interview I'm quite used to being in, uh, like data centers or server rooms and you are literally, shouting is so loud. 'cause the screaming fans, I mean, they like, they whine, but I mean, I'm stood next to a supercomputer that is running and lemme just put my microphone to it. Very underwhelming. It's, it's basically silent. I mean, there is, there's noise in here, there are other systems and fans and stuff running. But I mean, generally it's pretty quiet here you can have a conversation. So, um, yeah, it's amazing. It's, it's quite, it's quite incredible. SAM JARRELL It is. And you know, I was kind of surprised earlier in our conversation, I think I tossed out a figure like is there a terawatt of power potentially used? But to hear that there's only two megawatts of power, it's still a lot, but it is definitely less than I actually expected for a supercomputer. MICHAEL BIRD I was really surprised with how low that number was. I, I definitely thought it'd be like terawatts, some gigawatts for the 11th, fastest supercomputer in the world. Um, and to put that in perspective, I've got a heap up and an electric car, so I used quite a lot of energy for a home in the winter. I think I used about one or two megawatts, like per month. So it's not loads of energy, it's not, not a lot of energy, but it's not loads and loads of energy. So it is quite efficient. SAM JARRELL definitely not what you think of when you think of like supercomputers being very, very high energy consumption. and. like the sustainability angle. I think that that's a, a wonderful story. I also think, it was interesting to hear about, um, the direct liquid cooling piece of this. I believe that you all said that it's a closed loop. MICHAEL BIRD Yeah, it's a closed loop. So, um, they do use some water, as, as Emma said, in in the summer when it's really, really hot, they do then sometimes use some water to help, help with the cooling. You know, additional evaporation help with the cooling, but it's not loads of water. I think it was the equivalent of, uh, like 21 homes, like how much 21 homes would use in a year. Which, again, not an insignificant amount of water, but nowhere near as much as I was expecting. SAM JARRELL Not at all, and if you think about it, like, there's a lot of things that use definitely more water than that. I imagine just like crops and like to maintain like livestock somewhere, you probably use a lot more water. So like comparatively, it's actually much less than. Than I think the average person thinks about when they think of like supercomputing data centers, all of that. I hope that we hear about other supercomputers and data centers, trying to use a similar model. Now you guys also did mention something about heat reclamation being in the works as well, which also sort of plays into that whole sustainability loop. MICHAEL BIRD If they do get to do it, if they do end up doing it, it's such a cool story. It'll be such a cool thing to do, to use essentially energy that's just at the moment, not doing anything. If you can make use of that, do something intelligent, clever, useful with it, um, that's great. That's really, really great. And again, another, a model that could be used for other data centers. What did you think of going outside? What did you think of the, uh, all of the radiators outside? They were quite something, weren't they? SAM JARRELL Yeah, I feel like the entire like outside infrastructure area was huge, just a lot bigger than I expected. I suppose you do need like a strong cooling infrastructure for a supercomputer. From like a size perspective, it seemed like it might be, similar to like the data center itself. MICHAEL BIRD Yeah, it felt, it felt a similar size. Like, it was quite impressive. Cooling these big fans just running air, running air through the, uh, through the radiators was really interesting. So Sam, uh, obviously one of the big things that sets Isambard AI apart from other supercomputers is the modular nature of it. And so as a final thought, I wanted to know if Emma thought they might become trendsetters going forward. Will Isambard AI become a model that other high-performance computers and AI data centers could be based on. EMMA ROSE I think so. Absolutely. I mean, the modular data center approach is, you know, as we've, you've heard before, is fast. It's efficient. Um, it's, it's much cheaper than a bricks and mortar building. And heat reuse the ability to reuse the heats that are generated from such huge infrastructure such as this. Um, and yeah, use it to, to effectively, as you said before, as another utility generator is absolutely the way to go in terms of energy efficiency as SAM JARRELL Okay that brings us to the end of Technology Now for this week. Thank you to our guest, Dr. Emma Rose And of course, to our listeners. Thank you so much for joining us. MICHAEL BIRD And if you've enjoyed this episode, please do let us know. Rate and review us wherever you listen to episodes. And if you wanna get in contact with us, send us an email to technologynow@hpe.com subject line, mod Pod. And don't forget to subscribe so you can listen first every week Technology Now is hosted by Sam Jarrell and myself, Michael Bird This episode was produced by Harry Lampert and Izzie Clarke with production support from Alysha Kempson-Taylor, Beckie Bird, Alissa Mitry, and Janessa Ayache. Our theme music was composed by Greg Hooper. SAM JARRELL Our social editorial team is Rebecca Wissinger, Judy-Anne Goldman and Jacqueline Green and our social media designers are Alejandra Garcia, and Ambar Maldonado. MICHAEL BIRD Technology Now is a Fresh Air Production for Hewlett Packard Enterprise. (and) we’ll see you next week. Cheers! SAM JARRELL Bye y’all