¶ Cloud Adoption and Networking Professionals' Empowerment
Welcome to the Cables to Clouds podcast . Cloud adoption is on the rise and many network infrastructure professionals are being asked to adopt a hybrid approach as individuals who have already started this journey . We would like to empower those professionals with the tools and the knowledge to bridge the gap .
Welcome back to the Cables to Clouds podcast . Everybody , my name is Tim . I'm your host this week and joining me , as always , are my lovely compatriots Alex Perkins and Chris Miles . How are you guys doing today , chris ? What's going on in Australia ?
Not much , man . I like that intro you did there . Welcome back to the Cables . That was pretty nice .
I've been practicing all day .
Yeah , man , not much . I had a very busy work week , but I'm in lovely Sydney . You can probably hear the birds in the background . They're enjoying this lovely Saturday that we're having . Yeah , man , I wish I had more to report .
It's been pretty boring , boring a few days , but I think we , tim and I , are working on something very special that I think , by the time this releases , we can probably talk about it . So should we reveal that , tim ? Should we talk about it ?
Well , yeah , I mean it's not the purpose of the episode , but I think by the time this comes out , we probably will have let the cat out of the bag anyway . So yeah , yeah , absolutely .
So Tim actually got approached and reached out to myself and another previous guest of the show , steve McNutt , about writing the new certification guide for the AWS Advanced Networking Specialty exam from PACT .
So , yeah , you'll probably be seeing more on that , but yeah , we've kicked things off , we're writing and we'll hopefully have that out to you in a timely manner . But yeah , I'm really excited to do that . Tim , thanks for pulling me in .
Yeah , no worries , man , I'm glad to have you . What about you , alex ? What's going on this week ?
I also have had a pretty boring week . I don't know what it is about this week , but almost everyone is out of the office for some reason . So I've been just catching up on a lot more paperwork stuff , building out various documents , when I can't be bothered by all the people that are gone . So it's been kind of nice , but yeah , that's it .
Yeah , all right . Well , I guess that's all . So all the pressure's on me to talk about something interesting that happened this week no , that's fine , it's fine , I got this . So my youngest daughter did a Girl Scout STEM camp this week . So she's science , technology , engineering , math . I was really really happy to see that . We went over there yesterday .
Actually they were launching bottle rockets , so like different rockets that they created . It was pretty cool . Actually it was a fun little thing . They had a bunch of girls down from the . I don't remember all the Girl Scout ranks or whatever .
I'm sorry I don't , but like you know , kids are like six or six years old , all the way up to , I think , like 14 or 13 or 14 years old . It was really cool to see , so that was fun . The other thing is , of course , we're finishing up .
We got I got a little bit more time left before we pack up and go to Japan , so I just bought new luggage for Japan .
My condolences , I know right .
Yeah , no , it's not cheap anymore , that's for sure , but the old one I had I had taken all over the world with me and it was beat the hell . So it was time .
It's funny , you say bottle rockets , the immediate . I think of the , you know the fireworks that uh the fireworks yeah . It reminds me of dude . Like we used to . Like we were reckless with those things , dude , I remember , um , we , like you , remember the old lightsaber toys where , like you could unscrew the top and that's where the batteries went in .
We would take those , we would take the handles , we'd unscrew the top , take the batteries out , and it was basically just like a hollow tube and you put a bottle rocket and we just shoot them at each other . This is the most dangerous thing I've ever done . But um yeah .
Yeah , the girls got to do that . Yeah , that's good . Um , okay , yeah , anyway . So , uh , we have a new uh guest to the show . We're actually really excited to bring this guest on . I'm really glad we could connect and get them . This is Peter Jones .
Peter is a distinguished engineer in Cisco hardware , network networking hardware Sorry , if I could say that right and also the chair of the ethernet alliance . And the reason we connected with Peter actually is kind of funny story .
We had released our episode about the economies of scale stuff and in there , you know , we'd been talking about kind of the race to capture silicon , you know , but among the CSPs and all the hardware vendors and stuff , and Peter had some really good insights for that .
And then , you know , after we talked to him for a little while , he had way more insights than that and so we were like dude , you got to stop talking , we need to get you on the show so you can share that with everybody . So , peter , thanks , thanks for joining us on this one .
So you're welcome .
Um , I do want to mention you guys may not have had much interesting to talk about , but I'm just recovering from two weeks in Europe where the first week was the first week was attending a ethernet standards meeting in Berlin , and I can tell you that Berlin in July is lovely and directly outside the hotel was a nice big garden , oh my goodness .
And then we , my wife and I , we basically took the train from Berlin to Brussels , brussels to Paris and Paris to Campere in Normandy and we spent five days on a bike trip riding around Normandy , drink , eating crepes and drinking cider .
Oh my .
God oh man Wow . Dude . Yeah , I wish we'd made up for all of our lackluster .
I'll send you some pictures .
I love how you have to use . You have to use your term recovering from such a delightful two weeks .
Oh yeah .
Yeah , but I mean so coming back and getting over jet lag and stuff is always a little tricky .
It is .
And getting back into the oh geez , I got it . I actually have to work for a living instead of just like hanging around and drinking cider .
And cider and crepes are so cheap .
I think I bought a reasonable bottle of cider in a store and it was like five euro . Oh my goodness .
All right , so I actually this is not related to the topic at all but I have to know savory crepes or sweet crepes ? Yes , no , no , no , yes , the answer is yes .
It's interesting there is they call them different things . I think it's a galette , is the one that's made with buckwheat flour and that is always savory , and the crepe is made with wheat flour and is always sweet .
Oh , all right Cool .
And they serve you . They serve you the cider in little cups .
And that's that's awesome . Dude , I love crepes . That's that's really cool .
Yeah , so future episode recording ? Yeah , yeah , exactly .
Actually I am sort of wondering about this because I know there's some friends that do whiskey and wireless and I'm wondering on a Friday afternoon whether it could become what would run with the sea after figure this out . So for future , future episode , right , make sure there's some alcoholic consumption in the process .
Clouds and crepes , and cider sounds pretty good .
Clouds- and cider , I think . I think it's was whiskey and wireless was the show , and you was a couple of guys getting together and drinking bourbon and talking about wifi .
That's a great idea we got to . We'll have to see if we can work some some drinks into the next episode .
See , it's only 9am for me , but nothing's ever stopped me before , so
¶ Ethernet Development and Cloud Providers
it's fine .
Well , so here's the thing You're an Australian , You're in Australia and there's always passing well , it's always fast and midday somewhere in the world . Yes , exactly that's true , I mentioned this on the show . Should we get that contract ?
We will . But let me make this point I mentioned on a previous show it was amazing to me when the Super Bowl aired here how many people were still going to the bars at 10am to just get sloshed and and watch the Super Bowl . It was a true testament to the culture here . Yeah .
I remember that Dedication Well , yeah .
So my first job at a college was on a mine site in the in Northwest Australia , in the Pylbara , and it was not uncommon to get up having missed breakfast because we're in single main squatters missed breakfast and have beer for breakfast .
Absolutely Wake you right up . Very nice , all right , now let's get on track , tim , take it away , mike .
Besides the Well , no , I mean , I kind of set it up right . There were so many cool things that Peter had brought up that we really wanted to get him on the show and in front of everybody just to talk about it .
And , like I said , what kind of spurred us to have start the conversation was about the idea of , like you know , all the CSPs and the hardware vendors and businesses you know are all competing over the same silicon and the shortages and everything , and we kind of use that as a launching point .
So I guess , peter , is that , is that a good place to get started , or how would you like to start this off ?
So I guess the area where I'm most directly useful here is generally how ethernet is moving around , how it's developing .
Oh yeah .
So I mean , I've been building ethernet switches for a long time now and I also check the ethernet lines and we're at 50 years of ethernet , you know .
And on the sideline , ethernet has the biggest dynamic ranges ever had , because we're working on 10 megabits and 1.60 at the same time , clearly for very different purposes , and it's interesting to see how , you know , the transition has been .
You know , 10 years ago , maybe 15 , it was the service providers who were the bleeding edge and they were low volume and high margin , and now the CSPs , the web scalers , are the bleeding edge and they are high volume and very low margin and they really want to take everyone else's margin out of the business .
Yeah , that's actually so , just as an interesting sideline I was actually having as I was writing a thing for ethernet lines today , you know it was a piece for fast mode or somewhere else , so I'm writing about slow ethernet , someone else is writing about fast ethernet , you know , like high speed , and I just did a quick look with some numbers .
As it turns out , the predictions for 2027 say that , of ethernet switches to DC and campus , more than 75% of the ports is one gig and under Really yeah , one of guess 2027 . 2027 . It's , you know , because we often get we all get wrapped up around the .
Here's the really new cool thing , right , because this is exciting and we actually forget what actually makes the bills . So I just wanted to drop this in there as a , as a let's not get ahead of ourselves . It's really cool to understand where stuff is going , but it's also really cool to understand what matters now .
Right , and if you lose track of what matters now , then you don't serve customers anywhere . Right , because almost the worst thing is , you know it's if you have the right product at the wrong time , right , you're not helping . Yeah , if you're , if you're a long way ahead . And for customers , there's always the what's the risk reward ?
You know , depending on who you are , what resources you have , you've got to make very different choices based on your risk profile . You know , in the , in the enterprise world I live in , you know , refreshers on switches are like seven years .
Yeah .
Now that's very different to what you might see on a CSP , where they can afford to gut something every three years , and so I think you know the thing that you guys know and I try and keep in my head is there's not one size fits all and you saw , the first thing is what are you trying to achieve and you know what's your risk . Balance Right .
Where do you ? Where do you want to be in comfort level ? Because there's really good reasons why some people are like in the OT world they're a little bit behind adopting stuff because it really matters , right . I mean if , if I'm looking at the Cisco headquarters , like building 10 is where that all the higher ups sit .
So in the worst case , if we lose the network in building 10 , everyone can go out to lunch , All right . If the OT network goes down and the elevators stop and you can't get out , that's a bit of a different problem .
And so the OT work , I mean let alone if you're in something like an aluminum spelt or for Chris , and now you've been in this spelt when , when you lose power , everything freezes . So taking into account the risk profile , the customer , I think is really important because otherwise you get your recommended the wrong solution . That makes sense ? Yeah , absolutely .
I spent . I'm especially in terms of like OT or like you know . You also see this in government and medical and stuff where you know you tend to not see the bleeding edge adoption , because you need that Stability .
I mean . So there's , there is always the old thing about you never , you never want to buy . You know any of the first set of cars in the market , and there's there's a good reason why for most products you , you release the first one is like as version one , dot something Because you never , no one ever wants to buy 0.7 . Yeah , absolutely so .
Do we want to head back to ?
So if I , if we go back and think about ethernet in the cloud providers , I mean , first I'm going to separate and let me make it clear , I wear a bunch of hats , right , there's some of them in the background there's the Cisco hat , the ethernet alliance hat and there's the me hat , and so the me hat is sort of a synthesis of things I think .
Right , not speaking for anyone else but me .
Right , but if you look at , the cloud scale providers .
There's really , there's in the DC , there's really their IT network and their production network and they're very different . And in their IT network you'll often see the , you know , the normal , the normal vendors , the us , the Cisco , the riskers , etc .
But in the production network it's almost all their own kit , it's the white box stuff , and it's there for a bunch of different reasons . But I mean partly it's there because I think they don't really care about failure , because everything is everything can start up somewhere else at the same time , right .
So if you lose a racket , it's not a big deal because it just swaps over . And so their , their appetite for risk is much higher because eventually , if there's a failure they'll get , you know , it just loadsheds and no one cares about it . But they're also very picky about exactly how that is on that network .
You know , having looked at a few of them , right , every cloud service provider thinks they have exactly the right answer and they're all different , which does make it a bit of a challenge for the , for the providers . Now , it might be different in terms of what the radix is , what the they're all going to build . It different .
I know I talked to Russ White about this at one stage and it's like , yeah , there's a , there's good things , the bad things about how much money these guys have , but the bad thing is they all have enough money to really decide what , how to do it themselves .
Yeah .
And so from a vendor point of view , it gets really hard satisfying them all , because they're all different .
Yeah , and like every other year , they come out with a new demo of oh , we've upgraded , like Google has gone through so many iterations of their , of their internal like GCP network . It's crazy you can find literally almost every year they'll come out with a new presentation about a new like complete paradigm shift that they've done on their . Yeah .
And they can go and write their own protocols or anything else . And so it's a very different world to where we were , you know , even five to 15 years back , in terms of stability , what was happening and the amount of power that those particular customers have , but it's always .
I mean , you guys may have heard the saying that you know , one way to go business as a small company in America is to get a deal with Walmart right , because you have to . They will beat you down the price , and I think the cloud providers are somewhat the same .
They just have so much power Anyway , but that all being said , I mean they're driving to very high speeds as much as they can
¶ AI Networking and Language Model Training
. So this is where the push is coming to . You know , a little while ago so probably in 2015 ish it was pushing towards 10 gig to the server on 100 gig lift spine , and then it's gone through 50 . Now it's going to go to 100 . At the minute , ieee is working on 100 gig per lane , and then speeds of 200 , 400 , 800 and 1.6 T .
It's going to be interesting to see which ones of these end up being adopted , I mean if you think about . Well , no , they'll get ratified , but not all standards succeed . I mean , if you remember the first , the long time , the motto in 80 to 3 was 10 times the speed for 3 times the price .
That was a great motto , right Until we tried to do 100 gig and it didn't work very well , and so , if you remember , basically the first 100 gig was 10 by 10 and it was very ugly to use . We did 40 gig . 40 gig was easier .
Yeah .
And you sort of end up with a one lane , four lane story which then went , you know , 10 , 40 , 25 , 100 , 50 , 200 , and it sort of works well for mass adoption .
I mean , it's a little bit less clear in the data center because they could potentially reengineer that stuff much more , but it's still not easy , right , even if you can change a rack , changing the rack to spine connections is not a simple thing and , as you guys well know , inside the DC , you know , inside the rack , it's probably all DAC right , maybe active
optical cables , but once you get out of that it's probably it's multi mode fiber and of course what's happening is pretty much , you know , in general every new speed actually needs a new grade of multi mode . So it all gets . Even though they can change a lot of stuff , it actually does get a little hairy .
So I'm sort of a believer in the one lane , four lane story .
And so then you know , 800 gig , I think , makes most sense when it's four lanes of 200 , we're not there yet , but to some extent this is it's depending on what the SP , the , the web scalers , are trying to do , because the workloads for , you know , search and storage and everything else are one thing , but , as everyone knows right , everyone's little black dress
at the minute is AI with large language models , lms , because everyone's been going through chat , gpt , which is a great resource online , as one lawyer can tell you who used it to file his case and did not work out well . And so those networks are actually built , being built very differently to even the normal production network .
So in my head at the minute , you know you've got an IT network , you've got a , you know , cloud search provider production network , and then you've got the training networks and they're different . Okay , how ? are these and that's what I'm glad you asked him .
So I went to this conference the other day called the SmartNix conference , and the only reason actually went was because they asked Ethernet Alliance to come and give them like a 10-minute pitch on Ethernet and I tried to get one of my other colleagues who's in the SmartNix area to go do it and he couldn't . So I went instead .
But the whole discussion at the minute because , because of all the noise around this large language model training you know , as you will know , everyone has their own right . It's chat , gpt and this barred and everybody's developing their own here .
Yeah , everyone's doing this in a great hurry and I don't know if you've been tracking it , but there's been a bunch of stuff recently about there's people doing research and they're finding that in general , it's pretty easy to get around the quote guard rails that people are putting on these things . Yeah , because I think the guard rails are . Actually .
They're just trying to filter out input and , surprising enough , people doing input are really creative . Yeah , put that to the side . So , and you probably want to find someone who really knows this stuff better than I do . So here's what I understand about how you build that network .
It is a dedicated network that's fully homogeneous and is built specifically for this task In the ID world . It is not oversubscribed and you might have you know , somewhere between you might be serving somewhere between a thousand to 32,000 GPUs and you go . That's a big network .
Yeah , and here's the thing .
The model training can take a long time , where long time being in order of months to complete one full task . So a fully homogeneous network that's running for a month with nothing changing is a pretty unusual network .
Yeah , and not an easy task .
Just like you know , if you give enough money you can build anything . But no over-subscription at all is also a very different network .
Yeah , so I was thinking of that's the most foreign concept to me out of that is not ever subscribed . That's something you never , never , would consider .
I mean . So in the enterprise , we tend to think over-subscription from access to distribution is probably 20 to 1 , right , so we used to . But also in enterprise , you really optimize power for idle , where in DC , optimize fractiv , all right .
So , and here's why this is supposed to happen and again , I'm sure we can find people who explain this better so the way they do this is they break this workload up and they split it out to a whole bunch of different GPUs , so each GPU basically gets some data , is called to go and work for a while , and then it has to share its data with everyone else .
So it really is a control , work , control , work , control , work , and they can't start the next work phase until everyone has shared the data . Yeah , I think the data is quite large , so it is not just a single message .
So what they're trying to do is , every now and again I don't know whether it's 99% of the time or 9% of the time or something everyone has to send data to everyone else . So the whole goal is to make sure that everyone communicates , and what they care about is the time for the last packet to arrive .
OK , since they need a quorum , basically they all need to have the same .
No , not a quorum . They actually need everyone to bring it .
Oh , good point . Yeah , so it's like you said everything OK .
So that requires a lot of different things . It means things like the load balance who would normally use networking just isn't good enough , right ? Because , as you all know , ether channel or ECMP is a hash and flow polarization all this other stuff that's not helpful .
Coordinating when people send stuff is tricky , right , you have to deal with panning , and so I believe and there's a bunch of presentations I've read about this and you can put them in show notes .
I think I've sent you some already , yep , so they're trying to solve that problem , which is quite different and that's what I'm hearing about being called AI networking , and you're seeing stuff about it from Broadcom . They released , I think , the Jericho AI , cisco Silicon One . I'm sure Melox has a similar thing .
Everyone has their own flavors of this , but they're trying to solve that very specific problem . Now , for the time being , just imagine how much it's going to cost you to set up a couple of these clusters . Right , the cost of networking is actually trivial because the GPU cost is going to kill you .
You'll spend more on the networking because if you can reduce the time they spend communicating by 3% , you significantly increase the value . So this is going to be a very cool , sexy thing , but I doubt many people are actually going to go and build one themselves .
Well , I would imagine the CSPs might be among the people who might do that . Maybe not to that level , Maybe they will . I guess it depends on whether they think they can monetize , essentially pass that cost on along and make a profit on it .
Wow , this is , I think you'll find they're already doing it . This is what the Microsoft's will be doing for Bard , what Google will be doing for its one .
what Metal will be doing .
Yeah , I was going to say wasn't there an announcement around like was it open AI ? Someone was , I thought they were running theirs in Microsoft data centers .
Like in Azure . I think you're right , yeah .
And I mean , how long is it till all of the CSPs have agreements like that ?
So I think the pre-meeting at least , we were talking about this . We were saying , well , what size company would you have to be to set one of these things up ?
And .
I think the answer is unless you're doing this all the time , you probably don't . So there probably becomes a market for the cloud service providers selling basically letting you rent time on their training cluster .
Selling cycles . Yeah , that's just what they do now .
So we used to call this Bureau computing in the old days , right ? So it's sort of an interesting thing , but it's off to the side . And so then you say , if your goal is to build this with the minimum , with no jitter and minimum completion , you basically build it differently , and there's a whole bunch of ways people are thinking about doing it .
One is to basically use telemetry and basically start at running and then adjust things as you go to reduce the impact or collisions , and if you're running this thing over a couple of months , you can probably do that there are other people .
I mean , I think at the minute NVIDIA can do a bunch of this with NVIDIA Band , because NVIDIA Band gives you this sort of properties , but I don't think NVIDIA Band scales with a price performance that the CSPs want . So even though NVIDIA likes NVIDIA Band a lot , I don't think that's the answer . So the question becomes is what else do you do Now ?
I think since we talked , there's actually been a public launch of a group called the UltraEthnic Consortium with a whole bunch of people in it , including Cisco and Broadcom , maybe Rockom and a bunch of the other CSPs , and they're really trying to define the characteristics of what this network looks like .
Now they're actually they're listing the work they're going to do , including physical layer , link layer and transport layer , so they're looking potentially at all the layers of what they need to do to make this work .
There's not much information on the website , but they do have a white paper which sort of explains what they think it needs , so we can stick a link to that in the notes .
Which we will .
If I remember , the only CSP on there is Microsoft , and then you have Facebook , which I was going to bring up is one of the few companies outside of a CSP that's going to that could do it . Yeah , so it's interesting . It'll be fun to watch if more come on or build their own standards or how all that plays out .
Yeah , I mean so there's . So in the world there's roles for vendor organizations versus standards and they're sort of different . And the way I tend to think about standards are really bad places to develop technology .
Yeah .
But they're really good places to make sure it becomes common and usable . And so you see this in a bunch of places in terms of various consortia pushing together things , and basically you tend to see them as a eventually enough people go yeah , this is a good one . And so then you have to come in and make some compromises for standards .
But with this particular one , I think , they start up under Linux Foundation and they're claiming they can do , as I said , physical adjoint layer , transport layer and software layer . So they're trying to do everything . So it's going to be interesting to see , as you said , do they get some critical mass At the minute ? I think they said so .
Microsoft and Meta are in there , and so for the time being , I'm sort of consuming Meta as a web scaler , right , whether it's a cloud source or a cloud or not .
Yeah , right . That makes sense .
And the other guys listed are AMD , arista , broadcom , cisco , eviden , who , I don't know who , that is , hpe , intel , meta and Microsoft .
So a lot of the vendors yeah .
There's a fair bit of weight in there , but there isn't AWS and there isn't Google , let alone . Which one is guys ? Yeah , I mean those are definitely missing .
That's an interesting omission from that alliance or whatever from that group .
So often . There's often competing efforts rolling around the place and then you have to sort of see which ones end up winning . Not quite a lot of the flies , but it's sort of like that . So we'll have to see who makes progress and how it looks .
But , again , my understanding of this is this is a topic of interest , but it's a bit like I really want to understand how the Formula 1 car is built . It's a very cool thing , but I'm never going to buy one .
Obviously we'll . I mean , adoption is king , right . I'm sorry . Go ahead , chris . No , no , no finish with that . Well , I was going to say adoption . How many times have we seen vendor develops thing and then later on either it becomes the standard or the vendor has to adapt to the new ITF standard or whatever .
And it seems like generally adoption is king there . So no matter how good the technology is or isn't , it's going to end up who ends up using it and that determines
¶ The Adoption of Technology Standards
.
So I worked on a standard which is 802.17 , which was resilient packet rings . It was an excellent standard , it was well-written and it died . So it's not the case that you build a better mousetrap . So I mean you're right . I think the key thing becomes ultimately what drives adoption standards is how suitable they are for the use case , how consumable they are .
I mean , sometimes you get the standards , the tech geeks , we go and we find something . We say this is really cool , like 10 by 10 for 100 gig , but adoption is really hard . Or even single mode until recently was hard because terminating those things was nasty .
So I mean , if I go back in time , according to the books I read , when Metcalf so Metcalf invented ethernet , like 50 years ago him and Dave Boggs he spun out a company called 3Com which is the first ethernet company . So originally , when I started with 3Com hang on a second I have to show and tell . That's for YouTube .
All right , show and tell YouTube viewers only Yep , yep .
Sorry , We'll have to describe it for everyone else . All right , what do you got there , Peter ? Oh , I think he's having to dust it off he's opening .
It might be in a sealed package . This might be ruining resale value on where this is .
Is it OK ? Okay , this is the original ethernet cable , but this is this is it , and it's not nice and it's heavy , because the idea was I'm gonna run a cable down down and Corrupting officers now then put a tap in from each office . Yeah , after that we went to this , you guys . Yep , yep very exciting and as a side do you guys know David Bombal ? Yeah .
I do , yeah , one of things he's done it's he has a YouTube on the very basic builds a real functioning network from 10 base to and 10 base life . It's worth taking a look at it'll work oh yeah the caps and everything , it'll work right .
So for the audio viewers , peter's been showing off some really , really nice , really nice old old cables . I think he actually had to go dig them out of the Vault to bring them back up .
Where was it with this thought ? So the real question becomes is what probably trying to solve and how is it adoptable ? And the story I was going with is so early on . So three comma just got their first 10 base fire product I'd the cheapen that product working . And so they called up Steve Jobs and said Steve , we got this great thing .
You want to come down look at this because this is just wonderful . And Steve comes down , they show him and he gives them a normal Steve response with a large number of swear words and then says why don't you just plug it into the telephone cables ?
So this was the time that the direct the telephones were getting derogated in the state , so people owned the cabling plant . So 10 base T you could run over what was there .
Yeah right .
So that was the case , that's . That was the one that made sense . Yep and then you go okay . So for every new technology , what's its adoption path ? So when 25 gig was first defined , it was defined for 100 meters in multi-mode , for the DC and for campus . That was just not helpful . No so I actually was involved .
When we expect 25 gig single mode and we expect it to be the same Requirements for the 10 and 40 is 10 gig was with the idea of that . It ran 10 , would run 25 and then we went and built a transceiver that did the same thing because in the campus world your existing infrastructure is key . You can't change .
Oh yeah .
Now you go in the DC world things are a little different , but it's still not clear to me that want to change design patterns all the time . Now in the DC world . What you tend to be nowadays is it ? One stage it was like we need more and more processing in one , are you ?
That doesn't actually work anymore because now you're limited by the power you can get in and you can get out of a right Rack unit . So now we're actually becomes more interesting , right ? How much work can you do per unit of power ?
Yeah , that's right .
And , if you think about it , there's a lot of conversations about what we might want to do with liquid cooling and immersion cooling . I think these things are all fun , but I don't think they're actually broadly deployable . The the engineering challenges are just huge .
Yeah , absolutely get liquid cooling and all those racks .
Yeah , I would agree completely or run a will serve as a machine called right . It's a very cool . You know , this is the the formula ones , and there's the rest of our stuff .
Exactly like Chris said just build a data center into the ocean and that'll solve all your problems .
Well , actually I think I'm someone did that right . But so if you're gonna see broad adoption , there's this interesting window of what's consumable right , what is what is close enough to solve the problem at a price you can afford , that can actually be deployed at scale .
I think maybe then diagram right .
Yeah , and I think the cloud scalers have just changed what some of those scales look like and on sort of on similar technologies . You know there's a , there's a trend that says , look , we can't keep doing Module optics .
And the reason the problem is is that if you look at , for instance , the 25 or 50 , 51 terabit chips and you look at what the power in that system , a lot of the power is spent just getting the signal from from that chip Front optics module , because service is really expensive . So people say , well , why don't I put the optics on the board ?
That's what cobell was trying to do or why don't I pull it inside the chip itself , which is what co package optics does .
Hmm .
It'll be interesting to see again Whether those are , whether those are just the things you do when you can't do anything else , or whether they actually what end up being the dominant , and I don't know the answer .
But you always have to take a look and see this first thing that comes out Is it really deployable at scale by bunch of different vendors that it can stick ? Because often the first things that come out they're yeah , then they don't stick for a long time .
Not to derail this too much , but I mean , obviously we talked about the be be , be be aware of people building , making gifts , because if it sounds too good to be true , what probably is , yeah , yeah , so not to derail too much .
But I know what you know . We talked about this ultra ethernet consortium , right , and I'm sure we'll probably dive into the , the ideas that they've kind of put into this white paper on on how to solve this problem . But just just a side note is obviously this is this is like a standard space thing that that is going to .
Ultimately , the idea is to increase interoperability , right and and creates standards , right . So do we do ?
We think that these things kind of fall flat on their face because , like the , I mean , if I'm looking at this in which I'm , this is brand new to me , I don't know much about this , but , like I look at the founding members and a lot of them are obviously in direct competition with each other . So how does the interoperability worked in their favor ?
That's jfrabidinarticlecom . They're like all it's like the little bit vendors that hate each other .
So so let's let's talk a little bit about how stuff gets defined . So you have , you have Co-operative .
And you see this most clearly in in like the actually standard . So if I'm in a standards group , in 802 , the rule is everyone gets to vote in 75% of the room to agree to make the decision . So the room includes my suppliers , my customers and my competition . So getting 75% is always tricky .
The idea of a standard is where you decide that you better off making one playing field . Everyone can compete in the lots of playing fields . Now you look at the ultra ethernet consortium . It's got a bunch of people in it but about a bunch of people missing . So we're at the stage in general , most of the tech companies know they can't earn it .
So you can't earn it Because no one wants to be locked into one company . There's temporary things like in video , have a pretty much a lock on the gpu system . But you know , whereas at one stage x86 and Intel dominated , dominated the processing you now suddenly see almond . Other people come in .
So in general , no one ever wants to get stuck with one company , so there's a benefit for people to say , look , if we work together we can get a good solution . Now , if you go , look at that set of members , right , it is a bunch , it is chip supply system supplies and some consumers and you can , as a web sky , to build your own entire system .
But then potentially you end up in a bad place because if you have to build a chip that no one else uses , the volumes go down . Yeah , that's true . So the web sky is what they want to do , is they want to have common things produced at high volume that they can leverage .
So , for instance , if everyone of these groups tried to do something different , no one would sell any product . So and it's always a bit of a , it's always a competition , because you get the if we keep it to ourselves , will make lots of money .
And you look at people and say , well , if you keep it to yourself , most people won't buy it because they don't want to be stuck into a single business strategy . And so I think for the more mature competition , it's like okay , how do we make a market bigger ?
now , normally what you see is everyone will do standards plus they'll do the standard plus they'll do their own special thing Right , and I mean you can see this in the in if people using the cloud right . Cloud computing sounds like a great plan but , as you guys know , it's somewhat of a roach motel it's easy to get in , it's hard to get out .
Now , if you're trying to do a , you know , a multi cloud strategy , that's wonderful , but then you can't use all the nice optimizations everyone offers you .
No , that's true , definitely .
So I think what you want to do in the like a standards organization is get together , you know , towards the minimum criteria to be useful and then go sell it . I mean standards like documents right , you're not done , for there's nothing left to take out , because the more things that get put in the standards aren't used right , the worse it is .
So this consortium , which I don't actually know very much about , is obviously . They've been working for a while at this .
It'll be interesting to see whether they can get the other major parts of the world agreeing to it now , even if you look at the other guys being the Google's or the AWS is that's one side , but then you start getting all the Chinese cloud service riders in as well .
Yeah , so I think what's ?
going on stuff , yeah all those guys now .
Of course you know dealing with them at the minute is always tricky outside of a standards org . But you need to get down to a common agreement on this is going to work for a bunch of people and then we can start building a volume .
I mean before that , if you have to build phone into different products for every company right , there is no way that anyone is up ends up in a good state .
That makes sense . I wanted to step back just a little bit because I think we might have glossed it over a little bit before Something you said that I liked and I don't think we had spent enough time talking about it . Maybe you're just laying it out for the audience .
You mentioned that the network that we're going to build for low-level models , for like training , ai training , is going to look like Large language models , sorry . Large language models .
I'm thinking , yeah sorry , Large language models is a different network than , for example , the CSPs will leverage for its customers , which is different than an enterprise will build , different than the CSPs will use for themselves . One thing I think we glossed over a little bit is how would we consider them to be different ?
We know an IT organization for internal IT or something like that is going to look different than a CSP or network built for customers . We know that .
If we start talking about the large language models and throw that in there , how do those two Because the CSPs , if they're going to offer it as a service are you thinking they would be completely two different , separate networks and they would interoperate ? How do you feel ?
about . So you want me to start making stuff up .
Yeah , well , really , it's all speculative , right ? That's the whole point . A lot of this is very speculative , but what's your thinking about ?
that this is all on the record .
Yeah , we're holding you to it . By the way , we're recording .
So this opinion was worth exactly what you paid for it . So here's my gut feeling and again , this is based on my current understandings .
¶ Language Models and Business Networking Strategies
The training of the large language models requires a specialist , dedicated resource to go do it . The more normal processing and the inferencing runs on , that becomes the general production network . So the general production network has normal compute , some acceleration , some storage , short long-term storage , et cetera . But that's built how we built a DC today .
On the side , there's this other job that you're going to go off and run this one job for two months , and it's just a different piece of stuff . Now , the result of that is some large data file which is basically a language model . So there's going to be some transfer between them .
But from a fundamental operational point of view , what I think is going to happen is those LLM training clusters . They're just off to the side .
So almost like a container type of setup , where they just kind of have essentially worker nodes and they go produce some data and come back Because the worker nodes are fundamentally different .
Right , those worker nodes are just different . It's all GPU . So I think more like , if you think about , for the web scalers , there's the writing network and the production network . I think there's going to be the AI training network as well . Maybe that's not true at some low level , but if you're doing these huge ones , it seems to be that .
All being said , I was also reading some stuff the other day that people were able to take some of the output to the large language models like the chat GPTs and then re-implement them in smaller systems . So yeah , I have to go and find a link .
So I think the answer is we don't know , but if what I'm being told makes sense , then it becomes a fundamentally different network . So you stick it off by itself and you could basically carry it between . You carry the stuff between . The model is USB , possibly .
Sneaker net .
But I mean if there's a function where you need to go and run for two months and not touch it , this is fundamentally different .
That is .
Like I said earlier .
It's just a purpose built network , right Just for this thing . It's not oversubscribed and that makes a perfect use case for someone like the CSPs to offer this as a service .
You know yeah .
AWS probably have a slew of service names they would give you to offer this kind of thing , to build your own LLM right .
For your own customers , so do it themselves as well . Yeah , exactly , but I guess it's like , how do they do this on a shared medium too ?
Like that seems overly difficult If it's something that's not oversubscribed and what's shared medium . I mean if you're offering it as a service .
You can do it , it just runs worse , yeah true , I mean , like you said , it's all speculative .
I can only take what my 15 years of enterprise networking mind can grasp in this concept , but yeah , I mean , if you run this on a normal network , it's going to work .
It's just going to take longer . You're just going to spend more time communicating and less time working .
Right , yeah , yeah .
So the question is at what point does the time , what point does the time that working make a difference to you ? And if the answer is , if you're running 16,000 GPUs , you really want them working all the time ?
So the network .
I think the cost of network gets lost in the noise .
At that scale with that many GPUs . Yeah , that makes sense .
So your only purpose is how do you make sure the GPUs are working as much as possible ?
You know , what's funny is I was thinking about this when we were talking about it before , and especially about the idea of having all these GPUs doing this task , and I was thinking , man , if they're not training large language models , they'd be making a ton of money on crypto .
We're not going to get into crypto .
Oh , come on , Peter , tell us all . You , I'm just kidding , you don't want to know my opinion either . So , yeah , it's , yeah . Anyway , I was kidding , but the whole idea of task based GPU working on the same task it made me think of that type of so you know , I'm sure that there is .
you know there is , there's the inferencing required . There's going to be lots of cases where they'll be adding , you know , ai , small class to functionalities , stuff you can rent , but that's not these big , huge ones .
I think the big , huge ones , the reason why they're going to go and spend all this time defining fundamental new things is because they can't solve it any other way . If they could just use the standard protocols that today we'd be doing that already .
Right , that's a good point .
There's something here too . This might be a crazy thought , but it's . You know you're talking about having the CSPs .
A crazy thought on this conversation .
Well , I'm just , I'm trying to think of something over here .
Get your full hat on .
You have the CSPs that are going to do these really large ones , but this kind of almost opens up a new space completely where you could have like a different kind of CSP that rents out space for companies to do their training on right , If that makes sense .
I don't see why . I don't see why it's not the existing big ones . I don't see how you get to their scale without it .
Well for smaller ones , right , Like if you have a smaller company that doesn't need to use the entire like an AWS data center ?
No , I absolutely agree . There's going to be some people who rent this out .
The question is , is there a ?
business model for a second tier guy to offer the service that's cheaper or simpler than that .
Yeah , that's right . And where's the margin on that one that you're going to ? Yeah , I totally .
But I think probably the reality of this is is that sort of the second generation of stuff , because you have to see it work ?
first yes , you're right , that's a good point , for sure .
Cart before the horse on that one , because no one really knows what this is going to look like .
Yeah , I mean at the minute this is , I think there's huge speculative investments being made and I think the only people who can afford that are it's the web scalers , because they have so much money and they see this as so important .
It's like goldmine for them , right ? This is really right in their wheelhouse in terms of like provided service right .
So it's , it's sort of interesting . I'm not a chat GPT user . Part of the reason I don't is because it hallucinates , but no one would write something without spell check running , and so it's going to be interesting to see how this fits into the world . I mean , I think there's was , it was co-pilot , forget that got in .
Oh yeah .
Certainly there's . You know , in my company there was a thou shalt not use chat GPT message that went out , and I think it went out before the guys from Samsung managed to leak stuff out . But everyone is , everyone is adopting this and it's is it going to become , you know ? Is it revolutionary or evolutionary ? I don't know the answer .
We were talking to Yvonne Sharp about this as well . I don't know if you know Yvonne or not . I do , I do , we all know , we all know everybody in this industry . I know that's true , right , you know , and she kind of took the position that it was evolutionary , like it's here to stay and it's going to be part of whether we like it or not .
It's going to be part of the of technology moving forward , right , you know she took the evolutionary , I think . Response .
So so you guys like I mean , one of the things that's coming out a little recently is there's a really interesting question about intellectual property . So if you've trained your large language model on the entire web and you've sucked up some stuff that belongs to someone else , can you reuse it ?
And the answer is that's tricky no one , no one actually knows the answer to that . But probably you can't , and so both data protection and intellectual property , I think , are a huge problem that no one has solved .
Well , I think they're solving it by ignoring it . Like one of the I was going to ask you .
that is not , that is not solving no , no , no , I agree with you completely right .
So I was going to ask you what your own personal opinion you know , not the Ethernet Alliance , not the Cisco hat , the Peter hat on the fact that we have all of these . We have these very large companies with a lot of money hanging in the balance , you know , fueled by that .
You know that that money that are essentially racing to be the first to market and the the ethical and and privacy and IP type of problems that are necessarily going to happen when people just stomp right over .
Then necessarily going to happen . So I was . I also listened to some some legal podcasts about the Supreme Court and there was there was a case recently that was between the Andy Warhol Foundation and a photographer . So photographer taking a picture of Prince and then Vogue licensed it for Andy Warhol to draw a picture based on it for there , for Vogue .
Then Warhol drunk draw a bunch of similar pictures and then sold them . And the question in here was you know the whole thing about ? Is it fair use is is transformative , and how transformative is it ? I think the answer is we don't know .
I mean clearly , clearly , these models were trained on the whole web , right , and so they're trained on some material that was copyrighted by people . Now , even if you want to solve this , what would you do ?
I have no idea how you fix it anyway , but how do you pull that out of the learning model Like , how do you even identify it once it's in there , right Like it's ? It's like a Hoover vacuum . It just sucks it all up and you have no idea what's even trained on .
¶ Future of Large Language Models
What's funny about this , though , is , like you guys might remember this , but I would be within the last year when the whole like metaverse stuff was coming out . Isn't this what Zuckerberg was trying to do ?
He was trying to assign , basically , value , like you could pay in the metaverse to give , like , artist credit and stuff like that , and everyone was like , oh that's done NFTs , NFTs , NFTs .
Yeah , it's kind of similar yeah yeah , so so in an FT , right , you know you bought , you bought a URL .
Yeah , you bought a receipt . You bought a receipt .
So on the metaverse thing , do you guys read science fiction ?
Occasionally yeah .
Yes , absolutely , have you read Snow Crash ?
No , Is that Neil Stevenson right ?
Yes , yes , the originator of the metaverse term , so so so Snow Crash and Neuromax are absolutely worth reading . I started .
I love Gibson .
I haven't actually finished yet , but it's so good man , if you're into cyberpunk , you're going to see , like basically , where cyberpunk began . It was such a good book , oh my God .
And you remember ? Do you remember why Snow Crash was called Snow Crash ?
It's been a few years , okay it was brains .
Their brains crashed Nice . That's right . I mean the thing that I hear from this .
it sounds like this . Just , I mean , this builds the use case . Why it's ? Why would be valuable for someone to want to specifically build their own large language model , but the problem is like you got to have that amount of data to put into it and who has that ?
The only things I could think of maybe be like meta or or someone like that but has all this like information just about people , but then like they own all of that right , because you probably sign your life away when you signed up for that .
That's private privacy right yeah .
Okay , you can , you can imagine , okay , you could imagine you could imagine effectively taking a mostly trained model and then putting it on a vast amount of data you have yourself , for instance , let's say , let's say you were running a factory and you have a whole lot of data on that factory .
Maybe you basically take a partly trained model and you add your own stuff in right , which is not public data . That's your model to solve your problem , or maybe that becomes if you think about it . I think , as far as I remember , rolls Royce do not sell you jet engines .
They'd lease them to you , and part of the reason is they want the data , so maybe , maybe they end up training a model and it becomes part of their service offering .
That's interesting .
Yeah .
That makes sense . So anybody basically who could run this LLM network basically could even package up an offer in theory , the model itself as a product essentially .
Then someone takes over and basically adds their customization with their data and then offers on top of it . I mean , you can imagine lots of things right . Which ones are going to end up being product ?
Who knows who we're actually going to monetize and what's going to be the actual product ? You don't know that .
Google is over there just dreaming about the new angles for ads and stuff they're going to get from this .
So you guys remember they're saying from Yogi Berra right , Predictions are hard , especially about the future . Yeah , so it's going to be interesting . I mean I've now been in the business long enough to see a change from my 600-billion model to 1.6T Ethernet , but I think mostly it's been evolutionary . I'm really an evolution not revolution sort of .
I don't know where this goes , but certainly the volume of some of this stuff is going to affect pricing of other things . There is the original point you made , which is if these guys are buying huge amounts of this stuff , no one else can buy things . But then with supply and demand you're going to end up with more people building stuff .
So , the acquisition of large volumes of this stuff over a long period of time is going to reduce the price for everyone eventually , and so that's a good thing .
Interesting . I mean eventually we need the air quotes .
If you want the thing first , right it costs you money , that's the way it works for me . That is how it goes yeah , what's interesting here is that , compared to the original , the old service providers these guys are buying enough of a volume that it makes a significant difference in terms of cost of goods .
Okay , I think that's a good . I think it's probably a good spot to wrap up , because that's a good point to finish on . Yeah , we'll definitely have you back , peter .
Because you know that if you don't wrap up sometime , I'll just keep talking .
I mean , we try to keep this around an hour anyway , just so people don't turn us off .
Don't worry , we're going to have you back for round two and we'll do another hour , so we can just go back as long as we have stuff to talk about . So , yeah , this was great .
Yeah , it was . I really appreciate you joining . We'll see if we can get Lexi to join us next time . We tried to get her this time but she wasn't available tonight , so we'll definitely try to schedule it . So next time you're on we can bring Lexi in too , because she loves to talk about either queen , she is the either queen .
All right , let's go ahead and wrap up , guys . Any final thoughts ? Will everybody speak at once , please ?
I think their brains are .
I think they're melting .
That's right , they flagged their brains .
No , man , there's just so much , yeah so much here that I need to dive into , and so I think the goal of me coming here was not to actually tell everyone everything about .
it was to sort of try and set the scales so people can provide context , that people can fit in what they're hearing right into a world view , so you can sort of start to picture , you know , pieces of all .
This is where this goes and this is where this goes , and so you know I'll thank you for putting me on because I like doing this stuff and hopefully this gets good consumption .
Yeah , yeah , yeah , no . Thanks very much for joining us , peter . We'll go ahead and end it there . Of course , if you've been listening this long , the odds are pretty good that you're already subscribed to us , but if you are not , please feel free to subscribe to our podcast . Visit our website cables to clouds dot com .
View us on YouTube , subscribe to us in your favorite pod catcher . I don't know , buy us at Walmart , whatever the hell you can get us .
So we'll have to be happy to have you . We can't kind of deal with Walmart . Peter already said that we don't want to hear oh , that's right , we'll get driven out of the business .
Sorry , forgot about that . So , uh , all right , we'll see you guys next time on cables to clouds . Hi everyone , it's Tim and this has been the cables to clouds podcast . Thanks for tuning in today .
If you enjoyed our show , please subscribe to us in your favorite podcast catcher , as well as subscribe and turn on notifications for our YouTube channel to be notified of all our new episodes . Follow us on socials at cables to clouds . You can also visit our website for all the show notes at cables to clouds dot com .
Thanks again for listening and see you next time .