Ep 11 - Using AI for cloud networking

00:00
Welcome to the Cables to Clouds podcast.

00:15
Cloud adoption is on the rise and many network infrastructure professionals are being asked to adopt a hybrid approach. As individuals who have already started this journey, we would like to empower those professionals with the tools and the knowledge to bridge the gap. Hello, hello, and welcome back to another episode of the Cables to Clouds podcast, where we bring you the world of hybrid cloud networking. My name is Chris Miles, and joining me today are my beautiful, lovely co-hosts. I'm so happy to see them.

00:44
every two weeks. Tim McConaughey and Alex Berkins. How you doing guys? Tim, what's up? What's new in your world? You just got back from the pool I heard. I did, until I got the pool hair going on. It's summer, man. It's summer finally in North Carolina. It's been weird here. The weather has been strange because I think this is the furthest we've gone into June for a long time where the mornings are still in like the 50s and like the afternoons are just now breaking 80s.

01:13
is really really strange for North Carolina. But hey, it's here. Kids are out of school. They just had their last week of school. EOG is done and yeah, they spent the last week just staying up late and playing games, watching videos and sleeping until like 10 a.m. So I feel bad I'm gonna have to tell them to stop doing that crap pretty soon so they can actually do something with their summer. But for now, we're letting them relax a little. Nice. What's EOG again?

01:40
the end of grade, the standardized tests that the state is happy to do. Are they still on Scantron tests? Is that still a thing? I don't think so. I think it's digital now. It's all digital now. Man, I was trying to explain that to my girlfriend the other day, this Scantron, and she was like, she had no idea what the world was like. What about you, Alex? What's up? Yeah, Tim and I are having a pretty similar life, it sounds like. I don't know what the weather is like because this is my first real summer living here, but it's been awesome, man. The weather is just...

02:10
Like after 9 p.m. when the sun finally goes down. It's, ugh. It's amazing, dude. This fricking smoke, that smoke from all the fires from Canada is still, I was out at our observatory last night, it's like, I'll have to share pictures, man. You cannot even, the mountains, usually you can see so far away, it's so hazy, still, like a month later. And it's just. Wow, I didn't know the smoke got all the way down there. We'd leave early and come home. Oh yeah? Yeah, it's everywhere. Oh, dude. Yeah.

02:39
And you're going, you have a trip planned this week, right? You're going down to... I do, yes. I'm going to the inaugural GA Nug Network Users Group kickoff where one of our past guests, Will Collins, is actually doing the keynote. So I'm going there to heckle him a little bit. And it's like one of the closest big cities to me. So I figured I'd go give out some swag, some stickers and stuff and talk to anyone that wants to meet up and hang out.

03:08
That's really cool. Good old Billy Collins, as we call him. Tomorrow is Father's Day in the States. So happy Father's Day to all the fathers out there. Yeah. You'll be hearing this well after Father's Day, but just know that when it mattered, we wished you a happy Father's Day. Yeah, that's awesome. So if you're going to be at the inaugural GA NUG find out, you'll have some.

03:32
I'll have some stickers. Actually, I don't know why I'm saying this. This will have already passed by the time this comes out. Oh, that's true. But if you were there, hopefully you saw him. Hopefully you got some stuff. If not, hit us up. We'll definitely send you some stickers. Or if you just want to chat, reach out to us, cablestoclouds.gmail.com. We'll be happy to send those your way. Yeah, as for me, I've been.

03:58
I'm envious of you guys. You're talking about the sun going down at 9 PM here. The sun goes down at about 5.30, 6 PM right now. But I really can't complain. The winter here is so mild. Like, I'm wearing shorts right now. Like, it's fine. There's not much to complain about. But yeah. Yeah, I was going to say, you were talking about feeding birds earlier. So I mean, it must not be. They don't fly north for the winter there?

04:25
No, they very much stay right here. If you're watching on the YouTube, you might see some cockatoos come to this window because this is about time they really want to get food from us. So they'll come over here and just start banging on the windows. I asked for this. I wanted the birds to come to the window, but we'll see what the repercussions are. Cool. All right. So we have an exciting topic today. Very topical content.

04:54
Hopefully, we don't piss too many of the prompt engineers out there in the Twittersphere off today, but we're going to do our best. So, the topic today, we want to talk about how effective is AI in designing cloud network architecture, right? So it's no question to anyone that AI is a very prominent thing right now and very kind of game-changing for a lot of people.

05:24
It's a little kind of remedial task that we don't enjoy doing. So it's definitely taking some of the small tasks from us. But we want to focus on how is it going to be with taking the big tasks from us? How is it going to be with actually designing some large scale thing end to end? So just to kind of set the stage for this experiment we did. So we wanted to see how well it...

05:51
ChatGBT is what we used, obviously. ChatGBT 4 specifically for this experiment. So we wanted to see how good the responses would be in designing large-scale networks in each of the major CSPs. So we did AWS, Azure, and GCP. And basically, the premise was we

06:16
provided a simple prompt, which is just, you know, we gave very little context about what we wanted. We just made a kind of a blanket statement, which we'll get into the prompts as well, what they were for each CSP as well. But the simple prompt is basically just, you know, not providing a whole lot of context just to see, just some generalizations to see what we get back. And then we gave it a detailed prompt as well, which was kind of very pointed, you know, hey, we wanna use this product. We need, you know, separated prod, non-prod environments, things like that.

06:46
We wanted to see what the detailed responses were as well. So we will get into that. Some of the caveats we want to provide to this just for the listeners out there is, we use chat GPT-4 as I stated before, and the data that is used for chat GPT-4 is only relevant up to September 2021. So obviously there's been a lot of products and things like that that have come out since then.

07:13
that weren't represented in that and we will definitely take that into account into our in our responses and Another piece is when we did this each each prompt was a separate chat So it's not like we did, you know, we used a single chat to send the simple, you know query and then use that same chat to send the the detailed query because we know that chat GBT can use contacts from your previous chats to

07:38
to formulate the response. And we wanted to make sure we were getting the cleanest thing possible. So we used separate chats for all of these. And we, like I said, did AWS, Azure, and GCP. And we will also be publishing this document for the listeners to go out there and read. So you can kind of see the full detailed report in a document. So just have a look at the show notes for that. OK. Any other comments from you guys before we get into it?

08:07
No, I think what we need to do is dig in on it and yeah, we'll just kind of go in line comment I think based on what we get back. Right, right. Yep. There's a lot to talk about. Yeah, there's quite a lot to talk about. I mean, there's quite a lot in the responses that we got as well. So we're not going to bore you to death and just read this line by line. But like I said, if you want the document, it's out there to follow along with. So let's get started. So.

08:36
First one we went out, we went for the big boy AWS, right? So let's kind of get into the details of the simple prompt that we provided. So, and I will read this one word for word and just understand that the simple prompt did not really change per CSP as we went along. So let's see, AWS, we said, put together an AWS networking solution for a design or a networking solution design for a company called CornHub. They provide a streaming service, a vegetable streaming service.

09:05
Their customers are mostly in the Eastern US and Eastern Australia. They need multiple VPCs deployed with best practices for several three-tier apps. They also need to provide connectivity to their existing on-premises network as well. And we got a lengthy response, kind of bulleted items talking about separate things like how to set up the VPCs, application load balancers.

09:31
Auto scaling groups, Amazon RDS, oddly enough, was chosen for the database tier with that little context that we provided. Direct neck, Transit Gateway, et cetera. So I'll throw it out to you guys. How did you feel that this response was that we got back from GPT? Yeah, so like Chris mentioned, so this basically broke it out kind of into each little bulleted points for like different parts of your design. So like you said, VPC, right? It kind of basically, I'm gonna kind of summarize it.

10:01
basically said you're gonna create two VPCs in each of the regions, right? Because we said one region's in Sydney, and another's in Northern Virginia. So each VPC will have multiple subnets, public and private, right? That makes sense. That's kind of along like best practices for AWS. Same thing with the NACLs and the security groups, right, to control traffic. I don't think there's much to call out in the VPC section. That's pretty basic. I mean, it's very basic, in fact,

10:30
Where are your customers or where is your solution gonna be hosted and then bam, one VPC in each region, super, super basic. Yep, absolutely. And then the ALB section is super short. It's literally just one deploy and ALB in front. Yeah. Yeah, it's interesting that it says just an ALB will load balance in front of multiple targets such as EC2 instances.

10:57
It's just saying use a load balancer, but like it hasn't actually provided any real solution, right, besides just, hey, you should use an ALB. Yep, and then, you know, it actually makes sense the order they put it into. I do want to call this out because the next one is ASG's, or auto-scaling groups. Yeah. So it makes sense. Like this is a good spot to put in ASG's. Again, it's basically just saying use them. There's not a lot of detail and it's not expected, right? This is just a simple one. What I think is interesting though is

11:26
And you'll see as we go through this kind of again, line by line, but summarizing and not just reading the prompt, right, or not reading the response, is that each piece that it calls out as a part of the solution is contextually standalone, right? So like it says ALB, it says use an ALB, use an ASG, but like there's no context of how these two relate to each other in the context of the overall solution. So I think that's important. Yeah, like I mean, it is a good point.

11:56
it has broken them down into all these separate bullet points. And yeah, they don't tie them together, but at least it's got the elements that you would want, right? Like it's not the holistic solution, but it's all the different pieces you might use for a solution like this. It just doesn't tie them together in any way. That's like the Ikea solution, right? Like, you know, here's all your pieces. Yeah, it's basically like they said, like, okay, build me a cake and they're like, okay, well you need.

12:21
Here's eggs, here's flour. You know, they gave you all the ingredients, but didn't tell you like what the sizes or anything like that. They just kind of threw it all at you. So yeah. Yeah, I love it. That's a good analogy. The RDS thing is interesting. And you mentioned it, Chris. I think every one of us picked up on how interesting it is that ChatGPT assumes, not assumes, but like that it specifically says RDS is the solution you should pick for this.

12:48
And I don't think there's anything, I can't pull anything out of the prompt itself that says like why RDS is the right solution. This might be one where it might actually be useful to, also it's extremely short, to just say like what it said and maybe in the context of the response, it can tell, we can pick out like, why did it pick RDS over one of the others? Yeah, cause you can respond. So like if we had this live, like the prompt, you could just be like, okay, why in section four did you choose RDS?

13:18
And the only thing I saw in the prompt is that we called it, it's a streaming service. But again, is RDS, like I don't know the answer, but is RDS the best kind of database for streaming service? Yeah, right. I mean, it also probably could tie back. I mean, obviously it does tie back to this, the data that was used to draft these responses, right? So who knows if it's just going based on all the AWS documentation out there that says, RDS is a viable solution for your database tier. When we said best practice with three tier,

13:47
So I feel like that probably plays into it as well. Yeah, it's hard to say, but it is a very interesting decision nonetheless. All right. And then so Direct Connect, this is, yeah, obviously, right? Of course it's going to say Direct Connect. On-prem. Right. There's not a whole lot of detail there either. I'm actually surprised. So well, I guess not. So Transit Gateway was what, 2018 or 2019? Oh, it was out. Yeah, in 2021 for sure.

14:16
So it does call out using Transit Gateway, which is awesome. I don't think things like, maybe it's inter-region peering, stuff like that. Maybe some of the functionality might not have been here. There's no, well, specifically there's no context. Again, here's your bread, flour, and eggs about the fact that if we're gonna connect these to...

14:37
regions together, for example, if necessary. Like how would I use the TGW to accomplish this? Or do I need two different direct connects, one in each region, for example? So that's a good point. I don't think the DX gateway, I'm not sure the age on the DX gateway, because the DX gateway could split your VIFs across regions, for example, you'd only need one. Right, it doesn't even mention a VIF at all, right? Right. So it doesn't say what kind, right, yeah, so.

15:03
So there's definitely some, and I don't remember how old DX Gateway is, so that's probably definitely something that should look up, but it's interesting that it picked, it just says, like, you should use Direct Connect, but it, again, contextually forgets to mention that, like, you can only have one per region. Right, yeah, I feel like that's also an interesting point about AWS Transit Gateway, because if you think back to the VPC design that it proposed, there's only two VPCs, so you're telling me you're going to have two...

15:33
two AWS transit gateways because they're regional constructs, right? And then connect those together and they're right. So you can make other VPCs, right? So that's like, I mean, don't get me wrong. I guarantee there are customers out there doing that today. I'm sure that's actually a common deployment because, you know, absolutely everything under the sun for sure. But like it's kind of common to think like, yeah, we need two transit gateways for two VPCs. Right. That's just.

15:58
That's funny. I mean, it might be necessary in the context of the question, but it's, yeah, they don't really mention that. Just again, it's not like we're not punching down on the AI here. We gave it a simple prompt. It gave us kind of a very contextless answer. But I do think it's important for overall to point out that the context of all of it is missing, which is.

16:23
and we'll have to dig in on the detailed and see if there's any contextual information there. And it only matters in terms of like how you would use it. Cause if you didn't know about DX Gateway or that you'd need to peer Transit Gateway as a regional construct or whatever, you would read this and you would try to build it. And you'd lack a lot of the detail needed. Right. And I got to call this out real quick. So in the AWS Transit Gateway one, it actually says it offers enhanced security, which is like,

16:52
I mean, what doc did they pull that out of? I mean, where are you getting this from? Right. So I guess the VRF Lite type of route table segmentation is this maybe what they're thinking about. Like I'm sure some, some database doc out there says that's a security feature. Hey, and if that's the context, NAT gateway provides enhanced security. Yeah. Oh yeah. I'm sure that'll ruffle some feathers. NAT is security.

17:21
Sometimes it shouldn't be, but it is sometimes. Another point I did want to make here is, and we'll kind of talk about this later on the other CSPs, they actually do call out a CDN service, such as in this instance is CloudFront. And they say, CloudFront can be used as a CDN for streaming content, which we did mention in the prompt, this is a streaming service. So that was good to see that they actually used a CDN, because as you'll see later on,

17:49
that doesn't remain a consistent point that they propose. Well, they also mentioned Route 53, which is interesting. So there's actually a little bit of context there, right? A little bit of cross-service reliability. I mean, it makes sense to me because, I mean, at the end of the day, if you are going to have this simple solution, and if it's a greenfield solution, there's no preexisting things that you need to consider.

18:17
Why wouldn't you go whole hog and just use everything under the sun from AWS that you could, right? The Route 53 integration is going to make your life a lot easier if you can use that in every facet of what you're deploying in AWS, right? So I think that is a good point that they made. So I was... It's funny actually, now that I look at this, Tim, your comment made me think of this, but that's... So line seven is... Number seven is the first time they actually combine products,

18:47
actually all do the same thing. So it's kind of like it was getting smarter as it was getting near the end and adding more products in at the end. So it's kind of cool to see. I think that these particular products also, like WAF and AWS Shield are tied at the hip, right? They're not on, you know, and we're getting ahead of ourselves, but I, and I do agree with you, but I do think the more tightly bound the AWS services are in terms of the documentation, in terms of how they market it and all of that, then the more likely it is, I think that

19:17
The AI will pick up on that and use it, I think. For sure. All right, yeah, so we'll finish this off. So seven, right, CloudFront and Route 53. Eight is AWS WAF, Web Application Firewall, and AWS Shield for DDoS protection. And then the last one, of course, monitoring and logging, right? So I actually like this. It's obviously like they pulled it from some marketing page, but it describes why you would use CloudWatch. So there is actually some kind of context here. And then...

19:46
It adds in CloudTrail as well, right? So it's kind of cool that they added in and there is a little bit of context at least here. Well, what I think is really interesting is I don't think your prompt asked for monitoring and logging or any kind of- Nope. But like, yeah, I don't think there was and we'll talk about this later. I don't think monitoring and logging is even mentioned in the detailed response. So it's kind of interesting to see

20:16
that they proposed that without being prompted. Yeah. But that being said, let's maybe let's- Do we wanna- Yeah, let's go into the detailed prompt. Do you wanna have a look at that? Yeah, let's see what happens. We give it a little bit more to work with. Yeah, so all right. So let's get into what the detailed prompt was. For the first one, for AWS, I will read this ultimately word for word, but just understand, you know, later on we'll touch on that this doesn't really change except for maybe some product names and things like that. So.

20:42
For the detailed prompt, we said, put together an AWS networking solution designed for a company called CornHub. They specialize in a vegetable streaming service offered to customers around the world. Here are the requirements. Two regions will be used, Eastern US and Sydney, Australia. AWS Transit Gateway should be used to connect all the deployed VPs together. AWS Direct Connect should be used to provide hybrid connectivity to regional data centers in Ashburn, Virginia and Sydney, Australia. Each region should contain prod and non-prod.

21:08
environments to house public facing applications behind ALBs or application load balancer. These VPCs should follow a three tier app architecture. Prod and non-prod environments should not be able to communicate to each other and the data center should be able to communicate with each other over the direct neck links as well as be as well as use their as backup for the MPLS network. So obviously you can tell what we provided a lot more details about what we wanted from this design in this end.

21:36
The responses, yeah, they were still formatted, pretty much the same, all kind of bulletized items with specific product and construct designs. But yeah, so what did you guys feel was interesting about the detailed responses that we got? I will say that I gotta call this out before it gets lost and I don't know when to bring it back in. So the last point about, it's kind of like we were angling it.

22:05
for talking to each other over Direct Connect links. Now, a day is there is a service called Direct Connect site link. And that would be, I think, the obvious answer here. But that didn't exist at the time of this. So I just want to call that out as like, that's something that you're not going to see in the detailed response and I wouldn't expect it to. So it's interesting to see the other way that it would solve it back then. Right. Yeah. So, I mean, for the most part, I would think, having looked at the detailed response, I mean, it did more...

22:33
overall with because we gave it more to work with, for example, the VPC thing, obviously, if we say prod or non-prod, it splits it out. It says you have a VPC that's prod, you have a VPC that's non-prod. Otherwise it's exactly the same. That was like, because we gave it a different requirement, it took a different action, right? But I think, but it also mentions something like here's network isolation down there, which is not tied to a specific product.

22:59
Up until now, they've really only been talking about products and services. So it was interesting that they, here's a bullet point where we talk about a requirement, which is network isolation, but they don't specifically, they don't list it like a service. You'll see it when you, when the readers, when the listeners see it, see the prompt, maybe you can see that. I don't know if it just, you know, knew that was a requirement and didn't have anywhere else to put it, or if that was like a conscious, I say conscious, but you know what I mean? Like that was a decision made by the model or what? Yeah, right. Hey, hey, it's coming.

23:29
It's coming. I got to add for the VPC design section, they did actually say you need a public subnet, an application subnet, and a database subnet. But again, it's weird after Chris, your comment. There is nothing mentioned about NAT at all. It's just not on here. Like it doesn't need to reach. It's amazing it didn't pick up that NAT is the best security you can have for a VPC. Actually, now that you pointed it out, we're creating.

23:59
Now that you pointed out, I can't unsee this though, because we created this, it says create these VPCs, right? So these are not default VPCs. Internet gateway is not mentioned anywhere. Yeah. Yeah, right. That's another one. There you go. Yeah, good call. So the IGW, without an IGW, yeah. Yeah. It's also interesting that in this context, we didn't get a recommendation for Amazon RDS. There's just the call out of a database tier.

24:27
That's really the only thing that was mentioned with regard to that. They mentioned use of AZs, which is for high availability and fault tolerance and things like that. So that is good. The one thing that was most interesting to me was that, like you pointed out, Tim, there's a section specifically labeled for network isolation for this that is drawing a correlation to the requirement for separated prod and non-prod environments. They recommend using...

24:56
Network ACLs to restrict this traffic. It's not even saying anything about security groups or anything like that. Just network ACLs and then no mention of the separated security enhancement that they mentioned before about what you can do with Transit Gateway. Because the Transit Gateway is just talking about connecting all the VPCs together, but there's no talk about separation of environments, which is a very normal thing that you can do with.

25:23
TGW, right? So it was, I felt that was very interesting. That's true. The response about AWS Transit Gateway doesn't mention security in this one, which of course route table isolation is how you would isolate those two environments from each other with the Transit Gateway. So that's interesting. And so, this is also funny. They actually added VPC, like modifying the routing tables in this one, but that was not in the simple. So obviously that's something you're going to have to do. Again, this is another one that's not a product.

25:52
But it's something that you have to think about when you're doing these designs. So that's just another example of something they're calling out. Yeah, it's weird. They mentioned the VPC routing tables, but only the specific VPC routing tables. They don't talk about the TGW route tables. They just say, make sure you... Which is kind of a point that we wanted to cover in this, that we were very much focusing this discussion on design. Because we're not making, we're not necessarily targeting the argument

26:22
AI can replace implementation engineers or something like that. Because we're not asking it how to do these things. We're asking it to design it. And it's funny that they do get into the detail here. I like how we're referring to it as they, the AI is the perpetual they from now on. They respond and it's like they're talking about implementation details. Like this is a very small thing that you need to consider when implementing TGW, is making sure your VPC routing tables are updated.

26:52
And it's just, yeah, that's a good point, Alex. I didn't even think about that. They are getting into the actual deployment piece here, which is pretty interesting. Yeah, which they didn't in the simple either. Again, the simple was very much just a roundup of services needed to accomplish the objective, right? This is actually getting much more into the weeds, for sure. This next one might be my favorite. Oh, the VPN connection one? VPN connection. It just says.

27:20
Create VPN connections over your Direct Connect link so that if it goes down, you can fail over. If your Direct Connect goes out, you can fail over to the VPN. Well, I like that you can, if your Direct Connect, create VPN connections over your Direct Connect so that if your Direct Connect goes down, you can fail over to VPN. I see no fault in the logic there. That's amazing. Oh man. I hope, oh my God, that is the funniest thing.

27:49
And then you're gonna see a reddit post in the future. I hope somebody's setting this up. I Don't know what's happening guys Yeah, it's like a GPT told me to build the VPN over the direct net But as soon as it goes down everything's down, I don't get it. Yeah, they won't fail over. I can't get it to fail over guys That's really awesome. I didn't even notice like I hadn't read to that level I you know, it's scanned everything kind of read through it and my eyes just like went right over that but that piece

28:17
right there with building the VPN over the thing that we're failing over. That's beautiful. I love it. That's a really good one. Yeah, that's great. So, so they do mention security groups next. So I guess it was just in a different spot, but it's weird because they lump security groups and knackles together outside of network isolation. It's like they, I don't know, they tried it two different approaches. Again, secure, almost like a service roundup, right? Like that's how they're bulleting is like a service roundup. They can't decide if it's

28:46
it should be, like where it should go, right? It's more like a service roundup. But the data encryption thing is interesting to me because I don't recall the, did the prompt ask for data encryption? I don't think so. At rest or in transit? This is almost like the monitoring and logging recommendation in the simple one is that it's kind of out of left field. Like we didn't ask for this, but they're just kind of throwing it in. Like, is it a bad recommendation? No.

29:13
but we didn't ask for it. So it's just interesting that we got it like this. It's an interesting recommendation as part of this, especially because we were detailed about specifically what we were asking. Right. So yeah, it's basically, I mean, to sum it up, it's just saying, you know, use AWS Key Management Service or KMS to handle encryption of your, encryption and decryption of your data at rest and in transit. So yeah, it's like, we didn't, we didn't prompt it for that at all. So it was just kind of funny that it threw it in, but you know. And not only that, but like.

29:42
How much is missing from that comment? Oh, yeah. There's so much that. Yeah, they just, one sentence, so everything's encrypted. I'm sure it's that easy. Yeah. Just use KMS and it'll all work. Yeah, for sure. So like we said, with AWS, we would just kind of show you what the prompts were. And through the other CSPs, so Azure and GCP, we kind of kept the same format, right? Didn't change the prompt too much, maybe swapped in some.

30:09
different product names for the detailed one, but the simple one was exactly the same. So put together an Azure networking design for a company called CornHub, all the same requirements, two regions, three tier apps, and connect on-prem. It's really all we said. And I'll let you guys expand upon this, but it was kind of funny to me to see, especially for Azure, how different the responses were in terms of...

30:36
the like how the products were labeled and how the recommendations were laid out. So yeah, let's, let's get into that. What are your guys comments on the simple response for Azure? The very first thing was with the V net design, of course, V net, VPC, whatever the V nets are like VPC and AWS and Azure. They actually mentioned this and this is interesting because it's only now that they pointed out at that and that it wasn't in the AWS one that I'm I'm like, oh yeah, well, because there's so many little details that are left out of the whole thing, right?

31:04
They say choose an appropriate IP range that doesn't overlap each other or your on-prem network. That is an extremely important thing to do if we're talking about connecting all this stuff together, but it's completely missing from the AWS one and it's here just randomly. Again, nothing we asked for. It was very simple. We didn't say, anyway. No, you're right on. And the Azure one definitely does give a lot more of this nuance. Like you mentioned the IPs,

31:34
also network security groups, and it actually says why to use them. Right, it's like, it feels like, security requirements were not part of the simple prompt, but it actually says to use them for security to restrict access to other resources per your security requirements. Yeah, well, I mean, what do you think, Chris? I did feel like, I felt like it was a weird mix between like really, really useful information like we just talked about, and also just kind of like really high level like marketing level crap too, is kind of a weird split.

32:04
That's exactly the feeling that I got. It's like, I wonder how this correlates to the data that's out there for this to consume from AWS versus Azure. Does AWS have a ton more content out there just talking about the products and what they do and not getting into the meat of it? And then maybe Azure has kind of the same thing where they touch on the marketing piece but also get into the fine details on.

32:33
you know, make sure you do this or else you're gonna jack up your design, things like that. So it makes me wonder how that correlates to the actual data. But one thing that was funny here, what we will see in the detailed design or detailed prompt what the difference was, but they just flat out went and said like, yeah, just use VNet peering. Like, just peer the things together, you know, don't throw any kind of like Azure VWAN or anything like that in there.

33:02
proper contextual reasoning to maybe put that in there. So it was funny to see it recommend that. How old is VWAN? I'm pretty sure VWAN was right around the same time. TGVU. Okay, was it 2019 or something? I'd have to go look it up. Sure, it was definitely out by this time.

33:24
Okay, I wanna, cause I agree with you. I wanna give it that point. I wanna give it the check mark. And then I'm like, well, when, when did VWAN actually become a thing? When was VWAN GA? September 2018. So yeah, it was definitely. Oh, okay, wow, okay. So I'm, yeah. All right, well then awesome. Then yeah, like you said, Chris, like it actually understands that, hey, we've only got two VATs here. Why are we gonna involve something like VWAN? Yeah, it's like, I don't know. Because I've been doing a lot of reading of documentations lately between the multiple clouds.

33:53
Part of me wonders if this is because Azure has so much more detailed documentation than the other clouds it feels like. If you ever look at their cloud adoption frameworks, Azure's is like 4,000 pages. AWS is like 100. There's a huge difference. So I don't know if there's just more data for GPT to parse through or something. Maybe that's part of it. We do know that. I mean...

34:16
AI is as good as its data, right? The models are as good as the data. So maybe we're seeing an effect of that, right? Like just right in front of us here. Yeah, and it does seem like a lot of the, like, you know, in my day to day, when I'm going to Azure documentation to, you know, recall how to do something or look up how to do something, it feels like they offer, typically they offer a lot more context about the solution and like.

34:42
pointing out the caveats, and then they show you like three different ways to do it. They're like, hey, here's the click ops way, here's the PowerShell way, and then, you know, even like another audio. There's a lot of that, that's true. Yeah, so it's like, I wonder if that kind of adds to it as well? Yeah, I don't know, it's hard to say, but yeah, go ahead. I think to answer that one, we'd have to actually ask how to do it, and then see what BGPT spot out.

35:07
Cause like you said, they show you multiple ways to do it. So I'd be curious to see which way it would recommend. So that might be a future. That's a good point though. Yeah. So, and another one that this was kind of, I noticed this like right off as I was, you know, skimming through this, but the Azure one does not recommend a CDN. It actually recommends traffic manager. And just based on our last point about some of like the VWAN versus VNET peering.

35:34
It makes me wonder if because you, I think you specified in the prompt that the customers are only in those two locations, it didn't need a CDN, right? Like it's weird, it is like, maybe it's taking these things into account. Yeah, it'd be good to know that, you know, hey, the size of the deployment is so small that a CDN doesn't make sense. Like, is it thinking about that or, you know, is it that? Well, again, I say thinking, but obviously we're not, you know, you gotta use some words to, you gotta use some words here, right?

36:04
try not to ascribe too many human characteristics to an AI model, but you know what I mean? Is it considering the size of the deployment? I guess to be more accurate. Yeah, I will say throwing in the Azure firewall definitely seems like a marketing play here because I don't know. It's just weird that it threw that in there just randomly and then. We didn't ask for security, like you said. Like we never asked, the prompt did not say anything about security, so that's a good point. Although in Azure.

36:29
Things are a little different, right? Because in Azure, you can go straight out to the internet. You don't actually, there's no workload security, or whatever, like you, maybe that's why. Yeah, but we also- Yeah, but also control outbound internet access. Also the prompt does say that deploy multiple VNets with best practices for several three tier apps. Maybe it's a best practice question. Who knows how the best practice gets affected into it. Yeah. Who knows? Yeah, some very interesting decisions here for sure.

36:59
I do love that they added the private DNS zones. And they added the network monitoring on this one too. Yeah, they did. It's cool that they called those out. All right. Yeah, the private DNS zones. That, yeah. Yeah. All right, let's move on to the detail. We could spend a long time on these. Yeah. So like we said, all this will be in the document. So if you want to get into the meat of this and read it word for word, we'll have it out there for you. But let's move on to how the detailed prompt went. So.

37:28
As we said before. No, go ahead. Sorry, sorry, sorry. So with the detailed prompt for Azure. So basically the exact same one that we gave to AWS except we said, you know, instead of using TGW to peer all the VNets together, we chose Azure VWan. We set ExpressRoute. Ultimately, everything else is exactly the same. Proud non-proud environments should not be able to communicate and, you know, we want to be able to communicate.

37:56
as a backup to the MPLS over Express route. So, all right guys, let's get into it. What was interesting about the detailed response for you guys? To me, what's interesting is the simple response was actually longer for Azure. Same. And the detailed actually, instead of having bullet points, it actually has like regions and VNets, and then it's like one, two, right? It actually breaks it down more. There's less, but there's more, if that makes sense. More detail, but less total.

38:26
amounts of material, if you will. Like, it gets straight to the point and yeah. There's very little here actually. I mean, they're very specific. It's almost like the buckets are basically the bullets that things were categorized in are with a simple response, they were product focused. This one's more like construct focused at times, like talking about load balancers and security groups were lumped together. Regions and VNets were lumped together.

38:56
entire section. So it was kind of, I mean, there was ExpressRoute, VWin that had their own dedicated things, but the way things were kind of compartmentalized were a little different for sure. At least this one said, use ExpressRoute links to provide a backup for the existing MPLS network. I was going to call this a VPN. Don't build a VPN over the ExpressRoute. Yeah. As backup for the ExpressRoute. Yeah.

39:25
Oh, and sorry, real quick before we move to GCP. So I forgot to call this out. It's funny if you look, there's a couple spots where it actually calls them VPCs for Azure. Oh, really? I just thought it was funny. It's not a huge deal, but it's just funny that VPC is so like synonymous with cloud networking that it's even in the Azure one. Each region will host two virtual networks. VPC is as the acronym, that's interesting. Right. Wow, I did, yeah. Didn't even notice that.

39:54
Yeah, I mean, I think overall the Azure one's good. It is. Sorry to cut you off, Tim. One thing I thought was very interesting is the VWIN mentioned. As we know, it was definitely GA for quite a while by the time the data had run on GPT. But the VWIN response is very just marketing speak. It doesn't really touch on too much at all. It is.

40:23
Use Azure VWin to simplify your large scale branch connectivity. Like that is, that looks like I'm on the product page for VWin. That doesn't sound like it's any kind of design document or anything like that. So it was kind of funny to see that. There's no detail. Yeah, right. Like there's two sentences talking about why you do, no, is it one sentence? Two sentences on why you deploy it, but no real design capability piece to it. Yeah. Again, I...

40:52
You know, it seems like we don't have a list of services here like we did before. We do kind of a little bit, but like you said, it's like a little bit about the service and then more about like why to use the service or a little bit more detail. But again, the context is good. Like they, like when they're going to eat, like in context of like where to use it or why to use it is here versus like, you know, when we were looking at the AWS one, it was very much just like, here's the Ikea, you know, or the cake recipe. Here you have a little bit more.

41:22
more towards how to bake a cake, which is interesting. Yeah, totally agreed. All right, so last but not least, let's cover GCP, which as we all know, the way GCP operates from a networking perspective is quite different from the other CSPs. So I was excited to see how this one came together. So like we said, once again, simple prompt, didn't change anything, still the same requirements. So two regions.

41:49
best practices for three tier apps, et cetera, connect to on-prem. So, all right, guys, let's get into this one. So, yeah, I mean, ultimately, the way things were categorized were relatively the same. I feel like with this one as all the other simple prompts that we gave, but was there anything that specifically stood out to you guys as differentiators here?

42:12
I immediately picked up that, so I always felt like GCP is much more developer focused and I think their documentation is too and I think it's very, very clear. Remember the prompt we gave was just, you know, we need the streaming service with best practices. That was it, right? The very first thing they suggest with VPCs is set up three VPCs, one for development, one for staging, one for production. We did not ask for that at all. Yeah.

42:38
That's also, that was the very first thing I saw when I read this. I was like, oh, okay, yeah, this just is going straight into like the developer handbook of how to set up GCP. And especially caters to that. When they say the next point, each VPC should be set up to use auto IP mode to automatically arrange subnets in the region. Like as like, as, as networkers, I'm like, what? No, like do not do that. That just sounds like it's going to be bad, but. What's funny about this one though.

43:06
If you tie in that point about using auto mode, auto mode sets up a subnet in every single region. But in the very next point, it says each VPC will contain three subnets. So which is it? Are you using auto mode or are you using custom mode to set up three subnets? Yeah, it's like already contradicting itself. Again, it lacks the again, we have like a list of ingredients without the context, right? Yeah. Like each each one is an answer only in so far as.

43:34
that particular bullet point, right? Like it doesn't relate, like you said, right? Like if I use auto mode, I'm gonna get a subnet in every region. So you can't tell me now I need three subnets, right? Because I've already, yeah, anyway, so that's another good point. And then you got like things, just this last sentence for point two is the database subnet should have the strictest rules only allowing necessary traffic from the application subnet. Again, that's like.

43:57
developer, I mean, I guess it's for everyone. Yeah, I feel like that came from the best practice prompt as well. So it's weird. Well, not weird. It's interesting to see how that was interpreted differently by each, within each CSP, I should say. Also funny with interconnectivity and VPN, it doesn't flat out say use the hybrid circuit.

44:24
offered by the CSP. It says, you can either use Cloud VPN or Cloud Interconnect can be used. And it says, if you have this requirement from your on-premise network to have high throughput demands, then you should use dedicated interconnect or partner interconnect, which is funny that it made that distinction. It's the only cloud that did, actually, that said, hey, that's a last resort, essentially. Well, actually, it's the only one that specifically said,

44:53
if your requirement is this. So it had the context of like when you would use one and when you would use the other, which is interesting. So, yeah. I agree. Thanks, sorry. This one does call out load balancing in CDN. So that's good to see. But what's really interesting to me is this fifth one is actually enabling private Google access and CloudNet.

45:18
There you go. There it is. Hey, Chris, we got you. Cloud, we got Nat. This is the most secure solution by far. The private Google access, I don't think that was something that's specified. We never said that we needed to reach Google APIs and services, especially from internal. That's not called out anywhere. It's like a private link basically. Yeah, exactly. It's a private link is what it is. Yeah, that's a very good point. We didn't specifically ask for that anywhere. Yeah. Anyhow, this one- Again, best practice maybe. Maybe it's- Yeah.

45:47
when in the AWS and Azure one, we got these things offered to us that weren't in the prompt like monitor and logging. The GCP one actually gave us the most of that content, like the things that we didn't really provide any context around, but offered up as part of the, I'm assuming the best practice solution. So yeah, that was kind of cool to see that come out of this. It is interesting to, so for example, set up CloudNet.

46:15
to enable instances without an internet IP to reach the internet. So Azure, this is not required. So it's hardly surprising they didn't say that. I was surprised they didn't mention anything about it. Even the IGW itself in AWS, it was completely absent from all of the recommendations. And nothing about any gateway. Any kind of gateway. Any kind of internet access. Yeah. Yeah. So that's a, I would mark that as a big fail.

46:40
on the AWS GPT part. And I would wonder why, because I've read enough documentation from AWS to know they cover things like ingress architectures or egress architectures. And it's not like they never say the words IGW or NAT gateway anywhere. So it's, anyway, again, just, it's interesting to see how the model interprets and then what it chooses to include or not include. Right. It's funny too, because the...

47:07
We haven't really touched on this with the previous prompts, but as each one of these will at the end kind of do like a final thought or like kind of like summary. Like a roll up, yeah. Yeah, and this one points out specifically that the configuration should follow the principle of least privilege, which is this is the first one to mention that, I believe throughout this entire thing. That is true. Which was funny, because yeah, like the Azure one, says this networking design, the detailed one.

47:35
says what the aim of the design was. But the simple prompt from Azure was like, remember to consider these other things that we haven't really mentioned like scalability, security, high availability. So it was kind of funny that they, with the simple, at least in Azure, with the simple prompt was like, here's what we put together. But hey, you need to consider all this stuff. And then the detailed one was like, this design was built to do this. So it was kind of weird how that was laid out.

48:05
in that aspect. Let's go through the detailed GCP response and maybe just a quick roll up about what do we think about it and that'll be good. So this one, like I said, pretty much exactly the same except we mentioned dedicated interconnects specifically in GCP's network connectivity center, which I'll be honest, I don't even remember when network connectivity center was GA'd. This is what's crazy to me is I thought it came out after 2021, but I guess obviously

48:34
They know something about it. It's in the prompt. I mean, it's in the response. So it must have existed in some respect. I think what was weird for me is I didn't know much about NCC prior to the integration between Cisco's SD-WAN and NCC. I think that was the first real exposure I got to it. So that may be why I think it was much younger than I thought it was. But yeah, let's kind of...

49:02
But even then they don't cover very much about it. True. Yeah, very true. Yeah, this one again, the detailed response for this one is kind of like the Azure one where it's shorter and it's so straight to the point that there's not a lot of nuance except for the number six. But we'll get to that in a second, because that is super interesting. But for one through five, it's pretty generic. It doesn't seem like there's anything. It's extremely straightforward. Yeah, very, very straightforward.

49:28
Again, they chose in this one to do the network isolation piece as kind of its own, as its own little bullet point. But it just says like, okay, use security groups. It's interesting that I didn't see any where it really even mentions Google Firewall, which is the, like the NaCl thing in Google. I don't actually see it anywhere in here. I'm kind of surprised. Yeah. Again, we didn't ask for it. We didn't say use the security. But

49:55
You know, they go to mention network isolation using security groups. So like, why wouldn't you, you would have thought it would include a Google Firewall. So anyway. Right. All right. So the last point, who wants to take it? Go ahead. That's yours, baby. Take it. Yeah, this is, this is cool. Um, cause this is a solution that, um, there's a lot of actual context in this one. So it's basically says, um, your data centers in Ashburn and Sydney will be able to communicate with each other over the dedicated interconnect links, but.

50:24
How's that done, right? It's like, there's a lot of ways to set that up. So it actually kind of gives a bit of detail and says it can be achieved by setting up cloud router and BGP for dynamic routing. Which is cool, like it doesn't go into super detail, but it's cool that it calls that out as a solution. None of the other ones did. I don't think there's been any mention about BGP. There's been no mention of BGP in any of the other ones. No, not at all, yeah, no. This is the only time it comes up actually in the entire, in the entirety of it. This is the only time it comes up.

50:52
Which yeah, if you work with cloud networking at all, it's you're going to be using using PGP somewhere. Yeah, I mean it is good that it actually goes deeper than the other ones. The other ones just mentioned like well, of course, obviously we've already kind of beaten that horse on AWS to death that it's a completely wrong solution anyway. But the Azure one is just like, you know, yeah, you can use your express routes to to be backups, right? Like that's all it just assumes it's all going to work out. Everything's gonna be fine. This one actually does go.

51:22
further than the other two, which again, GCP having a lot of developer-focused documentation, I was actually surprised to see this level of information on the networking piece, actually. So. Yeah. So yeah, that was one thing that is very interesting to me. Because if we're thinking about this holistically, let's say you were trying to evaluate this for your company. And we want to see how valuable this stuff is to the actual people doing the work.

51:52
Like it's funny to think about this in the context that you are an architect and you're like, okay, well, you know what? I'm going to put my details of what my company needs into GPT and say, you know, write me a design for AWS, Azure and GCP. How much this could influence your decision on what you do, just, but like would not be really reflective of whether or not the CSP is the best for the job. You know what I mean? Like this, this kind of frames it in a completely different way.

52:21
which I found was interesting. Well, and like all things with chat GPT, and very quick aside here, I actually started using GPT not for work. I'm learning Japanese and I actually was curious, like how could chat GPT help me that with that? And I started having it check my grammar and stuff like that. And what I found was, 80% of the time it was good, but the times that it was wrong,

52:49
I could get in real trouble, you know, not knowing, not like he was telling me like bad things to say or anything, but what I mean is me not knowing better than the model. If I didn't know better than the model and I took the answer, I would be wrong and I would actually build a foundation of wrongness and continue to learn the language in the wrong way building on top of that. And I think that's very true here as well, right? Like a lot of this stuff and it's been said ad infinitum.

53:17
that like, hey, you know, all of these models, they can be wrong, but they have this extremely annoying ability, probably modeled very much after humans, where they don't know they're wrong and they think they're very right and they're willing to die on that hill or not. You know, sometimes you're like, hey, that's wrong. And they'll be like, yeah, actually you're right. I'm completely full of shit, right? I was gonna say it'd be really cool to do like a multi-cloud prompt in here too and see what it comes up with.

53:45
that's not so vendor neutral. We need to come back at this in six months and see, or whenever they release a new model with the new stuff in it, and try this again or expand on it and do a follow on, I think. Yeah, I think we originally wanted a multi-cloud option in here as well, but as we can see, then we would have ran well over the one hour mark for that. So we'll have to come back and try that. But yeah, any other?

54:14
So yeah, I know we're close to time here, and this has actually been really fun. I think we've kind of uncovered a lot. Because just as Tim pointed out, I feel like a lot of people are using AI right now to kind of help them build configurations and things like that. And very often, it does get small things wrong. Sometimes it gets very large things wrong. But I think we're seeing here that even in terms of design, um,

54:40
it's also getting things slightly wrong or just really framed in a weird way where like the human response to what this response from GPT is really is going to differ per the professional that's taking this and trying to make a decision on it. So it can very much get it wrong in this context too. But yeah, what are your all closing thoughts on this exercise we did here? Yeah.

55:09
I mean, I'm agreeing with both you guys. I think it's obviously a great tool to kind of get, like without context, right? It's like as a information gathering tool to, I imagine something, you know, you're like, for the GCP one, right? It gave you the four different options of kind of like interconnects you could do. It'd be cool to maybe not be so broad, be more specific and say, hey, what are my options here?

55:34
And then like, what is each one good for, right? It's prompt engineering, I guess. Right, it learns over time. That's the whole thing, right? Based on your previous asks, based on its previous responses, it gets context. Yeah, 100%. So it's, again, it's like Tim was saying, you don't blindly trust it, but I think it's super useful for getting ideas of maybe what's out there, how some other people might have solved some kind of situations, but it's not the end all be all. It's helpful.

56:04
but it's not going to be like, oh, well, yeah, here's, I need to design this architecture. Here's my answer. And just hand it over. There's still nuance. There's context missing as we went over many times. There's entire products missing. So use with caution, I guess. Or it's just freaking wrong, like with the VPN over Direct Connect. Like, God help the man that builds that and says, hey, we're redundant now. But that's also something that we

56:34
consider here is we're using this tool as someone, as people that ultimately already know the answers to what we're asking for, right? So we're evaluating the answers based on what we already know and how we would already do it. I think, you know, using the prompt engineering, as you say, like if I was someone that going into this cold turkey and didn't really know how these things operated, how these pieces of the puzzle fit together.

56:58
I'm sure I'm going to ask very... But I mean, I don't think I'm going to take it at face value. I'm probably going to ask to expand on each thing and it's going to give me more data, which could lead me further into the wrong direction or further into the right direction. So I'd be very interested to see how this would react if we were the person in that scenario trying to extrapolate more and more detail from each one of these bullet points.

57:25
That's honestly that's how it is with me in the Japanese learning Japanese thing because I am very much I don't already know the answers right and so I'm actually fact-checking a lot and saying, you know, whatever So I agree it is a different experience I will say that is a different experience when you are you know the answer and you're Analyzing it versus like you truly are trying to get help. I think it's a very different experience Yeah, I'll be honest part of me wants to go put in these same responses and ask

57:52
Like, why did you choose this or why did you choose this? Just explain more and give me some context. Yeah. We can add more to this document, man, before we publish it. We've got time, so let's do it. I'm going to go in there and ask if, I'm going to go in there and ask how good of a security solution that is and see what it says. Oh, OK. All the controversial comments we've made, we'll just add them all in there and see what it says. We should just, yeah, we'll just have a come after us for all these things.

58:21
All right, yeah, we should probably wrap up. We're getting a little long here, but this has been good. This has been really good. It's been good. Yeah. So we encourage anyone that's listened to this. Obviously, we know there's many self-proclaimed prompt engineers out there that are probably just so angry with us on how we've formulated these prompts, and that's fine. If you want to give us feedback, definitely reach out to us on Twitter, cablesclouds.gmail.com. Yeah, come talk about it.

58:50
We'd love to talk about it. We're just getting into this and as Tim and I said, we're using it within the context that we already know the answers and we're grading it that way. That may not be the best way to go about it, but we just wanted to kind of go through the experience and evaluate it. We definitely encourage anyone to reach out to us, talk to us about this thing. We love talking shop. So definitely reach out to us.

59:16
And I want to thank everyone for listening today. You know, like, subscribe, all that stuff. Go to the YouTube channel, hit subscribe there. If you want the video format of this, where you can see Tim's lovely background, his cool ever-changing posters on the wall, things like that. So if you want that, hit us up on YouTube. Like I said, we're always on Twitter as well. And yeah, reach out to us. We'd love to hear from you. Until next time, take care.

59:47
See you guys.