Yak Shaving, Live!

Hey, Ben.

Hey. Hey, Matt.

Um, I was just looking at our, uh, twos compliment org, uh, thingamajig

Yeah. Yeah. I, I was looking at that too.

I always type twos compliment.org into my browser. And it seems like Chrome these days does the automatically, Hey, you probably meant www dot that. Um, but if you don't do that, like if I use Curl or whatever, right. Uh, I get like nothing. In fact, I think it hangs, uh, let me just try. Now I know I'm doing that https compliment.org.

Yeah. I get a, I get a redirect if I use Curl.

So I'm literally just using Curl https s

Oh, that not doing OTPs. I'm just doing Curl Wells. Everyone does

HBS these days. All the cool people. True. It's just hanging.

Yes, true, true, true. But I'm saying like, if I just do curl twos compliments.org, I get a redirect.

Oh, okay. But yeah,

So I would expect, I think I would expect https

I Yeah, yeah. I see you are being redirected in. I see. Yeah. If you, if I use http,

Because there's nothing listening on port 443 for whatever that is. Right. Because I think it's

I don't even know how this stuff's done. I know that I check stuff into the GitHub and something something

Magic, magic happens. Yeah.

So should we try and fix that? Is that a thing

We could do? Um, well I think we should. I mean, there might be part of that. So like, like right now, the way this works is I think there's a cloud front distribution, and then when you check something in it like builds the podcast xml, the RSS feed, basically

the RSS feed. Right. Right, right.

And then, uh, you know, and the rest of the website, if it has any changes to load, it's all static. So there's nothing to build. The only thing that actually gets built is the RSS feed and then, you know, uploads that to an S3 bucket and then that, that cloud front distribution is backed by the S3 bucket.

Got it.

Uh, yeah, that cloud front distribution is backed by the S3 bucket and then you have to like invalidate the cloud front distribution to basically invalidate all the caches. Right.

Right because we are not doing anything clever with like cash busting URLs and things. And we definitely like, essentially index.html To always be like, just come and get it again. Um, I mean, I guess we could monkey with headers and things. I've done that before, but like the, the simple straightforward thing to do for a tiny site like ours is to say it's just all invalid. Like literally no one's really cares about the caching of this.

Exactly. Exactly. So I'm trying to like, okay, so if I do an NSlookup, like in a Yeah, I was gonna say,

Where does the DNS actually point at? Where

Does DNS DNS actually point at? Um, or actually I don't want to do a DNS lookup. I wanna use like dig. Right.

That's what the cool people do. Yeah.

Um,

But I, I'm still old school and I end up using nslookup, but let's do, let's learn Dig.

Yeah. So DIG is giving me an A record

A.

Yeah.

For um, twos compliment.org. Yeah. I think there's something funky. I think you can't have cnames for the top level domain. Oh, as I recall, that's what

It is. That's what it's, yeah,

That's my, because there's, so there's always like a thing. I know my DNS provider had some hacky way of pointing the, A record at where you needed it to be always. Where are we? Where is the, who's our DNS provider ours? Yours.

So it used to be Hover, um, and I am pretty sure that I moved it. Let me pull up the DNS or the AWS console here and confirm that my memory is not failing me because it very much, I think I moved to, to

My memory is still been,

And I have a memory of doing this because I was like, oh yeah, if we're ever gonna get this top level domain to work, like Amazon has some cool stuff that will make that work. And I actually reached out to the Hover support folks and they're like, yeah, we don't do that. Um,

Right. I was using dnsimple, dnsimple whatever. And they have like this fake, um, they have a name for it. It's like a made up type of record that's just like, it's itself, DNS look up something else and then keeps changing the, A record to point at it which obviously is not great for all the reasons, but, I think Route 53 or whatever it is on Amazon kind of does this out of the box.

Yeah. Now I'm starting to worry that I didn't actually move the domain.

Oh, that's smelly

Hold on a second. What is even? Come on browser.

What even are a domain. You just don't know.

Hover? dot com.

What are birds?

Uh, what uh, I mean are they real? I don't think they're real.

I don't Uh, we just don't know.

Status is redemption, what does that mean? Yeah. No, unfortunately two's compliment. So that is almost certainly what that A record is, is my, is hovers attempt to redirect you and that's why http works. But https does not, cuz hover doesn't have any concept of the SSL cert.

So what the, the, the, the A record for twos compliment org, that is the un the prefixed twos compliment org. is pointing at hover's own magic to do this. Is that?

Yes. Yeah.

Okay. And obviously they don't have our SSL cert, so they couldn't be able to, to serve up a redirect on port 443.

Exactly. Exactly. Yep.

So could we at least set up the CNAME in Route 53, like now, and then test it with Dig that it would work, uh, and then, you know, then set up, kick off the, the process of like moving the, the whatever the Main record and then we can at least go through that so you can Yeah,

Yeah. Yeah. How are

You, do you, how do you do this in the console or do you terraform for this kind of stuff?

Um, I mean, generally I terraform, but I don't actually remember what I did for two's compliments

. Fair enough. I don't think

I hate me. I don't think I did set up any char. I mean, it would be really cool. I would love to, um, add that Terraform into the CI build that does the deployment and everything. Because you

Can do just like our day job,

Right? Like you just have it do a Terraform apply, uh, and then do the deployment and then it's like, oh, I want to change the CNAME records or something. Like that's a commit along with anything else that just gets applied automatically. And

Just like, you know, when we publish an episode, we just check in an updated thing to our like Python program that generates the XML

Hacking

Up a Terraform thing. It's just there's a lot of credential nonsense to worry about and CI and all that.

My only, I I agree with you and the thing that pains me is that I did this before once for a bunch of my domains and I have zero memory of how I did it, which means I'm gonna have to learn it all over again. Oh. Because I didn't encode it into anything that is automated. So

It, it embarrasses me to say that, like I know that I have my, you know, hobby project, but I have enough other hobby projects that I have a God bolt Terraform, like personal Terraform set that we might want to crib from. So if you go to my GitHub, let's just have a quick look. Oh, is it public? Uh, it is, yes. And I'm after me carefully going, I'm so sure there isn't anything in here that, that

what's the name of this repo?

Has resources and crap in it? It's got whatever, I don't think this has root dodad. Cuz it kind of like jsb lives on bbc.gobo.org and, and godbolt.org as long been adopted as Compiler Explorers real domain. So it lives there. Really? Ah, yeah. I don't think this says any Route 53 stuff in it or blog. Aha. Hang on a second. I have, so there's xania.org,

Something like this, but I don't know if it's actually relevant to this.

CloudFront distribution has got all the crap for xania.org, Which is my blog. I don't know that this has got the root, the, the DNS for it though, which surprises me a little bit and embarrasses me also, cuz I don't, there I am touting like, Hey, well I did it. You're like, no you didn't

I've got something for this, for an old version. I, I have a recipe tracking website that I made called Tastypenny. Um, which

Is again with the Hobby project side

Project

So do you wanna just copy, do some copy pasta and see if it looks near enough? Like we could just paste it as a subter into the twoscp. Oh,

Cool. You know what? I think I actually have a better one. Let me read this through this one. So Earth is Dead. Oh. You know my board game.

Oh. Oh, I thought you, it was a statement of fact.

Yes. No, that's, I clicked on the wrong button. I, and now I've deleted the Earth. Well it was nice knowing you. Yeah. Um, yeah, no, my board game, so earth is dead.com. I have

Oh,

Something for that. I don't know if I have an SSL cert. Hey you.

Hey that's cool. Rady Set Go

Yeah, that was a friend of mine came up with that name. I was like that's a good name.

It is. How did the other ra I mean actually the same thing could be asked of me, right? Like, how did the other Rady's feel about you? Like being a Rady that's kind of stolen the name for your own machination.

You know my name, my last name is kind of made up anyway. My, my dad invented it.

Oh. Because most of 'em aren't though, you know most of 'em, right. Clearly

Some of them are

So, I mean some of 'em are like, yeah, I think we talked about this before. Like Taylor and Yeah. Baker and uh, Fletcher and Smith and all that kind of stuff. Yeah. But you know, ultimately it's all made up. Yeah.

Every, everything is, but

Yours is very specifically made up by like, I it's very recent history

Mine is very specifically made up.

Was it your grand grandpa or? That was

My dad. My dad. I mean, it, my dad, he just changed the spelling to be more Americanized. I say it's made up, it's like, you know, changed it from R A D I to R A D Y.

Oh, that's cool.

Yeah. But it's, it's funny because there's this like business school out in San Diego that apparently was paid for or founded or whatever by somebody with coincidentally the same last name. And we, uh, my brothers and I have got into the habit of collecting the swag from this, uh, business school. Cause it has, they have like stickers that say I love Rady. And it's like all, you know, backpacks and all those other crazy really bananas stuff.

That's super cool. Alright. Anyway,

Sorry

I'm

Distracting you this tangent, but I, so what I'm seeing right now is I don't necessarily have any, any terraform that we could drop in that would be exactly what we want. I definitely have a couple of things. And it sounds like you have a couple of things that would be like, close?

That are like 80%.

So maybe we, maybe we could do it this way. I think so backing up a second, I think the big thing is, is that I have never done the transfer over to Route 53. So if we feel like we can, if we feel like we can actually set up, cuz like, okay, so my, my memory is fuzzy on this and maybe you remember,

I, I actually, I don't even know if we need to transfer it, right. If if who your current provider, they let you set the, the name servers, they don't have to be their own name servers. They can still be the registrar to start with. And all you need to do, all you quote need to do is edit their name server record to point at the Route 53 Route Route 53

Okay. So let me look at, okay. Yeah. So right now the name servers for NS one. Yeah. nsone.hover.com and nstwo.hover.com. And can I see that if I use a dig command?

I think so. I think that's how you know, and, and it'll take a while to propagate, but we can actually make that change. But we need to go into the Route 53 and at least copy over what you currently have, which we can either do manually or we can terraform that up.

Yeah, yeah. Okay. Okay. Yeah, let's, let's start with Terraform. I like that idea is the cool, I don't even remember is the twos compliment. Is the repo public? Or is it?

Do we have any Terraform? I don't think it's public. I think it's private.

Okay. But you have access to it, right?

I do have access to it. And yeah, I've farted around making it all use poetry for the Python and all that stuff recently. Yes. And we have assets and

Is the public part. The public directory is the part of that that is actually like visible to the whole internet

but it's not light open to the world because, you know, well it is

Through the West website because we

Put stuff in it is, but yeah, like importantly it means, you know, when we put our, the reason why I think we shouldn't make it public is cuz that's where we do put our like episodes before we finished uploading them and farting around with them, editing them and whatever. Right.

Um, yeah, there's

A lot. Although in fairness, that's the same true on my blog post, but I don't blog quite as often as I used to. Like once a year is now my cadence. Yeah. So why don't we make a Terraform directory there and just copy paste in the main TF and the terraform.tf or whatever

Yeah, yeah.

From some other projects, you know, in the way that we do

And of course I have some makefile magic that I think I can steal to apply this stuff so we can make sure that the, that the Terraform that's checked in is always representative of the real world. Right? Right. Like, you never want those things to be able to get outta sync. So

That is true. That is true. Um, do we have this pre-commit hooks in? I'm a big fan of pre-commit hooks now, and I, there's, so I I use dot pre-commit, like, uh, we, well we use it at work and it'll be cool to make it so that you can't even commit if it says, hey, you know? Oh, no, no, no, no. That would be different because we want the CI to apply the Terraform. Yeah.

I mean, it's a nice sort of credential management thing where it's like you don't have to have the AWS credentials. All you have to have is access to the repository and the CI build is what holds the AWS credentials. So I'm not handing out like personal a, like credentials my personal AWS account to people in, I mean,

That's for absolutely for certain. I think that's a, that's that's a very, very good, uh, uh, thing to consider. But pragmatically I have found that like, it's very hard to test the Terraform config without actually applying it and, or like, especially if you wanna adopt existing things. So anyway, yeah. Let's start from, let's start from just getting it working at all. Yeah. Um, even manually I would say. So let's make a Yeah. Make a directory or something.

Yeah. Okay. All right. Uh, yeah. And let me, let me grab this. Hopefully I have some makefile magic I can throw into this thing.

Do we need the makefile magic? I mean, it doesn't do much. I mean, Terraform apply what you want to type.

Well yeah. But like, installing Terraform as a tool and knowing what version you've got.

Oh, neat. Neat, neat, neat. Yeah, that's better.

Um, which, and if I, I think I have this, and if I don't have this, I'm gonna give up on it, but I, I think I actually just have this handy

Terraform, I was gonna say, yeah. I actually use ozy for this, which is the thing. Oh yeah. Like as in our op our company's open source version of ozy, which has Terraform as a, as a an example thing.

Mm-hmm.

mm-hmm.

And yeah. So I have these like Terraform and Terraform and init targets that Oh neat. Like install Terraform and initialize the deal

Sounds Perfect. We're both on Linux, so Yeah. It's not gonna cost me No problem. We don't have to worry about like, oh, but what operating system. Yeah. And it'll get the right version and all that crap. So Yeah.

Especially Uh, do you have a major allergy to just putting the Terraform file like a single Terraform file in the root of the repo? Because this will only really ever be like a couple of things. Or do you want to make a a Terraform directory and put

I would make a Terraform directory cuz it, it just, cuz it's one extra command mkdir

Yeah that works for me. That works me. All right.

And especially as Terraform is like, it sort of globs the directory it's running in Yeah. To get all the.tf files. like the, yeah. Anyway, let's do it now.

It now. No, I'm, I'm down with that. Okay. Um, yeah, so I I you'll be able to see this in a minute when I commit it, but just to explain it to Yeah,

Yeah. I was gonna say like, this is,

Um, I'm making do this through

The medium of

through the medium of sound

Speakings

uhhuh

Like dollar open Squigglies terraform shout caps. No doubt. Yeah.

I

Mean we could share screen, but it's more fun to do it this way.

Yes. Uhhuh

Nice subtle thing that actually CURDIR is the right thing to use in makefile because I've used wood so many times on Yep. Yep. Some other thing. And it's like it own, it works by coincidence most of the time, but it's a, a sharp edge ready to cut your knees off. When you do make dash C or whatever that I, I can't even remember what breaks it under, but like now I've just learned. Yeah.

Just, just CURDIR is the right thing. Coo. Um, okay. And then I have a Terraform version variable just to make it specifically called out as like, yeah, this is the version we're using. Yep. And then I have yeah, this Terraform variable, which points to the executable and you know, because Hash Corp is Hash Corp, all of their tools are super easy to install with just

Because they are just a static executable you curl and put somewhere and then chmod or Unzip or whatever it is

It's, it's the best thing ever. I just wish everything in the world worked like that.

This is, yeah. We've talked about this before as being like a killer feature of, of Go for example, which I think they're using. Yeah. And, For our tool actually, so we, uh, did you see that we opensourced rpy

Um, Rpy? No, I think I missed that.

Yeah, so we, you know, uh, it is a read the nearest quote, nearest pyproject.toml, And then there's a little stanza in there that says, no, this is the, the, the, the right python to use. Like this is the right thing to do beforehand. So like, for example, um, you can put a stanza that says, Hey, before you run Python, do make deps in this root directory where you found the py project. Um, and then, um, uh, effectively it just means like I can go into any project and it's, it's, uh, rpy splash space and then whatever. But anyway, the whole, it's, it's a, it, it's a simple cool tool, but we did it in Rust and it's relatively straightforward to make that static as well, relatively straightforward. So that was a nice find recently. So you can just curl rpy down. And also, uh, rizzy I think is the other thing we've open sourced, which is another rusting that's similarly, we just want that, just copy the binary and it works feel nice. And that's all hashicorp and Go goes sort of like pushing

Yeah. That's good stuff.

That's my rambling covered enough while you're frantically tapping away there. Yeah, that's good. You don't have an offensive keyboard incidentally. I, I feel like when I, or maybe it's just Google Oh, filter. It's doing this amazing filtering of like non sound. It might be nonhuman sound. Yeah.

When you come back and edit this podcast later, you're gonna be like, why is this just 90% typing? Um,

Yeah. Right. So we are just agreed that instead of just

Yeah. I don't know. I mean, we'll see if this ever sees the light of day

. Right, right. We're boring the heck out of people as we like do our debugs in, but it's like listening in a pairing session, I guess. Yeah,

Yeah.

Of a sort.

Um, okay, so I got,

I'm gonna make you write tests for this Terraform

You know, I, that would actually be a really interesting exercise is trying to write tests for Terraform. Um, I've written tests for SQL before. That was a fun project.

Oh my God.

Um, I've written tests for bash, I mean, we should do an episode on, on, we should do,

Yeah, we talk about that. Yeah, it's in the list isn't it? About testing Bash? Although it's, there's a num Yeah. Yeah. I'd be interested in the Terraform test type stuff. Yeah. I definitely use Terraform validate in some of my pre-commit hooks to make sure that like, it's at least synt tactically valid. And I think it's Terraform. Terraform format that sort of makes it canonical formatting. Anyway,

Uh, okay. So I think, I think I gotta change something here because I'm running my target and it says nothing to be done. So I, oh. Usually what you have, oh, if

You, if you, you haven't made it dot phony have you, or have you made it dot phony if you haven't made it dot phony, you have a Terraform directory and that's what it thinks is up to date. It's like, yeah, I got a Terraform directory. You did make Terraform. And it's like, yep. Got that.

Yeah. Well, I don't think that's it, but I I I did not, that's called,

That's before now where I had directory.

Yes. I mean that's, I just like, I'm okay, I'm just gonna do this because there is no, the target I'm trying to run is called infra and there is no infra directory. Oh,

Okay. Great. But

I, I'm gonna make it phony just to, no, that's not it. Yeah.

It's not gonna be that, but like it was worth checking out, but that Yeah. Yeah.

Terraform init. I think the problem here is I have like an undefined target. So basically like terraform init. Mm-hmm.

Empty.

So it's like, yes. Wow. So then make was like, I, there's, you know, I'm not gonna make an empty thing, so it does nothing, but I also need to do terra CURDIR slash terraform dot terraform because we want to put this in a Terraform sub directory.

I think so. And then we can hide all squirrel away, all the dot yes dot files and drop them in there as well

And I think what that also means is that I need to do a CD in my Terraform init target before I, yeah. Uh, and because it's make, you can't do that on two separate lines, right?

No, let me just have a look. Does Terraform has a dash chdir equals Okay. So you can use that instead. So a bit like make dash capital C or Ninja Dash capital C where you're like, Hey, I'm running you, but don't run here, run there. It might make more sense to, to use that. It's nicer because it doesn't need to use sub commands.

Do I need to make the directory first?

Oh, sorry.

Do I need to make the directory first?

Oh, I think so. Yeah. Okay. It literally is like, Hey, you're gonna CD into that directory first. Okay. But it means you don't have to do the thing on one line. Yeah.

Like,

Which is essentially a sub shell. And, uh, and make doesn't know it, it just sort of executes it in the whatever the prevailing shell is and says, Hey, you do this.

Okay. So Yeah. Which

Is almost always sh or Bash, but for example, fish doesn't support that kind of nonsense. Yeah. Even though I, I love it dearly.

Um, okay. Yes. So I missed a target no rule to make Target source two compliment.tools. So I, I defined that tools directory, but I never made a target to create it.

To create it.

Yes. So, so I gotta grab that target, which is very simple. What're

Doing here is kind of working it backwards from like, I need this thing, so, and then mm-hmm.

Exactly. Okay. So now I have a tools home target, which is just mkdir dash p dollar @. So it's like, you know, just make the thing that you wanted,

Make the thing that you said I need needed to be, yeah.

Yep. Uh, okay. So I ran that and it installed Terraform and then unpacked it to the tools directory and then it initialized Terraform in an empty directory. So now I would expect to see a Terraform directory, and I do, uh,

fabulous.

Inside of that Terraform directory, I would expect to see a dot Terraform directory. And I don't

You need to make a main.tf Or just anything in there. Let's just create any old thing. Yes. Um, while we're thinking about it, I know there are various lock files and other nonsense, some of which need to be checked in and some of which don't need to be checked in. And I can never remember which ones are which. I

Think this project that I have has a gitignore that will

Has a exactly. Thank you. That's what I was asking basically is do you Absolutely. We can copy.

Yeah, yeah, yeah. Um, all right, let me try this again. And we're gonna see how good this, this make setup is if it knows to do the right thing, because I've already created some of these directories, but no, it might do nothing. Um, check infra. All right. Terraform has been successfully initialized. So now if I go into Terraform and I have my main tf, I still do not have a Terraform directory.

A dot terraform directory dot terraform that directory. What, what was the, what is the dot Terraform directory for?

So that is gonna be Terraform init. So when I run Terraform init, I'm expecting it to create a dot Terraform directory. Is

It created it in the root. Did he do the dot should ch d No.

Yeah, I have it. Terraform in it is CURDIR slash Terraform slash terraform. Am I just not looking at this,

Uh,

Correctly? Maybe it's like, because there's nothing to do cuz there's nothing in the main do tf it just doesn't bother to create it.

Yeah, almost certainly. It doesn't know what versions of anything to pull down because the main thing it puts in there is all the plugins that are gonna be like, here's the AWS provider and here's the whatever provider. So

Let's Yeah. Do a very minimal thing in there then. Yeah.

Uh, the most you, you're gonna need to have the Terraform thing and you need to have the state, um, management Yes. Part the thing that says this is where the state lives, which should probably be another Yeah. Another bucket somewhere. Like it always is in my world, dynamodb

Dv. Yeah. But definitely the best way to do that is with you store it in a bucket. Um, I have sort of gotten away with not doing that in cases, but like yeah. That's just, that's not the best. All right. I'm

Just, if you're gonna have to solve the credentials problem anyway for like me being to apply this, then

Yeah. Yes. I mean at this point I'm just trying to get to something where we can test this interest subjectively where I'm gonna commit this and push it and, and then I'm

Gonna take down and

Then I'm

Gonna put it down

And make sure that I didn't screw anything up,

Which has the credentials. Sort of caveat by, for what it's worth, I think I have AWS credentials. I'm gonna have to check, take a look.

Well, if all I'm doing, I mean we're gonna have to solve that problem eventually, but if all I'm doing is just trying to initialize this directory,

Then that will be intersubjective. You can

Just hopefully just do

That. Absolutely. Yeah.

Okay. Installing the backend, installing provider plugins.

Love it.

Finding hash cor AWS versions, matching blah, installing AWS matching, blah. And now inside of the Terraform directory, do I have, I do! I have a dot terraform directory.

drum roll

And I have that lock file that you were talking about. So let me go take a look at this gitignore and see what I did, at least in here. So in here I am ignoring a bunch of things. I'm ignoring the dot tools directory, which makes sense. I want, you know, I don't wanna check in any of these tools. Um,

The how ironic, ironically we could actually, because we both are linux, we could just check it. Terraform, you could, that would work, solve that whole thing. But you've done a nice thing for downloading it and changing it and upgrading it, all that stuff. So let's keep it.

Um, all right. And then I want to say terraform.tf state, uh, I've got an s thing in here. I don't think I need that. Terraform. And then we're not doing anything with Terraform plan. So Yeah, so the three getting ignores, I'm adding are the dot tools directory terraform slash Do I need that one? I was gonna say Terraform slash TF State, but I don't actually think I need that yet, so I'm not gonna put that in. And then, okay. Terraform slash dot Terraform, which will be that Terraform initialized directory. And I think

I'm just looking at my own final

Do it. Oh, I got, yeah,

That's the whole dot Terraform directory is all I've ignore in mine. And there is a dot terraform dot lock hcl, which I've accidentally been checking in, which probably I don't need to, we should, maybe I have to Google that.

No, I actually think you are supposed to check in the lock file because it's like

A lock file for when you do upgrade. Yeah, yeah, yeah, yeah, yeah. Okay then that's fine. The dot terraform dot lock hcl, I think you is like a stanza is when it's solved the Yeah. The constraints. You want everyone to be agreeing on what version Yes.

Check I, yes. I have that file locked into or checked into my other project. So I think that's actually correct. Okay. Okay. So I think I've got to the point now where I can push this.

All right

And you can give it a try. So let me say to go up into skeleton Terraform config with make target with With infra make Target. And you know, we can rename this later, but this is what I got right now. Okay. So that's pushed.

All right. Git pull, I see some things type make and there's nothing in the list because you didn't add the help because you're a bad person.

For No, I didn't. It's not, it's not cool enough for that list yet.

And I'm gonna do make what, what, what should you suggest? Make Terraform

Make infra or

Make make Infra.

Yeah, that's in, like I said, we're gonna have to rename that later. And that's probably not even gonna be the, like we're gonna need like a make plan and a make apply like Terraform Plan, Terraform apply I think would be good targets, but they're gonna have like base things that sort of maybe did do something.

It archived terraform.zip and Inflated. It made, made a, a Terraform sub directory, uh, which I guess we don't need to make it, you know, that maker is probably unnecessary cuz in you've made Oh

Duh, I've got the main file in there. Yeah, thank you. Yeah, perfect.

That's the whole point of that. But yeah, so we can get rid of that. Check that out. It says Initializing the backing initializing provider plugins Terraform has been successfully initialized everything. Quote just worked.

Awesome.

You may now begin working with Terraform. It tells me. So now we are inside that Terraform directory. I could do Terraform plan and it should do absolutely nothing. I know you wanna do makefile nonsense, but I like, I know old school, I'm just gonna go in there. No changes your infrastructure matches the configuration. That is the no infrastructure correctly matches the No configuration. Hooray

. Cool. Uh, all right. I'm gonna do a very small commit to remove that unnecessary mkdir

I'll allow it

Uh, but that requires that I spell unnecessary correctly. So I'm just gonna say unneeded,

oh, that's difficult. Yeah.

,

That's a pragmatic solution to the problem.

Okay. All right. I'll push

That. Well, my laptop fan has just kicked on, so when I edit this, I'm gonna have a big old hiss in the background.

Ah, sorry.

Listener.

Sorry listener. Okay. Okay.

I'm sorry. Listener also that you maybe are expecting a podcast and instead you're hearing this debugging

Session. We should just call this episode Yak shaving. That's what

Alright. So,

Um, all right.

The next thing that might be useful to do is either we um, we, we set up the, the, the, this Terraform Terraform State Storage mm-hmm.

That maybe that's a place to, to start, because I guess if this is gonna be like really onerous, there's a chance we might just want to bail on automating this entirely. Right. And just be like, you know what, we're just gonna do this one in the console cause I'm not gonna spend 20 hours getting this right.

I don't think it's gonna be that bad though. Like, I mean, let me, let me go. So if we go to Compiler Explorer, even, let me go to.dot compile devey compiler. No, infra Terraform, sorry. There's, there's a billion-ty files in here. Uhhhh, one of which is, is there a Route 53 tf? Yes. Haha. So I have, okay.

Okay.

A route 53 we can copy from. And let me just, I will send you the link. Oh, and I'll just hit my microphone.

Sorry. Oh, wait a second. Don't we want to do like a Terraform import?

You can, but it doesn't, it just says, now I manage that resource.

Oh,

It doesn't, but what I typically do is I manage, I import the resource, so I, you have to make an empty resource and then you import it and then you do Terraform plan. It says, oh, I would delete all of these things. You're like, great. Those are the things I'm gonna put

Okay. Yeah. Um,

It's pretty straightforward though. Um, like,

Yeah, no, no, I, I, so I hear what you're saying, like let's just get the Route 53 stuff in there cuz it's new Yes. And worry about everything else later.

I think so we can definitely import the bucket stuff and we, that's a pretty straightforward thing to do, but we are gonna need to be able to store the Terraform state first and foremost. And then the Route 53 stuff we can just add and apply and then you should be able to refresh the console and go, this looks good. And then we should be able to dig at Route 53 or whatever it is. Yeah. See that they're there. And then maybe that's where we can, I mean

We could just do local state for now and then I can move it into a bucket later.

Yeah. Yeah. I dunno how easy moving it is, but I It's fine. I think

It's, I think it's just, I feel like I've done this once already, but,

But I mean it's, it should be straightforward. Like, uh, hang on, let me just see what my own crap is. Main dot TF backend, S3 Key bucket. That's what it is. It's just a backend and then a bucket name, which can be something you we we'll call like, you know, infra dot twos compliment.org. just a region and a key and then we're done. Mm-hmm.

Yes.

So I'm gonna Yes. Paste this into, oh my God, I don't even know where to paste it into, uh, uh, discord,

Yeah. That's probably the best place.

So that is what I have in the com. I mean, obviously it's open sourcing, you gonna grab it. Um, so you, you already put a provider in there for AWS of you or have you not? I,

Yeah, no, that's in there. Although the version that you're using is way newer than the version that I have so much. Yeah,

Yeah. Yeah. I mean that's unimportant really. Um,

Now is the time to do it. Not gonna get any easier than now

The latest version, you know? Yeah. And then that is what I have for like the provider block above that. And that's kinda like my main.tf that is like the hey is all the versions, this is the whatever's and this is where I want you to store your state. And I think that's all it took, I think now I think there's something you can do with DynamoDB for locking and all that kind of crap, but I

Haven't done that. No, I see what you're saying there. So do you have another piece of Terraform somewhere that's actually creating that bucket?

No, no, no. The, the back end knows how to do that. That's like, you know, otherwise you've gotta chicken and egg situation. How do you create the bucket where this, where it's gonna store its state before you've stored the state to be able to create the bucket. So, okay. Okay. So I think, I think you can just type put anything in there. All right.

Well

I'm just, or in fact we already have the bucket actually as it happens. We can just use the, the existing bucket. Although the No, we don't wanna put the TF state there cause it actually goes to a website. So let's not do that. Yeah, yeah.

, I can't do that. So let's, okay, so bucket

State something two compliment.org or can be the name of the bucket. This is cool. If it works,

If it works

Retrospectively it'll be cool. But right now it might just be, as you say, another yak.

Okay.

We shaved this yak and then inside the yak was another yak

. It's

Like Russian yaks the stack and Russian and do Ys yak stack Uhhuh,

Probably better than that my friend is what it's all about.

Should I have typed enough times to be able to spell correctly? Okay. Um, okay. All right. So now if I run this, I would almost expect a credential, error. Cause I don't think I have any, I I tend to not just throw, have

Them one in the global

Credentials into like my Bash RC or anything like that. And I actually really kind of don't like the fact that the AWS client will discover credentials that you put in, um, your, you know, home directory. So I don't really like having a default profile

There, there are some actually, like, there's some like aws pseudo as in S U D O, pseudo, not pseudo as in PS U E D O mm-hmm

Yeah, yeah, yeah. Um, okay. So unfortunately I don't think I've made this makefile quite mature enough to where it will detect changes to the TF and rebuild. Um, which I think I have in number. Well

That's typically I make those things like if it's a, it's more, it is a phony target. So it's not that there are changes, it's because like, who knows? The thing is that Terraform, there are two sources of changes in Terraform. One is in the dot TFR that you edited and the other one is some clown changed it on the console and you wanna make sure that actually you apply that. Yeah,

Yeah, yeah. Okay. So yeah. So I need to change that.

So don't worry about that for now we can

No, no I got this, I got this. So I'm gonna change that in for Target to actually, I'm just gonna leave it the way it is and then I'm gonna make the Terraform apply Target or probably Terraform plan to start, uh, target. That's gonna be phony. Yeah. This is what we want do. And then, um, this is just gonna depend. I am gonna get rid of the infra target and replace it with the real one. Alright. Um, but it's plan, that's what I want is Terraform plan and then I just wanna see the plan, right. Which should be nothing initially. Okay. So then we're gonna do that and then I'm going to use my variable name for the Terraform executable instead of trying to guess where it might be. Be

Naked Terra, you're much, see I'd be in there just typing Terraform and then I'd do make fell afterwards. This is quite, I'm interested in how your process, how how, um, rigorous you are actually saying no, I never want to know.

Yeah,

Right.

Or right. I don't know. I've just done this enough times where it's like I just really like the result from it because it's like there's a very high probability that when I get to the end of this it will actually just work mm-hmm.

No, no worries.

Uh, make Terraform plan, what the hell does it do no configuration files, right? Because I didn't use the dash C

dash chdir. Yeah. Yeah. You can bake that into the Terraform command if you're being really horrible.

Oh yeah, good point.

I don't as in the, the variable Terraform that's slightly, Hey, run the Terraform in the way that I need you to run Terraform rather than being the executable is, is Terraform space dash chk. Mm-hmm.

,

Which is kind of naughty, but, but nice. Mm-hmm.

. Uh, okay, so I'm getting backend initialization required. Please run Terraform init. So my guess is that because I've already initialized this once with the wrong backend, it's now complete. It

Needs to be reinitialized. This is So

I'm gonna do the thing.

So honestly, I've had enough of these problems where I, this is one of the reasons why I haven't driven a lot of the Terraform by make, cuz this like knowing the subtleties of when you need to do the re init and is hard, so often you end up with makefiles that always re init all the time. And then that's painful. It means like the first 25 lines of every time you do a make thing, is it re initializing the same thing? Mm-hmm.

Yeah. I mean I've, I've definitely gotten this pretty solid on some of the other projects that I've done, so I'm not worried about that. Yeah, yeah. for this, I'm just gonna delete the directory and rebuild it. Um, sounds great because once you get that background right, it should be pretty stable. But now I'm getting a different error, which is unsupported dot terraform core version

The dot Terraform directory you mean?

Which is why, why do you this configuration

What version of Terraform do you have?

probably like an ancient one compared to what you were using? I have it set at 1.0.11.

Let me have a look. I am on 1.3.2

Well I will change the

Latest version is 1.3.7 though. So why the help

1 37? Yeah. Love it. Okay, so now this is another great exercise. And

Here's the funny thing, like when you check this in, if I try and test it, I can actually make a dot ozy dot yaml in that directory to point it at version 1.3.7 just for when I'm running in that, in that directory, which is just a me a saying. ozy pretty cool too. Mm-hmm.

I, which I I really like that idea by the way.

Anyway, sorry. We're, we're getting all all up in the confusion.

Uh, okay. So I ran Make Terraform plan installed the new version of Terraform automatically, which was cool. And now it's saying error configuring s3 backend no valid credential, which is the error that I expected to see. Awesome. Right. Okay. All right. So now I can go and

I wonder, you know, if you can specify where the credentials are expected to, to be.

Yes.

And then you can say, well, and then we can gitignore them and then we can make it so that there's a make file target that says, you know, you need to put the, you need to go get the credentials from exogenous place. Mm-hmm. And put them here. Mm-hmm.

Oh man. But that's

How we go, that's how we roll.

Yeah. Yeah. No, I think I'm gonna, I think I might do that. Um, cuz I, I actually have a, another example of doing that in a different project.

Oh. Even better.

Uh, where I say, uh, AWS shared credentials file, I think is the name of the environment variable.

Nice.

And so, and then I have a make target that is just that file and the, it just prints out like, Hey, you need to,

Yeah, I can't make this file because Yes.

, uhhuh,

Do we have vault, we can make Avol and then no duh. Now

We're right. Okay.

Oh, Myre

Done here. The, the, the yaks The yaks are stacking. The yaks are stacking. Yeah. Um, one pause. Yes. All right. So yeah, I'm gonna grab that. And then AWS shared credentials file is that, and then, um, this needs to be exported and that's why we had that AWS directory in the Ignore, because that is where it's storing the credentials. Um, so I will add that again because I absolutely, positively do not wanna check these credentials into uh, yes. git, under any circumstances.

I think something will alert you pretty quickly to it, but it's not worth testing it. Yeah.

Let's not find out, actually there

Is, there's a pre-commit hook for, for this. So if we do set up pre-commit, oh, as in the Python pre-commit thing, one of the hooks is, you know, it has an obvious look for like mm-hmm.

Uh, okay, so then I would assume I want my Terraform init to depend on this AWS shared credentials file, because otherwise Terraforms not gonna be able to do very much for you. Mm-hmm.

Done a typo or braino?

What?

You got two recipes with the same

Name? Yeah. Yeah. I think that's what's going on here.

I'm hungry.

Do that

Out. We,

We did this.

I'm gonna, I've lost my mouse pointer. There it is.

We totally did this.

I'm sitting funny cuz I'm trying to keep my mouth there, the microphone. But it means that now I'm

AWS shared credentials file is already in here. We did this already for something else. Okay. Awesome. What if I just Oh, use that? Yes. You must get credentials and save them here. Oh yeah.

What? Because that's how we tested some of, I'm sure. Hang on.

Yeah, yeah.

Config file.

It's like line 29 in the make file.

I don't have that. Which is fine.

How do you not have that?

Credentials? You must get credentials from Google Drive and save them in, blah blah, blah, blah. Yeah, that, look at that, look at that. Ben Ben Rady is a clever person, so I'm gonna go to Google Drive and go get them right now

About that. Did I put them there? Maybe I did put them there. I think I put them

There. Anyway. You worry about that. I'm gonna get them. Yeah. And then we can do an intersubjective test at the end of this, this all. Yes. I like this. Pat ourselves on the back and say mm-hmm.

Yeah. I would infer I have zero memory of anything that I've ever done, but I in would infer from this hint that I left my future self, that I made some, uh, two compliments, credentials.

There's a credentials directory in, in the shared gdrive where we steal all this

Stuff. Look at, look at me. There

It is. Aws December the 14th, 2020 by Ben Rady.

Wonderful.

Esquire.

Wonderful.

I'm gonna download those credentials right now. Yes. And I'm gonna move them into place. mkdir, aws,

Wunderbar,

Wunderbar.

I'm actually gonna do that too. I don't, don't even know that I have a copy of those credentials, locally.

Oh, that's not, I just typed vi instead of mv. That is just how stupid I am. Okay. Make credentials, nothing to be done for credentials. Look at that. Okay. How would I test that they work?

Uh, well, what are they used in?

I shared credentials, file credentials. Credentials.

I guess it's an environment variable, so it's gonna get used by like if you did a deploy, which Yeah. Maybe don't wanna do, but that would be one way to test. Um, I guess you could try to do like an s3 ls, uh, uh, on

The, but I'd have to hack that in because it would need to set the things Yeah.

Up.

Yeah. No, no, no worries. Anyway, I, I'm gonna quickly, I I will locally hack that just so that I can do it just to see that we aren't gonna fail while you try and get the next step done, right?

Oh

Yeah. No, no. We have a a website info. Okay. Put bucket crate bucket. Oh, no, no. Yeah, I thought we had on ls.

I mean, really like the test target list

Bucket. There we are. Make list. You should put it in Make list bucket, and it works. So,

The test target, Oh, list bucket. Yeah.

Okay. We, we've done this already. Wonderful. I just, it's

All coming back. Oh. Oh my god.

Dear listener, we are not the smartest. It turns out,

Uh, programming is a detective novel where you are both the detective and the murderer.

So very true, my friend. All right. What we got going on now then? Uh, so,

Okay.

You are just about to check in. The thing that points the Terraform config at this preexisting configuration, having made it also our, uh, terraform init depend upon the credentials. Mm-hmm.

Mm-hmm.

Oh. Um, it's in The Twos Compliment drive under credentials. Sorry. Under, yeah. Uh, credentials. Aws. Oh. I will paste a link to you right now in the chat, which says, chat with everyone. Yes,

I found it. I found it.

You found it.

All right. Wonderful. Okay.

I don't even know what that is, credentials, but you know, I'm gonna use it now to find all your secret projects. Mm-hmm.

Yeah. I, I had completely lost track of the fact that the Twos Compliment Drive was a drive you shared with me. I was looking at my own folders and I couldn't find it.

Oh, yeah, yeah, yeah. I, that's, that's me. That is,

Okay, cool. All right. So download it there and now. I

See, so you were waiting to get your cred.

Yeah. I just wanna like check to see if this actually works cause I haven't, you know, tried it. Um, mkdir AWS and then copy from downloads credentials to here. And I'm just gonna take a look at this thing real quick to make sure that it is the credentials that I expected to be,

Let me have a look.

Uh, nope, I don't. Cause I put in the wrong directory.

Can you just read out the, uh, secret access key to

Me? Uhhuh?

To check

It is, uh, no, I'm not gonna do that joke. All right.

Not even gonna do

All right, cool. Yes, that looks very reasonable. So now I should be able to do the same thing that you just did and say list bucket.

make list bucket.

And that worked. Awesome. Yay. So now if I do make Terraform plan, okay, so if I get Terraform,

These credentials may not have the right permissions to do whatever, but

So when I do this, it says S3 bucket does not exist.

Okay. Maybe we do need to create this and then maybe permissions, things about that for the state.

I mean, it would be very convenient if we could store these things in the same bucket. So one yak that, oh my God. The yaks, um, the yak that I would, that the, the programmer wants to shave right now is moving the public content into a sub key

Yeah.

So that we can store other stuff in this bucket safely. Um, but I don't want to do that right now.

Buckets are free.

Yeah.

Buckets are absolutely free and it's much, much safer to have a top level bucket thing and just say like, this is, there's, it can't even be public. Yeah. So I have no problems with you going into the console for this one task. Yep.

Yep. If

That's what it takes. I'm sorry, I I did think it could create them, but maybe you could, I

Mean, maybe there is a way I would, I would expect it to be able to do it, but maybe not

So dear listener as well, if you're not sure what we're talking about here. So Terraform is a way, and I'm gonna pad this while be Ben fiddles around

Perfect.

Have we, uh, have I sufficiently padded to

Get to this? Yes. Yes. And I realized that I'm gonna have to use a slightly different approach, um, for Okay. Right. Of course. I have to add access to the bucket. I created the bucket. Um, the, the backend attempted to initialize and it got an, an access error, which makes sense because I haven't actually,

We don't know quite what the credentials, I don't know what you did with these credentials in terms of like this The creation

No, I just,

The perennial problem of like, how do you. Grant the granter. Yes.

Um, yeah. I, I think I have some trade craft on this for that I can pull into this, but I'm not gonna shave that yak right now. What I'm gonna do right now is I'm gonna go into this user

And just say, Hey, you can do the thing.

Yeah. Where the hell is this user?

Oh, the IAM thing is a painter. I

Only, oh duh. I clicked on user groups instead of users. I'm like, where are my users?

That'll be why

No, this is Yes. Uh oh. Interesting. You want to guess what the name of this user is?

Uh oh,

this is funny.

Twos compliment would be my guess, but.

Incorrect!

Oh,

What is the name of this user?

Uh, is it, uh, oh, what is it? Something about argument that we had some other name

Constant arguments, which was the constant

Arguments, which was, so anyone who's made it this far into the episode gets a little bonus treat of learning what this podcast might have been called, which was constant arguments. How funny.

Oh my god. So funny. Okay. Um, yeah. Attached to existing policies and we're gonna go, uh, s3, uh, so I'll come back later and I will create a custom policy that. Only but for now, yes.

So what Ben is doing is granting the user for which these credentials allows us to log in as, or effectively access, uh, Amazon as. It's granting it the rights to rights to a separate bucket which won't be publicly available, unlike the website. So that Ben and I can store this state somewhere. Mm-hmm.

, uh, okay. So next error, uh, failed to query available provider packages could not retrieve the list of available versions from provider hashicorp. Block provider does not match constraint. Oh.

Oh, that sounds like another,

So I have re initialized Terraform and I'm still getting this error.

Should you do Terraform uh, dash init, dash update I think is the other thing which allows it to bring things forward.

Okay. I think. Okay.

Give it a go. This is now we're into hand waving Magical. Yeah.

Voodoo. Yeah. Yeah. In it. I'm trying the same So Dash update?

I think so. Uh oh. Upgrade dash, dash upgrade

Upgrade.

No, that's not right. Oh no, hang on init

Because I could just blow away the lock file cause there's nothing useful in there yet.

That's possible. Yeah. You wanna give that a go? Yeah. It is terraform init dash dash upgrade. We'll allow it to do that.

Oh well if that is what it does, I'd rather use that. So let's try that. Okay.

We'll give it a go. Does it make any difference.

It's, it's working.

Okay. Installing. So now we've made the lock files betterer.

Yep, exactly.

But the thing is about that dash, dash dash upgrade is you don't wanna check that in as being something cuz it will just keep pushing everything forward all the time.

Mm, yeah. Okay. Good point.

I think cuz it would've updated. Yeah. Yeah.

Okay. Anyway, so we have a new lock file, which I'm assuming contains our information about the bucket. Uh, and I'm just gonna take a look at that real fast. Uh, I guess it's just the registry a s corporate registry. What actually contains the information about the bucket. Oh duh. It's the main, like I, yes, of course. That's what you put in the main, dot TF file. You put the name of the bucket. Awesome. All right. So I think that actually worked. So I think we're finally at the point now where I can commit this and you can give it a try,

Right? I am just ha which version did you do? Uh, 1.3.7 I think of Terraform, right?

One through seven. Yeah. Mm-hmm.

. Okay. I am just adding an ozy dot yaml Okay. Into the two compliment things. So that I will Yeah, yeah.

Terraform No, that's cool.

Dash version. Lemme

Make sure I'm not doing anything.

Installing Terraform 13. Oh no, no, wait a second. Sorry. No, uh, it manages Terraform itself, doesn't it? Sorry. Oh yeah, I don't need to do this because you're doing it in the make file. That's the whole point of it. Right?

I mean though

That said, it's convenient to be able to type, I had

Kind of assumed that you wanted to just be able to run arbitrary Terraform commands in addition to this, the intersubjective stuff that lives in the make file. Alright, well, so I have no heartburn at all about it. Putting a ozy file on there. Check as well.

Alright. Right. Have you checked this in?

I'm about to, um, adding, uh, configuration for, um,

You're spending far too long on the, um, the thinking carefully about the, the checking comment for us. I know it's a good, it's best practice to have a decent checking comment and you very rarely

Pushed.

Okay.

Give that a, give that a try.

Pushed the ozy yaml as well. Okay, so I'm gonna do make terraform plan. It's inflating a new version of Terraform. It's initializing, the provider plugins says it's reusing previous versions of some things, but installing some newer things of other things and your infrastructure matches the configuration. Yay.

Also, we accomplished nothing

We've successfully done nothing, but it does mean that we should now be able to literally quote Yeah, just type a tiny bit, make a route 53 TF or something in that directory. Yep. I'm gonna try that right now. Okay.

I I, so this is, I actually, I think I actually need to bounce here in a couple of minutes, but let's, I wanna see what you do, do your thing and then we'll Okay. And then we'll see what happens.

I, okay. I'm literally gonna copy infra from Compiler Explorer, Terraform Module zone id. Ah, no, we have to get zone IDs and other things, plus the user doesn't

Have permissions to do any of this stuff anyway, so we're

Gonna Right then we are done for now. I think, well, we have successfully done nothing, we did not solve the original problem in any meaningful way. Um, we haven't even solved the, the simplified version of the problem where it's like we're not gonna get it to resolve. First of all, we're just gonna set up AWS so it's ready to resolve Uhhuh

uhhuh Right. Before, before we create more work for us to do

Right. But just to summarize, we have a place to hang future infrastructural changes that should be straightforward. We've gotta sort out some permissioning problems with the user so we can create a Route 53 thing. We're gonna have to actually make a Route 53 zone, which should be pretty straightforward. I have got the, so the, the compiler explorer stuff is in a module because I have multiple domains and so I set them all up the same with a module, which is cool stuff, but unnecessarily complicated for what we need. Um, then we can, with a zone which will be the twos compliment org, we can add some records, one of which will be, this is a, an alias record I think is what they call it to Yes, that's right. The cloud front thing, which we can also bring in and then we can just actually point them to each other. Mm-hmm.

All right, friend. Uh, well this was a fun adventure.

This was an adventure and, uh, yeah, I'll, this will be a fun one to edit. I think this one will be the least edited ever. So, um, and it'll just be as it came out. So apologies to our, our listener for the lack of, uh,

I love it. Alright. Cool.

All right. Until next time, my friend.

Until Next time.