Matt Toussain: From JAG Aspirations to Cybersecurity

The other thing is like copilot and such are really, really powerful these days, but they understand the languages that they've been trained on better than others.

And those languages are threefold. It is TypeScript. That is the language it understands best by significant margins. That web code, it just gets it. Golang, I would say, is another one. And then Python. Welcome to the Philip Wiley Show. Take a look behind the curtain of professional hacking and hear compelling discussions with guests from diverse backgrounds who share. Share a common curiosity and passion for challenges and their job.

And now, here's your host, Offensive security professional educator, mentor and author, Philip Wiley. Hello, and welcome to another episode of the Philip Wiley Show. Today, I'm excited to be joined by Matthew Toussain. So Matt and I know each other from the cyber security community. We've known each other several years, but we met for the first time, I think it was at Texas Cyber Summit in 2021, back when people started doing in person events again.

And that was like the first time that I got to meet you. And then also like Bryson was there too. Bryson Bork. That was the first time I got to meet Bryson in person. So it was a lot of fun because after going like over a year without really getting to go to conferences in person and to finally get to meet people again and to get to meet you was. Was pretty awesome. CyberTexas was a lot of fun too. I believe that on the Riverwalk they were doing the.

Was it Dia de los Muertos, something like that? Yes, as well. Yeah, that was pretty cool. They had this speaker event where I'll get to go out on the Riverwalk for us to be able to attend that. So it was a lot of fun. Yeah, that was. I think that was the best Cyber Texas that we've had. In fact, it was really good. Now I have to be against it because they've moved out of San Antonio. No, Austin's a good call. I'll actually be there in San Antonio next week. I'm going there for an API conference.

So I haven't been there in a while, but Texas Cyber Summit moved away from there to Austin. And so I kind of miss that because my. Although I go there once a year because besides San Antonio, I always teach a workshop there. So that's. I'm usually there for that and always have to hit up my favorite, one of my favorite Tex Mex restaurants. Happen to know what that is by name. Metierra. Metiera. Hmm. I don't think I've been to that one, which is Surprising. I was in San Antonio for nine years.

It used to be Casario because it was right on Riverwalk, and it was one of the first ones ever ate at. And I'd always go to Casa Rio, but then when I went to Miterra, it was actually better. So that's kind of the place I go now. A little bit easier to get into compared to the Casa Rio. That's right on the Riverwalk. Of course, you know everyone's gonna be trying to get in those restaurants.

Yep. When I was in the military, Casario was one of the places we would have a promotion ceremony sometimes. Oh, very cool. So is that how you ended up in San Antonio, through the military? The military, yes. Three times. First my father, then my dad again, and then me. Oh, wow. I've been trying to get out of San Antonio for so long and finally made it out. Oh, that's good.

So how are you enjoying Seattle? Absolutely loving it. As you can probably see from the video here, it's not nearly as rainy as people give it credit for. We have bright sunlight today, but we're starting to move into the springtime out of the rainy season. That's a little bit why. But I'm absolutely enjoying Seattle. I love the ocean, and I don't really like flat plains too much because I grew up in Colorado and Alaska for a fair amount of time. And I get all of that kind of back, which is.

Oh, that's cool. Yeah. It's one place I haven't been to yet, but I definitely want to get there, as I haven't been to Seattle. The furthest I've been on the West Coast, I guess, is Humboldt County, Eureka, California. That's the furthest west I've been. I've been through there. I don't even remember why I've been through there. Maybe it was just driving through to here because we drove. We drove to Seattle from San Antonio a few times as part of the move. Yeah, it's kind of crazy.

When I went there back in, one of my consulting gigs went there to do a pen test of Humboldt County Health Services. And when we flew in there the year, it was probably about 2015 or 2016, they had just started allowing jets, lightweight jets on the Runway. And I think they only had, like, one gate there. So it was really nice if you flew in and out of that place because there were no long lines and easy to get around. If you had to drop off a rental car, you could get in easily.

So I really like those kind of airports. Yeah, those are really nice. I'm the kind of person that shows up to the airport about five minutes before the gate closes for my flight. So then I, like, walk straight into the airplane. And you can't get away with that. At Seattle Tacoma, you can almost get away with that. I've got it down to a little bit of an art science. But smaller airports. I used to live in Colorado Springs when it's the Air Force Academy.

That one, you could just walk straight through. There's never a line because they have more planes than they have people. Yeah, those are nice. It makes me think of some of Dave Kennedy's stories about how close he cuts it to getting to the airport. That would kind of wreck my nerves a little bit. Yeah, yeah. Tim Medine does the same thing.

I think if you travel just enough and you've missed enough planes, a lot of the stress starts to get away from missing another flight because you know you're just going to get on another one. Yeah, good points. So I guess we'll kind of get into the conversation. I always have my guests share their hacker origin stories. I know you have a really interesting one, so if you wouldn't mind sharing. Absolutely.

So I was supposed to be a humble Judge Advocate General, which is to say an Air Force lawyer. That's what I wanted to be growing up for the longest time. I was really into the show Harmony or. No, that's not the name. That's the main character, I guess, in the show. Forget the name of the show. Actually, JAG show is called jag. Right. And so that's what I wanted to be growing up for the longest time. And I'm a military brat.

My dad was in the Air Force 21 years, so I figured the best way to do that would be go to the Air Force Academy. And after about two years there, I joined the cyber competition team, and I absolutely loved it. And one of the things you have to do in the military is if you want to become a jag, you have to do two years in some kind of career before you're allowed to switch into that school. So I swapped out of intelligence into cyber warfare. As a result of that, I also got a little bit lucky.

The SANS Institute at the time was giving scholarships away to some of the service academy cadets for spring break. And a lot of my classmates, they just didn't want to give up their spring break. And I was like, hey, I'm going to be Harry Pottering this and staying at the academy anyways over spring break. Because my family lives in Alaska and I can't afford to fly there. So yes, please, I will take a all expenses paid trip to Florida for a week to learn stuff from. Who is that again?

Sandstone. And I absolutely loved it. I absolutely adored the cybersecurity stuff. I picked it up super duper fast. The logic just made a lot of sense to me and I just didn't really look back afterwards. I did end up applying for JAG school. I got in and you're not really supposed to not say yes when that happens, but I like cybersecurity enough that I decided to just stick with it and say goodbye to the legal stuff.

Yeah, I guess origin story wise after, after the Air Force I started up working with a bunch of different cybersecurity companies. Black Hills and Counter Hack Challenges. And with Counter Hack we actually got to do a little bit of military collaboration with private sector which had been something that I'd been really pushing towards while I was in the military. So when, when that didn't work while I was in the military.

But suddenly after I was with Black Hills for a year, it just sprung out of nowhere, out of the ether. That's when I joined Counterhack and we started working on this product called muscatatuck which was taking this big mount range for like urban combat training and cyber refying it.

And so effectively there was a hospital there, there was its own water and electrical production and we just made that into a fully cyber enabled range so that we could do exercises and such with the military and they could train in a range like that, which was really, really fascinating. Awesome project to be a part of. After that I kind of went off on my own and did open security and we've been doing all kinds of penetration testing work ever since. Very cool. So yeah, that was pretty.

Sounded interesting about the cyber range with Counter Hack. So was that actually a physical type of range as well? It was, yeah. So when we were doing exercises there everyone would actually be in, in person one of the, one of the capabilities that the military really building over the past, I guess it's a decade now, man. I guess it's over a decade now dating myself here. But it was the CMF construct and Obama came out with this and that's where we got the CPTs from.

And those are cyber protection teams or CPBs if we're talking about Army Cyber Protection Brigades and their idea is that we've got this like 39 person team and if some event happens in the country, typically the country, because we're talking about protection though overseas military Bases technically if an event like that happens, like let's say somebody hits the power grid and they've knocked out a state owned facility that does electrical distribution, they might deploy to that and then

start doing immediate defense. But like active defense because you can't just turn off the things that have been compromised since they're critical infrastructure they need to be on. So you need to learn how to fight through the compromise.

And the entire purpose of this range was to build a very realistic, a hyper realistic scenario where they could actually deploy out to the field physically and then have to fight through a compromise where a Red Teamers like myself were, were hacking the planet and we got to pull up some pretty fun hacks against the, against the army.

One of them, they, they, their TTP at the time had been to just join their, their systems to the Active Directory environment so they could operate on the Active Directory environment which a little bit of sense if you're trying to do threat hunting. But at the same time if your threat hunting machines are now connected to Active Directory and that Active Directory is owned completely by an adversary that can just pivot into your machines. Right. Which we, which we did.

And they didn't have any antivirus or any EDR or any detection on their own platforms because they had been planning to just orchestrate that into the remote environment. So that was a big gap and those kind of gaps were what we were trying to look to identify and then help them close. Yeah, that sounds really fun. And it's kind of interesting to see something that's more real world scenario because you see some of the events.

Because whenever I used to work, I used to work for US bank and we annually they had this cyber defense competition. So each team would try to attack each other and defend against each other and a lot of the scenarios really weren't realistic outside of your hacking each other. But that must have been pretty interesting to have stuff that was more real world. Actually have an Active Directory in the environment opposed to just standalone hosts and stuff. Absolutely.

Yeah. That was kind of a very wholesome experience if you will. Because in so many situations it's, it's always contrived. Right. In the military context we often talk about it as white carding, an exercise where we say, hey look, here's this thing that needs to happen and so we're going to white card it.

So it does happen, but it takes so much of that realism out of the actual exercise and engagement and then after a while you start doing it too much and the, the, the exercise becomes so degraded that all you're doing is running through a standard operating procedure, proving that you know how to follow the checklist and then what is the real value proposition.

So being able to take an exercise and really push learning into it, where training object kind of the first primary outcome that you're pursuing, that's a peerless experience I've found. And I'm of course a little bit biased because I'm CTF learned. So at the Air Force Academy when I joined that cyber competition team, we were competing in ccdc, made it to nationals a couple times with that. We have our own little cyber defense exercise across all of the different government universities.

So West Point, Annapolis, Air Force Academy and then the military war colleges and such like afit. And so those are fun. But what's really nice about it is that they have this like player versus player beyond just the environment kind of context. So as people develop their skills and get better, the bar gets raised and then you have to do the same if you want to stay at the very highest level.

I think that there's very few things in cybersecurity, particularly on the training side, that offer that kind of experience. And for those who learn more hands on if you will, like I do, it's an amazing way to pick up skills. Yeah, it's very cool. And one of the things that, that you know, I've kind of noticed recently there's some people that, that aren't fans of the OSCP or Offensive Security Offsec is they're called nowadays, kind of complain about the OSCP being ctf like.

But in my opinion, you know, I've worked as a pen tester knowing that CTF machines sometimes more difficult than real world scenarios because sometimes in a CTF they kind of purposely try to get you going down a certain rabbit hole when that's not the vulnerability, throw you off and you miss the actual exploitable vulnerability, which I think it's a better learning experience. So what are your thoughts on that? I would very much agree.

I do think that a learning experience or even a testing experience that's lab oriented is much more. Let me back up.

I think that if we're looking at certifications, there's a certain point for the certification to exist and that is if like if you're an employer or if you're somebody who wants to evaluate somebody's skills and you see a certification of some sort that might give you some insight into what their capability or experience might be and the other side is if you're pursuing that certification, I Find that studying helps you get better at doing cybersecurity work.

And I don't think that's really a novel concept. But, but when you're getting a certification, when you're pursuing that certification, you do acquire additional skills in the effort, if you will. Now what do those skills look like?

And I think that if you're dealing with a certification or exam like OSCP versus the SANS exams that are much more multiple choice, let's call it, there's a fundamental difference in what the skill progression is when you're preparing for that kind of exam for one of these versus the other. And I do feel like the hands on is really important, but even more so, what are the skills that you're trying to get out of that?

And I think that if I were to say one thing about the OCP exam that I, that I do appreciate, it's that there's a report at the end. I think the report is not just key, but I think it's the most important part. Because if you're really thinking about skills in the real world from a cyber perspective, you can spend tons and tons and tons of time bashing your head against the keyboard and then eventually stumbling upon what you're looking for. And so if you're passionate, you're going to get there.

But are you dedicated? And I think there's a big difference between passion and dedication because dedication means that you are writing that report before the engagement is already over. And if you can write great reports and if you can get them in on time, I almost don't care about your cyber skills because I feel like you can, you can accomplish those, you can get there. Yeah, very cool.

And it's just really kind of interesting to see that because one of the things I've also witnessed too is some of the best hackers I know are really good at ctfs and that's how they really sharpen their skills doing, doing CTFs. Absolutely. Yeah, that's what did it for me. I haven't done CTFs in quite a while, but I kind of hit my, my top point on ctf. So where there's no like going up from here and then I quit because going down from there was definitely possible.

So I did like CCDC when I was still in university and then I was really, really big in the net. Words Networks is actually my favorite CTF specifically because Net wars is a capture the flag exercise that's designed to teach you how to learn. And I don't think that there's a better way to Figure out how to do cybersecurity skills. Because I've been an educator for about a decade now and I can. I've taught so many people how to do certain things and they can repeat those things for certain.

But in the real world, every environment is unique. And being able to figure out how to twist what you've learned to fit in a, like a square peg into a round hole, I think that that is a very novel, difficult skill. That's probably the X factor, if you will. And I think that CTFs can be really helpful in accomplishing that. I think that of all the CTFs that I found, most CTFs aren't designed to train you. Most of CTFs are designed to test you and then you get a score.

And like, it might not be a test as in it's hardcore, it might be fun. But still, the CTF itself is more of an evaluation than it is an educational experience. I do think that educational experiences for CTFs are one of the most optimal ways that you can learn. So, you know, based on your unique perspective, you know, being a hiring manager and also having the education background, what would you recommend for someone that wanted to get into offensive security? Open source project support?

Yep, absolutely. And it's been that way for about, I don't know, 20 years, maybe even more. Back in the day, it was pretty straightforward. We would always say, hey, go contribute to the metasploit project. Because Metasploit project has just this huge group of people, people who've contributed to it, and it's got so much documentation about how you might go about contributing to the metasploit project, that's no longer true.

And so it can be quite a bit more difficult to figure out where you fit what projects you're interested in that you might want to contribute to. But I think that if you have a resume that reflects GitHub results and you're looking for boutique cybersecurity work, that those things tie together. And I should really specify boutique cybersecurity work is if you want to join a firm that does security work as their main line of effort.

So most of the time in a cybersecurity job job, you are a cost center for the organization. So you're there to reduce what the risk is and the likelihood and the impact of a negative event occurring to the actual organization, this is fine. But for a lot of cybersecurity people in that kind of situation, they do feel like they're a second class citizen inside of the company because the company's objectives aren't their personal day to day objectives.

And so for a lot of these folks, their objective is to eventually move into a Black Hills or a trusted SAC or Spectrop, something like that. These boutique cybersecurity firms, I think it's a lot more difficult to get employment with them because the primary thing that they need you to do is to slot into the team right off the bat. And without demonstrated experience, that can be really, really tough.

I think that if you can prove that you can be valuable to the team on day one, that can make up for a lot of that difference. And if you are a contributor to the tools that the team is using every day, then that makes you valuable on day one. Very cool. And speaking of open source, I know you've got a open source vulnerability scanner that you've created called Serious. Would you mind sharing about that? Absolutely. It's a transition.

So I've been writing for the past five years a open source vulnerability scanner and I've been an open source dev for, for a long time. My first DEF CON presentation was on a tool called subterfuge back in 2012 and I was shocked actually to have been accepted at DEF con, because I was, when I got accepted I was still 20, I hadn't turned 21 yet. And you know the tradition at DEF CON is your first time speaking you have to do a shot on stage that was a lot of fun.

And actually that shot turned out to be very necessary because I got in front of like 1400 people at DEFCON and oh my gosh, my nerves were to the 10 degree and I was like shaking in my boots. And while I had practiced this presentation so much, I knew like to the minute where I was supposed to be at the 20 minute mark, I'm like 40 minutes into the presentation and at that point because I've been talking so fast, I run out of air and I start coughing.

And if you know anything about Las Vegas, all of the hotels are full of smoke, right? And this was the final day of defcon. So I'd already been at the Treasure Island Hotel, which is the worst for smoke by the way. Don't go there. I had already been there for days and days and days. So I'm on stage in front of of 1400 people and this is all recorded and it's online for the rest of my life and I'm coughing up a lung on stage that I'm like, Chris, my co presenter, take the microphone.

And if only it had sounded like I was a dj. Chris, step up to the microphone, but it did not. Fortunately though, right about that point, because I'm a lightweight, I was like £130 at the time, maybe less even. Yeah, I was probably less than 130. So that shot actually starts to take effect and so Chris finishes up whatever he's saying and then I just start doing demo. I'm like, hey everyone, let's hack some stuff.

And the tack, the talk ends up going pretty well from there, but I really needed that shot. The moral of the story is that I have been very passionate about open source and open source development for the cybersecurity community ever since. It's been a passion of mine. But finding a project that really scratches your own itch and can really have an impact on the community can be difficult. Subterfuge had over 10,000 users, about 12,000 users at its peak, which was just absolutely amazing.

Replicating that is the goal. And, and the reason that's the goal is because if you're doing open source dev, you're not necessarily just doing it for yourself. Like I, I mentioned scratching your own itch and I'm very passionate about that ideas. And if you're building a project or a program and it doesn't do something that you need it personally to do for you, you're just not going to understand what it is that it's accomplishing as well as if you did. So I'm a firm believer in that.

And Serious Scan has both of those kind of ethos from the, from the jump. The first part is, as a penetration tester, I am so darn tired of hacking the planet, finding all the vulnerabilities inside of an organization, then coming back a year later and all those vulnerabilities are still there.

Yikes. But there are two reasons why this might happen. The first one we all know and maybe don't love so much. The client is maybe a little bit lazy or doesn't actually care too much about the report. Maybe they wanted the report for PCI purposes and that's it. So they just need the annual penetration test accomplished and they're good to go for the rest.

This is a little unfortunate and it does feel a bit defeating on the penetration tester side because we're doing good work here and we want to see organizations heal as a result of it. But the other side is actually, I think, even sadder. It's these organizations who go a year. Maybe they actually tried to fix the problem, maybe they tried to remediate the vulnerability, but it didn't work. Maybe they Missed a registry key, maybe for print nightmare.

They used the first patch and the first patch didn't do anything. So they needed that second patch which also didn't really work very well. So they needed the third patch which removed the vulnerable feature and then log 4J hits them the same year later on down that year. Can they get a little bit ahead of the penetration test by finding some of the basic vulnerabilities that we're going to obviously exploit and fix those so they can get maximal value off the penetration test?

That's one and two. Can they, after the penetration test has been performed, do the remediation and confirm with a vulnerability scanner that the remediation was successful. And you might say okay, just go ahead and buy Tenable or Rabbit 7 or Qualys to accomplish this and you could do that. And if we're talking about a Fortune 500 organization, they probably have licenses to these tools, if not potentially more than one.

But what about mid cap firms, firms about 50 million to 150 million. I also do incident response and I have done a lot of incident response for organizations that are right in this size of market capitalization and they're getting ransomware left, right and center.

And the tooling that exists for them at the moment either is very expensive, which is to say that they could probably spend their budget better somewhere else, maybe remediating things, maybe putting more defensive utilities in place, maybe a little bit of defense in depth, those kind of things. So their money is probably better spent on that than a huge contract with Tenable. And so as a result they're just lacking that as part of their grid. What can they do instead?

Openvas just really doesn't cut it at the moment. NMAP scripting engine, nice, but more of a hacker tool really. Don't expect an IT sysadmin without security background to be able to employ that. They could definitely run a tenable if they had it, but that doesn't exist. And that's exactly where Serious Scan comes in. So Serious Scan is a MIT licensed fully open source vulnerability scanner, specifically a general purpose vulnerability scanner.

It's got a ui, looks and feels quite a lot like nessus. And that was the first kind of component of the tool. First focus, need wise. The second side of it though is the scratching my own itch part. And that is to say that I despise using vulnerability scanners if I'm doing a penetration test. Absolutely hate it. Why? Well, the vulnerability scanner doesn't have me in mind and for good reason. If you were tenable or rapid. 7. Your big ticket clients aren't penetration testing firms.

They're Fortune 500 enterprises that are going to give you a contract for upwards of a quarter million per annum. That's great for you, especially if you're charging agent ways, agent wise by the system. Wow, you're having a great time there. But if I'm a penetration tester, my needs aren't being accomplished by these big major vendors. And so the second part of SeriousScan's ethos, if you will, is to be a tactical vulnerability scanner to get you as close to your vulnerabilities as possible.

And so that when you're interacting with them, when you're scanning them, you're going to see information like what ports and services are open, really, really quick, front and center.

You're going to see information about vulnerabilities that is more filtered towards things like the known exploited vulnerabilities list so that you know that maybe there is a metasploit module that's available for a vulnerability and it highlights those because those are more actionable for you as a penetration tester. And that's kind of the goal, if you will, of a tool in general. I know that was a lot. Very cool.

So as far as how does it kind of differentiate from the commercial tools as far as like speed and performance? That's a good question. So technically it performs way faster than the commercial tools. If you were scanning something with Nessus, it takes about 20% that amount of time to scan it with Sirius. But this is not because my tool's better. In fact, it's because Nessus is doing a whole lot more.

Currently Sirius doesn't have a huge repository of scanning scripts, so it does a number of things in order to find vulnerabilities. It'll do common platform enumeration and then lookups to do vulnerability correlation, which the other scanners do as well. And that is the majority, I would say that's 80% of the vulnerabilities that you're going to get from a tenable. And Sirius does do all of that. So you got that kind of 80% coverage.

I've actually used the tool in some of our corporate environments or client environments so far. And what I found is that I am getting about 80% coverage, maybe more of the critical vulnerabilities, but we're missing quite a few of the mediums, the lows, the informationals, those kinds of things. And that is really to do with the depth of the scripts that we have to do scanning. And so when I say it's way faster. That's technically true, but it's also doing quite a bit less.

That is something that we're trying to fix into the future. This isn't live yet, but one of the things that we're really hopeful about is how artificial intelligence might integrate. Specifically, I would like to have automatic creation of scanning scripts based off of vulnerability data for new vulnerabilities that come out and we've got in the works right now we're calling it Validity GPT Labs.

We're building something that should be able to provision vulnerable systems and software and then automatically provision potential scanning scripts to identify if the system is vulnerable or not. And then try those and then send that straight to a GitHub CI CD pipeline so that if a new vulnerability comes out from nist, we'll immediately have a scanning script that gets put into Core Sirius.

That is a long term project that we're using to hopefully catch up to something like a tenable and a Rapid 7. But in the short term we're focused more on the Kev list. So if it's a vulnerability out there that is known to be exploited and used by adversaries, that makes that a first class objective for the serious Scan project to be able to and for.

The listeners that aren't really don't really have a pen testing background things that around the Kev list stuff that's exploitable is really more important in a pen tester than just finding a bunch of CVEs that, that may not really amount to anything, just the exploitable type stuff. That's absolutely right. Last year for example, there were about 40,000 CVE numbers issued. 42,000. I think that's a huge number of CVEs. Right.

And if you're thinking about intrusion sets, adversaries out there, they're not picking up 42,000 new techniques every year. They're just not doing that. They're picking up more like a dozen a piece. And those dozen are probably shared like 90% of the dozen that your average intrusion set picks up is probably the same, you know, dozen that the other average intrusion set does.

So that means that every year we're really talking about like 2030 vulnerabilities that actually matter, but there are 40,000 plus that come out. And so what that tells us is that the vast majority of CVEs of common vulnerabilities and exposures that are identified will never be exploited by anyone ever, for any reason, period. With no exceptions. Like that's just a fact. So when it comes to penetration testing, you don't want to be going through 42,000 vulnerabilities per year per test.

That's just not really feasible. So how do you filter that down to things that really matter? And the known exploited vulnerabilities list by CISA is a really powerful place to start that and then say the metasploit list where you have listed vulnerabilities that have exploits available for them too. Yeah, that's very, very cool.

And I think really important for people to understand because people get too caught up in sometimes on the CVEs and not what's exploitable because you take some of the lesser experienced pen testing companies that come in that it looks like a NESSUS scan. A lot of cases, the majority of what they're doing is running a NESSA scan, see if they can exploit some things. Sometimes they're not doing a lot beyond that.

And there's some companies that are basically putting a NESSA scan in their boilerplate template and selling as a pen test. And then the customer has got to prioritize through all that stuff and figure out what needs to be remediated and they've just got this laundry list of things that really don't matter. That's absolutely right. We like to colloquially call them pen test puppy mills, but.

But yeah, they're a big problem for the industry because of course it's also rather cheap to be able to do an engagement like that. You just spin up your NESSA scanner, you send that out. Maybe you have Ghostwriter for a reporting tool, you import your NESSA scan, Ghostwriter's got a template, you just send that to your client and then boom, wham, bam, you're done. But while that does produce theoretically the same kind of output as a real penetration test does, it lacks all of the depth.

For example, when you have a client who doesn't know what they're getting from a penetration test, maybe it's their first pen test. They're expecting to get a report with a list of things to fix. And for better or for worse, if you run a NESSA scan at an environment and then you just say, you know, here's your highs and criticals, here you go, that's a list of things to fix. It's just not a very good one. It's not a prioritized list of things, it's probably missing a lot of stuff.

There's likely a lot of false negatives, which is to say that the vulnerability does exist in the environment, but the penetration Testers didn't find it because automated scans are really bad at finding the vulnerabilities that actually matter. This is another thing that I'm trying to do with Serious Scan to kind of change that paradigm. For example, the agent in Sirius should be your pen tester's best friend. It should be your pen testing agent.

And so there's a tool out there called Crypt Breaker. A friend of mine, Jeff Pamello, he wrote this tool and it's uses aws, but it's a password auditing, password cracking tool. And we're working on integrating that this summer into Sirius. And so the idea there would be you've got your agent on the systems, that agent can do automatically identification of what the password hashes are for the user accounts that are there.

And if you have this agent throughout the environment, you can just say, find me some duplicates. And if you find duplicates, then you have a confirmation that you have reused passwords between multiple users in the environment. This, in my mind is a vulnerability. A vulnerability is a flaw or weakness in a system that could be exploited. And if you know anything about attackers, the most primary flaw that they love to exploit in systems is that you have weak passwords or reused passwords.

Vulnerability scanners today don't have any concept of looking for vulnerabilities beyond. Well, that's not necessarily true. They do have a concept of looking for vulnerabilities beyond the cve, but they have no understanding of looking for vulnerabilities inside of core features like your login system. And I really feel like your vulnerability scanner should be able to help with. Yeah, that's a good point.

So kind of for the folks listening that may be on a management side or maybe not pen testing, how would you recommend them find a good pen testing company? It's not a puppy mill, trust me. That's a terrible advice. So you can always ask them if they have references. I will say this is not necessarily the most successful approach. It's because with a lot of these pen testing firms, we're under very tight NDAs with our clients. They don't necessarily know who.

So the adversary, let's say, wouldn't necessarily know who's gotten tested by us. And they might know our methodology and then try to attack a client based off of that. But you could ask for some of that because some of those organizations do have clients they work very closely with. So in my organization, for example, if you were to say, hey, look Matt, I need you to prove to us that your organization can do really, really Solid penetration tests.

One of the things that I would love to do in that situation is say, hey, look, here's a client of ours. They've been with us since they got ransomware and they're very, very happy with the work that we did to heal them from that ransomware and the work that we've been doing with them since. I would recommend that you get on the phone with them and talk through how the engagement and how the relationship with your security partner has been. And I think that that's really key.

If you've got an organization that's looking just to do a service for you and then they're going to get out of dodge, that's probably not the kind of organization you want to be your cybersecurity partner. And that is absolutely what you should be looking for in an organization that's doing your annual penetration test or your quarterly vulnerability assessment or annual vulnerability assessment.

I'm not sure what your tempo might be, but whatever it is, that cohesion between one engagement to the next engagement, that cohesion between the engagement itself and the reporting of it to you, maybe remediation, testing, all of this stuff should be a much more partnered relationship than it should be transactional.

And if it is more of a partner approach, or if you're getting the vibe, if you will, from the conversation that it is more of a partnered style approach, that is likely a really high thumbs up. Because one of the things to note with Pentest Puppy Mills is that they are very tight margin organizations, which means that if they wanted to be your partner, it would blow the budget for them, much like it would for an mssp.

So if you're looking for a security partner as opposed to a vendor to simply do a transactional assessment, you're probably setting yourself up in a good way to find an organization that'll treat you right. Very cool. So you kind of back to, you know, talking about doing the open source work, for working with open source projects to kind of help get your start. So what programming language would you recommend someone learn? Great question.

I have changed my minds about this very, very heavily over the past couple years. It has entirely to do with artificial intelligence. So I used to say that I don't care. There's no such thing as programming languages. There's only one programming language and that language is logic. Everything else is syntax. That's what I used to say, and that still remains true. But there are certain languages that AI understands the syntax of much, much, much better.

So the first thing that I would recommend is that you use a garbage collected language. A garbage collected language is one that you don't have to do the memory management of. So that takes things like Zig, Rust and C off of the table. Because you just don't want to be dealing with memory management for two reasons. First off, it's hard and a pain in the butt and you might break it and it could be very frustrating.

The last thing that you want to do when you pick up a programming language is get frustrated and quit early. The longer you can do that first, sprint, if you will, that first experience, the longer that you could stretch that first experience and still have a good time, the more you're going to fundamentally understand and learn. So I would recommend, first off, not picking something hard. And so a language that doesn't have memory management, that's, that's key.

The second reason why you don't want memory management is because AIs hallucinate a lot. And today if you're doing coding, you might be doing a fair amount of it by hand, but you're almost certainly using something as far as AI assistance that might be just. You have Chat GPT open on the side and you're asking it questions and then you're using its answers instead of Stack Overflow. Because it is fundamentally better than Stack Overflow for helping you get through bugs and issues with your code.

Because with Stack Overflow you have to wait for another person to respond, and that person is probably not going to respond nicely, let's say. Did you even read the documentation with, with the AI, you don't have that. The other thing is like Copilot and such are really, really powerful these days, but they understand the languages that they've been trained on better than others. And those languages are threefold.

It is TypeScript, that is the language it understands best by significant margins. That web code, it just gets it. Golang, I would say, is another one. And then Python, I think Python and Golang are pretty close. But the thing about AI is that if it understands the plumbing, so let's say you had code that was very heavily documented, in that case, AI is probably going to work with that code base pretty darn well.

And so if you have AI making your code and you tell it to document as it goes, whatever code it generates, it's likely going to be able to continue to work with, which is fantastic. But the bigger thing is languages basically have built in documentation in certain cases. And I'm talking about types here.

So if you have a language that is very oriented around types, maybe it's pretty type safe, then your AI experience is going to be a lot better because when it looks at one function's inputs and another function's outputs, it can marry those two together because the documentation is basically in your type system itself. And this is one of the things that makes it so much better at TypeScript than JavaScript.

Even though TypeScript just compiles into JavaScript or transpiles, I guess I should say, in a JavaScript. And so I think the best language to learn is probably one that's somewhat visual. I would probably recommend people start with TypeScript, and then my second recommendation would probably be either Golang or Python. Me and Python don't go very well together anymore.

I'm just not a fan of Python 3.7, so unless I'm using something where I need a specific Python library, I'm completely avoiding that. I don't like to work with the virtual environments that we have to deal with Python these days either. So I'm personally just a Python hater, which is funny because Subterfuge, my first big tool, was like 30,000 lines of Python 2.7. So I've done a lot of work with Python, but these days it's just not the language for me. I think Golang is very powerful.

It's. It's really easy to use the standard library. And Go is very, very strong as well, and AI understands it too. Yeah, one of the nice things about Go, yeah, one of the nice things about Golang is be able to create the portable executables. How, you know, with. With Python, you have to use Py2exe or something like that to. To get the same, same type of output. But it's. Yeah, it's pretty interesting. I know Jeff Foley, the creator of owaspamas, is a big, big fan of Golang. Is he? Yeah, Go is.

Go is so easy to write. It feels a lot like Python minus the indentation needed. It took me a little while to get used to it because there are a couple idiosyncrasies within Golang that get a little annoying.

Like it has mandatory error handling and if you just wanted to, like shove code into something and then get a response, which is what Python really is good at, and you're used to that, then Go feels like you've got these golden handcuffs on or these silver handcuffs, but once you get used to it, you'll find that all of the things it forces you to do that you were skipping before, are really actually helping you out in the end. So. So there's that? Very cool.

So we're getting down towards the end of the episode. Is there anything you'd like to share that we didn't cover?

Maybe we have time to talk about vulnerability management a little bit? Yeah, yeah, definitely. Outstanding. Well, so vulnerability management has been a kind of a sort of aligned passion project of mine with seriousscan. So my life has kind of been consumed with these higher level vulnerability things for the past five, six years, maybe longer than that, maybe even call it 10 years if we include the SANS class. So I used to run a SANS class for enterprise threat and vulnerability assessment.

And what I found from that is that people don't know how to do vulnerability management. And it's not because they don't want to, it's because of how we've kind of built it.

And I find that a lot of enterprises today, the way that they're doing vulnerability management is that they just have a person who gets these big, big laundry lists of vulnerabilities and they're just a product manager for finding out how those things are going to work or where they're going to go, are they going to get patched, those kind of things. But I think that vulnerability management is the major missing feature or missing gap in our cybersecurity grid today.

Because we talked earlier about those zero days, how 53% of vulnerabilities that are used in mass Compromise events are zero days before that event actually happens. That's from Rapid7's 2024 trends report, by the way. This is startling. This is super duper terrifying. Because if we think about vulnerability management, it is our core to resiliency on the cybersecurity side, it's what we use to know that we have a certain level of defense and depth in place.

For example, if a penetration tester comes in and they just absolutely wipe the floor with the environment, I can already tell that that's probably not an environment that's had their vulnerabilities managed particularly well, because it means that we're probably not doing customized risk assessment. Maybe we're doing a lot of indiscriminate action. We are patching, but we're patching the wrong things, not the things that matter the most.

Are we using threat intelligence within our vulnerability manager program? Are we doing threat modeling? Are we even doing vulnerability validation? Or do we just trust anything that comes out of the scanner to be gospel?

And I find that almost all of these things to some degree, and in a lot of situations, to a major degree, exist in almost every organization that has a vulnerability management program out there, and then there are all these other organizations that don't have a vulnerability management program at all. And it's really unfortunate because vm, I think, is how you extend the time from initial access that an attacker has to their effects delivery. And I think this is really key.

And it's what red teams exist to test. Right? Because the idea there is, if we detect and we respond inside of that continuum, inside of that window of time where the attacker gets that initial access and then delivers those effects, it doesn't matter that we were vulnerable, we still kind of won. But that takes time. And so with vulnerability management, we're really trying to stretch this window to make it take more time.

Unfortunately, I don't find a lot of vulnerability management programs focused on things like compensating controls today that can really help out with that. Very cool. So what are some steps someone could do to improve their vulnerability management program? Because a lot of people, I think, just think vulnerability management is just running reoccurring scans and they kind of leave it, leave it there. So what are your recommendations on running a successful vulnerability management program?

Absolutely. I would start by tracking down the lifecycle of vulnerabilities in your environment. So, for example, when you have a vulnerability that enters your vulnerability management program, how does it get there? Maybe you're using tenable security center, and maybe 80% of the vulnerabilities inside of your vulnerability data management are from tenable. Okay, that's fine. Is it the only way? Are we getting vulnerabilities from SAS scanning, from DAS scanning?

Do we have application specific vulnerability scanners? Do we have penetration tests that are performed? Do those find vulnerabilities? Do we treat all of the vulnerabilities that our program is identifying with the same measuring stick, which is to say, do they all go into ServiceNow or Archer or Brinka? Do we do our vulnerability management with Excel? Whatever that piece is, is likely going to define how you can consume, process, and work through your vulnerability data.

And if you're missing something there, or if it's not all funneling into a single point of truth inside of your vulnerability management program, you're going to be doing a lot of that indiscriminate action. So tracking the lifecycle of vulnerabilities from initial discovery inside of your program all the way through exit, which might be patching or it might be risk tolerance, it might be risk mitigation.

There's a lot of ways that you can have a vulnerability to be exiting your program, but we tend to look at it as if it needs to be fixed. And I think that that's another thing that hamstrings or vulnerability management programs a fix or a solution and a fix might not be mutually exclusive. Yeah, that's interesting.

One of the things I kind of see too that I think companies could optimize what they're doing because there was a situation where I worked at this company and one of the things we saw company wide was like an ILO vulnerability across every system. And they're going to continue to pen test every infrastructure in scope throughout the year. And they got these vulnerabilities.

And so kind of my opinion, I think sometimes you find something that's a broad affecting issue is to go ahead and start remediating those before it's pen tested. Because if you know everything is this across the environment, if people take a proactive approach, they can, they can reduce the risk. You don't want to wait a year or several months to remediate that finding. That can be exploited possibly. I think it gets more value out of your penetration test too.

Because I like to always talk about pen tests as if it's a pen test story. Because that's really what we're doing, penetration testing versus say vulnerability assessment. They really have a big difference in scope. And so if I'm doing a pen test, I might try to find a story about how I can make risk actual for your organization. And then if I have more scope and time left, I'll just try to do it another way. I'll try to find another story where I can make that risk actual.

And we'll keep doing this until we run out of scope. But by the end of that, have I found all of the stories? Well, obviously not. And if we have a lot of these, these baseline vulnerabilities that can be the root of a story, starting then suddenly we have tons and tons of potential work for your penetration testers to do, which is fine and dandy until you realize that the outcome you're looking for is the more fine grained, the more fine tuned in depth work the penetration tester provide.

Nipping a lot of those more broad scoped vulnerabilities that maybe hit a lot of systems inside of your environment to reduce that attack surface or at least mitigate the impact of that attack surface is extremely valuable. Yeah, you mentioned risks made me think of another, another topic.

So how can security leaders leverage risk and other ways of communicating with those in upper management and outside of technology to be able to kind of sell the case for their security budget, sell the case for remediation to be able to, you know, actually help reduce the risks in their environment. Absolutely. I think twofold. First off, I think you should lean on your penetration testers. If you have third party pen testers and there's some kind of risk that you're looking to sell.

Excuse me, that there's some kind of risk you're looking to sell, why not make, make sure the penetration tester focuses on that and writes that up in the report and perhaps even writes up what your recommendations are. Because if you have a third party coming in as a champion for what your solution might be, I think that that has a lot of weight with management. It seems a little weird that management wouldn't believe their internal teams without that external validation, but it's true.

And so your penetration tester is likely going to be over the moon about that because they still want to provide value and if this is another way for them to do more of that, that's outstanding. The other side is I think it's really good to connect it to real world events. So if you can say, hey look, we have this vulnerability in our environment. Equifax had a similar vulnerability because it's a Java based application.

Let's say we should fix this this because we don't want to be like them and here's the amount of damages that it caused them. These days we are in the best place that we've ever been in cybersecurity to have those kinds of case studies. You even have some of these case studies being publicly revealed like PricewaterhouseCooper's report on the Conti breach of Ireland's HSE, their healthcare security service, Healthcare service executive.

And so this is this like 120 page report that's on instant response on everything that the attacker did. All of the improvements that the HSE made in order to make sure that didn't happen again. And if the improvement that you're trying to sell to management is on that list, why not reference that report as well? I think that that's a very potent way to get eyebrows raised and minds changed.

So what's a good source for someone to get like this threat intelligence to be able to leverage that and kind of do what you're describing there? Honestly, I think bleeping computer is probably the best source because if there is a major cybersecurity event of any sort that's happened, there's going to be an article on there about it. It might take a decent amount of reading. To find all of the ones in the past.

But I would certain paying attention to the articles that come out into the future so that you can make a note of them. I'm not sure that there is a single repository. Well, Wikipedia is a single repository, actually. So if you're looking for major cybersecurity events, there is a Wikipedia page on those. And the reference in section on that will have references to articles on every single one of those. Now, which ones are the ones that are most valuable?

I think this is another thing that can be a little bit rough. The trend reports can be really useful here too. So, like Verizon's annual trend report that they produce. I read that one religiously every single year because there's always some degree of statistics that I can pull out of that. That and then leverage. So I think trend reports can be really useful. Those are a bunch easier to find because you're just looking for the major vendors and the ports that they put out, like Rapid7.

Verizon's report is pretty infamous. CrowdStrike creates a bunch of these every year as well. And then on the breach side, you're really looking to see if there is public reporting about what occurred during the actual breach and if there's public reporting about the monetary result of that. And the monetary result might come out much more in the future. If you take Merck's breach by WannaCry, it wasn't until 2021 that they settled their lawsuit with their insurance firm for $1.2 billion.

So it can be important to go back and look at a breach that you maybe have heard of, because more information and more evidence might come out after it's already left the news site. Yeah, great. Great information there. So we're getting down towards the end of the episode. I really appreciate you joining today. It was great to catch up. We hadn't got to speak in a while, I guess, really last time we spoke was probably at Texas Cyber Summit. I'll know that.

We ran to each other since then, I think. Wild West. Okay. Yeah, yeah, that's true. Yeah. Yeah. Are you gonna be in Deadwood this year, by the way? Yeah, I need to go. I submitted a talk and if I don't get accepted, I need to go anyway because that's a. A fun conference. That's been one of my favorites so far. I got to go in 2022. Yeah. Definitely planned to be out there. That's probably the last time then. Yeah. Was it 2022? Wow, that seems like so long ago.

Yep. Yeah. Would have went in 2023, but my daughter got married and her wedding was going to be. So they're, they're that end of the week where Wildwiss Hacking Fest was. Friday was their rehearsal dinner and Saturday was the wedding. And I thought that would be cutting things too, too tight. And with the way the airline industry is, you know, flights getting canceled and stuff, I thought if I miss my daughter's wedding, I would never be able to forgive myself. So I, so I didn't go.

But definitely want to make sure to get there this year. Yeah, absolutely. Yeah. Yeah. Well, I'm sure your talk will be accepted and I'll be in the audience. Yeah, sounds good. Look forward to seeing you. Maybe we'll see something before. See you before then. And definitely want to get up to Seattle sometimes. That's still one of the places I haven't been to yet. And so. Yeah, absolutely. Well, don't be a stranger. I'll. I'd be happy to show you around the city. I do.

I'm new to the place, but I do love it. Okay, very cool. I'll hit you up. So thanks again. Absolutely, Phil. Thanks, everyone. And we'll see you on the next episode. Thank you for listening to the Philip Wiley Show. Make sure you subscribe so you don't miss any future episodes. In the meantime, to learn more about Philip, go to thehackermaker.com and connect with him on LinkedIn and Twitter @ Philip Wiley. Until next time.