Welcome to the Azure Security Podcast, where we discuss topics relating to security, privacy, reliability, and compliance on the Microsoft Cloud Platform. Hey everybody, welcome to episode 56. This week is another light week, it's only myself, Gladys, and Mark and Sarah are all on vacation. But this week, we also have a guest, Michael Malone, who's here to talk to us about Microsoft Defender Advanced Hunting. Before we get to Michael though, I do have a few little news items.
Azure Advisor for MySQL has now just gone in preview, and it allows for basically Azure Analysis, giving you things like performance tips, and giving you security ideas, and just some advisory notes about securing and improving the cost efficiency, and the efficiency in general of your products in this case, MySQL. We've now also added the ability to support custom certificate authorities in Azure Kubernetes Service in AKS.
This is really cool because that way, if you decide to create your own certificate authority, you can now install your own root certificates in there, and then use them just as you would do. Say, I don't know, GoDaddy or a various sign root certificate. So that's another nice thing to see, and I know a lot of customers have been asking for that. Azure Application Gateway now supports private link that's in preview. I've talked about private link, I think just about every single podcast.
More and more past services are supporting private links so that way you can have private IP addresses, private DNS names, and essentially having traffic flow between two past services without going over the public internet. So that is Azure Application Gateway. We now also have in public preview, continuous backup enhancements in Azure Cosmos DB. I don't pretend to be an expert in backing up Azure Cosmos DB, but having some enhancement there is probably a good thing.
We've also now added to API management, content security policy, and core configuration support. So core is cross origin resource sharing. It is a critically important defense in browsers. It allows you to share resources across domains without violating the browser security policy. You got to be very careful there though. Don't go putting a star as that wild card to say, I support absolutely everything from absolutely everywhere.
That defeats the whole purpose of course, but it's great to see that because now you can actually set it both in the actual portal and you can also do it declaratively. So that's the news that I've got this week. Let's turn our attention to our guest. This week, as I mentioned, we have Michael Malone, who's here to talk to us about Microsoft Defender, Advanced Hunting, and KQL. Michael, welcome to the podcast.
Would you care to take a moment and give our listeners a little bit of background on who you are and what you do? Thanks for having me, Michael. So yeah, I'm currently on the Defender 365 customer experience team. So I work helping large customers on board and operationalize the Microsoft 365 Defender XDR Suite. So XDR is an expanded detection and response capability for that's helps customers out finding suspicious and malicious activities at their enterprise.
It's kind of like an upgrade, if you will, from a traditional antivirus. Prior to being on this team though, I spent about seven and a half years with Microsoft Dart. So essentially what we would do is we would go investigate, determine human adversary or human operated type attacks. So when you had a human at the other side of the keyboard, it's not just purely a malware problem, if you will. And actually, I heard you had a new book coming out on Azure Security, Michael.
Yeah, I've been talking a little bit about that. I'm not kidding, we're actually literally four or five pages away from actually having the drafts completed. So we're really excited for that. It should be out in November. And it's designing and developing secure Azure solutions. Yeah, I actually just put out a book last year. It's called Designing Secure Systems, which is kind of a theoretical based approach on system security overall.
So it's like, essentially, if you can imagine a unified model that lets you look at physical human process as well as cyber systems and how a vulnerability in one area can lead to an issue in another. So the topic of this podcast is advanced hunting. So the question is gonna be, so what is advanced hunting and what does it sort of entail? Absolutely, so advanced hunting is kind of like a roll your own detection capability inside of Defender. So Defender, as I mentioned, is an XDR.
It's essentially a capability to identify not just malicious but suspicious activity. And the reason why I make this clarification, traditionally, when you looked at security products, you had antivirus. And antivirus is tuned to specifically find known bad things. It's very difficult. The way I like to describe it is, from an antivirus perspective, if you clean up a legitimate file, you're likely to make news. Whereas if you miss a bad file, you're just a bad antivirus solution.
EDR and XDR kind of helps you cover that gap because we are essentially tuning to avoid false positives. If you have a false positive, you can take down a customer's enterprise. So EDR enables us to identify suspicious activities that may not necessarily have inherently malicious files involved. So for example, if an attacker was to perform lateral traversal, depending on the tooling they use, they may or may not use malware.
They also may be performing some other suspicious administrative activities during the attack. So EDR covers that gap. EDR being endpoint detection response. And then more broadly, so about two years ago now, we launched Microsoft Threat Protection, or MTP, which eventually became Microsoft 365 Defender, which is our XDR, as I mentioned, it's the expanded detection response. And the big difference between EDR and XDR is the breadth.
So in an XDR solution, you're not just looking at endpoints, you're typically looking also at things like identities. So Microsoft Defender for Identity is our UEBA and also Identity Detection Solution. There's Defender for Office, which is looking at suspicious and malicious email activities going on. And Defender for Cloud Apps, which is looking for shadow IT and suspicious use of cloud applications.
So Defender Advanced Hunting gives you the ability to do raw access hunting to the data we use inside of Defender to create detections. So we create a whole bunch of detections out of the box for things that we know are suspicious or malicious, but sometimes you may have something that's contextually unique to your organization.
For example, remote administration tools are not inherently malicious, but you may know inside your organization, you only use one product or another, you might only use remote desktop to manage these systems. And as a result, you wanna identify things that are not remote desktop that enable you to control those systems.
With Advanced Hunting, you can create these queries that'll identify when these processes have been created or when files were written or activities occurred on these devices and also enable you to create custom detections. So if your query is really good at identifying suspicious activity, you can have Defender automatically respond by isolating the device, quarantining the file. It basically gives you raw data access to create your own customized detections for your enterprise.
So detections is one of the things you can do with Advanced Hunting. There's also a whole bunch of other capabilities that we use as well. So customers sometimes wanna have kind of a picture of their entire enterprise, i.e. find out where this particular software package is installed, see where this individual user logs on. So it gives you this kind of cool reporting capability as well.
Since we have all these process creations and logon events and network communication activities and file creations, et cetera, emails and such, you have the ability to kind of create your own report of what goes on in your environment. So you may be able to do your own baselining. You can also identify anomaly-based activity as well.
Another thing is from on the baselining front, if you get out of the box and what Defender really is, you can do some neat things around, for example, we have queries that'll help you design your app blocker policy.
If you're not sure where, like what the impact of a particular app blocker or a WDAC policy is gonna be, you can very quickly report on where, what processes would launch from that particular path that you're allowing or declining, or you can also use it as a mechanism to figure out how many different executables and how many different devices have things launching from that. So it helps you kind of tune app blocker, WDAC, or like firewall policies is another great example.
You can see where inbound and outbound communication happens inside the enterprise. And when you wanna determine the impact of allowing or blocking a specific IP address or application, it'll help you get that picture of what's going on inside your enterprise. So you know it before you actually implement it. Hey, what's WDAC?
WDAC or Windows Defender app control enables you to build a policy that can define what applications are or are not allowed to run on a given endpoint, a given Windows endpoint. For example, you can say files that are signed by a specific certificate are allowed to run, or things from a specific path are not allowed to run. You can also use it for an auditing capability as well.
For example, you can say if a process launches in this particular context, I wanna log this and see it from your Defender Advanced Hunting Console. So this is kind of interesting, right? So you could potentially use it for sort of alluding to this for more than just detection, right?
Because you could actually just run a normal clean system, a known clean system, and get an idea for which applications are running, where they're running from, if they're digitally signed, which hopefully they are, what network ports and protocols are being used. And you could actually build that up to be like a list of what good looks like. Is that a reasonable way of putting it, or is that just way too simplistic? So that's one approach you can use.
You can definitely, you can also, it's worth noting there's, because we're using KQL or Kusto query language under the hood, there's also the ability to bring in external data. So if you have a data set that has like, for example, systems like maybe IP address ranges, or list of file hashes or signers or whatnot, you can actually dynamically import it into your queries.
So there's definitely one way of doing it, of basically saying, this is a known clean box, and this is what its profile looks like. But you can also, a lot of times, enterprises are very complex. There's gonna be some sort of drift that's gonna happen across the board. So what we can do with advanced hunting is build a picture using summarizations and pivots and joins and such, that tells us dynamically what your enterprise looks like today.
The interesting thing there is, you can also dig into those outliers. So that long tail that everybody talks about in statistics is a very real thing in an enterprise. But it's also where a lot of the really interesting aspects of your enterprise are. These are where your attacker tools might be, for example. Or this might be somebody who's just violating policy for one reason or another.
It gives you this really cool ability to dive into those and rationalize and figure out what kind of policies you'd like to create with tools like WDAC, or ultimately, if you wanna create a custom detection to respond to such a thing. Can I get pre-built hunting queries, or do I have to start from scratch? I mean, let me put it another way. So I'm not really an expert in threat hunting by any stretch. I can't imagine as a little API I can run that says, show me how I got whacked.
Show me bad things that are happening. I mean, because I don't know what I'm looking for. I'm gonna be honest with you, I don't know what I'm looking for. So do we have built-in or sample queries that people can use that might show sort of indicators of either malicious behavior or even indicators of compromise? So there are, there's a couple of really great resources out there. So the first piece of advanced hunting is understanding KQL itself for Kusto query language.
We've got the Kusto query language referenced. So this is gonna be the same query language using Sentinel if you've used it. It's also what you're gonna use on what's called an Azure Data Explorer cluster. So if you've got experience with those, you're using KQL. But there's also really great web-based doc that references the entire language. It's pretty easy to get used to. I saw a shirt the other day that actually says, KQL is a new PowerShell and invite me stop and think.
It really is across our security stack. It's really the key language you need to understand. And as well as, as I mentioned, it's really great for if you have an Azure Data Explorer cluster. So the second thing is understanding the data landscape that we've gotten advanced hunting. Depending on which products you've got from Defender, you may see information from devices, emails, apps and identities, et cetera. So understanding those is really important.
And for that, we've got our schema reference. There's a public face, there's some public facing docs that are out there on it. But ultimately my favorite spot to go, if you open up the advanced hunting page in Defender in the upper right-hand corner, there's a button for schema reference that breaks down every single table and every single column and provides some really great detail.
One that I would like to highlight, especially if you're a Defender for Endpoint customer, is the device events table. We have a lot of really good information in there. For example, scheduled task creations, plug and play device activity. So if you wanna know if somebody's plugging in a USB drive, you can see that, you can see volume mounts, a whole bunch of really good information for creating your own custom detections or researching activity inside your environment.
And if you wanna get started with advanced hunting, there's a really great series I put out about two years ago with Taliash called tracking the adversary with MTP advanced hunting. We basically start off with very one-on-one approach. So we cover the basic operators, like take, for example, the where clause, et cetera. And we cover and cover each of the individual datasets that we've got advanced hunting at that time. And ultimately build their ways up to things like joins.
So there's all different kinds of joins inside of KQL they're important to understand. And last, we start moving into summarizing, pivoting and joining. So essentially, if you wanna build these reports or summarizations of what's going on inside your data. And the very last episode we dig into a contrived hunt. So we've gotten a targeted adversary situation we replicated inside of a defender tenant.
And we go through and use the things that you learned in those different episodes to hunt down the adversary and figure out all of its attributes. So one thing I mentioned at the beginning that really sort of piqued my interest is he talks about tracking down human adversaries. Is there any sort of tips, tricks? Is there a format or structure that you go through to help with that? So one of the tools I like to use is I like to call it the ABCs of instant response or the ABCs of security.
Essentially the ABCs represent the things that you wanna look for in any target attack or really any cyber situation you run into. So to get started, so the ABCs are authentication, backdoors, communication channels and data. So authentication represents the identity aspect. So what identity did the attacker use when they connected to the service? After they contacted this particular service, what identity were you using on the device itself?
So for example, if it's a web server, you might be anonymous coming in, but if there's a vulnerability in that web page, you might be able to run code as the web server. In that case, the identity you'd be looking for is the identity of the IS, Apple or perhaps the Apache service. The third piece of identity is what identities were compromised as a result of that event. Now you've got on the box, do we have credentials that are exposed to memory and are you running as admin?
Is there perhaps a password file or something on those lines? What types of authentication does the attacker now have access to as a result of that attack? The second one is backdoors. Now, backdoor is a malware term, but ultimately it's the mechanism that the attacker uses to control the endpoint itself. So a backdoor can be a perfectly legitimate tool and an intended capability. And our web server, the backdoor, the initial backdoor they connected to is essentially the web server itself.
It could also be an exposed RDP port or it could be a piece of malware. If they've already infected the device, you might have a remote access Trojan or something along those lines on the device that enables them to control the system. So just like authentication, there's three time frames. You've got the initial access, so what backdoor did they use to control the endpoint itself? Was there any backdoors installed as a result of the event?
So for example, did we see a downloader or a dropper install a backdoor into the system? And as a result, what backdoors do they have access to as a result of the attack? So now you've compromised this device. Presumably you can use a device to pivot to other devices that were protected by a firewall or a security mechanism. So it's essentially the mechanism for control. The third one is communication channels. And that essentially describes the way that the attacker communicates with a device.
It's the path between the attacker and the backdoor. So in our initial access scenario, when the attacker contacted the web server, it might be a user agent string or a source IP address. It could also be more generic things such as countries or ISPs or things along those lines. These are all things you can use to profile that attacker activity inside the enterprise. You also have, of course, the post breach scenario where what communication paths are open as a result. And last, you've got data.
So data, it really represents the impact. It's the confidentiality, integrity, and or availability that you've lost of information as a result of the breach. So if this is a ransomware or a wiper attack, some of your data may have been encrypted or destroyed. Or if it's a tamper attack, you might have some additional rows in your database you didn't intend. Or last, if it's a data theft or intellectual property theft type case, it's what data essentially went out the front door as a result.
Did we lose some intellectual property? Some secrets or sensitive organizational information? So at the start of this, you mentioned using KQL to perform the hunting. Is that the only way you can do hunting? I mean, there are other APIs or anything that you can call or you really restrict is just using KQL. So there's two main ways you can really use advanced hunting inside of Defender itself. So the first way is to use the web interface itself.
That's probably the easiest way, but there is some limitations. You have a cap of about 10,000 rows. So if you need to bring back a whole bunch of data, you may want to use the API. So the API will give you up to 100,000 rows worth of results and enables you to essentially programmatically call advanced hunting. It's really great for hooking to things like, for example, if you have some Azure automation out there, you can pull your data directly into reports or whatnot.
The third way you can kind of use, and this is sort of advanced hunting, is really referring to the data itself. So we've got a couple of different APIs out there that enable you to pull the data into other systems. So the first one is going to be your Sentinel connector, which is a native connector between Defender and Microsoft Sentinel. So you can actually pull your data into Sentinel and then join it with all the log data that's in for instance at your SIM.
Or you can also use the streaming API, which lets you pull it into either directly into blob storage, or you can pull it into an event hub and then do it with what you like. I'm not gonna be right, I mean, I love KQL, but I always like the ability to call an API, just in case. I mean, I want to build like some kind of custom tooling or something, and it just gives me that extra level of flexibility. So that's really great to see. I mean, is there a community of folks out there?
Cause I imagine this stuff's relatively complex, especially if you're sort of learning this stuff from the get go. Is there other people I can talk to about this? We also have a get hub. So we're actually inside the Azure Sentinel repo. If you look under the hunting queries, you're gonna see Microsoft 365 Defender. And inside there is a whole bunch of YAML formatted queries, which is gonna be the same ones you see in the advanced hunting portal under community.
So it'll help you get started if you want to see how other people are using advanced hunting today. All right, so one thing we ask all our guests is if you had one sort of final thought to leave our listeners with, what would it be? So probably the biggest thing is when you're looking through your EDR or XDR solution, you see activity, make sure you stop and determine if it's gonna be commodity stuff.
So the stuff you get just surfing around the web, or if there's might be some targeted intent behind it, i.e. if you're looking through seeing some suspicious activity. If you do see targeted activity, remember your ABCs, authentication methods, backdoors, communication channels and data. And be ready to bring in help if you need it, like from Microsoft Dart or Microsoft Defender Experts for hunting. And if you're looking for resources for hunting, check out my book, Designing Secure Systems.
It's got a lot of good content in there that'll help you think like a hunter and help you track this adversary using the ABCs and what I like to call authorization theory. Well, thanks again for joining us this week, Michael. I know it's an interesting topic. I admit it's not an area that I'm particularly familiar with, so it was always good to learn something new. And to our listeners out there, thank you to you also for listening in. Stay safe and we'll see you next time.
Thanks for listening to the Azure Security Podcast. You can find show notes and other resources at our website, azsecuritypodcast.net. If you have any questions, please find us on Twitter at azuresecpod. Background music is from ccmixter.com and licensed under the Creative Commons license.