John Hubbard 00:00
are you looking for the best in depth training for your cyber defense team look no further than sans blue team courses whether you focus on network or host data windows linux or even specialize in open source intelligence seam sock or defensive architecture the sans blue team curriculum has the course for you from longtime classics like sec 503 network intrusion detection to the newer sec 530 defensible security architecture and engineering and sec 47 open source intelligence gathering no matter what your specialty we've got you covered with an extensive archive of free webcasts on the sands site and free online demos available for most courses you can easily check out the sands blue team catalog and see which course is the best fit for you and your team check out the constantly growing list of available courses at sands url comm slash blue team ops
00:50
this is the blueprint podcast bringing you the latest in cyber defense and security operations from top blue team leaders blueprint is brought to you by the sans institute and is hosted by sans certified instructor john hubbard and now here's your host john hubbard
John Hubbard 01:06
today on the blueprint podcast we have josh brower from security onion solutions if you've seen josh speak at any conferences recently you know he's been working on some key contributions to the new version of security onion in the form of the new playbook application regardless of whether you use security onion or not i wanted to dive into the design principles and workflow he planned out and coded up for the new open source playbook app because i honestly think it's amazing work and there's something to learn from it for everyone on the blue team a lot of sox struggle with use case organization analytic documentation unit testing metrics creation and more and i think what josh has done here wraps up solutions to many of those problems into one tightly integrated solution if you've ever been an analyst staring at an ids alert and wondering what to do next or a sock manager trying to generate metrics on your detection capability and an easy way this episode is for you combining sigma rules attack navigator the hive and other open source blue team favorites the new playbook software just created was designed to address common sock problems and much much more stick around to hear the details coming up on this episode of the blueprint podcast all right welcome everyone to the blueprint podcast today i have josh brower i had josh on the podcast today because i'm really interested in his involvement in creating the playbook functionality for security onion as someone who's always trying to organize analytics and use cases and things like that josh came up with a really innovative solution here and i think he's probably come up with something that might be more efficient than anything i've ever seen before and i love the way it integrates and everything like that so i wanted to interview him and pull out some of that very very useful information on workflow and the deep thinking i'm sure he did to build this application so welcome to the podcast josh
Josh Brower 02:45
thank you so much john great to be here
John Hubbard 02:48
so before we get started let me hear a little bit about your background and how you got to where you are now
Josh Brower 02:53
sure sure so i actually started in it when i was a teenager probably a lot of you the same where you fixed your family's computers and things like that and so very early on got into it and then the first part of my career and operations more from a sysadmin and a network side of things and then got my first sans cert back in 2008 and from then on really focused on security at that point and spent most of my career in the international nonprofit community doing information security among that particular community then about a year and a half ago joined security onion solutions specifically focused on engineering of the platform and the training of run security onion in production for many years in my previous organizations and so it was just a really natural fit and it's been a really good year and a half with security onion working on the platform itself
John Hubbard 03:51
fantastic what was it that brought you specifically to want to tackle the problem of playbooks
Josh Brower 03:59
yeah that's a that's a great question so well first of all let's define this idea of a playbook right so a lot of people in the industry will use different ways to define and playbook and playbooks and for me i really look at it from the perspective of a detection playbook we have plays that make up a playbook and plays are just essentially custom reports or detections that you write and they make up your detection playbook and i really got into this because a few years ago again in my previous organization i was working on a particular issue where we had we had a technology stack that required a high level service account domain service account and i was concerned about monitoring that and i went to try to write in the last alert rule specific nodes figure out a way to write a rule for that so we can monitor anytime we see usage of that account outside of where it should be and i realized that even if i could remember this rule and get this alert to generate the next person who sees alert is gonna have no idea what this means we just know that this you know this account was accessed on this machine and there's no context and there's no next steps okay so i've got this now what do i do and that was really the beginning of me trying to figure out a way in my work of how can i write detections in a way that would give context to what i'm trying to do and give the analyst next steps because whether the analyst is me six months from now because i've slept and i don't remember what i did or you know someone else on the team so that was kind of the original idea is building up detection plays in playbook that gives context and next steps does that make any sense john
John Hubbard 05:47
yeah absolutely that gets to i think one of the biggest problems in a lot of security operations centers especially ones with a lot of kind of retention or turnover problems is the tribal knowledge thing right you get new people coming in and they're like first day on the job this alert goes off what do i do right and so i'm having a documented place for those kind of things i think is a fantastic way to eliminate that and standardize you know what is expected and understanding what a rule is where the data comes from what false positives might be created and all that kind of stuff and that's kind of how i look at playbooks too so that's why i was really excited when i saw like everything you had in here i'm like oh this is perfect that's super cool walk me through in terms of the new application for security onion like what is the workflow for someone who is maybe just thought of a brand new idea and is going to turn that into a rule as opposed to maybe an analyst that is going to be referencing it what's how does interaction happen
Josh Brower 06:41
sure sure so playbook is an application the backend is just a redmine and open source like issue tracking and project management application and the idea is that the person who has this new idea let's say i'm really really basic good one here is i want to get alerted anytime there is a rdp connection from an external ip address to one of my internal endpoints right so that means that i have maybe an rdp listener remote desktop for the listener publicly accessible and so i got this great idea of where do i go well first i got to figure out where my logs are right and so i know let's say i have zeke logs that are available for me to learn off of insecurity onion so i know i've got the logs next up is you're going to open up playbook and you're going to write a sigma rule for it sigma is a language describe detection think of sigma just like we have sericata and snort alerts for network data we have yara for files we have sigma for log files and so you're going to write a signal rule that says anytime i see a zeke connection record or excuse me a zeke rdp record from an external ip address to an internal ip address i'm going to go ahead and generate an alert you write that sigma rule and then you can test it within the within the playbook application make sure generates the elastic search query that you need and then you actually click on create play and that's going to create what we call a detection play inside playbook and this play gives you some metadata about what's going on you can put you can tag the play with mitre attack information and things like that but really the three components of the play which i already mentioned is number one context why am i doing this what am i looking for number two the elastic search query that we need for the play and then third is a next step so next steps may be is this a valid rdp listener do we have a terminal server on premise if we don't who should i be contacting next what should i be checking to make sure that this is valid or not once that is actually in playbook then you can make that play active and that goes out and does some automation and creates in the last alert configuration so that we actually will now get alerts for that it also creates a case template for the hive the hive is a case management application and so we'll talk about the analyst part of that next but it creates a case template next it also updates to application called the attack navigator which gives you a visual indicator of your coverage across the mitre attack framework that was kind of a real quick overview but the idea is that you've got this idea you got to figure out where the log is you got to write the sigma rule for the log and then create the play from that and then put it into production does that making sense as we go along there yeah
John Hubbard 09:35
yeah definitely and that was one of the things i loved is like all of those automatic actions that happened with that palantir has has made public like their eds kind of process and it's very similar right you come up with an idea you come up with like the core is it going to work you're testing and all of that but what you did was take it a step further and say like we're going to push out a case template we're going to turn an actual sigma rule into a rule that's active in the environment we're going to make it easy to see what you've done with attack navigate and so that i think is where all of this integration becomes really really nice as a analyst looking at an alert that has gone off what is your kind of ideal interaction with this environment and like how might that go if they like they're open up the alert and they're like okay i've never seen this before i'm going to playbook then what sure
Josh Brower 10:17
sure yeah so from the analyst side now that let's say i get an alert generated for this rdp traffic i pull up in the alert and from right there i can pivot to playbook there's a link you know i can play with the playbook and i can see context okay this play was written because we want to know when we have an rdp listener publicly accessible so okay i've got context and that's even without having to read the elasticsearch query itself if you've ever had to i mean i think very normal is that when we're looking at sericata alerts or something else because there's just no context typically to what's going on we have to spend quite a bit of time if i'm not familiar with the title i've got to go research the rule i've got to go try to dig into it and figure out what it's doing why is looking at it and so the point from an analyst perspective is that i should be able to quickly and efficiently pivot to the context understand what's going on why i got a an alert generated and then quickly and efficiently pivot to my next steps okay now i know i need to go check this system go check in with this person validate it and then escalate it to a case or you know suppress it or you know close out that alert so the idea is that can be able to quickly and efficiently get context on the alert and validate it from there
John Hubbard 11:40
very cool that's one of the things i'm repeating all the time in class is like you know in this triage stage right you have a whole mountain of stuff you need to know what's most important and when you pick one you know you need to see like very clearly this is something that is clear danger and very high risk and providing that context and what i didn't realize is it's linked directly from the alert so even better right it makes it very very easy to just say oh i don't know what this is one click or there and love that too that's really really important especially maybe the newer you are in infosec it really helps out in that respect as well as the
Josh Brower 12:12
you know the tribal knowledge problem let me just say john in relation to that when you click over to the play in playbook you also get the history of what's happened with the play if we've made changes let's say we've tweaked it because we've seen some false positives and we also get commentary from other analysts on there and so like i said if you're new you can very quickly look at the history of what's going on and figure out you know is this something that's outside of the norm or is it something that's legitimately something i need to take care of so yeah all that tribal knowledge i think is more easily self contained in something like a playbook yeah absolutely
John Hubbard 12:48
i totally forgot about that angle too yeah that's a really important point is you know every analytic goes through many many changes and many people find issues with it and having a single place to say like this alert misfired in these situations have that tag to a time and a person that made that and then maybe when you make an edit have that time stamped and put in there as well that's a workflow i think works really well with a ticketing solution and you know just by chance i happen to use a ticketing solutions in my past as well for use cases and found the same kind of benefits which is why i was really stoked to see you know that was what you were going with the other things i wanted to ask about the sigma piece sigma is one of those things that i think is is catching on for sure i'm starting to hear more and more about it see more and more support for it what has been your experience with the the sigma part and where do you think do you think and do you see that becoming a standard within the industry
Josh Brower 13:40
i do i do i i really i really liked sigma so originally i was when i was working on playbook from an application perspective i started growing or using a homegrown solution for describing a play as i'm a really big believer if there's something out there that works and works well and is got pretty broad support and let's just go and use that you know why create yet another quote standard when there's something already out there and so that's one of the main reasons why i went with sigma and certainly there are some edge cases here and there but from what i've seen and used in production sigma works really well again if you're not familiar with sigma you write your sigma rule in yaml and then you use the sigma converter which is a python script and you feed that converter your field mappings for your back end system as well as what you want your back end system to end up being in our case in security onion it's elasticsearch but it could be splunk or something else and then the sigma converter will then spit out the query you need for that particular rule and so the issues i've seen are mainly around not having field mappings set correctly or you know things like that which inside again the platform that i'm working with security onion we do all that for you you don't have to worry about about the sigma converter or anything that's all done on the back end for playbook
John Hubbard 15:06
so one of the things i noticed when you create new rules with a sigma rule and you make them active within playbook is that it pushes out a new case template into the hive and for those that aren't familiar with case templates in the hive this is one of those things i guess there's maybe a little bit of a collision of terms here but a lot of people would call it a playbook right but in terms of the hive it's a case template is a series of tasks that are expected to be done you know any given situation and so you can take this play as we call it in playbook and then tie it directly to a set of steps within the incident management system which i think is another really awesome thing how did you make that part
Josh Brower 15:42
work side note here actually originally looked at creating a playbook in the playbook type thing inside the hive and i just couldn't get like you said the case templates wasn't quite there with what i was looking for and so that's why i ended up going with an external application but i really do like the functionality of having your case inside the hive and having those tasks that you go down and so the way that we do that in security onion and in playbook is that you add an extra section to your sigma rule called tasks and then when we convert that over and we make that play active there's some backend automation that will just take those tasks and make them part of the case template inside the hive again the idea here is that when they escalate a an alert from the interface in security onion it will create a case inside the hive and there will already be pre loaded next steps so that the analyst or whoever's working this can see okay i need to go you know check this out another let me give you another example what this may look like if i have shared drives and by environment os query at open source endpoint agent allows us to really get some interesting information about endpoints in the environment and one of those is shared drives like windows shared drives and in environments that i've worked in it's really common for you know temporary quote temporary shared drives to be made and then forgotten about backups or i'm just going to share this information but specifically backups is what i was thinking of and so one of the early on plays that i created was anytime there is a new share created in the environment but it has the term like hr or backups or temp or something as that share name or it's included in that go ahead and generate an alert based on that and then the case template next steps would be go and confirm you know was this a recently created share what are the share permissions they should be locked down you know go find out who is the owner of that of that system whether or not this is a legitimate share can be removed so there's some very specific next steps that we could take in a case like that and that's why i think the power of adding tasks to your play is really important yeah
John Hubbard 18:00
definitely one of the other things i was curious about is from a manager perspective you know having the guidance for analysts and all of that sort of thing is fantastic but what kind of organization and like metrics and reports can playbook produce for those who are maybe managing the sock and trying to understand the field of things that are available for detection and the state that they're in and things like that
Josh Brower 18:23
sure sure yeah great question so specifically right now there's integration with the with the mitre attack navigator application which is put out by navigator or excuse me mitre and allows you to as i mentioned every time you make a play active or so a detection is active if that detection is tagged with a particular technique or something like that then we'll actually get color coded cells on that on the application so from that perspective a manager should be able to quickly get a view of what kind of coverage they have from a detection perspective you can also right click on one of those cells and one of those techniques and say view related plays and you'll be able to bring it'll bring up playbook and it'll show you all of the plays that are tagged with a particular technique or tactic outside of that within the application itself there is the ability to generate some basic metrics and those would be based on rule sets so let's say within the sigma community repository there are over 500 sigma rules that you can import and that we import automatically and those are tagged with like is this a windows is a sis mon related things like that and so we can do some metrics around that as well
John Hubbard 19:43
is there any capability right now or any plan to do any capability where you can tie the actual amount of times the alert fired and back to the playbook interface itself i'm sure you can probably produce that directly within the interfaces but is there like a you know this play
Josh Brower 19:58
has fired this many times you Yeah, that was that was something early on that I was looking at. And it's certainly doable. Everything is accessible via API's, all that data is accessible via API. And so the plumbing is all there. We just got to tie it all together. It is something we're looking at. But it's not something that is currently implemented. But yeah, I would love to, I would love to be able to pull a report and say, this play fired 50 times over the last three months. And out of those 47, were false, positive three led to incidents, you know, in something or other. I think that's definitely a possibility.
John Hubbard 20:32
Yeah, keeping my manager hat on for a second here. Now my mind's kind of running wild and like thinking, well, if you could track how many times the thing fired, and you can think about, you know, like, when it was acknowledged, versus how long it took to go from whatever state to closure, you could get times and all that sort of stuff. And then all of us would become one really, really nice, like, yeah, metrics generating package for me. Yeah. So
Josh Brower 20:55
yeah, he's really up lots of possibilities.
John Hubbard 20:57
Yeah, one of the things I was wondering, there are a ton of, obviously, rule sets of all sorts that are out there, whether it's sericata rules, or, you know, anything based on any rule set, right? There's hundreds of rules and analytics that any environment they have at any given time, how do you decide what of those rules become plays in playbook versus the ones that maybe you just kind of leave as an IDS signature, and
Josh Brower 21:19
you don't independently document that's one of the things that we struggle with trying to figure out how to do? Sure. When you think about in particular, network intrusion detection system signatures, there's, they really run the gamut. Some of them are really looking more for atomic indicators, right? Like they're looking for particular IP addresses and things like that. I think those you could possibly disable inside of the knids engine and do a play just for atomic indicators in your environment. But there are others that are really specific and may be looking for a C to communication, or some particular malware that communicates via the network. And it really makes more sense just to keep it as a, let's say, a, you know, sericata or snort rule. So I don't think any of those are ever going away anytime soon. I think it's going to be up to the environment and the the team that's looking at it to decide whether or not this should only live in something like a knids rule, or it should be pulled into a playbook play. I definitely think with tuning you can make your knids rules much more manageable. But I was actually looking at one this morning for it was curl curl outbound, there were a number of different applications that were setting off one of them was from Cisco Meraki it was it was doing a port 80 HTTP curl outbound. And I mean, I don't care about that. That's not great my environment, but I don't care about it. So I thought about maybe could we create a play that filters out some of that kind of noise but brings and some of the other more interesting kind of curl events? With a little bit more nuance? And I can do with the other tools right now inside knids rules? I'm not really sure yet. But my point is that I don't think there's a one size fits all when it comes to that. I think there's always going to be other alerting engines and the alerts that they produce that we're going to have to figure out how that integrates into something like a detection playbook. Does that making sense? Yep. Yeah,
John Hubbard 23:22
I think one thing you said kind of reminded me of maybe an approach or, you know, kind of maybe how I've unconsciously done this myself in the past. He said, you know, specific indicators, right? Thinking of the pyramid of pain, right? Yeah. It seems like a lot of the Sigma rules are oriented more towards those ttps. And the things that are mitre attack, you know, navigateur layer style, you know, some techniques or parent level techniques, and things like that. So maybe that's a good way to kind of divide that in between, you know, this is super specific. And these are generic kind of processes and procedures, you don't expect is the new navigator. Stuff supported in there as well, the sub techniques and that kind of stuff. Yep.
Josh Brower 24:01
Yes, specifically for sigma. There was some recent changes. So the sub techniques are all supported. And in playbook itself, that's all supported as well.
John Hubbard 24:10
Fantastic. Are there any other unique challenges are things that you kind of had to think really hard about in terms of process or otherwise, when you were developing this? What do you think other teams might run into? And where can this help from, from an implementation perspective?
Josh Brower 24:23
So specifically with playbook and within security on a week and Port 500 plus sigma rules from the community repository? One of the things that I think can be a challenge is just like any other rule set is, should I go ahead and and just enable all 500? Right? And then how do I deal with the delusion of alerts that come through? And that's where I think when you're looking at doing something like playbook, keep in mind that there's really two ways to approach this. There's the predefined rule sets, there's lots of places that are now producing their own sigma rules. that you can easily import and turn those on but then there's also more organizational specific sigma rules or detection plays that you'll write those will probably be a little more high fidelity because there's specific for your organization and so those i wouldn't necessarily have a problem with saying let's just get all those turned on and make them active and and go from there but be cautious about just enabling you know hundreds and hundreds of detection plays without really understanding what you're looking for and why keep in mind that sigma rules really needs specific log sources so we're in the sigma repository you know you have zeke log sources you have sis mon windows event logs powershell syslog and a bunch of other types of log sources so unlike let's say an ids like sericata where you can just enable everything i wouldn't recommend it but if you just enable all the rules is just one log sources your network data versus the log sources for sigma are much more disparate and so you just have to be more cautious and aware of what you're enabling from that perspective so that would be my my thought is you look at moving this into production using something like playbook
John Hubbard 26:13
yeah i'm glad you touched on that that's actually another conversation i like having with classes is you know obviously in the false positive and tuning section of the courses on teaching one of the things i always bring up is you know what's the right or best approach and i'm not sure that there's a solid answer to this but i think i know how you might fall in this question but i wanted to ask it anyway when you get any sensor whether it's you know the list of defaults sigma plays or anything else is your approach that in general you start from like turn everything off and turn on the things only that you know you want or do you kind of leave the default set on and prefer to like work down from the false positives that are generated and just turn those things off
Josh Brower 26:53
honestly really depends on the environment and what i'm looking for um i've been in small college environments and i've been in more organizational executive office environments and i think depending on the environment i would probably do it a little bit differently but i would typically start with just enabling the things that i know i want rather than you know turn on all the policy violations i don't want to have to go through all the work of disabling all the rules that alert on you know instant messaging and things like that in a college environment because that's not a problem in that kind of environment right and so that would be my answer it's gonna depend on the environment and you know what you're looking at but i would typically start with nothing and enable what i'm looking for at that point
John Hubbard 27:36
yeah that's usually the kind of my approach as well as i find at least in my experience when you leave the default rule set on endless we'll see an alert for the first time maybe it's never gone off before and they're like well do we care about this or is this just the first time it's gone off and now right this moment we're realizing it's irrelevant and a false positive you know and so that's kind of a clever approach to kind of getting around that and cleaning things up right from the start right
Josh Brower 27:59
yeah yeah in terms of
John Hubbard 28:02
trying out playbook what is the easiest way that people can kind of see the application in action is there a way to do it kind of outside security onion do they just drop into the iso and put it in eval mode like what's the shortest route to seeing what you've done here
Josh Brower 28:14
sure sure so playbook is an application i was working on previously to security onion but and you could certainly still run playbook with a bit of work outside of security on but it's really best use inside security onion because that's what it was designed the last year plus of integration with security onion and so best thing to do is go to our specifically there's free training out there called security on essentials on youtube and that gives step by step on downloading the iso and installing eval mode and you can get up and running within an hour and check out playbook from there so that would be that would be the best way to go at it
John Hubbard 28:55
fantastic final question any plans for new features going into the future where it's the roadmap do you have anything exciting coming up
Josh Brower 29:04
sure it's so there are there's quite a few things some of its already implemented it just hasn't been widely pushed yet one of them is play unit testing so that when i create a play i'm also will put a copy of the log that you're trying to alert off of as part of that creation process and then anytime the sigma changes it's going to run and verify that you still get alerts based off of that log and if you don't you'll get errors associated with that because again environments change over time right so where am i make a mistake when i'm editing the sigma you know six months from now and so i want to make sure that i've got a valid detection for the logger that was originally working with if the log changes that's fine you can update that so unit testing is one of them for detections secondly is this one is coming in the next little bit as well when we're bringing in the 500 plus Sigma rules from the signal repository, some of those needs to be tweaked for your environment. And we also keep those updated. So if you're trying to modify a rule for your environment, and then you know, we'll just overwrite those, unfortunately, every night when we do our updates, and so we have some logic in play that we're working with, from a contributor that will allow you to, will either merge the changes or make it so that we won't overwrite that that's in a rule to make a little bit easier to customize it for your environment. And it's one thing that I don't think I really mentioned much is that with the community repository rules, you need to make sure that you customize these as needed, like the remote desktop protocol detection, I mentioned earlier, if you've got a terminal server publicly accessible, please don't ever do that. But if you do, you know, you're going to need to specify in your sigma rule, you know, this is legit to see RDP traffic to this IP address. Anything other than that, though, you know, shouldn't be a problem. So those are two main features are certainly some other automation things that I won't get into now. But you know, we're really on the ground floor of playbook. And I'm looking forward to continue developing that in the near future.
John Hubbard 31:10
Very cool. Love that unit testing thing. I need you to think about that one yet and asking. So yeah, glad you brought that up. That's a really important thing, right? It works when it started versus it works three years later as to
Josh Brower 31:21
exactly, exactly.
John Hubbard 31:24
Where can we follow projects or doing security onion, where's the best place to keep up to date with us online.
Josh Brower 31:30
So my Twitter account at defensive depth also have a blog, defensive depth calm, those are really the two best places to keep up with a lot of the work that I'm involved in specifically with playbook OS query or security onion in general.
John Hubbard 31:44
Well, I think that wraps it up for today. Thank you, Josh, for being on the podcast. Your teams of the world certainly appreciate what you've produced here. Really, really nice. Well thought out integration of all these tools with metadata and case templates and tack navigator and metrics and unit testing. Really excited to see where this goes in the future. So thanks for being on the podcast.
Josh Brower 32:04
Thank you much really enjoyed it. Hey, blue teamers.
John Hubbard 32:07
Hope you enjoyed today's episode of blueprint. If you've got a second and wants to help support the podcast, please subscribe and leave us a review on Apple podcasts. It would be really, really meaningful to us and if you have any ideas or suggestions, I would love to hear them. Your reviews are going to be one of the best ways to help others find this podcast so anything you could do would be a big help. As always, thank you for listening. You can connect to me on social at sec hub sec HQ, BB on Twitter, on LinkedIn. So until next time, thank you for listening to the blueprint podcast.