Is keeping your network secure feeling like, uh, just this constant battle against a flood of data.
It really can be overwhelming.
Yeah, and are you confident you've plugged every single hole against attacks? Is your sound system, you know, really seeing what it needs to see?
Or maybe you're just starting out right and wondering where do I even begin with intrusion detection exactly?
If any of that sounds familiar, well, you're definitely in the right place today. Our source material kicks off with those exact questions.
Finding those critical signals, the important bits amidst all that network noise.
That's the core challenge.
Really absolutely, and that's what we do here on the Deep Dive. We take dense information like this fantastic guide we're looking at and try to pull out the key insights for you, you know, those aha moments.
So our mission today is pretty focused. We want to understand how you can build a robust intrusion detection system and IDs and also a SIME environment, and.
Crucially doing it with open source tools that are freely available. That's right.
Our guide for this is Richard Medlins A Network Defender's Guide to Threat Detection using zeke Elastic Search log Stash, Cubana, tor and more.
Quite a title promises a lot, and we're hoping to make that journey a bit smoother.
What's interesting is a guide itself points out a big hurdle. You can find info on installing these tools one by one, sure, but getting them all configured to work together effectively clear up to date guides are surprisingly hard to come by.
Okay, so let's unpack the toolkit we've got. Zeke used to be called BRO. Some might remember that name. The elastic search, Logstash, and Cubana. That's the ELK.
Stack, the classic ELK stack.
Yeah, finally, tour the Nanimity network. The source says getting these to play nice is tricky, so let's try and clear that path.
Okay, let's start with the foundation Zeke. Our guide rightly calls it a cornerstone for a solid IDs.
So Zeke formerly BRO maybe a nod to Orwell's nineteen eighty four seems kind of fitting for network monitoring. Maybe a bit ominous, huh?
Maybe it's open source Network analysis software functions as a network intrusion detection system, and an IDs basically gives you a live view of network happenings.
What makes it special though. It's not just basic pattern matching, is it.
No, that's the key thing. It's smarter. It analyzes live traffic, yes, but also files moving across the network, even old recorded traffic and pccapy files, and it generates these neutral events based not just on known attack signatures, but also on network behavior that seems unusual.
Right, So it's like noticing someone acting suspiciously near the building, not just spotting a crowbar.
Exactly like that.
Yeah, and it has built in analyzers for common protocols, HTTP, FTPS, mtp DNS, the usual suspects.
And you can add more if needed.
You can.
Plus, it integrates with other tools like snort and crucially for US today, with elastic search.
The guide mentioned some basic dependencies libcap for packet capture, open SSL for crypto, usual build tools like make c plus plus RAG.
Yeah, the standard stuff for building from sores. But it quickly gets practical focusing on prepping Aubuntu for good packet capture.
Right, you need to see the traffic clearly first, So step one, according to the guide, disable network manager. Why turn that off?
Well, think of network manager as like the automatic pilot for your network connection.
It tries to be helpful manage things.
Which is usually good.
Usually yeah, But when we're doing passive sniffing just listening in, it's active management can actually interfere, maybe drop packets or change how we see them.
So for a.
Dedicated sensor, you want it off. The guide gives the system trick to ural commands.
Okay, that makes sense, less interference. Next step disabling NIC offloading functions sounds technical? What's the risk here?
Modern network cards and icees they do tricks and hardware to speed things up right, take some load off the main CPU.
Like handling checksums or segmenting large data packets.
Exactly, things like TCP segmentation offloading TSO, Generic segmentation offloading GSO, Large receive offloading LRO all designed for performance, but they can bundle packets together or change packet timing in ways that distort the picture for our analysis tool. For ZEKE, you might even see packet sizes reported that are technically impossible.
Ah, so we need the raw, unaltered stream. We need to turn those hardware shortcuts off on our analysis machine precisely.
The guide shows using the f tool command with a bunch of flags rx tx, TSO, UPHO, GSO, grow l, row, et cetera, setting them all.
To off, and the guide stresses you probably need to run this f tool command every time you start zeke.
Yeah, that's a key point because those settings might not persist across reboots. You use f toolpinae to check their off. Some might say they can't be changed, but you verify the important ones are off.
Okay, Then it mentions ensuring the DNS network service is running but not really changing its can fig Yeah. Then the big one, promiscuous mode sounds sneaky.
Hu Well, Normally your network card only listens to traffic specifically addressed to it, like a polite receptionist only taking calls for their extension. Right, Promiscuous mode tells the card listen to everything, all traffic hitting the wire on that network segment.
Doesn't matter who it's for, which is absolutely essential for an IDs. Otherwise you'd miss most of the conversations.
Totally essential. If it's not promiscuous, it only sees traffic to the analysis box itself, useless for monitoring others.
The guide shows if canfig interface promises to turn it on and ip A show interface rep I promised to check handy check YEP.
Then it lists dependencies core ones we mentioned, but also optional stuff like GUOIP support.
Ah geolocation so zeke can tell you where traffic is coming from or going to geographically based on IP address exactly.
Using the limex mindb library and the free geolite two database super useful for spotting, say, unexpected international connections. Has that set up, The guide walks through app to get installed lib max mindyb dev, then using curl to download the geolite two city database, putting it in our share gip and it wisely says check for updated download links.
Things change good tip Okay, next up. Pfriing sounds fast.
It is.
It's all about boosting packet capture speed, getting higher data rates than standard Linux methods.
How does it work?
Basically, it uses a Linux kernel mechanism called NAPI new Api. Essentially, it creates a more direct, efficient path for packets from the NIC into memory buffers that Z can access quickly reduces CPU overhead. Think of it like an express lane for packets.
Gotcha, and installation looks like compiling from source again.
Yep, get the code from GitHub, compile it with Make install the kunnel module the guide puts it in opt which is common for optional software.
So the steps are installed. Build tools like bison flex right.
Get, clone the pfring rebo cd into the kernel directory, Make Pseudo, make install, then Pseudo in smithmad to load the kernel module. It even covers building userspace libraries and TCP dump with pfr ing support.
Okay, lots of prep work. But finally installing zke itself.
Yes, clone the.
Z cre go from GitHub, use the recursive flag that's important. Gets all the submodules right. Then the classic dot configure, make make install. The configure step is key. You tell it where PFI an G is, where guyp is. The guys suggests installing.
To opsec compilation can take a while, I bet can do.
Then a reboot is needed. Update your system's path so you can just type z commands easily.
Almost there.
Okay, installed. How do we tell zeke what to monitor? Basic config that's next.
Two main files and ops can secut dot no dot cfg and networks dot cfg.
No. Dot cfg is four.
Defining your zke setup for a simple single sensor. The main thing is telling it which network interface to listen on the one we set to permiscuous mode earlier.
And networks dot cfg.
That's where you define your local IP address ranges, you know, using cid R notation like one ninety two point one sixty eight point one point zero two four or ten point zero point zero zero point zero eight. This helps zeke know what's internal versus external traffic.
Makes sense. It also mentioned local dot Zeke.
Yeah, that's like.
The main control panel for enabling or disabling different Zke analysis scripts.
You load the scripts you.
Want to run there installed configured basics. Time to fire it up right.
First, you need to give the zeke binari's permission to actually capture packets using the set cap command. It's a security thing, okay. Then you use zke toil the command line controller first time ever. You run zeke toil install. After that to start it with your config you use ziketil deploy and check if it's running zeke til status and you can watch the logs live like the connection log with tail opt sk logs, current con dot log. See the traffic rolling in phew.
Okay, solid foundation with zeke running. Now the guide switches gears to tour. Why bring an anonymity tool into an intrusion detection discussion seems backwards.
That's a great question, and the guide tackles it directly, understanding tour, how it works, what its traffic looks like. It's actually idle for defense how so, well, partly it's no your enemy right, understand the tool attackers might use, but more practically, seeing tour traffic inside a typical corporate network, that's usually a big red flag.
Because employees generally shouldn't be hiding their activity using TOR for work stuff exactly.
There's usually no legitimate business need, so detecting it could indicate policy violations or even malicious activity like data exfiltration or command and control channel.
Okay, that makes sense, understand it to detect it.
The guide explains onion routing briefly, how tour encrypts traffic and bounces it through multiple random relays to hide the source and destination, and.
It warns about running an exit relay.
Definitely, big legal disclaimer there. Exit relays are where the tour traffic pops out onto the regular internet, so you could be blamed for what others do through your exit node.
Good warning. It also mentions law enforcement uses TOR too.
Sometimes yeah, for investigations anonymous tip lines things like that, and critically it advises don't do anything illegal and don't log into personal accounts if you're using tour for anonymity.
Common sense, but.
Important, absolutely so setting up Tour on our analysis system.
The guide covers installing Tour itself, plus a helper tool called privoxy sox it's a web proxy here. It's used as a middleman to easily forward web browser traffic into the tour network, which listens on a soocks port.
How's install work?
Add the Tour project's repository ear system, import their signing key, update packages, then APT install tor, guipdb, torsosdeb, dot Tour project, dot org, desh gearing pretty standard package. Install a provoxy just APT install privoxy simple enough.
How do they connect? How do you tell provoxy to use Tor?
You edit provoxies, canfig file, et cetera. Provoxes can fig add one line at the end, forward socks five local host point nine zero five zero zero that tells it to send traffic to tours default soocks port ninety fifty running on the local machine. The guide also suggests commenting out the log file line, then restart the provoxy.
Okay, installed, configured, How do you actually use it? The guide mentioned torsocks.
Yeah, torsocks is neat.
You just run torsocks command and it tries to force that command's network traffic through tour. The guide shows an example with.
Curl, but not everything works with torsocks.
Right, especially graphical things like web browsers. For those, you often need to go into the browser's network settings and manually configure the SCKS five proxy point it to local host port ninety fifty.
The guide then shows a script tour switch to toggle tour on and off easily. That sounds useful, super useful.
It uses x settings to change the system wide proxy settings for Genome desktops and system cordal to start stop the tour in provoxy services.
Saves a lot of typing, and.
It provides the script code how to make it executable with chew mode. Even handles potential ownership issues YEP, and it.
Covers letting a regular user run this script without needing the pseudo password every time.
How editing the sudoer's file gotta be careful.
They're very careful use with pseudo always. It checks syntax before saving. The guy gives example lines to add allowing specific commands system sort, telecorner, stop, status, et cetera for a user without a password.
Prompt Finally installing the Tour browser itself.
Yeah, that's the prepackaged browser designed for Tour. Simple install apt get install Tour Browser launcher. Then you run Tour Browser Launcher as a normal user, not root.
So now we can generate Tour traffic on our.
Network exactly, which means we can test the Zeke signatures designed to detect it, which we'll get too later.
Helps you understand it and spot it.
Okay, Zeke is capturing We understand Tour now making sense of all that data. Enter the ELK stack, Elastic Search, log Stash, Cabana.
Right, the powerhouse trio for log management and analysis, open source, very flexible, scalable.
Let's break them down. Elastic Search is the search engine the database sort of.
Yeah, It's a search and analytics engine based on Lucine, highly scalable, distributed. Its stores and indexes all the data so you can search it incredibly fast.
It's the heart of.
ELK and Stash the data processor exactly. Think of log Stash as the data pipeline or like the prep station. It ingests data from many sources filebeat for instance, then it can parse it clean.
It up, enrich it.
Enrich it how like adding GOIP info based on an IP address or translating cryptic log codes into human readable text. It uses things called grock patterns to understand structured or even unstructured logs. Then it sends the processed data to elastic Search.
And Cabana is the shiny front end the dashboard.
That's it.
Cabana talks to elastic Search and lets you build visualizations charts, graphs, maps, tables, lets you explore the data interactively. It's how you see the patterns and anomalies.
The guide also brings in Filebeat. What's its role?
Filebeat is a lightweight agent, a shipper. You install it on servers where logs are generated. Its job is just to collect those logs and send them efficiently, either straight to elastic search or to log stash for that extra processing.
Why use filebeat instead of just having logstash collect directly.
Filebeat is much lighter, uses fewer resources on the source machine. It also has pre built modules for common log types like zeke logs, which simplifies setup. And it has this back pressure sensitive thing, meaning it won't flood log Stash or elastic Search if they get temporarily overloaded.
Okay, so the flow is typically zeke writes logs, Filebeat reads logs sends to either log stash to elastic Search or elastic Search directly, then Kibana reads from elastic Search for visualization.
That's the common pattern. The guide then walks through installing ELK plus filebeat, big warning, use the exact same version for all of them, avoids headaches.
Right for elastic Search, add the elastic key in repo app to install elastic Search, then enable journal qualcogging. Why is that needed?
By default?
Elastic Search is a bit quiet in the main system logs. The guide shows modifying its service file elastic Search stop service to remove the quiet flag. Makes troubleshooting way easier using journal calls. Then you check it's running with corol local hosts point nine to two zero zero, and we.
Need Java, open jdk.
YEP, elastic Search and logs Dash run on Java. The guide recommends open jdk eight for the versions it discusses. Simple app to install open Jdk eight.
Jdk log Stash install is similar, but it doesn't start automatically.
Correct install the package, but you need to explicitly start the service. The guide shows the commands and where the config files live.
Kubana install dot enable and start the service. Configure Kobana dot AML. What are the key settings there?
Server dot port usually five six oh one, serve it dof host often local host or zero point zero zero zero to listen on all interfaces, and crucially elastic search dot hosts, which tells Kubana where to find elastic Search usually http dot local host, dot nine two Z or A zero and.
Filebeat install enable it to start on boot okay stack installed now making it work with Zeke.
The guide shows two main ways. First the simpler one filebeat send z clogs directly to elastic search.
How is that configured?
You edit filebeat dot AML, tell it where z clogs are, opseclogs, current dot log usually enable to built in filebeat z module. Then configure the output dot elastic search section with your elastic search address.
And what changes on the zke side. For this direct method.
You need to tell Zeke to output logs in JSON format and importantly use eph time stamps that's the number of seconds since nineteen seventy. You modify aii dot z and local dot z to set this up. The Kabana zke module expects that format.
Then restart everything, go into gabona, create a filebeat index pattern and.
Boom, you should see the pre built zke overview dashboard populated with your data instant visibility.
Cool. But there's a second, more powerful method using logsdash.
Yes, filebeat tas logsdash elastic search. This lets you do more sophisticated processing in logsdash.
What changes for zke in this case timestams right.
For the logstash route.
You can figure a zeke jets an output again, but this time use ISOSO eight six to oh one time stamps. The more standard yy y y m M d d t h m s SC's format set that in.
Local dot z and logstash needs configuring to receive from filebeats exactly.
You edit pipelines dot mL to define the pipeline maybe tweet logstash dot mL.
The core is a new config file, say zke dot com.
Well goes in zeke dot com.
An input section using the beats plugin listening on a port like five thousand and one. Then filter plugins, remove headers, parse json, use the date filter for the isotimestamp. Crucially, add the geop filter for location data. Maybe translate constate codes. You take field types, rename fields.
Lots of power.
There finally an output section pointing to elastic search.
So filebeat needs to send to logstash port five thousand and one instead of elastic Search.
Correct change the output section and filebeat dot aml from elastic search to logstash, specifying hosts Localhost dot five zero zero one, test the config, restart everything.
Then in cabana, create a logfash index pattern yep.
And now, because logstash added the goip data, you can build visualizations like coordinate maps showing traffic origins. The guide also adds a good troubleshooting tip. If data isn't showing upright, sometimes delete it and recreating the index pattern in kipana fixes it.
Okay, Zeke is logging, elk is processing and visualizing, but Zeke isn't really alerting yet. Is it just logging connections each TTP, et cetera. We need signatures for active.
Detection, exactly signatures. Tell Zeke hey, if you see the specific pattern, generate a notice. It moves Zeke from passive observation to active alerting.
The guide points to installing signatures from GitHub.
Yes, there's a repository linked you get clone it, then run and install dot Sher script provides this copies the signature files into the right place in your Zeke installation op zeek share Zeke site.
Usually important warning from the guide here back up your existing site directory.
First, absolutely crucial, especially local dot Zeke. That install script might overwrite your local customizations if you're not careful.
Back up first.
The guide then lists like forty eight example signatures from that repo.
That's a lot it is, and it briefly explains each one. Some are basic canfig checks, make sure certain scripts are loaded and Jason logging is on. Others are more detection focused, like what scan detection, YEP scan dot seek also detect trace route detecting vulnerable software advertised by servers, noticing when software versions change unexpectedly, finding Windows, reverse shells, sequel injection attempts.
Wow, okay, it looks really comprehensive. Signatures for specific protocols FTP, SMTP, SSH, HTTP detecting web apps.
Tracking known good hosts and services, validating SSL SERTs, detecting SSH brute forcing, flagging interesting host name.
Hashing all transmitted files, detecting non malicious file hashes. MHR heart bleed detection.
Even yeah, the heart bleed one comes with the performance warning, though it's intensive. Also asset tracking, VLNMIC logging, spotting, clear text, HTTP basic.
Off, and the tour detection we talked about.
DDP scans, detecting web directory modifications, more, SSH attack detection, a whole data exfiltration detection.
Framework, CRYPTO minding detection, DNS tunneling, RP.
Monitoring, SMTT analysis, DNS zone transfer attempts looking for potential credit card numbers, and plaintext with a redaction option thankfully, FTP brute force HTTP stalling attacks, general HTTP impacts, finding passwords sent over clear HTTP.
It's a really solid starting set.
Okay signature is installed. Any zke reconfiguration needed.
The guide mentions two things.
One ad port ninety fifty CCP towards default to the sl ports set in Zeke's SSL canfig. This helps Zke analyze encrypted tour traffic.
Makes sense.
The other an option in theh GTP canfig to actually capture passwords sent via basic authentication. The guide mentions it exists, but rightly avoids recommending turning it on in production. Logging plaintext passwords bad.
Idea, very bad idea. Okay, signatures are in Zeke potentially tweaked. How do we see the alerts?
After any config change.
You go to Zeke's bin directory, run dot ZULC install them the Z two hell deploy. This loads the new config and restart zke. Check status with ztail status.
And the alerts themselves appear where an elk.
Primarily in the notice dot log.
That's where Zeke writes these alerts generated by the signatures. So in Kibana you'd filter for logit dot notice or look for the notice dot log source file if using filebeat modules.
The guide gives examples like seeing an nmap scan alert.
Yeah, or alerts about large file transfers happening after hours, you'd see the show up as documents in qubana source from notice dot log. And of course you can still tail dh fndice dot log on the Zke box itself and.
The concrete example analyzing tour traffic we generated earlier.
Right start Zeke, fire up the Tor browser, visit some sites, then check notice dot log in cabana. You should see notices pop up. What kind of notices things like ipoddress identified using Tor based on unique SSL certificate characteristics. Tour uses often followed by SSL invalid server start notices, possibly referencing the same certificate fingerprint.
And the guide even shows how to double check if an IP was a tour relay using the tor Project's exonerator website.
Yep metrics dot tourproject dot org exonerator dot html. You put in the ipn date, it tells you if it was a known relay, then really closes the loop on verifying the alert.
So this shows the signatures working actively flagging specific potentially risky traffic exactly.
It turns Zeke into a much more proactive defense tool.
Okay, nearly there. The guide finishes with a quick look at Kubana dashboards.
Yeah, just emphasizing that Kubana is where you bring it all together visually. You create custom visualizations geoheat maps showing where traffic comes from, goes to top talkers on the network, top applications used, top destinations, maybe track misbites for network health, total data volume, and crucially summaries and trends of the notices Zeke generated.
And you combine these onto a dashboard right.
Arrange them all into one screen to create your custom SIME dashboard gives you that single pane of glass view of your network security posture built with these open source tools.
Wow. Okay that was a lot, but incredibly useful. We've gone from installings Zeke, understanding Tour, setting up the whole ELK stack, processing logs, and now actively detecting threats with signatures and visualizing it all.
It's a powerful combination.
You really do get a foundational grasp of building this kind of open source IDM set up from the guide, Understanding Zeke, integrating with ELK, leveraging signatures like the Tour detection, it's.
All there and for you listening, the learner. We hope this deep dive hit that sweet spot thorough but also you know, digestible, giving you those connections, those aha moments without getting totally lost in the weeds. Yeah.
The goal is to show you can get really well informed about network monitoring using these free tools. It's not just for big companies with huge budgets. This guide provides a fantastic shortcut.
Absolutely. So here's a final thought to chew on something building on this, How could the kind of visibility you get from this setup, seeing graffic flows, seeing alerts, how could that actually shape your organization's security policies, or improve how you respond when an incident does happen.
That's the you'll pay off.
We'd definitely encourage you to dig deeper into Zeke's features, explore more of Kebana's visualization power, and maybe just maybe think about writing your own Zeke signatures, tailored specifically to your network's unique risks and normal traffic patterns. That's where it gets really powerful.