What's going on? Everybody? Welcome back to another episode of Adventures in DevOps. I'm your host Will Button, but I'm flying solo today. Warren is at the DevOps conference in Zurich, so he'll be back with this next week. But meanwhile, joining me in the studio, I have the founder and CEO of net Data, Costa Sausis Costa. Welcome and I did Butcher your name didn't I? Yes, it's man. Well, welcome to the show. Thank you for joining me here. Thank you very much for inviting me.
It's very nice to be here. Yeah. So you're the founder and CEO of net Data, a monitoring solution for simplifying and modernizing infrastructure observability, which is that's a that's a huge task. I was you know, as we were talking before the show, we've been doing this for a while and so you know, we've learned a lot of lessons in the last few decades about doing this. So can you give our listeners a little bit about your
background and how you found yourself at this point in life. Yes, So that's a funny story because you know, the data is a monitoring solution that was let's say it happened by accident. I never wanted to build a monitoring solution, even actually when I was starting, when I had started building it, my intention was not to built a monitoring solution. So the idea, guys, is the following I was migrating some infrastructure from prem to cloud.
We had several several problems. It was very early in the cloud industry, let's say, after spending a few big budget actually and building a large team of skills and consultants and advisors and the likes, and with the help of the cloud provider six months past, no outcome. Problems were still there.
In the early days. I don't know if you remember this, we were talking about that cloud is a little bit alive and it behaves a little bit differently, and all this kind of all these kind of discussions, which my understanding, it's a little bit of garbage. It's it's at the end of the day. But anyway, after spending quite some time there, I found myself, you know, it was very painful. We had it was a
fintech company. We were doing transactions, payments on the poss et cetera, you know, cards, and we had many retail chains that we we were serving and the CUE the people that were waiting in line to actually finish the transactions, pay for the goods and go home. It was going around the blocks. It was stressful times. So by that time I started thinking,
come on, what is wrong? Why we cannot find what's happening? Why monitoring systems are so I had the impression that everything that they had built, you know, all the dashboards, all the tools, everything there was just something to make me feel happy. It didn't provide any value. But so initially this is how I started. I was so picked off that I started building a tool to consolidate all the consoles. So what I wanted is not to build a monitoring solution. I said, okay, we have Metrix and
data all over the let's build a tool to aggregate everything. That was the goal. The goal. So instead of having people being on the console of the database and the console of the systems, and the console of this or that, aggregate everything into one environment that we replaces the consoles. So the monitoring tool replaces the consoles. Then in order to do that, I said
a few goals. The first is, come on, I need the same fidelity if the consoles are per second, I want this thing to be per second if the if the consoles have I don't know ten thousand metrics, I want ten thousand metrics. So whatever the consoles do, no, no, no discounts at all. Once I started building these, and you know, the first generation of data was born, and actually I did the same thing
that the console tools do in many cases. So if you have a freeze or a point is missing, a sample is missing, and something cannot because elected, then I have a gap. It's not something that smooths out today's monitoring systems, most of them smooth it out. Yeah, but in the data from the first day, this was a gap. I failed to collect that thing at that time. So the idea is that I worked alone weekends and nights, and you know, I was a COO in the day and
an open source maintainer at night. So after building this for a couple of years and enlisted so and the people loved it, of course, mainly because it gave them all the fidelity that they were missing from monitoring tools, all the information. It's also fully automated, so does boards come up by themselves. Everything happens by itself. So with the discovery of metrics, everything so database or no moving parts, plugins, that the base everything isn't there.
So once I saw the low it's kirokeatet. I released it and it's kirokeated immediately, probably one of the fastest growing products on githubs. So it got ten thousand git caub starts in two weeks or something like that. Oh wow. So once I saw this, I said, okay, you know, initially you feel the you feel proud, but at the same time you feel the responsibility. You say, well, now I have built something that is in total thousands systems around the globe. I hope I didn't mess it up
somehow. So you feel the responsibility. So I started getting you know, ideas, and new people came in and they were contributing and all this kind of you know work that happens in communities. That was amazing. So a couple of years later I decided to start a company. I said, okay, with we have something here. This is not this is not a toy anymore. This is something important that people use every day to actually monitor their
systems. So the I this is how Neata was born. Of course I needed to find a plan for you know, because this is funded the funded companies, so you need somehow to make money and be in the market, have to go to market strategy, et cetera. So what I did then is that I decided that the most important thing to do is to keep this high fidelity nature of data, real time, high fidelity across the board everywhere. How do you scale that thing? That's that's a tricky part. How
do you scale it? Most of the monitoring solutions centralize everything, even the commercials and the data dogs and Dina trace and you really can all the all the all providers today, even the open source from et cetera. All of them centralize everything to one database and then query the database or this day basis one for medicine, for logs, et cetera. How do you scale it? The first thing is that I said, okay, we have something that
works at the edge and it's high fidelity. It collects everything, has a lot more information, amazing coverage of technologies, it is real time. Even the latency is amazing. So on the data has one second data collection visualization latency. So your presenter on the console to do a change and the next second that does work goes choosing the result. Oh wow, So I said, okay, how do we change that, how do we scale it? And then I said, okay, let's go distribute it. So instead of
centralizing everything to one place that becomes a hooge. After a while, it becomes overwhelmed that I said, okay, let's let's try a completely different approach. Let's have the data spread out across the infrastructure in little islands here and there, and let's figure out a way if we can do it to actually merge everything at quiritie can we have the data all over the infrastructure and at the dashboard you have the feeling that this is one thing you can see everything.
So that was the idea, and this is what we implemented. We spent a few years, we implemented the thing. So today in the data is a modern solution that you installed to discovered. Still all the same stuff exists today. So it's a modern solution that you don't need to configure anything, mainly because we don't cherry pick information. So on the centralized infrastructure monitoring systems you have to cherry pick. You have to know beforehand what metrids,
which methods you need, how frequently you need them. Since we eliminated this factor, so let's have everything in high resolution. Again. The next goal was, okay, why to configure anything? Since we can ingest everything, let's to discover everything. Let's just ingest everything by default. The next goal was, okay, since we now we ingest everything, why to go through the process of configuring dashboards metric by metaic insult by charge. Let's find a
way to create the meaningful doashboards out of the box by itself. So we attached metadata that allows the data to correlate the metrics run time and present them in beautiful dashboards meaningful doshboards. The same happened with alerts. Since we collect everything, we have everything instead of having alerts, you know the default threshold alerts that you have If the aggregated bumps goes above this trigger an alert. Instead of doing this, what we did is that, okay, can I
monitor component by components bottom up the entire infrastructure? So can we alarms for a disc, alarms for a netwical interface, alarms for a container, alarms for a posters database for an instance of posters, for an instance of an ENGINEX. So today we see about three hundred and fifty alerts that monitor components
of your infrastructure. Right, So, The idea is that you install a data and suddenly and out of the box in minutes, in seconds, you have a fully functional politory system that you didn't do anything to get it, apart from installing in the data. That is the beauty. Wow. Okay, we got to pause there because I'm just trying to wrap my head around this. Like hearing you say it, it just makes perfect sense. The
part that I'm struggling with is like, this makes so much sense. Why am I three decades into my career and just now having this revelation Because you know, like you said, like when you install a monitoring system, it's like it's like getting grilled by an interrogator. What day do you want? What stresholds are? What frequency? And you're like, man, I don't know. I just got here when you had these answers. Yes, and you know, we went a lot further. So, for example, let's
assume that you use that. So you installed it, you have one hundred several, two hundreds of thousand servers, you installed the data. Everything works by itself, you have alert you have doshboards. Everything works. Okay, now you go and see the dashboard. Wait a moment, you see the matrix for the first time. These are metals you are not familiar, right, right? Can we make the charts easy to grasp, to digest at first sight? And what information do we need on a chart? So we
invented the needle framework, the middle frame. Google is a little tool but above every chart that are allows you to understand where data are coming from, which notes, which instances, which them, and what labels they have and give you statistics about the sources that contribute to the chart. M hmm.
Once you have this, then we can we started discussing about Okay, now that we have really a lot of metrics and everything is automated and everything is visualized by default, can we come up you know, it's it's the trouble shooting that we say that there is a troubleshooting to why because we try to solve the problem of being efficient a troubleshooting time. So how how it works
for more for most monitoring systems. So you have an infrastructure and you face a problem there is a dive in your shales or your users or whatever you say, a dive or a chart. Okay, what do you do next? You start speculating, Oh, probably it's it's database. It's going the database if you don't have charge, let's figure out how to build charge for that. Let's validate the assumption. Oh, it's not the database, it should be the network. I think we have a problem in a network.
Right. This is how it works. You're speculating all the time and you hope that your experience will help you pinpoint the right cause because the monitoring itself cannot tell you. So what we did is that we added to the data supervised machine learning. So we train multiple machine learning models for every metric, multiple for every metric. And then what we do is that data is able to detect a normal list in real time based on the past of its metric,
so it is the path of its metric trends models. And then during a collection, it decides if they just collected value is an anomal orner it's an aler okay, And we store this in the database. So together with example, we say oh, this was a normalogy or no, no,
this was long a noomas. Now the beauty of this is that we create a tool we call it a dobe advisor that you highlight a spike or a diver, whatever is interesting, and we have we build a scoring engine inside inside the data, so it goes for that timeframe that you highlighted across all the metrics. It doesn't matter how many how many metrics are there, and scores them based on their anomaly rate. So your your your ham moment, Oh the disc did that is within the list, so you don't need to
speculate. We just go there, press a button and the data comes up with a list. You show the list. You say, okay, if this happened, this happened, I know, I know, I know what is this? What happened here? How the database did that, or why did this or why the network did that. So the idea is to simplify tremendously trouble to allow yousers be a lot more efficient in the resolution of problems. So not only do you have access to the data points from the metrics,
but you're also putting them in context. So when you look at the when you look at the graph or the dashboard, you see the numbers, which you're providing context to say is this a good number or is this a bad number? M HM. And we in our visualization we have added an anormally ribbon where you can see in real time what the anrmally the anomalists do
the machine learning does how the machine learning detects anomalists in real time. What we found also is so when we had all this infrastructure that was training all this kind of stuff and detecting anomalists in real time, we realize that anormalist happening clusters. So you go there and you see that they are normal lists
on across nodes happening in clusters within a node. So a lot of methods get anomalous wearing something or when there is an anomally within a node, but also a lot of notes together in very short time, one after the other, they get anomalous high in a great percentage. So we didn't know that. We realize this by we're reviewing the data and then we build a tune
to actually to allow people to review a normal list across the infrastructure. So now we have a chart, for example, that gives you for a line for every node that you have, and it's a percent that the percentage of metrics being anomalous concurrently, so you can see the strength of the anomally and the spread of the anomally that at the same time. So this is the
story. This is a short the story of data. Let's say we are trying to make observability a lot more, a lot easier for people to tell you that there was when develops started, there was this little diagram that the consultants or the consultants films. They were saying that, you know, devlops is a joint between data science, software engineering, and IT infrastructure, and were saying, at the at the the three bubbles, at the point they
join, all three of them, you have DeVos. My understanding, this is in theory it is okay, but in practice to have an extremely good data scientist that is also software engineer and it knows about IT architecture and the depth of the IT technology that exists. Come on, guys, this guy does not exist. I don't know if the world may have a couple of them, three of them, I don't know for sure. You cannot have one next to you. The idea is that monitoring needs to be simpler,
and it can be simpler. No need to learn query within the data. We don't. You don't need to learn query language so you can filter, slice dies the data. You know, it's like a tube. You can change the tube the way you see I fits by point and click to create
those words by drag and drop. So the idea is that we are trying, let's say, let's say that we we try to bring the technology in the monitoring technology that the best organization of this world have so real time per second high resolution machine learning everywhere and bring it to everyone in a very simply affordable package. Because the data also is mainly because of its distributed design,
is the most cost efficient solution. Oh right, yeah, because you're not stuck with a couple of years from now having to run these monster servers just to maintain the amount of data you got. Actually, we use resources, computer resources that are available and spare. It's your servers. They have two PERCENTPE to spare and I don't know, two hundred megabytes of RAM, that's easy. And this is what we use. Two percent cype of a single
core, three percent cype of a single core. This is what data needs. Resources and two hundred megabytes of RUM and I don't know one gigabyte or two gigabytes of this that's it. Well, So not to mention that if you compare with commercial offerings, all of them require tremendous mount of egress bandwidth. The data does not stream anywhere. It's it's inside there, so it out egress. Banquet will be used only when you view the dashboard. If
you don't view, there's nothing there, there's no egress. So that's the whole point. Simplify. Take something that is the best out there. We integrated in the data technoledge for example that we use a variation of Gorilla compression. But Facebook developed so Facebook has developed a real time high resolution monitoring system data base, time data base, and it's called Gorilla. We took the concept, we adapted to the data and now Gorilla compressions in the data.
So that's across the board. That's what we do. Across the board. We're trying to bring the best and give it to a This is why we say that we democratized monitor So when you get to this level where you're bringing in this granular data from all across your infrastructure, how do you how do you determine what's a surface for the user? Because that seems like an avenue
where you can get to information overload really really quickly. So the idea is that, Yeah, so the idea is that we group everything into meaningful stuff, so you have you go and it says, here are medical interfaces, the top information about network, the faces, this package and bandwidth for example in terrors. But then that are all that is that you may need in order to explore what's happening. The same happens everywhere, so your database,
ever, your even your tables. Do we go down to the intex level? Now all this information mainly because we have this scoring. Eagine when when you don't know what to do, so you are in a dashboard it has five hundred charge on it. Okay, what do I do now here? The first thing is that we there is a button that says, okay, identify for me the ones that are currently in the visible time frame, the
most unormalous for example, so to give you something to look at. You know, when you are looking for something, you don't know where to start, but it will identify the doshboard sections. Okay, here I have twenty percent. Normally this is bad, go look here. So the idea is one we've developed tools to help people with the information overloads, let's say. But the most important thing is that there are people that are afraid this information
overload. There are other people that enjoy the depth and the detail. What we hear from users is that when you use the data for some time to explore your infrast actions. Let's assume that you're not troubleshooting anything. You just want to understand. It's like feeling the pulse and the breath of the infrastruction. You feel it because you see it on every second. It's extremely high resolution, so you can understand what is really happening there. And I think
that this is the most important. Of course, it's a tool for people that want to learn. If someone wants to just traffic lights, oh it's healthy, it's not healthy. If someone wants just this, it's very whelming for them. But if someone wants to learn to dive, to understand to fix the problem, this is where in the data steps in. Wow,
that's wild, that's wild. I mean, you know, because I feel like from my own personal perspective, like you know, we've been approaching monitoring and observability the same way for so long and then this just completely flips all of that upside down. But that's the whole point, and I think this is the monitoring system that is missing today. So PROMI, for example, are amazing for customizability. You can build whatever imaginable you imagine it, you
build it great. The big guys data dog Dinah, Trace and they likes. They try to give you a silicopter view of the most important things. Although they value high resolution, they charge high resolutions, they are not by default. So I think that there was no tool to cover this, this this area of real time deep dive monitoring that you can go and take some in everything in high resolution and very detailed. To tell you the truth I have. The simplicity that we added to the tool is one thing that our
users love. So when I go and speak about data, I see in many I was enforced them, for example a month ago, and it was amazing because you see people that are skeptical about observability. There are some that are the point of observability denial. The idea is that what they have an argument. What they say is that come on, this is too complex,
too expensive for what I get for sure. So with what we tried, the the fact that we solve the cardinalti granularity problem and we can scale infinitely without actually becoming a problem, this allowed us to become a lot simpler. So it does a lot more. But the thing is simple. You don't have to do anything, you just have to use it. So this is I think that this is the booming factor, is that it is higher resolution and at the same time extremely easy. You don't have to do anything.
It doesn't require from you, not even resources. Just give it the resources that you already have and that's pair. So I think this is the This is the combination of things that make the data so appealing to you. Yeah, for sure, especially the ease of use thing, because a lot of the other observability tools and just my own personal experience with them is they they rely on me too much, and and they rely on me to know what questions to ask. And I'm like, man, if I knew a question
to ask, I probably wouldn't be asking. You wouldn't be Yeah, yeah, you know. The interesting part is that some people when it is when they hear, for example, a distributed monitoring solution that everything is at the edge, the first thing that they say is way the moment, man, this is going to be more heavy than the other agents, right. Yeah. We did a comparison on our side. There is a blog post where I could compare the agents of all monitoring solutions that we could find. The
data is one of the lightest. So the core of the data is written insane. It's severely optimized to be performance m of course we have plug ins and the likes that are high leveled and written in goal or whatever, but the core is x extremely optimized to be very efficient. For example, we have we did a stress test against against Prometheos, mainly because Promethels is the
industry studdard. So one third, let's SPU same lod. So we gave them three terabytes to almost three million metrics and we said per second, everything per second, and we said again, let's let's see the resources that both systems need. The data and Promethels the data used one third LESSIPU, half the memory, it used ten percent less bandwidth, ninety eight percent less diskyo, almost the disc was idle all the time with the data, and seventy
it managed to fit seventy five percent more data into the same store. So we have for example promythees has either uncompressed data on disc at two points something two point ten and if I remember correctly bites per example on disc or compressed with Gorilla that goes to one point three one point three bytes per sample. The data has zero point five zero point half a byte per sample on disc. It's extremely efficient. This is this is why we did it in c
to make it extremely efficient. Yeah, for sure, for sure. And you know the data traditionally was only metrics. But last year I said, we had built our own logs management thing, so it logs database that we're integrated into in the data. But then last year I said, wait, the moment I think we're doing it wrong, I realized that all of us have a new system, the journey. System the Journal is part of system G fort the logs, et cetera. This thing is amazing. Probably we
don't know it, but it is amazing. All the logs management systems they sufferre from cardinality. So how many streams of data are right right? System the Journal does not care. So every log line can have its own fields and its own values on these fields, and all of them will be indexed, all of them. So cardinality not the problem. Full indexing on everything the only problem that has. So even if you centralize, you put a lot of logs, web server logs for jump to it. The ingestion process,
et cetera. Are extremely optimized for system the journal. The only problem that it has is that they need disk space, so it requires more disk space than the rest, but it does more with the disk space, and at the same time it is secure. It's the only logs management solution that has us fall tolerance and high availability and even secure ceiling for securescess for ceiling,
so it ensures that the logs cannot be tempered. So the idea, the way we managed to use system the journal is exactly the same way with the data, so we use them in a distributed way. You don't need to centralize your logs. Why the whole world tries to build a logs processing pipeline and they try to install real time tools to do analysis and the likes and all this kind of stuff, you know, in order to get insights from the logs. And what we said is why do we need the pipeline
in the first place. We have an agent next to the logs that can quate the logs in real time, extract all the information needed and report it. So why why to move the logs from there? Let them be there. Yes, that's the same same approach as you were talking about with the
metrics collection you've got. You can just take a little bit of the CPU and the memory that's not being used on as system and just do the work there rather than the transport method of shipping it off someplace and dealing with this stuff in volume. Of course, we have centralization points, so if you have a femeral, you have a coubernetus cluster, you can have any data
partent. This is how we call it. It's the same software thet but you have a parent now, so all the ephemeral notes push metrics to it in real time. But these are small and only do the degree need it, so you don't need to centralize your entire infrastructure to this. And you can have multiple centralization points, one here, one there on if you are a high if you have if you use a hybrid cloud, you can have one on a WUS, one on a P one on Azure, one on
prem whatever. So the flexibility that you have. You can have the same system the journal, you can have centralization points which system Digenal methodology is not in the data system, the journal remote and system the general upload, but it's exactly the same philosophy centralized to the degree required for your operational needs. Yeah, and not because the system, the monitoring system requires it. So
it's your operational needs that require some centralization of some kind, gotcha. Yeah, So that puts the that puts the decision back on you to meet the needs of your business rather than putting the burden on you because of the tool that you chose. So and also you can centralize for high high availability. Do you need high availability? Of course, push the metrics somewhere, the logs pustem somewhere, to have two copies of them, of course. But
this is the idea. You centralize for operasonal needs, not because the monitoring system mandates it, right, Yeah, yeah, I got you. That's wild. This is just this is just I feel bad saying this, but it really is mind blowing to me because it just makes so much sense. And I'm just I'm just shocked that it's so obvious. But here we are. That's the beauty of it. You know that most of the monitoring systems, you know, the monitoring systems evolved. There is a Initially we had
nagios and the likes vise were check basis. So they run a check, they take a result. Together with the result, probably they have one or two metrics or some logline or something, a string to say what's wrong, and the check has status one or more status actions there it may be healthy and healthy or warning or whatever. All the systems, not the zubis, Essensiu, Insigna, solar, Aguain, spirited, the all of them are
in this the first generation of check based systems. Right then the world went to from technology that is metrics logs, so metrics databases, logs databases. It's not checks anymore. We gather the information, we put them in the database, and the go analyze the database. The problem with this is that it struggles at scales. It is expensive generally to run, so you need
to filter, you need to slower the resolution. Cherry pic metrics lose some metrics some information in order to have a performance system and a performance centralized system. Of course, this philosophy the guys, the data dog and Dinatries and the likes, they took it and they built integrated environments, very nice, very nice, inte great environments. So what we do is, I think
it is the next generation. It's the next evolution. So we take this philosophy of metrics, logs, et cetera, et cetera, where you don't do checks right, but the checks are on the logs and the metrics and the data, and we did it distribute it. So we did it in a way that it's still integrated, it's still one infrastructure, but we eliminated all the problems there that forced people so far to provide low resolution insights or eliminate several insights from it, et cetera. And I think that we did
it in a very efficient way. So the way I think of it is that the data is the next evolution of monitoring systems, and it's nice to see. You know, one aspect that we didn't speak so far is the following the data has. You know, monitoring systems are metrics, logs, traces. We don't do traces yet. I think we're when I started soon and it's going to be also distributed. But metrics locks, let's say,
for the number and traces when you have micro services. Developing micro services, what we found out is that there are a lot of information that is neither. It's not metrics, it's not logs, it's not traces. It's all the show cats, for example, all the network connections that the system has, it's all the the processes that run. It's so it's all the files that are open. So the idea is that we created a mechanism where are
our collectors the click ins that collect data. So they collect to a postgress for the connect to posts to collect for some some metrics. But they expose a function that allows the dashboard to say, okay, show me the slow queries of postives. So query posts the slow queries and give me a list of the slow which is that are currently running. Similarly, give me all the network connections or the outbound connection or the inbound connections, or the listening
circuits. Show me all the processes that are running. So the idea is that at the end we created a monitoring tool. It's the original idea when you're trying to kill all the consoles, who still do right the consoles?
Yeah, because that's like that's again that's additional context. You know, you get your list of slow queries from postgrass and eventually, if you don't quickly see oh well this is this is a query problem because we're missing an index or whatever, you do end up you know, sshing into that system and
running top and checking the open files handles and all of that stuff. Because no one can afford to run full time monitoring of that level of detail using some of the other monitoring tools like you you you would bankrupt your company trying to collect and store all of that data exactly, and this is exactly what the data changes. So it's more it's more cost deficient to run the data than running any other Commercially, it's more cost deficient because we don't need resources,
you don't need skills. It just works. Yeah, and and the information is just there when you need it. But it's not hurting when you don't need it. It's not costing you when you don't need it. Exactly. Wow, So what's the what's the onboarding process look like to start using that data? So today we have so there is the open source agent that you We have two paths how people can join the community. Two paths. One is you are a did have fun open source fun or whatever and enthusiasts.
You go and install the data the agent. The agent is monitoring in a box, so by itself on a single node, you will have it will do everything, does watch, alert everything, it will do that do them for you. That's one path, and then you will you do the second installation the third and then you realize that you can build parents or you
can use nedata Cloud to unify an infrastructure. The other part is that people that are looking companies that are looking to replace their monitoring system, they go to the first five through Google or whatever, they find our site. So they went to our site. There is a trial where they sign up to nedata Cloud this time and nedata Cloud instructs them to install agents with certain keys et cetera. In short, in order to link them with their own space,
their own account. So in both cases you start for free. Nedata Cloud is a thing do centralize data. So the data are still distributed, but nedda Cloud is a layer that builds a map of your topology, so it knows where are the nodes and what retention they have and what metrics they have without centralizing the metrics themselves the values. And then when you go to queer data, ne Data Cloud says, okay, I'm gonna quer this this and that notes to get the data, merges the data present them to you.
Okay, all these are in real time. All these yea, there's no latencies there. All these are like these very very quick. So the idea is that people can start either by the open source world, go to GitHub, download the software, install it, or they can from a commercial world go to nedata cloud, sign up a trial. Again, you're gonna download the agents, et cetera. So Neddada Cloud, our commercial offering, let's say, uses the agents as a distributed base, so we don't have
different enterprise agents. It's the same thing in the open source software. But what it provides is the following. The first it allows it allows you to skill your infrastructure horizontally. So if you don't use an data cloud, the only thing you can do is bigger, build a bigger parent, bigger and bigger and bigger to aggregate all the infrastructure there. Like the old traditional centralized systems, it skills better than them, but still it's one system. This
is one way with n data cloud. You can have as many independent centralization points or individual standard loone servers and all of them will become one at query type. The second is success from anywhere, so the agents. You need to hit the IP of the server to access the dark ware for the data cloud. It doesn't matter. You look into up in the data cloud and
then you access severity. The data cloud gives you a mobile lab for push notifications, so all the alerts across the infrastruction will be pushed to your mobile lab adroid and dios. And also it dispatches alert centrally. So instead of dispatching all the ages to dispatch alerts to slug or pay your duty or whatever email whatever you use, the data clouds receives all the transitions that dapplicates them and then dispatches the alert central. There is a free ti air two for
home users, et cetera. There is a small free air on data cloud But overall, you know, mainly because we decop it the cost of observability from the monitoring itself, it is a lot significantly more cost efficient. So if you go for the commercial offering, if you go to data with the legs, you start at twenty and thirty dollars a month per not right three oh wow. And even with data Dog, like when you start with that level like that, quickly you quickly learn that that's only the monitoring costs.
Like once you start bringing in the data, there's data charges as well, and those others I have actually here because we did this comparison about the resources that data Dog for example needs versus Data Sepe. Usage of the Data Dog agent fourteen percent the data through three point six memory usage data DOG almost a gigabyte nine hundred and seventy two megabytes are from the data one hundred and eighty one egress pernude eleven gabytes per month perndes. The data nothing doesn't need an
egress barnment. So it's you. It's more expensive and you put more resources to it compared to Yeah, wow, So are there are there you? You know, we mentioned early on in the podcast that your background started with retail transactions point of sale systems, which are very high throughput and if you've ever worked in retail with a customer, they're they're not the most they're the easiest things to deal with. So you come from a high stress environment.
Are there is that? Is that indicative of the type of customers who really embrace net data or is there a specific industry that you work really really well with. I think we have people, we have businesses from all over the old industries. So we have people from health care, we have people from manufacturing, we have people from technology, a lot of technology of course, right, So I think that the key point here is, Look, if you go to a DevOps guy that wake ups at three A at three am.
We have another like this that says, I love three am. If you have a guy that wake ups at three am, you have to understand that at three am has he's pissed off? So you have you have to be real time. We have many big companies, really among the Fortune five hundred say companies that they don't accept the latency that others provide. Yeah, a minute latency is what. No, at three am, I will not wait one or two or three minutes to see if fixed it or not.
I wish say it now. Yeah, okay, So that's the that's the idea you have. People have to value the fidelity the insights, so they need to. It's like people that want to learn more, to understand more, to feel the infrastructure more. This is our best clients, best customers, best users. On the other hand, people that they are afraid of this thing. If it works, don't touch it, let's rebut it. That's the funny thing because many people do that, Oh it doesn't work,
let's put it. It's okay. Then the data is not that useful for you to you you need Actually the first kind of monitoring, the check based the check based systems for sure you have just the light or not. So another thing that's that strikes me as being really cool about this is that you've got all of the observe observability data in one spot, because that's been one of the other challenges I've had over the years is you get the alert,
but now it's just an alert. You don't really have the context, so you have to go somewhere else to get the context of what what, why did this thing alert? What does this mean? And there's another spot for every alert that we see. We have a community site that people can go and see what others did about this. So you we have a CTA on the alert that goes to the forum about this alert where people have this alert. We have an introduction with that that we wrote ourselves. This is what
the alert means. This is how you what you should do if you have the system do like this, if you have the other system to like that. But then people discuss forum that's wild because like even even you know, there's just you get alerts for all kinds of things, and then you add to that that you're getting paged at three am. You know you you might need a little a little help there to understand the context. So thin within the data is that the community is vast. So so far we count more
than ten million users. The community grows with about five to ten thousand new users a day. Even Docker have downloads. For example, we have about one hundred and fifty two hundred thousand DOCKERHB downloads every day. Even on our Sash offering, we have about one hundred and fifty two hundred sign ups every day. Business that's that's that's a lot in a lot of project. And not only that, for example, the love that we see from users is
extreme. The data in terms of user love starts. For example, we lead the c and safe observability category. We sur passed Elastic in October. Now we are leading the observability category although we are not incubated where it seems if does not endorse the data. But it's the most loved project. Let's say that. Yeah, that's just crazy. I mean, and just af you're talking with you, I can see why you're getting those kinds of numbers. It's just I'm still just trying to wrap my head around it. It
almost seems too good to be true. I'll just be honest with you. It's like what's the catch? Yes, what's the catch here? I will tell you all the data is not not mature yet, so we're building that does boards are pretty new, it's less than a year old. Even our database, it's the third version of the database. We feel is the last year. So it's not that we have not built that many tools on top of the infrastructure of the monitoring infrastructure yet. So the baseline is there.
You have high fidelity, you know, unlimited methings. You have all the building blocks to actually do the work. We are lacking in a lot of high level tools right that we're building. Now that's okay, yeah, yeah, for sure. The good thing is that the foundation is very good. Yeah, and it's it's open source. Do you what's your what's the open source community look like? Do you find a lot of do you get a lot of poor requests and input? I will tell you we have a lot.
We have about four hundred fifty five hundred contributors. Some of them very few, are very skillful. Because we write in sea. If you remember the course in c this is pretty, this is pretty. It's not easy. In the early days, a lot of community used because come on, I used to be an engineer in the nineties. Then suddenly in twenty fourteen fifteen, I became an engineer again, so I was rusking. You can understand that. So sub in technology all these years, I was a coo
all of all my time, it was city o stuff like this. But suddenly I had to write quode myself and people helped me. So a lot of people stepped in and showed me how this is done, and you know, pushed me a little bit beyond my limits. All this happened. But I think that today in that I is a mature open source project. It's very robust. We also wanted to crassies and the likes of that. It's very nice, it's very reliable software. So I think as time passes it
becomes incrementally more difficult for people to contribute code. Yeah, and actually what happens now, we're going system the Journal D. We submit it to system D repositories patsies to make System the Journal fourteen times faster. So we are also a community contributor to that thing. So the System the Journal that you have in your in the next version that you're gonna have there, it's going to have parties of a data inside to be fourteen times faster than it work.
See, and that's cool that part, that part of open source really gets me excited. You know, where you you're consuming an open source thing and you're like, ohw this can be better, and rather than forking it or creating your own, like, contributing back to that, I think is a more idea. You want them to. You want the software that tend to be maintained and high quality. So and actually, the work of applying
patsies to your version, come on, that's not good. Yeah, It's better to have it there and they maintain it from now on for sure. Yeah, because yeah, I've seen that trend over the last few years of people patching or forking or creating a competing product, and I just I think we would all be so much better if you just contribute back to upstream, because you get all of the benefits of your work, plus the benefits that
everyone else in the community is contributing as well. If you think of it in the data of the day, because it's monitoring out of the box, it's an opinionated monitoring. So what happens is that when you're install the data in your infrastructure, you're monitoring team is the data. We are your monitoring things. You're just a consumer, right, using it the same way we want system D to be there for us exactly. It's utilizing the whole community
in order to provide high quality to end users. Yeah. Because and that's that's really key because very few of us, none well none of us really except for the people who work system D A are are getting paid to create you know, the journal Like our customers aren't interested in our journaling, they're interested in the product. That I think for projects like system D, this is easy because most of the engineers that work there are paid by other companies
by their company and work on that thing. So other commands red hat or mund this or that, they contribute resources that are working dedicated on this stuff. Yep, that's okay. So their job is to work on system D, but they work for some other company that uses system right for a startup like us, because we are startup still, so we just started we started monetizing five months ago. Oh wow, okay, this is pretty new. Yeah, very so for us, we cannot dedicate so much resources. But
for sure, the whole point is the community. The whole point is to aggrediate a lot of to unify the community, gather together a lot of value the users. Yeah, so early stage start up. What's the future? What are your what's on your roadmap for data? Oh? You know, monitoring is endless, endless, and there is no way for this to finish. What we are trying to do currently cover us much because we want to grow our sales. We are not sustained, so we need we still need
to prove that the data is sustainable and it deserves more. So when this process. The idea is that currently we are looking to help the users that have a budget. This is our main thing because this will allow us to survive. Once we pass through this, I think the next steps will be to address tracing, provide the high level tools that people need, become more
contextual in the US interface, and the likes. I think that the data has the winning mix, the winning combination in product design, easy, higher resuls hire insights, higher fidelity, the overwhelming part of you said, or we already know how to fix it, so people not to feel overy well by the amount of information if we switch to a more contextual approach at the
presentation level. So we're going to switch from overwhelming to very deep. Right, So these are these are changes that we need to do as we progress. But I think the first thing is people to to try in the data, seeing the data, if they use the data in production environments for commercial purposes, to buy a license because this will allow us to continue guys who are here to actually we are here to provide value. And for sure, if people don't buy the value, then what are we doing here? Yeah?
No, And and it's a it's a proven business model, you know, data Dog and and Dine Tracing those other companies have proven that you that there is a market for this, and so if you're able to tap into that market and provide better service at cheaper price. Yes, it's not only cheap, it's the overall cost that is better, all of it across the board, so even cost optimization. We see a trend for the unification of
the monitoring tools. So mostly because tools. Monitoring tools are not that comprehensive. Let's say they have certain aspects that are very good, but they're very bad at others. Mostly for these people use a lot of tools, right, so people are trying to consolidate tools. They are trying to save money, become more efficient in troubleshooting. This is what we do. This is
exactly what we're trying to do with the data. Simplify monitoring, make it accessible to everyone, let them focus on their job, not a monitoring tool, nothing to learn, just some familiarity with the tool and you're done. Stuff like this for sure. Yeah, Like simplification of the tool is huge because we're all getting increased responsibilities and increased scope of work. So and that technology becomes more complex. Really, absolutely, our time passes the complexity skyrokids.
You cannot you cannot fit it in your head anymore. So you need the tool to be smarter. Instant looks, get smarter tools, tools that do stuff by themselves. Yeah, and that to me, that's a huge, huge win here. Just providing the information in context, what do I need to know at this given time, rather than me having to go find it something that presents it is huge. Well, this has been an eye
opening conversation. I'm I'm excited. I mean, I monitoring gets let's be honest, it's one of those hard things to get excited about, but this is kind of exciting. It changes the dynamics, it changes everything, Yeah, it does. I mean I think back to like a few decades ago configuring nagios and zabs and thinking about how that just led to this here.
It's pretty cool. It's definitely definitely something worth checking out. So thank you for joining me today, Thank you for being here, for inviting me. It was great. I hope you enjoyed it. I did. I thoroughly enjoyed it because this is a perspective on monitoring and observability that I never had. I feel like I'm walking away from this conversation with a completely different set
of goals for how I'm going to approach this problem in the future. Like you know, you hear, especially in a start space, you hear the term disruptive throne around a lot, But I mean this one, it kind of like it fits that category and it's just such an odd place, like who thought, you know, you could disrupt observability, But here we are. YEA, Let's hope it works. It remains because you know, the devil is in the details for sure. Work hard to smooth everything out to
to make it as perfect as possible. Yeah, but it's a big only the integration that we have, you know, it's eight hundred integrations. Come on, it's it's and we're a small team. We're thirty people. Wow, that's impressive. That's impressive. So for all of our listeners, if they want to find out more about data, where can they go data dot cloud or no data monitoring on Google, your shirts on GitHub. The data and data is that Apple didubs last data, slast data. I think it's
quite popular, so it's easy to find. There's a big community, they speak about it and we have read it. Also there is an army data so yeah, right on it sounds good. Costa, thank you so much for joining me today. This has been opening and I really appreciate it. And to all the listeners, I hope y'all found this useful. I hope you go check out net data. I know I'm going to and I will see y'all next week. By it