Barr Moses on How Data Observability Can Save Your Company Millions

Welcome to Data Driven, where we dive into the thrilling world of data, AI, and on occasion, misbehaving chatbots suggesting glue for your pizza. This episode features Bar Moses, CEO of Monte Carlo. Not the casino, not the car, but the company keeping your data from quietly wrecking your business. We talk observability, the chaos of unreliable data, and why one tiny schema change cost a company

$100,000,000. Ouch. So buckle up. Because if your AI bots are making decisions without reliable data, well, hope you like eating rocks for the minerals. Hello, and welcome back to Data Driven, the podcast where we explore the emergent fields of data science, artificial intelligence, and, of course, data engineering. And with me today is my favorite data engineer in the world, Andy Leonard. How's it going, Andy? It's going well, Frank.

How are you? I'm doing well. I'm doing well. I was in Raleigh last week, drove down, rented a car actually, to save mileage on, on ours, and, spoiled because it's been a while since I bought a new car. And this is the second time I rented a car, and I'm getting tempted. I ain't getting tempted. It was a Chevy. It was a Chevy Malibu. Not a Monte not a Monte Carlo.

See what I did there? I don't even know if they still make them. I I was driving, the little one off and dropping the little one off at daycare, and I was behind a Chevy Monte Carlo, like, a two early two thousands vintage. But that is actually quite relevant to our discussion today because with us today, we have Bar Moses, who is the CEO and cofounder of Monte Carlo, the data and AI reliability company, not the casino

or the car, I would assume, or the town. Monte Carlo is the creator of the industry's first end to end data and AI, observability platform with $236,000,000 in funding from Accel Iconic Growth and others. They are on a mission to bring trustworthy and reliable data and AI, to companies everywhere. The company was recently recognized as a enterprise tech 30 company, a CRN emerging vendor, and an inc.com, best workplace and accounts Fox, Roche,

Nasdaq, and PagerDuty, among others, as their customers. Welcome to the show, Bar. Thank you so much. Great to be here, Frank and Andy. Awesome. An intro. No problem. Do you drive a Monte Carlo? Because that would be epic. You know, I really should be driving a Monte Carlo. I do not, and I've never actually been to Monte Carlo either. So I will tell you if you're into cars, like, I'm like a recovering car, nerd. Oh, very cool. It looks like a car show. Like, honestly, I went to Monte

Carlo, and we had rented, like, a Saab convertible. And I felt like we were driving. We were driving driving, like, the low end of the car thing. I mean, there were I mean, I've never seen Bentleys in the wild, like, just parked on the street, like, no big deal. Wow. Like, I mean, every luxury car if you're in a Saab and you feel like you're slumming it Yeah. It is clearly a high money area. But, so welcome to the show. So Monte Carlo

why'd you get the name? I I'm assuming it might have something to do with Monte Carlo simulations, but that's in the Great question. Yeah. The unofficial story is that, one of our CO, founders is a fan of formula one and, you know, as, you know, formula one crisis. So right. That's, you know, clearly the, the, that's the unofficial story. The official story is that, you know, we

had to we had to name the company. We started working with customers when we started the company, and we we had to choose some name. And, I studied math and stats in college, and so I sort of opened my my stats book and sort of looked through and, you know, reviewed my option and, you know, Markov, chains didn't seem like a great name. And next up was Bayes' theorem, which was similarly kind of not great. And

and then, you know, I was reminded of Monte Carlo and Monte Carlo simulations. I actually I actually did some work with Monte Carlo simulations earlier in my career. And it seemed like it seemed like a great name, a name that would speak to, you know, data engineers, data analysts, folks that have been the space. And, you know, I think naming a company is a very difficult thing to do. We decided to go with it. And the spirit of Monte Carlo, One of our values is ship and iterate. And so, the

name has sort of stuck with us since. And, it's quite memorable. People either love it or hate it. So I think it works for us. I think it it works. Like, I think of the car. I think of the casinos. It has a certain amount of, high class, maybe more so than Markov chains, Markov chains. Although I did for a time flirt with the idea of of also starting a company called Markoff Chains, but, like, have see if we could see if we can get money for mister t to be the spokesman. That would

have been epic. Yeah. Jeez. He did you. Ideas, Fran. I was the only one I was the only one that thought that was a good idea, but, you know, I was a big fan of mister t as a kid. Marketing. Yeah. That's funny. That's what I do in my day job now. Oh, yeah. I swear, folks, I didn't pay her to say that. So so you you talk about data and I AI reliability. And to me, when when I hear that,

a slew of things come to mind. Like, there's security, there's the veracity, like, the five v's and all that or four v's or whatever it was. What exactly is kind of Monte Carlo's, like, wheelhouse there? Yeah. Great question. I'll

actually sort of anchor ourselves in in kind of the metaphor or sort of a corollary that we like to use here, which is really based on software engineering. So we didn't reinvent the wheel when we say data and AI observability. We really take concepts that work for engineering and adapt them. So, you know, when we started the company, the idea, the hypothesis, the the thesis that we started the company on was data was going to be as important to businesses as applications, as online

applications. And, they were data was going to drive the most critical sort of, you know, lifeblood of companies through decision making, internal products, external products. And, while software engineers had all the solutions and tools in the world to make sure their applications were reliable, and so some, you know, some off the shelf solutions like Datadog, New Relic, Splunk might be

familiar to you, data teams were flying wide. So there was literally nothing that they could use to know that their data was actually accurate and trusted. That's sort of, like, the the problem the core problem that we started. Fast forward to today, you know, we created the data observability category. We're continuing to create it. AI is making this problem just infinitely bigger, harder, more important. Why? Because data and AI products are now you know, there's a proliferation of those.

An AI application is only as good as the data that's powering it, and the AI application itself can be inaccurate, can be unreliable. Right? And so at a very high level I know this is, you know, very vague, but at a very high level, the idea was the same diligence that we treat software applications, we should be treating for data and AI applications. Now, what does that actually mean? How do we do that? Enter the concept of

observability. Observability is basically understanding or assessing a system's health based on its output. And so basically, the thesis was, can we observe end to end the data and AI estate, learn what the patterns are in the in the data, bring together metadata and context, lineage, for example, about the data, derive insights based on that to understand and determine what the system should behave like, and alert if that gets violated. So that's sort

of the first part. The first is actually being being able to help data teams detect issues. The second part is actually being help, helping data teams resolve issues. Now here's the interesting thing that we sort of learned over over the years. We've worked with hundreds of of enterprises. So, you know, we mentioned a few. We real really work with the top

companies in every single industry. So, you know, in in, in health care, in retail, in manufacturing, in, technology, in each of these areas, the, data in the state obviously varies, but there are actually interestingly commonalities. And the commonalities is that every single issue can be traced back to a problem with the data, problem with the code, problem with the system, or problem with the model output. Can go into detail into more each of those, but that's sort of the high level,

framework. We basically provide end to end coverage to help data teams understand what the issues are and help them trace them back to data issues, code issues, system issues, or model output issues. So when did you get the idea that I'm sorry, Andy. I cut you off. Okay. When did you get the idea when you realized that data is gonna be as important as applications are to businesses? Oh, great question. Yeah. Great question. So so we started the company in 2019.

And, actually, what's interesting, it was pretty clear to us then, but we had to prove that or we had to convince that of people. Definitely. Yeah. It was not obvious. It's it's still there's still a lot of people that are kind of, like, I guess, they'd be in the quadrant of laggards where they realize, oh, I guess this is important. A %. I would imagine in 2019, that would have been you would have sounded insane. Like We we sound I

sounded insane a %. People are like, what? Data is gonna be important? Are you sure? Now a couple of things happened since, which I think helped. First is, there were some large acquisitions in the data space, like Tableau and Looker earlier on, and then Snowflake IPO'd. Snowflake was the largest software IPO of all times. It was quite interesting that the

largest software IPO of all time is a data company. So I think those things sort of help kind of convince that this you know, convince, at least, externally, you know, to the market that data will continue to be will will be important and critical. I think the things that I noticed is, you know, before we even started the company, we spoke to hundreds of data leaders, and I

speak to dozens of data leaders every single month. They continue and I think what you hear from them is more and more data teams and software engineering teams are building products hand in hand. So they're actually they're side by side building. Right? And so, actually, almost more and more critical business applications, revenue generating products are based off of

data, and they're being powered by data. I'm not even talking about generative AI, which is a whole whole other story why that matters, but just data products by itself. Think about reports that people look at internally. You know, just give you an example. You know, we work with with, many airlines, for example. Airlines have a lot of data that goes to internal operations. Like, what's the connecting flight? What's your flight number? How

many flights left today? What time did they leave? How many passengers were on the airplane? Where is your luggage? Right? That information is powering internal and external products. You know, it's powering the application that you're using in order to onboard the the plane, in order to connect to your next flight. If that data is inaccurate, like,

you're screwed. Right? And that hurts tremendously. Your brand is an as an airline, your reputation, it leads to reduced revenue, increased regulatory risk that you're putting yourself. Right? So so the data, what we see from our customers is powering critical use cases like airlines. I'll give you another example. You know, we work with a, you know, a Fortune 500 company, perhaps your your favorite cereal. I don't know if you're you guys are big cereal. I I, like, eat cereal

for breakfast, lunch, and and dinner. It's, like, my go to. You'd be surprised into how much data optimization, machine learning, and AI goes into actually optimizing the number and location of cereal on the shelf. So there's a lot of data that goes into supply chain management to make sure that you're actually, like, fulfilling the right warehouse, demands on time and, you know, making sure that everyone gets their serial on time. There's actually a lot of data that goes into all of

that. So I think what gave me conviction was in speaking with so many companies across so many industries, data was actually allowing data teams, allowing organizations to build better products, to build more personalized products, and to make better decisions about the organization. So I think that really sort of made it clear that the future was going to be based on on data. Well, I I like that you pointed out, the importance of observability.

My career path winding as it was, I made a a leap from being a software developer to being a data really a database developer. When I made that transition, one of the things I had noticed, this was two two and a half decades ago, I had just started in software development doing test driven development and it had just come out, it was called fail first development. I remember thinking this was perfect. It was a big deal. Yeah. It was. Yeah. Twenty five

years ago. And I remember thinking this is perfect because I'm always failing. So this this will work nothing ever runs the first time and if it does, it's suspect. But when I got over into data, I had just become, you know, kind of a a big believer in the power and and and really the the confidence that test driven development gave me. And I was like, we need that over here. And so it was, just a field that's fascinating me. I have an engineering background, and so it kind of flowed

right through. Instrumenting the data engineering, was a big deal so that, again, you could achieve what we now call observability. But being able to watch that data flow and when I would mention this to people kinda like you in 2019, I I would get all sorts of responses. Most of them kinda raised eyebrows. And I would, some of the more interesting ones were things along the lines of, well, the data is sort of self documenting. I mean, it's it's just there. And I'm

like, no. No. It's not. It's I especially when you've moved it through a bunch of transformation to put it into a business intelligence solution or data warehouse or or any of that. And that now feeds, you know, modern LLMs, AI, and and the like, those same sorts of, I guess, old school processes, I do. Or at least that's my my understanding. Maybe I'm reading too much into that, but I love the idea of having observability go

all the way through. You mentioned lineage. That's huge. You wanna make sure that when you, you know, you make this one change, that's not gonna affect anything else. Usually, it does affect other things, and having that lineage view is huge. That is spot on. That's exactly how we've we've thought about this as well. So, you know, I

think there are specific things that you can test for in data. Like, for example, you know, specific thing that you can declare, you can say, like, you know, you know, a T shirt size should only be, you know, small, medium, large, extra large, whatever. Right? But then there are some specific things that, you

know, you you don't necessarily know. Like, for example, if there's a particular, you know, pattern that the data is being updated, you can actually use machine learning to automatically learn that pattern and then forecast when it should get up updated again. So it's not necessary for someone to manually write a test for that. Right? And so I actually think it's a combination of both of those things which really give confidence to to data teams over time. So there there's sort of a

couple components to it. The first, I think it really starts with visibility, sort of call it end to end observability, but it really includes, like, you know, you mentioned a few of these parts, but, the data lake, the data warehouse, an orchestration, BI, ML, AI application that can include the agent, the vector base if you have a prompt. Right all of those

components you have to have visibility. The first thing is actually to to your point, like, having lineage into what are the different components that can cross this. So all the way from. You know, sort of ingestion of the data to consumption of it. And the second is to start observing. And and, you know, you there are some specific things that you can declare and test and based on your business needs, and there are some things that you

can do in an automated way. And and, actually, I think this is an area where AI can help. So for example, what what oftentimes teams end up doing is spending a lot of time trying to define what are data quality rules. And, actually, you can use LLMs to profile the data, Make some make some, yeah, make some inference, based on the semantic meaning of data and then make recommendations. So for example, I I love this example. We work with lots

of, sports teams. And so you can imagine that, you know, you have a particular field called, like, let's say this is in baseball, a baseball team and sort of, like, you know, pitch type. And and then, like, the the speed that matches that. And so you can imagine that, like, an l m can recommend or infer that a fastball should not be, you know, less than 70 miles per hour or whatever it is. Even though I don't know what the real number is. I just made that up. But there is, like, some you

you can infer based based on that and make a recommendation. And so, actually, it's a I find that AI and LM is a really cool application of how to make observability faster and and and easier for for teams. So, yeah, I'm I'm very excited about about what you just shared, Andy. Well, I I love what you brought up about machine learning being able to to make basically make predictions about things.

And and one of the terms that, you know, as a practitioner of, business intelligence is especially the data engineering that supports it Mhmm. Is data volatility. Mhmm. So if I'm especially if I'm looking at an outlier. So I'm consuming this data day in and day out, And let's say, you know, 10% of the data is new stuff, and maybe another 10 or 15% are things that are have been updated, old stuff that's been updated, and the rest of it's relatively

stable. If I see those numbers go crazy out of bounds, you know, and machine learning would be able to pick that up right away and say, there may be a problem with the data we're reading today. You know, I would I that that sounds like one of the problems that would solve is that volatility, expected ranges of volatility of data. That's exactly

right. Yeah. Cool. Interesting. I think there's also something you said was, you know, when you have LLMs, because, obviously, we have to talk about GenAI because it's 2025, and I think you're in Silicon Valley. I think if you don't mention GenAI every twenty five minutes, the cops come and knock on your door and check it out. Welfare check. Could get in trouble. Or they make sure you're okay. Make

sure you're okay. But I think one of the things that really kind of makes me worry about GenAI is that it's not immediately obvious. Like, if you're at the airport, obviously, it's not a good look for you. Like, if the if the and this has happened to me where the app says one thing, the screen says something else, and my ticket says yet a third thing. So I'm not really sure where I'm supposed to go. Generally speaking of those, the app tends to be more accurate. But, that depends on the airline.

But with with LLMs, it's a the latency between you seeing the data where the cons the bad consequences of the data tends to be a lot more I'll use a $10 word today. I can't even say it, but it's not it's not immediately obvious. Right? There goes my my fail and my $10 word. But, like, it's not like it there's a lot more steps in labyrinthine. I'll go with that one because I can say that.

But, like, what so how do you provide observability in something like LLMs where the, the input and the output time tends to not be quite as straightforward as a data as an old school data pipeline? Yeah. Such a great question. And maybe I'll just share some of my favorite wonders if that's helpful. And and I think I'll share them because it's helpful to explain the gravity

of these issues. So, for example, you know, if you're in an airport and, you know, the app doesn't say the same as what you have, hopefully, you arrive early at airports, Frank. I don't know if you have enough time to, like, figure out the discrepancy and you won't miss your flight. Right? But oftentimes, those things can lead to to really big disasters. Even three gen AI. So so I think this was in 2020.

Unity, which is a gaming company, they had one schema change, resulting in a hundred million dollar loss. Their stock dropped 37%. Oh my gosh. Pretty meaningful. Right? Fast forward, I think this was 2023 or 2024, but not so much related to AI yet. Citibank was hit with a $400,000,000 fine for I remember that. For data quality practices for lack

of data quality practices. So think about all the regulatory industries like health care, financial services, like, you know, wherever there's, like, PII and and, And and the, like, you know, the the implication there are pretty grave. Some fun examples for more recently. I don't know if fun. I shouldn't call them fun. Some other examples from yeah. You mentioned Chevy. So I think there was a user that convinced a chatbot to sell the Chevy Tahoe for $1. I I commend the user from being able to

do that, but that is terrible. Right? That's terrible that, that happened. And that chatbot went down the next day. They they took it offline the next day. I think it was in Fremont, California, so not that far from the bay. Yeah. So right. So that's pretty pretty consequential. I'll just give another, like, example. This is my favorite example. This is what it went viral on x couple months ago. Someone googled, what should I

do when cheese is slipping off my pizza? And Google responded, oh, you should just use organic superglue. Great answer. They they had some really good gaps. There was the, eat eat one rock a day to get your, minerals and stuff like that. Yeah. So I I love that because that's an example of where, like, the prompt was fine, the context was probably fine, the model was fine, but the model output was totally not fine. Right? Right. And so and by the way, maybe Google can get away with it

because it's Google, but, like, 99.9% of brands can't get away with with the mistakes. Right? And so what, you know, what do you do? How do you provide observability in in that world? What does that look like? First, I'll just say, I think there's still human in the loop, and there will be. So, actually, you know, it's interesting going back to 2019 when we started the company. People would tell us, oh, you know, I have this important report that my CEO looks at.

But before they look at it, I have, like, six different people looking at the report with, like, you know, sets of eyes to make sure that the data is accurate. So, like, people use manual stuff back then. Today, what I hear is I was just speaking with this head of AI, Silicon Valley, and I was like, how do you make sure the answers are accurate? And they were like, well, we have someone sifting through dozens, hundreds of

responses every single day to make sure they're accurate. So I don't think human in the loop evaluation is going anywhere. There's more advanced techniques, you know,

comparing to to to ground truth data, using LLM as a judge. There's various sort of, things that we can do, but but I think human isn't going away. In terms of observability, I talked before I'll explain a little bit about this sort of framework of, you know, data issues can be really traced back to these four core root causes, and I think it's important to have observability for each in in sort of this world. So the first I mentioned is data. And so by that, I mean,

you know, let's use another example. Credit Karma, for example, has a financial advisor chatbot where, basically, they take in information about you that they have, you know, like, what kind of car you have as being of cars and, you know, where you live and whatnot, and then they make financial recommendations based on that. If the data that they are ingesting from third party data is late or isn't arriving or is incomplete, that messes up everything downstream. So one

root cause can be the data that you're ingesting is just wrong. Maybe it's all null values, for example. The second can be due to change in the code. So the code could be like a a bad like a schema change, like in the Unity example. It could be a change in the code that's actually, being used for the agent. Really, code change can happen every anywhere. And, by the way, not necessarily by the data team. It can happen by an engineering team or

someone else. It has nothing to do with the with the data state. Right? So code changes can contribute. The third is system. A % of systems fail. What what do I mean by system? I mean system is, like, basically the infrastructure that sort of runs all these jobs. So this could be, like, an airflow job that fails or a DDT job that that fails. You know, again, a % of systems fail, and so you would definitely have something that goes wrong in systems.

And then the fourth is you could just have the model output be wrong, kinda like with the cheese in in Google, example. And so when we think about sort of having what does it mean, what does observability mean in this in this age, I think it has to have coverage for all four of those things. And here's the problem. It oftentimes includes all four together. So I don't know if it you know, it's typically on a Friday at 5PM. You're just about done, and then

everything breaks at the same time. That's an interesting point. Like and and it's you also use the a term a couple of times, which, you're I can count on one hand how many non Microsoft people have used this term, data estate. And I'm just curious about I know where I pick from Microsoft. No. No. No. Like, I'm like I mean, I always thought it was a, you know, Microsoft invention. I don't think it is.

But, like, where did you pick up that term? Because I've only like, seriously, you were, like, the third or maybe fourth person who is not never worked for Microsoft, never worked with Microsoft. I I mean, I don't know if you work with Microsoft, but, like, I I always whenever I hear someone say data to state publicly, I'm like, so who'd you work for at Microsoft? What division? Like, like Oh, wow. Yeah. It's like that. And at first, I

didn't like I'll be honest. I didn't like the term at all, but eventually, I kinda grew to like the term because it there's a lot behind it, and I'd be curious to get, like, one, where'd you where'd you where'd you pick that up? Like, I'm just, like and then two, what does it mean to you? Like, what does that term data state mean to you? Great question. For what it's worth, I actually didn't like it either. For the record, I didn't even

like data observability to begin with Mhmm. To be totally Really? English is yeah. English is my second language, and observability was such a difficult word to pronounce. When we started the when we started the, you know, the company and and the category, we had to give it a name. So we didn't really know is this you know, we used we we coined the term data downtime, you know, as a corollary to application downtime. We thought maybe data reliability. There are lots of

options. At the end of the day, I always try to get gravitate towards where my customers are, so whatever language my customers use. And so customers started using the word observability, so I started using that too. And same with the state, they started using the data state sort of as a language. And so Interesting. Full disclosure, have not, have no ties to Microsoft, but but just have heard

mostly enterprises sort of think about that. I I think my understanding, you know, for for what they mean is, you know, wherever you store aggregate process data. And so that, you know, can include, you know, you know, upstream sources or upstream, data sources. But, you know, it could be, like, an Oracle or SAP database. It could be data lake house, data warehouse like Snowflake, Databricks, AWS, Redshift, s three, all the

way to wherever you're consuming that. That could be a BI report. You know, Power BI. Sorry, Microsoft. Right, Looker, Tableau, you know, various, various options. And, honestly, the, you know, the most common enterprise has all of the above in some shape or forward fashion. And so to sort of include all of that, I think the some of the thesis that we have around observability is that, by the way, each of those by themselves has some concept of observability.

Right? Like, you can, for example, with Snowflake, you can set up some basic, sort of checks, if you will, like a sum check or whatever. Right? You you could do that in Snowflake. However, we think that observability needs to be sort of third party and to be end to end. And, again, that draws on on software corollary. So, you know, like, AWS has CloudWatch, for example,

but that's probably not sufficient for whatever you're building. You're probably gonna use, again, like, New Relic or Datadog to connect across the the board to, you know, variety of of, integrations. Right? They have hundreds. So that's what I think about when I say data estate. But it's a great question. It's definitely not my word. No. I was just curious. Like like, you know, because whenever because first, I hated the term too. Right? And I can't maybe it's

Stockholm Syndrome. I don't know. But, the more I kind of sat on it and kind of digested it, I was like, I like it because it explains, like, you know, you know, historically. Right? Like, a state is, you know, whoever owned the land got to call the shots and whoever called the shots owned the land. Like, there was a very, you know, you drew the food, you you cut down the trees, you, you know, you mined for, I think the Minecraft

movie is coming out. So you mined for all these things. Right? My kids are into it. But, like, and it's really kinda like it's just the idea of seeing it, like, it's land. It's kinda like land. It's kinda like a natural resource. It's not really natural, but it is a resource. Right? And if I say unnatural resource, that's really weird. But it's a resource. Right? And if you you can either you have it. You already have it. You either develop it or you don't. And, you know, do

you, you know, do you grow food on it? Do you, you know, like so see, I I liked it because it was the idea that it's already there. Right? Mhmm. And it's it might be in forms you don't really think about. Right? Like, you know, PDFs in a in a SMB share somewhere. Right? Mhmm. I mean, that's part of your data to state. Yep. Right?

And it's that's how I kinda, like, came to terms with it. And, like, I really kinda like it because it helps you to think holistically about data because I think a lot of business decision makers and even technical decision makers don't see data as a as a as a as a resource. I think that's changed over the last maybe five, six years. But it really became something that they don't see it as a resource they could mine, they can get value out of. Right? The

smart people did. But, for the most part That's right. Yeah. You had to convince them. Right? Exactly. It sounds like based on what you say because, like, you know, my wife works in IT security. Right? So, so we're a two engineer household. So the kids are super nerds. But, like, I was telling her after chat CPT came out, I was all excited about it. And I was

telling her about how this works. I was like, you give it this big corpus of data, and they chews through it, and it comes up with these these vectors and stuff like that. And then she looked at me and it's like, so all the training data is now a massive attack surface. And Yep. When that's just why I love my wife. So I

I'm wronged. She's never wronged. Well, that's true. But at first I was like I was thinking but but you're missing and then I was gonna say you're missing the point which one is never a good thing to say but Like midway through I was like, oh my gosh, she's right. Oh my gosh. She's right. So then when I started talking to other data science and AI types, and I was like,

but but don't you think this could be, like, a big attack surface? I look like that meme with the guy from It's Sunny in Philadelphia with, like, it's always sunny where he had, like, the conspiracy thing. Like, I swear I will like that meme. Yeah. And, you know, and if you look at the I think OWASP has, like, the top 10 vulnerabilities of LLMs

that is either two or three. Right? So it's kinda like there's a fine line between, like, thinking too much about problem, but also kind of thinking ahead of the problem. I don't know. No. Oh, I think you

cut off a little bit, Frank, but, Andy, to me, that resonates a lot, and I think it's sort of really the overlap between data and engineers. And, by the way, like, we didn't even talk about security. Like, all these concepts also exist in security. Right? And I think in the same way that we sort of manage, like, you know, sub zero, sub one issues in security engineering, data issues should be treated the same way. You should have a framework to understand what's

a sub zero, what's a sub one for data issues. You should it should be connected to pager duty. Like, people should wake up in the middle of the night when you have data issues. I think I think that's right. It's improving, but, we're not quite there. It'll happen. No. You're right, though. Like, they don't think about this in terms of they don't does it I wouldn't say it's not disciplined. Sorry, Annie. I cut you off. No. But my experience we talked to data engineers. Sorry,

Andy. And I I I I am a former data engineer myself. Like, I thought of it in terms of schema structures and pipelines. Mhmm. Not necessarily securing those pipelines. Right? Mhmm. Sorry, Andy. I'll go. No. I was curious. I wanted to to shift back to you. You mentioned the four areas that your software, looks over your AI and the observability software does. What happens when it detects something amiss? Great question. So not even talking about Monte Carlo specifically, but rather

an observability solution. I think an observability solution needs to have coverage or an observability approach, by the way. Like, some people build this in house. An observability approach should take into consideration your data estate, should take into consideration, right, your entire data estate. I think, oftentimes, the mistake is people will even if they build it in house or do anything else, they'll really just focus on, like, the

data and their data lake or the data in a particular report. Like, that's not sufficient. Right? It it just isn't. And so people waste

a ton of time trying to understand, like, what's wrong and where. So I think the first is, like, you need you need visibility across the data state, which hopefully we've defined an unnatural resource that should be managed securely. And and I think that's right because I I by the way, Monte Carlo doesn't doesn't do the security part, but I similarly believe that in the same kind of diligence that we apply to data as engineering, you want data products to be reliable but also secure, scalable,

like all those concepts should adapt. By chance, we happen to focus on the reliability and observability part, but all the other, principles of software engineering should apply. We specifically don't do it, but very much believe that should be the case. But back to your question, you know, so so what happens when there is an issue? Very similar to workflow that you might find in Datadog, New Relic, and and PagerDuty. So there is an alert that goes out,

often you know, in whatever flavor of choice. If you're an enterprise that has a data state, this is likely Microsoft Teams. If not, this would mean Slack or an email or what you know, some teams like to have it connected to to Jira and and pager duty for for sev zeros or sev ones. And, you know, the first thing that people will do is start, you know, typically an analyst. I was I was in, you know, prior an analyst. The first thing you start asking yourself is, why the hell is the data is wrong?

Right. Yeah. You're like, well, was the report on time? Was the data accurate? Was it complete? You start going through all and then you start you basically come up with hypothesis. And then you start researching those hypothesis, and you're like, well, let me let me trace the data all the way all the steps of the transformation and start looking. Was the data okay here? Yes. Check. Okay. Move on. Was it data right? You literally you started this, like, recursive process. Gotcha.

Before we started the company, I used to do this all manually. So I remember, like, I would go into a, you know, into a room. Maybe you did this too. And, like, on a whiteboard, I would start, like, basically mapping out the lineage. Okay. This broke here. Was the data here okay? Let's let

let's sample the data and make sure it's okay. Okay. Move on. Let's like, literally, we have this, like, very every morning, actually, you know, that this became such such a problem because we were so reliant on this particular day dataset that every morning, me and my team would wake up, and we would basically go step by step and diligently, like, make sure that the data is accurate, which I felt like was I was like, this is, like, total, you know, crazy.

So, you know, I think, particularly in Monte Carlo or, like, what observability does is provides the information that you need in order to troubleshoot and understand where the issue is. And so we can surface you information like, hey. There was at the same time that this dataset you know, maybe the the percentage of null values in particular field was inaccurate. And then at the same time, there was a full request that happened. Maybe those are correlated, actually. Gotcha.

Maybe, you know and maybe, actually, you can use you can also do a code analysis. So you can, like, basically, you know, analyst what we used to do is, like, sift through lines of code and try to see what the change. Hey. Why did few surface to you that, like, there was a particular change in the, you know, name of a field, at the same time as an example. So bringing all that data into one

place can help you sort of troubleshoot that. And sorry for another LLM plug, but you can actually have an LLM do this for you, which is pretty sick where it's like an early beta test for us. We haven't released it yet. But, basically, what we're testing internally is for every like, for data incidents, there's basically, like, an in like, a troubleshooting agent that spawns agents for each of the hypothesis. So there's, like, an agent that

statement. Yeah. I it's really cool. There's an agent that looks into, like, the code change, the data change, the system change, and then and then it does it recursively on all those tables. So you can actually run up to a hundred agents in under one minute. And then there's a larger LLM that takes all that information and summarizes it and synthesizes it. So, again, early days, this is like we're still

building it. Very cool. But the early results are really cool. Yeah. It's like basically turbocharging your your data analysts and your data stewards. Sorry. I got all excited. No. It's it is That's really cool. Fascinating, and I love that you're excited about it. And what one of the jokes that I make when I'm I'm working with my kids on something, if they nail something, I'll I'll say to them, you know,

something similar to this. It's like, if you can only, you know, if you can only run a hundred in one minute, I guess that's if that's the best you can do, we'll just have to live with it. Yeah. Exactly. That's that's an amazing stat. Yeah. Yeah. That is interesting. And I also think too I also think too that, like, observability could help with secure the security story. Right? Because if, you know, you're looking at a

pipeline and it's like, hey. Weren't there a bunch of sketchy looking IPs, like, poking around our system about the time that this pipeline ran? Maybe the rest of the data that goes out of that pipeline run is a little bit suspicious too. Yeah. A %. Like, we we you know, for example, you work with a, call it delivery service, and there was a very suspicious tip very suspicious

amount of tip that was given. Like, you know, you can imagine, you know, the range of tips can be between x dollars and y dollars, and suddenly that's, like, you know, 10,000 times y, like, 10,000 times the upper limit. Yeah. You know, triggers off a suspicious alert. It's not a normal tip, and it's not a mistake. It's actually, you know, security issue. So that's an example. Yeah. Interesting. Yeah. I

love the anomaly detection aspect of that. I mean, it just it it's it's something that we've been doing for a long time, but then at wrapping it with automation and then combining that automation with what you just described with all the agents running down all of the permutations, that that just sounds amazing. Yeah. It's really cool. I can't take credit. This isn't me. It's it's it's my team. But, but I I was like, woah. It's like a hundred bars

running at the same time under one minute. That's amazing. There you go. It's really cool. Probably smarter than me. But yeah. That is so awesome. That is cool. So we we generally have is, we have kind of our our stock questions that we ask, if you're interested in doing them. They're not we're not Mike Wallace. We're not trying to I don't even think anyone gets that reference anymore, but we're not trying to catch you in a,

I gotta come up with a new one, in a thing. But it's mostly, like, how'd you find your way in the first one is I'll get the rest of them, up for you in a second. But the first one is, how'd you find your way into data? Did did the data did you find the data life or did data life find you? Oh, that's such a great question. You know, it's funny. I grew up you know, my my, my mom is a meditation and dance teacher and my dad is a physics professor. And so,

yeah, and so I, I, you know, grew up with very sort of like, yin yin yang in my family, if you will. At a very early age, I used to, like, hang out in in my dad's lab and, like, do scientific research and stuff like that. So or, you know, like, very at a very young age, my memories are, like, sitting in a cinema, watching a movie with my dad and trying to, like, guesstimate how many people are sitting in the in the audience. Right? Yes. Just like, you know, I think for, like, a five year

old, it's sort of like a fun fun thing. But, you know, throughout my my adulthood, like, always sort of had that in in the background. And, you know, I I think later on in life, I sort of always gravitated towards data. And when I decided to start a company, I was actually debating between various areas like IT and actually blockchain, or, you know, crypto for a little bit and and data. I think at the end of the

day, like, my heart was really in in data. If I look at, like, the next ten, twenty years, it's pretty clear to me that data is gonna be I think it still is the coolest party, and I think it will be the coolest party to be in. And I personally, like, you know, it's it's it's funny. Like, throughout my my career, I've I've also learned the limitations of data. Right? So so data can

tell you whatever story you want. It could tell you, you know, for every question, it give can give you a yes, and you can also tell a no story. Right? So so there's also limitations to data, but but I always have been fascinated, by by data and space. So can I say both? That's Yeah. I mean, that's fair. That's fair. Good answer. That's fair. Yep. So what what's your favorite part of your current job?

Oh, that's hard to choose. I love my job. I just love it. I think, you know, the ability to work with customers and actually, like, change the way they work, I I think that's probably the biggest gratification that I get, you know, from from my my career. Like, the fact that you can actually work on something that matters is pretty insane. You know? And when I think about, like, the future, I'm like, what? So data is gonna be wrong? Like, we're

just gonna be, you know, making decisions off of wrong like, what? I don't wanna live in that world. You know? And so Yeah. I think there's something that's, like, really fulfilling and helping, you know, drive a mission that I believe in that has an impact on customers. And, you know, when customers will tell me, you know, I started sleeping at night because I know that, like, I have some coverage for my data. I'm like, yeah. Oh, wow.

I'm glad you're sleeping. You know? Like, good for you. I love sleeping. So What a cool thing to hear. Yeah. Exactly. I think that's that's probably, you know, maybe one part. And then the second is, like, just working with an amazing team. You know, I I spend most of my my day maybe kinda like, you know, you guys, like, hang out having fun, laughing. So, you know, I I I'm very grateful that I get to work with the smartest people on on worthwhile challenges. Oh, very cool.

We have, three complete these sentences. When I'm not working, I enjoy blank. Sleeping. I yeah. I I have a I we recently have added we we had two kids, and we adopted a cousin. And I forgot how draining a toddler can be. And I'm I'm eight to 10 years older since the last time I had a toddler, so it's like I, I have two kids, on two under four. So I, respect the sleep even more. I I can't even I can't even wrap my head around that. It gets it gets better. I can say

that. It's my own role. I appreciate that. So our second one is I think the coolest thing in technology today is blank. The coolest thing in techno I think the pace of innovation. I think that's really freaking cool. You know, you can, like, work at a problem today and you're like, you can't solve this. Two days two days later, a new model will come out. Boom. You're done. So it's harder. Right? The bar is higher in order to, like, actually like, it's it's harder to it's

harder to know what to bet on. It's harder to know what the future will look like, but it's a lot more exciting. So I'm in it. Cool. Our third and final complete sentence is, I look forward to the day when I can use technology to blank. I was always a big fan of teleportation. I think teleportation is really freaking cool. That would be nice. Can't wait for that. That would be cool. That would be cool. You know, you're not the first person to answer with them.

Oh, really? Yeah. It's pretty cool. Pretty cool. Sorry. Number six is share something different about yourself. Something different. Yeah. Something different. Let's see. I mentioned I have two kids. I meditate when I don't sleep. I like to meditate. I, what else? I'm married to my cofounder. Oh, wow. So we, yeah, we're fortunate to share our lives both at work and at home. That is cool. Yeah. I can imagine that would work out really well or not. Like, there's not a lot of

middle ground there. High risk, high reward. High risk, high reward. I get, like you know, my wife is, you know, she's a federal employee, and she's, you know, reevaluating what her career futures look like, you know, and she's like, you know, I was like, well, you know, you could help. You can start a new podcast. I can help you with that. She's like, yeah. But then I have to work with you. And, like, I know what she meant. I know how

it sounds. I know how it sounds, but I know what she means. Like, so when she did work from home, like, there was literally a, like, an entire floor between us because Yep. Like, it's too loud. I'm too loud. Yeah. Yeah. Yeah. Yep. We're very loud too. So where can folks find more, learn more about, Monte Carlo and, and and what you're up to? Probably, I'm the place where I hang out is LinkedIn. So,

I know we just got connected on LinkedIn. That's great. Probably follow me on LinkedIn or, honestly, reach out to me directly, me, Moses@MonteCarlodata.com. I hope I don't get a lot of phishing now because of that. But Well, hopefully, make sure it's the right account because we found out

in the process that there's there was another suspect in suspicious looking account. And I also think that for our listeners, it's worth pointing out that I think that people have realized that LinkedIn is a is a major security vector because I've been getting a lot of weird a lot more lately. Now I don't think it's related to the, the refrigerator scandal. Andy and I will do a whole show on that later because there's there's actually an interesting AI component to that. Okay.

Good to know. And finally, last but not least, Audible is a sponsor of the podcast. Do you do audiobooks? If so, recommend one. Otherwise, just recommend a good book you recommend. A good book. Let's see. Thinking in bets by Annie Duke. Professional poker player. Interesting. In in how lessons from poker can be applied in, in life and in business. Interesting. I once worked at a financial services company, and one of the big shots used to play online poker. And

they're on company, not on company money, but on company time. And a lot of people Not a lot of people took a dim view of that. Rightfully so. But he was making so much money. You know, people that matter didn't take a damn view to it. When he stopped making so much money, people everyone took a damn view to it. And it they don't that does end the the story. It is on I don't see if it's an audio oh, it is an audio book. It is an audio book. Awesome. I'm gonna add that to my

list. I'm done. Okay. And if you you know, they are a sponsor. So if you go to, the datadrivenbook.com, you know, you'll get a free audio book on us. And, you know, if you sign up, we'll get enough to, you know, buy a coffee. Maybe not tip them $8,000, but, you know, we'll get enough for a Starbucks maybe. Maybe. Yeah. I just tested the link, Frank. Every now and then, we had trouble early on

with the link coming and going. So I just when you saw me turn away a minute ago when Frank started to this question, that was me typing in. It worked. It worked. It's always DNS. That's the Always. It's interesting you mentioned that. I read an article. Actually, it was a newsletter recently that talked about, betting being the first stage in, kind of the path to minimally viable products. And I thought, now that's curious, and I don't know again, I haven't

read the book. I will listen to it. But the idea of engaging your team I I manage a team, as well. And engaging the team by having them do interesting things and making taking these very large bets that look nearly impossible, perhaps. And it's like you said, the the the problem comes up, and you're thinking this is this is unsolvable. And two days later, it's solved. And over and over again, I've had that experience, but I never tied it to the concept of

bets. And I saw this this newsletter that talked about do that first, And it reminded me a little bit of Collins talking about, the the big hairy goals, you know, back in the day. It's very similar to that maybe in concept. I don't know. I'll have to listen to the book and check it out, but I was intrigued by the newsletter. Yeah.

There's interesting concepts. Like, I think some of the ideas is, like I mean, even when you start a company or sort of, you know, start working on a team, like, you basically have you have a set of cards, which are, like, your strengths, your weaknesses. And so how how do you play your cards? Like, you can't you know, if you wanna win around, you can't play with someone else's cards. You are what you are. And so the best thing you can do is play

with your card. I think that's true for a team solving a problem or startup or whatever it is. I love that. Yeah. Interesting. Any final thoughts? This was so fun. Thanks for having me. Thank you. Thanks for, and you did mention kinda offhand early on. I don't remember if it was in the green room or not. You have a podcast yourself? I do not have a podcast myself. Alright. That was my mistake. Maybe I'll end it tomorrow. Okay. All good. Life goal one day. There

you go. There you go. And with that, we'll let our AI finish the show. And that wraps up another data packed episode of

data driven. A massive thank you to our brilliant guest, Bar Moses, for taking us deep into the world of data observability, sketchy LinkedIn impersonators, and the dark arts of tipping anomalies. Who knew a dodgy schema change could cost more than a luxury sports car? Now, dear listener, if you've made it this far, you clearly have excellent taste. So why not put that good judgment to work and leave us a rating and review on

whatever platform you're tuning in on? Apple, Spotify, Pocket Casts, Morse code, however you get your fix, would love your feedback. And dare I ask, are you subscribed? I mean, you wouldn't want to miss out on future episodes filled with more wit, wisdom, and the occasional fridge based conspiracy, would you? Until next time, stay curious, stay observant, and for heaven's sake, keep your data tidy.