Navigating the Complexity of Operationalizing ML Models

Welcome to show 350 of data driven. In this episode, Frank and Andy interview Chris McDermott, VP of engineering at Wallaroo. Wallaroo helps customers operationalize machine learning to ROI in the cloud, in decentralized networks, and at the edge. It's a fun conversation on MLOps and the future of intelligence systems and model management. Now on to the show.

Hello, and welcome to data driven, the podcast where we explore the emergent fields of artificial intelligence, Data science and of course, data engineering, the fundamental thing that kind of underpins it all. And with me on this epic road trip down the information superhighway Is my favorite data engineer of all time, Andy Leonard. How's it going Andy? Good Frank, how are you? I'm doing alright. I'm doing

alright. We just, we're chatting in the, virtual green room about some of the logistical challenges we had, with Microsoft Bookings and how Kind of like you can only have, you know, like, remember that the pick any 2 triangle, right? Good, fast, and cheap? Yep. Yep. Like, we can only have 2 things, 2 features of what we

needed to do. Right. Alright. Despite logistical challenges, we are excited here to have, Chris McDermott who is the VP of engineering at Wallaroo, and, he is a passionate, and intellectually curious professional With excellent communication skills, he loves hard problems, then he must have definitely loved the process to get on the show, And, have yet to meet 1 he couldn't solve somehow. Maybe we should get you, Chris, to help us with our scheduling stuff. Really? You visit

later? Yeah. So welcome to the show, Chris. Thank you. Thank you. It's great to be on. It's nice to meet you both. Well, likewise. Likewise. So you're coming to us from the, Mile High City. That's right. Awesome place. It's, I was there once, for internal Microsoft conference actually. Oh, nice. And beautiful town, like, it was just really cool. I think it was the 2nd biggest event that in Denver history was the Microsoft thing. Wow. And they they literally ran out of hotel rooms like it was.

Oh, wow. It was pretty wild. Yeah. I think it was, just before one of the big parties had a convention there. And, they Oh, yeah. Yeah. Yeah. Yeah. I was so I'm actually slated to head back there next year for a Red Hat conference, so we'll see Let's see if the hotel situation has improved. I think it's improved a little bit. The city's been growing a lot. So, Yeah. Lots of government. Isn't Denver the place that has, like, the large bear up against the conference center that

Yeah. Yeah. Yeah. Yeah. Yeah. That's exactly right. A giant blue bear appearing in the window of the conference center. Yeah. I was there. And, and and I remembered that. That was the That was the first thing I remember. It was, I was there in 2007 for a Kind of a Microsoft conference. It was a, Professional Association for SQL Server. That's what it was called back then. And, I was actually the first one I spoke at. I've spoken at a bunch since then, but 2007 in Denver was the first. And,

yeah. Like, I echo what Frank said, beautiful city and, just very picture picturesque. Yeah. Yeah. The weather in the mountains are beautiful. Mhmm. Yeah. And it's funny, like, you know, on the East Coast, we talk about mountains, but It's nothing like that. Like Right. Yeah. We quite the

same. You would laugh at what we call mountains. Yeah. Right. But I remember a Robin Williams bit Where he said something like that people he admired the people in Denver because they got to Denver and they looked at the mountains and went, Well, I can't say what he said, but he had a kind of an Elon moment. There's so many of those. There's so many No more. We're stopping right here. We're not going over those mountains. So,

You're VP of engineering at Wallaroo. So tell us a little bit about Wallaroo. Mhmm. Plus you're also ex data robot too. That that's interesting. Yes. Yep. Exadata robot. Yeah. So I've been working in the machine learning and AI space for, about 7 years now, I guess, or 6 years. And, it's been really fun. You know, it's, it's a good time to be in the business. There's a lot of development happening, very fast pace of change, which I appreciate. And, you know, Wallaroo has been really great. Like,

the team is fantastic, and the people are wonderful. And it it's a lot of fun working, with people that you enjoy hanging out with and and you respect and everything, that's that's very important to me. That's awesome. But I also just I think the product is awesome. It's really, I think, playing well in the market. Like, we are focusing on making it as easy as possible to deploy And manage machine learning models.

And the focus is really on any model using any framework and being able to deploy onto sort of any architecture, any hardware, and being able to leverage GPUs if you need them or different kinds of CPUs, different acceleration libraries that people have tailored to the different architectures. And, honestly, there are not a lot of other solutions that tackle those 2 problems for people. Right.

A lot of the other companies that we're competing with, they are trying to be like an end to end solution or, like, really force people into, you know, their platform. So you train on their platform, you deploy on their platform, you manage on their platform. But it's very limiting in terms of what you can bring on to the platform and and being able to, deploy on the different types of architectures and, platforms and things like that. So it's really

exciting. It's fun. I think that's really important that you bring up the CPU solutions. As I've been tinkering, you know, the past couple of years with, you know, with the different, different platforms that are out there, it's That's definitely a smaller market, but maybe it's emerging now. I'm just not sure. Mhmm. Yeah. I wonder yeah. Sorry. Go ahead.

Well, I was gonna say, you know, a lot of the time people conflate training and inferring, which is, you know, sort of the 2 different stages. Like, 1st, you have to train a model, but then you use the model to make inferences, which, you know, it's really like asking the model to make a prediction or you give it some input and it gives you some output. And,

they're they're very, very different tasks. And just because, you know, like, you may wanna use some hardware GPUs for training Doesn't necessarily mean mean that you need the GPUs when you are in production and you're asking it for predictions. A lot of the time, you know, The model is small enough that you really don't need to, but there's so much hype. It it's hard sometimes to separate the hype from the, you know, The real stuff and Yeah. Yeah. The hype the hype

machine is real. I mean, like, it's and and and I I wanna get your thoughts on, you know, I mean, I love generative AI. I'm not knocking generative AI, but it feels like it's taken all the oxygen out of the room for All the other kinds of AI. Yeah. Yeah. Yeah. Because there are a lot of, you know, great models. I like XGBoost is a very standard one. It's been around for, you know, a long time, meaning at least for, you know, 5

or 10 years now or something. But, that really honestly solves so many problems, and it's such a Small, easy model to deploy. I I wish people would focus more on on that kind of thing rather than hype. Right. No. That's a good point. And I think you bring up an interesting point because not all not all AI workloads are created equal. Right? Obviously, there's, I heard this term the other day and I had to spit my coffee out because it was just so funny. Legacy AI. Yeah.

Yeah. There's generative AI now. There's legacy AI. That's crazy talk. You know? And I was just like, wow. But, you know, because, you know, legacy AI, basically,

you're not using deep learning, you're not using neural networks, Generally, you don't get a good boost from GPU's. Correct. Right. And that's something that when when you tell that to Even tactical decision makers, they they they kinda look at you like, you know, what sorcery is that? Like, you know, because they'll they'll They'll say, like, oh, we don't have enough GPUs. There's no budget for GPUs. Like, what what what types of workloads are you running? And I

tell them, it's like, well, it's not really a concern for you. Like, you don't need them. Yeah. And you see, you know, the the the the people who are doing the actual data science, they're like, yeah, duh, that's what we're trying to tell you. Yeah. But you see, like, the leaders of these teams are like, like, you know, it's, now Just for my own education, there wasn't there something called RAPIDS, and it was an acronym that let you use GPU's For

certain types of like XGBoost, I think was one of them. Random forest there. I don't know. Oh. You See, it's funny because it was an it's an NVIDIA thing and obviously it only optimizes on. But, like, it was I remember Hearing about it in 2019, and I'm thinking, wow, this is gonna change everything, and you haven't heard of it. Only, like, one ever per other person I met in the wild has ever heard of it, and he was at the same conference I was at where we heard about

it. So I'm like, That's kind of unusual, but, you know, we gotta watch so fast, you know, and it's really hard to tell sometimes What what, which new developments are gonna end up being the future and which ones are gonna end up as dead ends? Right. You know, and even all the transformer stuff that that is powering GPT and and those similar types of models, I think that was originally written up in a

white paper in, like, 2017 something. Mhmm. And it just kinda sat around for a while, and nobody paid a whole lot of attention to it until OpenAI really ran with it. So yeah. Pension is all you need. I think that's was that the paper? Sounds right. Yeah. And then we're gonna go. Oh, sorry. Go ahead. Sorry, Andy. I cut you off your point. No. I I don't wanna go too far downstream before I say cred boost for using the phrase I don't know. Oh, nice. Somebody with your

credentials, you know, saying I don't know. That's that's super cool. So cred Honestly, there's way too much to know. There's no way anyone person could know that. I I like to joke. I haven't checked my phone or, like, news Feed in like half an hour and I'm like woefully behind now. Yeah. But it feels that way like in the whole Oh, no. It does. Yeah. Especially it was especially interesting when the whole drama on OpenAI, and I

don't wanna go down that rabbit hole too far. But when all of that soap opera kinda unfolded Yeah. Yep. It was kind of like, what's the latest? Like, is he back? Is he gone? Is he working at Microsoft? Like, he did work at Microsoft for like 10 minutes, and now he doesn't. Like, Yeah. You know, at at some some point down the middle of it, it's like, call me when this is over, and I'll deal with the, things yeah. I'll

check-in again. But that's just the human side of it, let alone the let alone technology side of it. So Operationalization. I think that's gonna be the buzzword. Obviously, chatty b t and JennyIs, taking all the air out of there. And I think the next buzzword It's gonna be operationalization. 1, because it's kinda hard to say, and I'm not gonna lie, I've had to practice. But, it's something that I think companies and organizations that adopt AI, whether it's legacy AI

Or generative AI. They're gonna have to realize, like, it's one thing to build the model, and then it becomes a, okay, now what? Yeah. Yeah. Well and models

really are just like any other software. It's not something that you just write once and you, you know, Throw it out there, and it runs forever without being touched. Right? All of it requires care and feeding, and and machine learning models are no different. So, I think part of it is, you know, how do you deploy it? And then, you know, how do you keep that that deployment up to date, you know, getting critical

patches and vulnerability fixes and things like that. But also how do you monitor the model and how it's performing and how it's performing relative to the real world, Because the world doesn't stand still right. So even if the model was trained on some data and it was 98% accurate when it was trained, as the world shifts and and the situation around it shifts, that accuracy will almost certainly start to degrade over time. So You need to monitor that. You need

to know when to retrain the model. And you have to be kind of keeping track of, new training data. Right? So the the the new environment that the model is operating in, you need to be recording all of the the inputs and also paying attention to the ground truth of, You know, what was the outcome of that prediction that the model made? Was it accurate or not, after the fact? And and correlating that back into your training data So you can retrain the models and, you know, keep them going

over time. And that's just, you know, assuming you're gonna be using the same model forever. But as we just finished talking about new models coming out all the time, new approaches, new techniques. So, yeah, it really is is something you've gotta pay attention to. It's an extremely Yeah. It's an extremely dynamic space. Mhmm. I've heard this called MLOps for the longest time. Mhmm. Mhmm. But I've also heard a new term kinda pop up on the radar called AI ops Mhmm.

For this. What do you call it? I generally call it MLOps. You know, one, I I sort of per like, AI and ML, there's an interesting, you know, difference there in in terms of who uses the different terms and when they use them. For me, AI is more of a general term that I use conversationally. And most of the time if I'm trying to be fairly technical and specific, I'll usually revert to ML,

Because in fact, most of these things are machine learning. AI is a much more nebulous concept, and I I don't even think everybody agrees on on what AI is or What the threshold would be, you know, if you're doing statistical analysis, I think most people probably would not call that AI. But there are a lot of machine learning models that do work that way. And and that's definitely, like, part of the gradient. You know? I've

noticed that too. Like, there it is a gradient too. Like, there's not a, like, a hard, like, You know, typically it depends on the audience. Right? If they're if they're BDMs, business decision makers, they're gonna use AI. Yeah. They're technically focused people. They tend to prefer the term ML. Yeah. That's also been my experience Interesting. Quite often. So I like MLOps because, one, it sort of grounds you a little bit more in that technical perspective. Mhmm.

And, and it's sort of a like, To me, I think I came up through DevOps a lot of my, you know, first half of my career was DevOps and infrastructure and things like that. And, I think part of the appeal of the term MLOps is it taps into a lot of the DevOps, associations. Right? And Right. The concepts and the themes of DevOps, which is really about, merging different skill sets and breaking down silos and getting different teams to communicate with each other and And to collaborate more,

being more dynamic. Not just, you know, putting software out there and and letting it run forever, but Keeping it up to date, monitoring it, recording the logs, you know, all of that kind of stuff, and and getting into a flow of continuous deployment, you You know, continuous integration, continuous testing, continuous

deployment. And I think on the ML side, that's also where MLOps really shines and and is bringing those themes to the party, rather than a data scientist training a model, deploying it, and, you know, Throwing it over the wall to to, like, an operations team or something. It's getting all these different teams and skill sets to work together. It's building a continuous, you know, pipeline, with monitoring and and feedback loops and so on. So that's that's why

I like MLOps. No. I like that too. So in order to prevent any hate mail come in or or but actually comments, AI ops is also used, I've heard, in, the telcos and network operators tend to have a term called AI ops, where they use AI to help operate their network. So that is Got it. It's it's a it's a namespace collision,

which I've free further which I prefer MLOps for to avoid the namespace collision, plus all the reasons you said. You know, what's interesting is and I came from a software engineering background and, you know, and I'll be honest, I was not initially a big, believer in in DevOps, but kind of as time went on, I became a convert. But I think that, you know, when you look at how AI models, ML models, whatever, how they get operationalized.

You look at it And I I often I often can tell who the fans of the new Star Wars movies are by using this analogy, because I'll say it's The 2015 Star Wars movie and the 1977 movie, DevOps. DevOps being kinda like the original, episode 4 And then the new, the the first new one, right? It's the same plot. I mean, the characters have changed, some things are different, But very effectively, it's the same plot. And, you know, some people will laugh like you did, and some people I can

see will, Their their faces turn red. But, but I mean it's like it's it's the same plat plot. The names, the places have some have changed. But you're right. I mean, I think and there's a lot of lessons we can learn Yeah. In the ML community from the DevOps world. Right? Because, You know prior to DevOps, you know, the developers and operations had a very antagonistic relationship for the most part. I'm sure there's always

exceptions. You know, I was I was joking that they would only meet, they only have to interact 3 times a year, and one of those was the holiday Christmas party. You know what I mean? And Yeah. But if you wanna deploy something in a far more agile way where they have to, you know, you put it In some extreme cases, every few hours, some new bit of code gets gets pushed up. That's obviously on on one fore end of the spectrum. But for the most part, you know, a

couple times a month is not unreasonable. You have to automate that. You have to have processes in place. Yep. And I I see a day, and if that day has not already come, I would be surprised, That AI is gonna be the same thing or ML. Right? You're you're gonna have to get but to your point, right, this is a continuous process, You know? Yep. Yeah. We can't get away with, you know, you have this isolated team of data scientists. They they they kinda go off to their little area 51 type labs

in secret. Right. I then come back with some model, and and I'm guilty of this too. I've done this. Right? Where I'm like, I built the model. I'm done. I did the math. I did the hard part. How do you get the play it? Not my problem. Not my problem. And it's funny, like Yeah. You know, I caught myself. Right. I caught myself doing that as I, you know, you

know, doing that. Like recently, I had to I had to do a demo and I had to work on a kind of a It's basically a predictive maintenance type thing, and I took all this data, had the model, and I just said, here's the here's the link to the model, Have at it, pal. Mhmm. And then as I sent that, I was like, you know, I should probably be more involved in getting this on a race for it. Right. Yeah. Yeah. Yeah. Yeah. No. I think that's a big part of it.

Another big part of it though is, scale, you know. And I think scale scaling of compute and, how How people were using compute and how much compute was required was a big part of what drove DevOps. You know, if you were a sysadmin responsible for a 100 servers, That's, you know, challenging, but it's feasible. Like, you can do that. You can keep them all up to date. You can keep them all in sync with each other. Make sure they they all have the same patch levels and

and so on. But you scale that up to a 1,000 servers? That gets a lot trickier. You try to go to a 100,000 or, you know,

if you're doing Internet scale things like Google or Facebook or somebody, We're talking millions, tens of millions. And Right. That level of scale requires you know, everything has to be automated. Everything has to Has to work that way and it has to be resilient and it has to, you know, have automatic fail over and stuff. You know, there's the, x k CD where they're, You know, they get to a certain point. They're just

roping off entire data centers and being like, alright. We're throwing that one away and moving on to the next one. And for AI, I think a lot of the same stuff is happening. When, you know, 10 years ago or so when when people were just getting started on this journey And as an entity, as a business entity, if you're talking about 1 or 2 use cases, you know, you can have humans curate that stuff and hand

craft it, hand roll it, hand deploy it, and hand manage it. But if you're a a big enterprise company and you you wanna have hundreds of use cases in production or thousands or tens of thousands, there's just no way. You have to automate it. No. That that that's a that's an excellent point. Like, one way I've heard to describe is that if you're baking a loaf of bread for your family and friends Or loads of

bread. You can do it in your kitchen. Right? You don't have to do anything special, but if you're the Wonder Bread Corporation or Mhmm. And you wanna deliver at that scale, that's no longer an option. Mhmm. And I think that we're at that point where and correct me if I'm wrong, where I think AI and ML adoption or AI adoption is still new enough where there's enough of naivete out there of, oh, we don't need to scale to that degree. Like, we don't need

the production line. I think I think that's ending. I think we're getting close to the the end of that era, but that's kind of been my yeah. I think so too. Yeah. Because they're they're more and more, ML tools in everybody's toolbox. Right? So you were talking about telcos routing network traffic using ML models. That's not gonna be 1 model. Right? Like, with latency and and

everything else, you're gonna need, you know, Very small. Lots and lots of very small models deployed on every router, every top of rack switch, every, you know, whatever 5 gs cell phone tower, whatever you're talking about. There are a lot of cell phone towers. So you're not managing 1 model. You're managing a fleet of models, right, across different geos and all kinds of things. No. That's that's an excellent point. Sorry, Andy. That's okay. It does seem to scale like that,

though. Right? It's almost it's almost tectonic. There's a whole new layer going down. You know? That's that's the new surface. I noticed on the website, I I popped over to wallaroo dot, aiwallar00.ai. And I noticed a familiar looking, blurb just below the top of page. And it's familiar to me because, I started off in business intelligence. I'm still working in BI. And there's a note, 90% of AI initiatives Failed to produce ROI. And I saw this in, you know, it's very similar number, 85% in,

in BI back in the day. It's probably still true. So where does that number come from? Well, I think it reflects a lot of things. You know? Some of them we were just talking about and and where MLOps is coming from is is, a lot of the failure modes were teams not really working with each other. Right? Somebody decided we should be doing AI, so they hired the data scientist. And the data scientist works in the corner for a while and,

You know, 1, they don't have access to all the data. They don't know what the data is, where to find it, how to access it, how to clean it, what it means to the business. There there are a whole set of challenges there. And then, you know, they may train some models and and get something, you know, to a point where they think that it's gonna solve a problem. But Then you've got to work with an IT organization to stand up infrastructure. You've got to

work with somebody to package the model and build, you know, an API around it or a UI of some sort And figure out how to deploy it, train people on how to use it, and and actually, like, somehow integrate it into your business process So that it's it's driving business outcomes. And all of those are really tough

challenges. And all of them require breaking down those silos and getting a bunch of different People within an organization to talk to each other and communicate and to work together to solve something. I don't think ML or AI is is a magic wand that you just wave and magically provide value to a business. You've got to really

think about What is your business doing? And, you know, machine learning at at heart, it it's really just like a a more efficient way of Making decisions, you know, faster and more accurately, and with less human input. And so you've got to look for places where your business can either save a lot of money or make a lot of money by being able to answer a a simple question repeatedly very, very efficient. And that sounds easy, but in practice,

defining a business problem is often one of the hardest parts. So now I'm seeing even more parallels. Uh-huh. Yeah. You know, that was the problem we were trying to solve, with business intelligence as well. So didn't mean to cut you off. Sorry about that. No worries. So I yeah. I think I agree with you. It it there are tons of parallels there. I think there are a lot of similar lessons to be learned, and I think we are applying them in this In this space in

ways that we've applied them to other spaces in the past. I also think there are technical challenges. You know, part of it is the field is moving so fast. So there's just this constant stream of of new frameworks, new models, new techniques, and you have to kinda stay on top of that. You have to be careful with your tool selection, to make sure you're not, you know, going whole hog into some tool. That sounds great today, but it's just not flexible, and it's not gonna be able to support,

like, all these new things that are coming out. Yeah. Or that company could have internal internal political strife, which was crazy talk. Right? Cast Absolutely. Right. Cast doubt on their future. Alright. That would never happen. That would never happen. Sorry. Yeah. You were talking about privacy, which I think is another key thing. Yeah. Data residency, data privacy, see data security. You know, all of those things matter tremendously. And for for a business trying to, get

value out of AI and ML. You know, a lot of it, depends on having good data and, Cleaning it and curating it and getting it ready for things. But then it it forces the the organization to really kind of do an inventory. What do we have? What's useful? What's not useful? Well, how much do we store? How much do we not store? How do we comply with various regulatory environments? Right? GDPR is is the big one everybody, you know,

loves to throw out there. It's it's big and it's complicated, but, you know, things like that matter a lot. And And there's 300 +1000000 people behind that. They're covered or whatever. I think that, you know, that that is not only do they have a big stick, but they have a big arm that they can wave that stick wet. Yes. You know, if if a small country with, like, you know, 50 people in it, and

that could something like GDPR, people would just walk around it. But I think that, a block with I've heard different numbers, but it's for, you know, pushing 4 to 500,000,000 people. That's a huge that's a big enough market nobody can really ignore. Yeah. What's interesting is on the LinkedIn page for Wallaroo I love the website, by the way. I checked that out too. Thank you. It talks about decentralized networks Mhmm. And at the edge. Yes. What how would

you define decentralized network? Yeah. This is a big new push for us that we've

been focused on for, I mean, we've been focused on it kinda for the last year, but it was a lot of, development on on the back end. And we just released kind of our 1st edge features and product,

in October, so it's kind of a new thing for us. But, As you think about ML and edge or ML and AI, and the the fleets of models that we talked about and all these use cases And, you know, telcos and and five g cell phone towers and all of those types of things, intersecting with data and data residency and privacy and security, It it really seems to indicate to me and and to us at Wallaroo in general that the future is lots and lots of models being deployed in lots of

locations. And I think that one big sort of industry wide theme that I'm seeing is if the last 20 years, let's say, was the story of Everybody picking up from their colos and moving to the cloud and centralizing all of their IT, I think that the next 20 years are gonna be Not like deconstructing the cloud. I think the clouds are here to stay and they're gonna continue to grow, right, year over year. But there will be more of a push out to more edge computing environments. Cell phones

are getting more and more powerful. Cars are getting more and more powerful. Like, there's more computer stuff happening, all over the place, and the compute available, the memory and the storage available is all through the roof compared to what it was 20 years ago. And, I think we're gonna see more push for smaller, more specific machine learning models, And they're gonna be pushed out to all these edge locations so that they can run

close to where the data is. So you're not schlepping this sensitive data all over the Internet and other people's networks. Yeah. But, you know, you're taking advantage of of compute resources that you have local to the data and making very fast decisions, you know, very efficiently. So I I have to jump in because, you you just made me feel really good. About a year ago, I built a large server here at home, which I hadn't done in a decade. Actually, my my

20 year old son built it. But he and he helped me with, with picking out the new shiny fast parts, on it because I was so out of practice with this such confessing. But, and it's really cool to see, you know, all of his All of his skills. He does edge. We just picked up the Raspberry Pis are back in stock, finally. Yep. And I just picked up, like, 3 for $35, You know, the 1 gig force.

Yep. Anyway, super excited about that. One of the things I built at the time I built a box About a year ago, you couldn't do a local GPT or anything close to that. And I said, Eventually, we're gonna be able to do this. I I made that guess, and it was a guess. Yeah. But about 6 months later, about 6 months ago, All of a sudden, I started seeing these 7,000,000,000 token machines showing up and it started clicking. It was like, holy smokes, you can do this. I did make one stupid mistake

and he didn't catch me on it. I bought a 12 gig GPU because that's super crazy huge From 10 years ago. And that wasn't super crazy huge at all. No. No. No. But it's interesting. Now they're back now. They can run on, You know, on the 12 gigs. And like you said, you mentioned the CPU models. So I just learned a ton as I've been going through this. And, That it's it's very encouraging to hear that. I had not heard anybody say edge and running small ML models on the edge.

That's, I mean, that's what we've been trying to do here. And I I love the redundant you know, the idea of a redundant array of whatevers, you know, MLs. It's almost like a swarm of MLs. I've heard, yeah. Yeah. Yeah. That's true. Right? And, you know, I think there's a lot of interesting stuff happening on the battlefields in Ukraine right now drones. And Right. That Yeah. Was also a fascinating space and very much, I think, heading in the direction of lots of ML running at the

edge. It's it's funny you mentioned that. So I live in a DC area,

and, I was at a government tech symposium about 2, 3 weeks ago now. And they were talking about that that, you know, edge is gonna be much more important in the future of warfare. And he said presumably elsewhere too. Right? He was permanent primarily a government in defense. It was definitely a military industrial complex, type of type of event. But he was explaining like, you know, in the past, you know, 20 years,

we've not dealt with adversaries. We've only dealt with adversaries in in battle space conditions where it was, you know, we controlled the airwaves. Mhmm. And he, I think he used an interesting term. We had airspace and electromagnetic electromagnetic dominance. I was also like, Wow. Yeah. That was yeah. Yeah. I was, like, oh, that's interesting. So, like, the whole idea of these disconnected decentralized networks, I mean, I think

you're I think you're spot on. It's the future for geopolitical reasons, but also just for, you know, Privacy and just kind of flexibility reasons. Yeah. The question I have though is, like, Organizations can barely manage the infrastructure they have now and barely manage the software they have now. What are they gonna do when the software starts Not thinking for itself, but, like, this becomes another workload Yeah. On

top of that. Like, what Well, for one thing, that's why Wallaroo It is focused where we are, and we're trying to build this platform to help people, you know, with this capability of being able to deploy models and manage a fleet of them at the edge. Because, yeah, there aren't a lot of good solutions for that today. Yeah. Interesting. I I think the general answer to your question is probably some combination of cloud and edge.

You know, like, it does make sense to centralize a lot of things, and it makes the the maintenance easier and, more efficient. And You can get some economies of scale and, you know, all that kind of stuff. But, we are gonna have to get good at managing a bunch of, disparate types of things in desperate locations. I think all of us. Interesting. So this is the part of the show where we'll switch over to The premade questions, and for your convenience, I will, paste that in here.

Hopefully, paste it. And there we go. So You had an interesting career looking at LinkedIn. You were at SendGrid. You were then you were at DataRobot, and you said you made a switch into the the data world, which begs the question, How did you find your way into data? Did data find you or did you find your way to data? I I

guess that is a good question. I think that, it was probably a little bit of both. Finding my way to data, I think that the beginning of the story is probably at SendGrid. And I joined SendGrid as a DevOps engineer. And to be honest, I had not really heard of SendGrid at the time. I knew a little bit about it, but it, you know, I didn't really understand what it was, too much with the scale. SendGrid, by the way, is now owned by

Twilio. But they have an API for sending email, and they make it just really easy to integrate with, websites and applications and and software so you don't have to worry about SMTP and, you know, DKIM signing and all the other, like, gnarly bits of of email. Turns out that Sengrid had a ton of data. They're handling billions of emails a day, and, you know, there's a lot of metadata there. The the actual data of the email and and so on, the recipients and who to send it to and all

that stuff. And so working in that space, I was dealing with tons and tons and tons of data. I mean, we had, we were using mostly MySQL, and we had these massive massive clusters. I think we had, like, 30 or 40, you know, schemas under management. Each schema was a cluster of anywhere from, Like, 6 to 40 plus servers, Wow. You know, with lots of compute and everything else. So that was probably my

1st foray into, like, really thinking about data as a first class citizen. And, and even to the extent of, like, You know, building an architecture around the data. Right? So that you can optimize the flow of the data, and being able to store it and process it and transmit it fast enough to keep up with, with the flow. And so, yeah, from there, you know, had a lot of fun, learned a lot of things about, startups about industry, about, DevOps and and all kinds of

things. Management as well and leadership because that's where I first, started managing teams And then moved to data robot and, into the ML space. And then it was a whole another learning journey about, you know, data, engineering, feature engineering, transformation tools. How do you curate your data? And how do you really, like, know what you have and inventory it and, make it available to people within the business so that they can get value out of it.

Interesting. Very much. So our next question is what's your favorite part of your current gig? I think it's actually, I'm gonna cheat and I'm gonna say I have 2 favorite things. And I I kind of always have I I Figured out this formula a while back, in terms of what motivates me. And it's one part the people that I work with and another part, the problems that I had yet to solve. So I wanna work with smart people. I I really don't like being like, feeling

like the smartest person in the room. I much prefer to surround myself with people that are smarter than me and I respect and I can learn from. But that also, you know, I enjoy. Right? We spend a lot of time at work, so it helps to to enjoy the people that you're working with. True. So that's a big part of it. And then, finding tough problems, hard challenges. You know, if I don't have hard challenges to keep me, to keep my mind

engaged and occupied, I start to get bored and, that's no fun. I prefer to to always have something new to to to, you know, be chewing at. So, yeah, good people, smart people, and hard challenges. That is that is really awesome. I feel the same way about about both of those things. The, for me though, I I, Trying to find people that are smarter than me is really easy. So I I enjoy that part a lot. Like Frank. Frank is smarter than me.

Well, thank you. So we have a couple of, complete this sentence, questions. The first one is, when I'm not working, I enjoy blank. When I'm not working, I enjoy reading. I Enjoy movies. I go biking sometimes. That's part enjoyment, part exercise. You know, it's good for me, but, There's a lot of good, road biking in particular around Denver and a lot of beautiful scenery. So you can, you know, just ride for a while and find yourself up in the mountains or something, which is great. Yeah.

Traveling, cooking, all these things are good. Our next fill in the blank is I think the coolest thing about technology today is blank.

I I don't think it's necessarily something about today, but I think the coolest thing about technology is how it builds on itself. I remember Years years ago, I was studying for the CCNA exam, and that was such a formative moment for me to suddenly understand How networks worked all the way from the physical, you know, sending electricity down a copper wire, and it can be on or it can be off.

And that's it. Right? And you can do that really, really fast. Switch from on to off, on to off, on to off, all the way up to, web 2.0 and and Ajax and, you know, Asynchronous JavaScript stuff happening in Google Maps. Right? And I can just drag my map

around. It's just mind blowing. And, honestly, like, that That journey from the zeros and the ones up to Google Maps, that was, you know, what, 50, 60 years of, technology building on itself of people solving very small simple problems, but you add up all those small simple solutions and you get something incredibly complex And absolutely mind blowing. Excellent. Very interesting. The last, the 3rd and final, Complete the sentence. I look forward to the day when I can use technology to do blank.

I I would love, a Personal assistant, you know, like Jarvis from from Marvel Comics or something or, I don't know, from I I'm big into sci fi and and things like that when I read. So, there are plenty of examples, but some kind of a smart personal assistant that, you know, I can chat with and it keeps track of my calendar and reminds me of appointments and, you know, when to call my dad and whatever else, stuff like that. I just think that's

so cool. And I don't you know, with Especially with all the new LLMs and and GPT stuff that's happening, I don't think we're super far from that. So it's kind of exciting to me. No. You're right. Like, I you know, if you watched, you know, when I was a kid, Star Trek next generation was on, And the way that they were able to interact with the computer just through their voice. Yep. And I mean, the 1st Star Trek show had that too, but, like, the way the conversations I thought were more

richer and more kinda interactive. Mhmm. Mhmm. We have a lot of that now. Yeah. I think some of the fundamental pieces are in place now. Yeah. It'll probably take a little while to put them all together and make it work right. But yeah. Agreed. So our next one is, share something different about yourself. But we, always remind guests that we're trying to keep our clean

rating. Yeah. On Itunes. So I don't know. I think one of the more interesting things about my Journey is that I don't have a background, like a a degree in anything technical. I went to college and I got my undergrad Studying Greek and Latin and classics. And so it was mostly history, archaeology, languages, and things like that. And Computers have always been a hobby of mine and and I

definitely did some computer science stuff in high school. I took 1 or 2 classes in college, but I didn't really make my way into that Professionally until a few years after college. And, you know, honestly, I I don't think it's hurt me at

all. And in many ways, I think it's helped me partly because, you know, it it helps a lot with management and leadership, just to To kind of have a broad background and and understand, you know, different people and perspectives and and where they might be coming from. And I'm sure that some of the languages, you know, studying languages helped me picking up computer languages as well. I think there are a lot of similarities in

In, human languages and and computer, you know, programming languages. But What? Yeah. But, yeah, it is somewhat unique, and I don't run into too many other classics majors, At, you know, tech startups. I could definitely see the convergence, especially now when we're talking about LLMs and the like. Right. You know, the the nearest neighbor algorithms and all of that that are that are being applied because my understanding is that's that's, You know, that's how that

works as it picks the next best word Right. You know, in a in a sentence. And so syntax and grammar and all of the things you studied in-depth, That should be very helpful. Yeah. No. That that's awesome. There is that good value in, like a classics education. I I went to Jesuit High School and Jesuit College, you know. Mhmm. I was forced into studying Latin and things like that, like, didn't do it voluntarily. I'm not gonna admit that, not do that. But but like as I get older, like, it's

definitely like, Oh, I get this. Like, you know, especially when dealing with a lot of lawyers, there's a lot of Latin in that. And so I'll hear them, like, you know, Excuse some words. I'm like, I think I know what that means. Yeah. Audible sponsors data driven. And you mentioned you read a lot. Do you do audiobooks and sci fi? Do you have any recommendations? Yeah.

There was a really good book that I read recently. Like, this is maybe a year ago or something, but, best book I've read recently. It's, The title of the book is called Seeing Like a State, and it's by, James c Scott. The the longer subtitle is, something like how Some

schemes to improve the human condition have failed or something like that. But, it talks about this concept of legibility and how a lot of The developments over the course of the enlightenment, the industrial revolution, and, in the last few 100 years in in Our society have been primarily driven by the centralization of power in states And the state needing to administer all of these people, taxes, lands, land ownership, and all these different things.

And, you know, as part of, like, the the enlightenment, the scientific revolution, we all got very enamored with, like, rational thought and Logic and and all of this stuff. And we thought, we're understanding the principles of the universe. We can predict the motions of the planets and all these things. Well, we can solve all these problems about, you know, around human civilization and humans as well. And in a lot of cases, it failed. Right? And we didn't know as much

as we thought we did. And one of the sort of basic, like, premises of the book, I guess, or arguments that it's trying to make is that we routinely Underestimate, the complexity of the natural world and how necessary it is. And we think we can Simplify things and strip out all these variables and go, you know, monocultures in our in our agriculture, for example, and do industrial scale agriculture. You need timber for building ships. Great. We'll just plant Norwegian pines in straight

rows. This is gonna be great. It's so predictable. We know exactly what, You know, an acre of that will yield after 10 years. But it turns out you can't strip out all the variables because the whole thing falls apart. You need the complexity of the ecosystem to keep all those trees healthy. And so all that predictability you thought you had disappears within a couple of generations because, it can't sustain itself. Wow. So, anyway, it it's a very, like,

complicated book. I'm not really doing it justice, but I definitely recommend it. Interesting. It's on Audible. Yeah, yeah, so definitely check it out. The show. So if you go to the date is ribbon book.com, you'll be routed to an Audible page. And if you choose to get a subscription, to Audible. You will give us you'll get a free book, and then we'll get like a little bit of a bump on the head, and pat on the back, and Probably enough to

buy a cup of coffee. It started Which will share. Which will share. Yes. Yes. And the final question, Where can people learn more about you and Wallaroo? And they even made that rhyme. Yeah. Great. I think the best place to go is the Wallaroo website, which, as Andy mentioned earlier is wallaroo.ai. So wallar00.ai. And we've got a ton of great stuff on there. Lots of, you know, documentation and and white papers and, tutorials and things about the

product and what we're doing there. And for myself, I'm on LinkedIn. That's probably the easiest place to find me, Chris McDermott. And, I think I even have that as my, like, LinkedIn Profile name or whatever sits in the, you know, in the URL. Cool. It is, actually c s m c s McDermott. Okay. Well, thank you. Close. I was just looking at it, and I was also looking at the website. It's a very nice website. Thank

you. Great design. And, although I can't design great websites, when I look at one, I can tell whether it's great or not. Me too. Me too. Same boat. I can't do it myself, but I definitely appreciate it. I I can't cook, but I appreciate a good meal. There we go. Yeah. That's it. And with that, we'll let Bailey finish the show. Thanks, Frank and Andy. And thank you, Chris, for putting up with our broken

calendaring system. Satya should really look into that now that the drama around open a I is over. Well, over for now at least. Maybe g p t five can fix it.