How Box Builds AI Agents, Redefines Technical Vision, and Balances Speed with Security w/ Ben Kus #230

We're doing a special in-episode feature on the future of AI-powered incident management with our friends and sponsor X Matters. People as a primary integration layer is really fragile. With multiple people and all of that coordination, you become slower to find the root cause. The slower you find the root cause, you then don't know what action you need to take to resolve it. Getting to that fast is the goal.

Later in the episode, Mike Bennett, who leads the engineering team at X Matters, shares why human-driven coordination creates outage risk. And how AI powered orchestration can dramatically accelerate your path from event to resolution. The concept of agents is so powerful, and I really believe that. And I believe that you can do so much with it as you think of these like a multi-agent system in these like teams of agents doing these things.

And then and so somebody who might come to me and be like, I have this really complicated process. And you told me that these agents just do all sorts of great stuff. I wanted to replace My loan illustration process, my uh my uh process of the way that we do I don't know, insurance claims.

Like we want to do MA due diligence, like all these things that are like, that's a really hard problem that is going to take us a while to get our agents to think this way and do this in a way that you'd ever want to utilize. Like, wait a minute, I thought AI was smart. The you know, the look on their faces like, I thought this was supposed to change everything immediately. Like it's supposed to be so easy. No, no, no, it's not that easy. Okay, but I do believe that agents.

I'm in the future, I'm just expecting that we're all going to be like, I've got this job to do, and um I'm going to kick off an agent to do work.

I don't even think agents are gonna replace big people's jobs. I think they're going to have like people are going to be kicking off these agents to do all sorts of work and that becomes like the same way a manager is one of the more important people in your company. Like everybody's gonna become a manager, like managing all these agents who do all this work.

Hello and welcome to the Engineering Leadership Podcast brought to you by ELC, the Engineering Leadership Community. I'm Jerry Lee, founder of EOC. And I'm Patrick Gallagher, and we're your hosts. Our show shares the most critical perspectives, habits, and examples of great software engineering leaders to help evolve leadership in the tech industry.

For decades, 90% of enterprise data has been locked away, unstructured, and unusable in contracts, presentations, and videos. But what happens when generative AI provides the key to finally unlock it? How do you reimagine? And reorient your organization toward a new technical vision. In this episode, Ben Koos, CTO at Box, joins us to deconstruct their enterprise AI innovation journey and how they reimagined and reoriented the company toward a new technical vision.

So in our conversation, we cover things like how to reimagine and then set your technical vision. How to operate a multi-speed organization that balances startup innovation with enterprise level stability. Fox's platform-first approach for building secure, scalable AI systems. why security must be foundational from day negative one and a deep dive into the future of agentic AI and multi-agent systems. Let me introduce you to Ben. Ben Coos is the chief technology officer at Bach.

Where he leads technology and AI strategy to help enterprises securely unlock insights from their unstructured data. Ben's career spans engineering, product leadership, and startup innovation, including co-founding Subspace, which was acquired by Box, and being an early employee at BigFix, which was acquired by IBM, where he later served as chief architect of mobile security. Enjoy our conversation with Ben Coot.

Ben, welcome to the show. It's great to have you here. How are you doing? How are things? Doing great. Uh uh busy times uh and uh but I'd say exciting. Love it. So just to give a little bit of a walk up to folks listening here about like what we're talking about and why and what's going on. So there's a lot of massive transformation going on.

People are rethinking their vision. They're rethinking what's possible for their companies. They're also like questioning and rethinking where value is at for what they're creating with the company. It's a wildly disruptive moment. And what's I think special and unique is engineering leaders are sort of at the heart of this. They are the ones driving a lot of what's possible here. And I think it's super special. So we kind of have four goals here. One is reimagining technical vision.

Two, practices around becoming AI first. And I I know there's a lot of thought you and Box have put into this. And then there's like some deconstructing what you're doing with AI agents. And then I couldn't think of a better person also to talk about balancing innovation, security, and responsibility and the insights and trends around there. And so that's kind of the world we're gonna be in today.

So I think to begin, you know, why don't you bring us into your world a little bit? So, you know, if we're talking about vision, like what's been this vision around box that you've been reorienting around and and maybe give us what's sparked this shit. Yeah. So um so if if you don't know much about box

So box is uh we we call ourselves a content management company or a unstructured data platform. Um and so the things that we deal in were things that companies use to run their business, but um and we call it data, although it's it's not like data, structured data databases. Structure data. It's data that takes the form of movies or PowerPoints or documents or contracts or all these names that we provide to these like kind of like uh representations that don't have a

schema, right? This idea of unstructured data has been with us for like the history of computers, right? Like everybody has a file System on every single device ever. And at some point, we, you know, a while back, Box was founded on the concept of being able to collaborate on these across company using the cloud-based uh based uh approaches. And critically, um, Box is a B2B company.

something, right? Like we focus most of our efforts on large enterprises or on enterprises across the board. So mo most of everything I'm gonna talk about will be related to the idea of like, what does um these like major corporations like Fortune 500 or Fortune 2000, like what are they thinking about? so these kind of challenges because that's the natural set that we think about in this enterprise world.

Now, interestingly with generative AI, it was kind of born on unstructured data, right? You read every book, it read every uh every it reviewed every web page, it had all these different things and and you could use it to understand, you could use it to create.

things that historically never were able to like you can act you had to be a human to process it. And that was one of the most severe limitations we've ever experienced, always experienced at Box. Is like what you can do to automate things is relatively limited in the world of unstructured data or was.

Like in in the in the data world that I was like got machine learning, I have like data lakes, data warehouses, this whole new like, you know, like twenty years of like massive innovation, like uh where it's just like you could, you know, I'm a data oriented company, use insights, you get all this stuff. And then in the abstract today world it's like,

Yeah, you know, search it better, of course you collaborate, secure it. And for companies, like ninety percent of what they do is unstructured data, right? Like and and so like the ten percent, they were killing it, and then the ninety percent was like technology really doesn't help that much.

Like and so we would go through and be like, oh, I find a machine learning model that could take this contract that could do this and then structure it. And and that's what we did a few years ago. But it was like, oh yeah, like that's

And let's get another one for the point, you know, let's get to point oh two. But then when the uh generated AI models came out, it was like this like moment, like we've kind of been watching it for GPT two and then like kind of seeing it go. But until it really worked, which is around chat GPT style, like GPT 3.5. You you you you start to see, wait a minute, it can do everything.

Like it it can understand what we understand. It can it can and then and then then the image models came out. It can look at this picture. It's not about OCR. It's about looking at something and and giving the kind of responses human could give. And then suddenly the world of unstructured data really opened up. And for us, For our inner our customer set, if you ask them just

be like, what are the things that make you believe that AI is going to matter to you in the future? Usually one of the quick examples they'll give is around unstructured data. Because those were the things that you really couldn't automate previously. There's one like they they had solved many aspects of data problems over Um and so then our job and the thing that we do is to bring all of those AI capabilities to our customers.

So we had we had to kind of like reimagine and rethink what we did. Uh, you know, Aaron Levy's still our founder. He was uh so box been around for a while. Um, you know. Got a hundred and twenty thousand enterprise customers over an X by data, over a billion in revenue. So we we're not a small company, we're in Republic, but we uh have the mentality of a startup in some forms. And so we say ourselves almost every day, it's like if we were building box today.

Like what would we do? And it was like, well clearly it would be an AI first. It would clearly revolve around every uh like, you know, be platform esque, it would clearly solve all these

pro problems uh that like basically if you say the word AI and then say the word f content or files or any of that in a sentence, we're gonna wanna have whatever you say next is we're gonna be wanna be the best in the world. Data extraction, rag, uh like uh being able to to do agentix search, be able to do a debris search. Like all of these things are like That's what we're gonna do. That's what we wake up every morning to to focus on.

A lot there's so much to unpack there. Okay, so a couple different directions I want to go. So number one, you you mentioned so you were talking about some of the big trends of just like finally seeing sort of proof of concepts and like real use cases of of this technology, like the the chat GPT two point oh moment. Like I I've you know distinctly remember this too because this was like

2022, people are playing around with like first like little chat interfaces and and it was it was a spark moment. Like you look back at the use cases then it was kind of silly almost. Like it's like, oh, like it just kind of had this like capability and like wasn't really right, but like you could start to imagine the possibility. And then what I thought was cool was you you're talking about like

specific conversations with customers where you're starting to survey in in those different features. So I want to dive into like maybe some of those specific moments. Like were there customer conversations or maybe internal conversations you had that really started to ignite like this is the direction we want to go with box? Like this is really shaping the thesis for where we want to go.

So there there's some fun internal conversations that happened right before that. And and so like so we had spent years like with these machine learning applications and we we came a little bit disillusioned with the idea that like if you wanted to do something automated with your content, our first question was.

Let's look through 300 models and then pick the right one that works for you and then and then and then have it work until like, you know, somebody moves, you know, a field somewhere. Like and and we had tried early stuff and it didn't and it was like interesting but didn't really work well.

I remember Aaron came by, he's like, Hey, you know, like there's this interesting new thing. I was talking to uh you know, he's well connected and and and I'm like, Yeah, but like I've been trying it, it's not really there, like you know, and and then and then then we had this like bit of an argument about things that could be done. Well let's just try.

And so we got an early version of some of these things and we we applied it to some of these things that were like our examples of stuff that didn't work. And it worked. It was like it was able to deduce whether that cost was risky. It was able to pull out um the data without having to give it the format of the structure. It was like, and then you just had this moment of like, oh my God, it it understands.

Like so for us, like most people would look at the early version of ChatGP like, I can't believe it's able to chat with me and talk to me. And we looked at it and we're like, I can't believe that if you make it read a document, it'll then answer questions about it. Like that was amazing. That was like this like disruptive technology. And then based on that, like suddenly all the stuff that we had tried and failed for years to could to general purpose use suddenly became like available.

Things like data extraction, things like search, things like being able to get answers instead of just getting documents, like this whole variety of things that over the years, We had tried, made a little bit of progress, but couldn't make, you know, really work enough to be production. Like suddenly we just had this like long list of things to try and do. And then and then interestingly, as we went through, we were like,

Oh, that's not quite good enough. And then like, well, new one came out. Like, okay, try that one. And like, and then is it we just and we've been on this little like roller coaster of things that are just becoming more and more available like uh over the last couple of years. And then that's just partly why it's exciting time.

What's interesting was like I'm thinking of like the pattern of that in 2022 and then like where we are now. And there's still that same sentiment of like test and experiment with all these different things. Like I'm thinking of my recent experience with agents. Like I just use like Chat GPT agent to book a flight.

And I was like, well, like it kind of worked, like, but like it's still a little clunky, like a little slow. But I was like, man, if that same experience and pattern happened in twenty twenty two, like in six months, like it's gonna be such a different experience. Like it was just it's just cool cool to see those trends.

Yeah. I mean, we're very platform thinkers, uh or at least that's kind of like a a part of our what we do is to think first in platform. And by that I mean that like you could have built a feature that was like chow your documents or like uh like do better search or whatever. But like rather than do that, we we early on said, let's build a new platform layer. We call it AI platform. And and this thing will be fully integrated in a box.

so that like in the same way that box allows you to upload and download and unless you view, unless you convert, unless you do all these things that people just do naturally sync with your data. Like we added like a fundamental like API layer that said, I want to ask AI in to be able to do these things for I wanted to extract these fields. And then we we we basically added it that way. So you could think of it like a platform that we use ourselves.

And so um in the early constructs of it, it had to be very secure. That used to be the big challenge, right? Like was that like no enterprise. They they were so scared and they even some are to this day, of like this idea of like what you're gonna do what? You're gonna have like AI run on my data? That sounds like the scariest thing in the world because I don't want to.

to train. I don't want my stuff to turn into like some training run somewhere. Like so we had to lay in the platform capability to do secure compliant AI on your data. I mean this is for box. This is like it's it's just second nature to us. We have to be the most secure, most compliant, most trustworthy place for your data. So part of what we'd say is like your data is probably more vot more safe in box than in some computer that you have locked away in your in your

organization. And we needed to apply that to AI. So we couldn't just build it in whatever we felt like. We had to like make principles like contractual obligations that we would tell our customers is to say, we will guarantee these things. And then we get make sure that like the all the techniques we use will guarantee this kind of security and these principles.

And then only then could they even consider using them. So then if we built a new feature, we didn't want to have to go re-verify all that. We wanted to lay a platform that then we would then use our own platform to sort of build on it. And so um one of the things that we did over time was to continuously upgrade this concept of this platform.

Including bringing the idea of AI agents relatively I like we started to use that term a while ago. And then um it was kind of crazy to see how it blew up. And then we systematically updating our agentich platform to do more and more, including things like multi-agent systems, including things like being able to like uh like reflect more and just like the the newest trend and the newest techniques around AIH.

And so for us, this is is just an ongoing set of things we need to continue to do to make uh like everything better. The models that we offer, we offer models from like, you know, pick a great model and I I then I'll say, oh, okay, we already have that. Trusted, trusted uh posting layers.

In addition to the techniques that you need to do, like RAG, like OCR, we call it secure RAG because it's very permissions aware, um, data extraction, being able to output the formats, be able to make your custom agents that let you bring their own destructions and their own sort of plans to these agents and so on and so on. So we think platform first.

Make a feature that that revolves around that and then continue down the path of uh making your platform better and better. And usually along the way, people find more and more uses for that platform.

Oh so many different things to to get into from there. So uh one of the questions I was gonna bring up was like w at what point did like security come into the vision role? And it sounds like from day one it was like security, compliance, and trust is is a primitive for for everything that we have to do.

Uh it was it was yeah, negative one. Like we wouldn't have even thought of it um without that. Like and like and it I this still shocks me to this day, by the way. It's like and if you're startup uh like out there, I would I would uh advise you to like be careful about this topic. Some people like will say, I built a cool feature. Now I gotta figure out how to add security.

And and like you already lost to that point. Um I saw that we uh there's just like examples quite telling early on. Um in some company I'm not gonna name

Went through and they built their own internal little uh rag system where they had like they they dumped a ton of data they've had internally, just documents and things, into this thing. And then they they they vectorized it, you know, embeddings. You do the traditional put a vector database, do a cosine search. And so and then it was able to answer questions. And they were like, this is amazing. I love AI, like you know, this is like early days.

So the people who were working on it were like, man, this is, you know, this is a public company that you would recognize if I told you. And then they asked a very sensitive question and they got an answer. And they were like, oh, that's crazy. And then and then um it turned out that like them knowing that and doing that was a serious offense. Like AI doesn't keep secrets.

If it sees it, it'll report on it. It was very critical to first filter all the list that goes into the AI from a reach like from a rag perspective to then give to the person.

And and you can't train a model because that train model will know. Uh like some people are like, Oh yeah, I'm gonna add like role-based access controls later and things like that. You're like, No, no, no. You first of all you don't need role based, you need user-based access controls. And then second of all, like don't even tell anybody about it until you have that in that.

Like uh because no enterprise will want to use it. And if they do, because they're not sophisticated enough, it'll turn into a disaster that won't reflect well on you as a company nor the person using it.

We're taking a quick break for a special feature on the future of AI-powered incident management with our friends and sponsor, X-Matter. Mike Bennett, who leads the engineering team at X-Matters, shares why human-driven coordination creates outage risk and how AI-powered orchestration can dramatically accelerate your path from event to resolution. We're the ones that are correlating the alerts across the platform.

We're the ones that have to remember that a similar issue happened six months ago and this is what we did about it. We're the ones that have to figure out this is a symptom in service A, but it has a dependency in service B that we need to know what that dependency is and how that could impact this thing. We decide on who is going to be page based on some informal knowledge.

It's it's not scalable. I mean it that all of that works in a in a very small scale environment. But as as systems grow, as teams grow, people as a primary integration layer is really fragile. So the outage risk is with with multiple people and all of that coordination, it you become slower to find the root cause. The slower you find the root cause, you then don't know what action you need to take.

to resolve it. The risk there is not knowing immediately what the problem is, so you don't know what the route for that mitigation is. With all of the information that is out there, getting to that fast is the key goal and is the key problem when you've when you're relying on people to do it. When a signal comes into X Matters, the first thing that you can do is based off of that signal, you can then make a call out to the right people.

Rydyn ni'n ymwneud â'r cydnol sy'n sy'n sy'n sy'n sy'n sy'n sy'n sy'n sy'n sy'n sy'n sy'n sy'n sy'n sy'n sy'n sy'n sy'n sy'n sy'n sy'n sy'n sy'n sy'n sy'n sy'n sy'n sy'n sy'n sy'n sy'n sy'n sy'n sy'n sy'n sy'n sy'n sy'n sy'n sy'n sy'n sy'n sy'n sy'n sy'n sy'n sy'n sy'n sy'n sy'n sy'n sy'n sy'n sy'n sy'n sy'n From there, the incident commander can then use automations that are set up in the incident because it it automatically creates an incident for us.

It's linked to the ticket that generated the incident. And from there we can determine, okay, well I've seen I've seen this before because my incident suggestions is saying this looks similar to this incident you had last week. We've got built-in automations that can do stuff. So within an instant, you might have an automation that says automatically restart pods or automatically rollback services. Like I mentioned before, we can also do that as part of a response.

to the signal that comes out to say, okay, this has happened, do a rollback and I can just Touch my phone and go back to bed without even getting out of bed. all of the automation, the flexibility of the tool and all the the things that you can build in along with the data that you've got with the service catalogue, with your on call, with your who's on duties and gets you to get the right people at the right time on the call if you need to get to a point where you're in a conference.

X Matters automates the entire incident lifecycle, taking you from initial event to final resolution. To see how their purpose-built AI slashes your resolution times and gives your team the context to stop disruptions before they start, head to xmatters.com. That's x-ma-t-t-e-r-s. Dot com

From talking to a lot of people, like when we're thinking about like what are the next bottlenecks going on? And so we think about like let's make an assumption like execution and the ability to build things is going to continue to accelerate and people become more sophisticated, tools get better, and like all of the different assistants and coding agents and things like that are going to continue to work.

We're I was talking to somebody we're we're kind of aligning around like, oh vision is going to be the next bottleneck because like how fast can you start to reset and and reimagine what's possible and get the vision of the product aligned and then get people aligned on that and then get execution going? Seems like then like all of a sudden the coordination, the human work starts to become the bottleneck here.

So for me, like what I get really excited about as you're sharing the story is like one, how do you in in the collective group at Box reimagine like what's possible for the company? And then two, start to set that technical vision in a way in which it can align people. So can you bring us in a little bit to like what did that look like and and how did that work?

Uh it's it's on my mind. I just uh uh walked out of a of a meeting just a moment ago. It's like and we had this like really interesting debate, which is um and it and it it revolves around this platform topic. So when you say vision, I think of not only features and not only the pitch and not only the

marketing uh aspect of it, but also how are we going to respond to things that we don't know about yet? And that's the key right now, like in for me, is that you'll notice that some companies who used to be leaders in certain areas, like fell off. And and there's a number of reasons for it. But one reason might be that as everything evolves They couldn't evolve with it because they had a uh their platform wasn't ready.

this whole agentic world that we're in now. And when when I say agents, I think of an intelligent system that can do a more complex task and specifically like something's got objectives, got some instructions, it's got uh tools, it's got context. Um, and then and then I think about being able to put these together of multi-jit systems and so on. So like So this concept is is is is very, very powerful. So there are examples of things that you want to do early on.

We're doing deep research uh on your data. So you can like query things. This is a multi step process. We're doing agentic search where you can have the agent go look for things for you and pull it back. Like there's all these interesting like first step opportunities. But

The thing I'm really interested in is not the things that we're doing right now, but like in the future, we might want to build something differently. And sometimes you have this trade-off, right? Because we could go build something the way that is like the easiest or the most sort of like I'll hack it together and make it work. But like if you can instead say No, I'm gonna like lay down the basics of the platform.

platform and and and then and then and then lay a layer of abstraction so that I can solve things in the future. Then suddenly the next time someone wants an agent, they just build an agent, as opposed to building the system that then powers it and is secure at scale and so on. And so to me, like uh the the the trick of this is you have to predict

what kinds of architectures will be the thing that will matter in the future for your customers and decide if you want to pay up front that some of the cost of laying that that platform and slow yourself down a bit. And that's really hard. Um but it and if it fails, like if you build a wrong platform it's quite expensive to replatform.

If you get it right, then you'll see like I can't believe how fast this oh now I get it. Like and and and and to me, agents are a bet. Like the like the idea of building an agent platform where you have these little like entities, these little sub agents that can then run and do things, and then you rethink everything in that form, that's like a bet that you have to make. You have to believe it's going to work in different ways.

A year and a half ago, nobody was talking about agents. Like the auto GPT had failed, like maybe AGI didn't quite work out well. And so now everybody's loves agents. But if you do that system wrong, then you will almost certainly fall behind as you have to like re-replatform into whatever the newest.

way agents work or whatever replaces agents. And so these are the kind of things that I think company leaders, technology leaders in particular, have to just be constantly talking about. Like, and I love agents. I'm super in the mindset of agents that have this whole like, I'll tell you all about agents.

And at the same time, I have to always worry that maybe agents are not really the thing that we're gonna talk about in two years. Um I actually think they will, but but but that but that's I have to worry that I'm wrong.

At what point then is there like the like a clear vision of like we're gonna follow the we're gonna follow this path, we're gonna make this bet, and maybe this is sort of the timeline or how we're going to revisit this? Like what is it what does it look like to then kind of have that moment where it's like, okay, this is the route we're gonna go, and maybe we'll revisit later on?

So interestingly, um I do believe that uh if you're trying to keep up with AI, you have to do it in the form of a multi speed business for something like The traditional way that like we would deliver features is you do some research ahead of time, you make a business case, you then decide at the high level what you want to do, you you get funding for it, put in a plan, get approval that they go through and kind of this yeah, there's like process. It's like your development process.

And then along the way, people kind of expect that you know answers to questions ahead of time, right? Like uh how much money is this gonna generate, uh, how many customers want this, uh, like what is it gonna do? Like uh like normal questions for for software development. With AI, A, it's so disruptive that it's a whole new paradigm of how it works. And so nobody quite knows.

what it can do or what it can't. And the trick of the whole thing was is that like you can probably get something working pretty quickly, but it's unclear if the thing you got working quickly will actually survive into a production experience that people want to use and and be accurate enough, you know, they're inherently non-deterministic.

Will customers accept them? Is the way that you implemented it work out well? And so your normal software development process just doesn't really work that well in this case. Or or if it does, you probably end up with this trick that I just mentioned, which is by the time you're done implementing it, you should realize you were wrong when you start it. And so you're much more into not only a more agile sort of philosophy, like um, but not not just in the specific sense of the development.

You have to constantly reassess what you care about as a company. Most companies have a lot of stakeholders. You know, we got people who are in the customers, we've got the, we've got the salespeople, you've got your marketing people, you've got like all these different places. And they all have, they typically have input into something you want to do.

But it's very hard to engage all of them constantly as the world changes. In particular, like, okay, we're gonna talk now about uh MCP servers and like.

What the hell are you talking about? Like like uh oh, is this a new trend from the last three weeks? You know, and like and it's just like they're like, No, um, let's talk about something else because that sounds confusing to me. Like, no, this is probably the future of how all integrations will go forward, but yet it's it's hard to keep it on. So the two speed system is

This idea that you have a smaller group of people who are almost like running a startup inside of a bigger organization. And so what you do is that you quickly iterate on some ideas. When they survive for a little while, you then turn them over to the normal process. Because no matter what, in a company like Box, you can't deliver things that are ideas or hacked up.

It must be like, you know, this is our thing. We we have to have platforms, you have to be uh secure, you have to be compliant, it must be audible. And and so I think there's it's hard for any company over, you know, maybe even over a hundred people to be able to deal with this. And so the way we've chosen to deal with it is to have a little bit of a testing ground.

where we are delivering things earlier that aren't quite done to test them ourselves to then turn around and then decide now to put into its it's like full normal cycle.

Do you have an example of maybe what in what one of those projects maybe that worked out and one was a go, one was a no go? Um yeah. Originally when we were doing uh this idea of uh data extraction turns out to be this like really critical thing that a customer.

It was one of the first things I'd ever tried and I was just amazed that it worked and it just keeps getting better and better. Is that you just say, Here's a document and then here are some um fields that that you care about and then most AI can actually just

uh look at it and be like, okay, I'll fill it in. This is what human would. If I gave you this, like here's a contract, and then like who signed this? And like, you know, like what are the dates? Um or and then and then if you were a lawyer, I could be like, is this a risky contract and what are the clauses? And then and then AI can do all of these.

So you you almost think of it like a little like uh JSON sort of set of structured data associated with every piece of unstructured data. That's what we call metadata, and then we have that built in. So we made it work originally. We were like, this is amazing. The first feedback we got was no one's going to

because it might be wrong and it won't tell you its confidence score. Somebody's like, no one's gonna want to use this thing. And so that we almost killed off this project because it was so new and weird to people. Super happy we didn't and we kept testing in, we kept getting feedback and and and then then the overwhelming feedback was I can't believe how easy it is to do this and how it starts to work. And then and then now this is a major set of of things that we provide to our customers. Like

hundred millions of these like things going on right now, like to like, you know, extract this data from these kind of uh files. Along the way, like there's interesting little sub sub thing is we we thought it worked for everything. And we could not find an example or didn't until we found those examples that our customers brought us. And they were like, hey, you said it would always work. And in this case, it's not. And then you start to realize, oh, sometimes the models have limits.

Sometimes uh the way you instruct them has limits. Um so then along the way we had to then say, Okay, we need a new technique to handle these extra complicated things. This is where we started going the path of agenticness, where you actually had AI like taking multiple steps. and and basically multiple different like a like a like a state diagram of graphs and this is

how we think of what a agentic means. Um and then and then that started to solve the problem. So that's the example of it like the kind of feature that like

turned out to be really great, but then w it was working out. We we've also had some other examples. I'm almost embarrassed to tell one of them um it because it was of how cringy that I I would think it would be like But then then first way that everybody thought of, the way to solve um all problems would be to make an assistant that was always in your product.

It says just imagine the corner. You always have a little like uh little AI person there, you know, and then on the the it's just so natural to want to name it. You're like, I've got this little boxy or whatever it is in the corner, just like a little and you you give it a name, you you like talk about it that way, and its job will be to just always be there and it'll be like

I'm waiting to help you. Like, and so we actually early on had this idea that like no matter what, anything you want to ask it, AI is smart and enable, we'll teach it, we'll train it, and it'll just always be there for. And and I'm pretty sure had we done that, I don't know if I'd be here, I suppose, but like it would have been um it would have been embarrassingly bad.

Because nobody wants to think of it as like a little helper thing in the corner for you when you're in a enterprise class production product. And instead you want to do much more sophisticated ways to integrate it into the review.

So at the time, like people were really gung-ho on on oh AI, it's e you need to name it, you need to brand it, you need to put it in your system, but it would be lame. Um and and and so it's so hard in early days to figure out what was is cool and going to work and is going to be revolutionary versus the things that are going to be lame.

And and it and this is like a weird time in my in my uh twenty five or or more years experience is that like most of the time you're very clear ahead of time. For the last few years it's been a little bit harder to figure out which one's which. I it's so funny is you're mentioning this browser extension piece. I I have like three of them open right now that are just sort of there waiting to be engaged with. Yeah.

Yeah. I I I I appreciate the ones that are like, Hey, uh, you're on this marketing page, please or I you have customer support. Like th those are fine. But but like a like an enterprise class product. Like you don't want an email to be like your little thing and they're like, Well, you're an email or an on your messages or anything to be like, Hey.

Hey, what can I do for you now? Like it's like if I had an assistant like sitting on my shoulder right now, I'd I would almost be annoyed by him, like even though maybe they're there to help me.

Yeah. I want to switch up to talk more about agents. You you've mentioned this a couple different times, and I I'll say like building out agentic systems, figuring out some type of agent component within people's products is like probably a top P0 for a lot of people right now. It's like they're they're they're diving into it. So what's your vision for AI agents in the enterprise? Maybe we we start there and then like let's get into the story behind how Box built out its Agenc AI system.

If you were to ask me like What are you excited about this year for and what you're building and what you've built? I would tell you about agents. If you said, what about next year? I'm gonna tell you about agents. And even you're like, what about what I'm probably gonna tell you about agents? Because um I consider the technology paradigm with agents.

to be incredibly powerful and to have a very long runway about what you can do with it, the promise of it seems to go extend far beyond what is currently possible for a number of reasons. So uh for us in the early days, uh there's this idea that if something's so complicated, um that you run you very quickly, we we very quickly ran into things that were too complicated for a single pass agent.

Uh our first example would be like we would we would find these like very long complicated documents and we would ask the AI to answer a complicated question and then we would see if we could prompt it properly, you know, only use the answer and the information you find in this, uh here's a rag base, here's uh pieces document, and then we'd ask the AI to be like, Can you answer?

answer this question. And then we we we would start to hit the limits of this where you'd be like, Oh, if I add a period

then it doesn't answer anymore and like um in in in in oh the context window I need to make it bigger, smaller, you know, and I only had a few thousand tokens to work with at the time. Like so so you start to realize that like there's a limit. Now and then then Chief T four came out and then the new Gemini's came out and the quad's great and so you start to like it started doing more and more but

But we still maintain examples of things that nothing can answer. These days is actually uh the humans have a hard time answering them too. And so um then you realize that when a human answers these, they spend a lot more time on it. And they like go through and they do this like project. to like think through and do these things. And so for us in our early versions of of AI, it was all single package.

Here's some info, answer. You have seven seconds, you know, or whatever. Like um and and and then you start to realize that like If I could just feed it back. So we had early on we had graders.

So we built our own LM grader before it was a like a cool idea. We were kind of proud of that at the time is that they it became like a standard trend that we were happy to have done early. And so we'd have these graders where you d get a question, let's say um uh answered about it, uh or you extract data and then you have a grader look through and be like can be very helpful when you give them the question and the answer at the same time, right?

If I were to ask you like some random fact that you may or may not know, like, you know, when was the Magna Card signed? Like that it's actually much harder for you to answer than if I said was the Magna Card signed in 1215 or you know, like and then it because you're you're like, that's not right. Like um and so you you get better feedback from a greater in this case. And so we did this.

And the grader came and it would say, This is totally wrong. And we had like s long since given the the answer to the person. And then so then I assume that person was just like would have been annoyed that um they got the wrong answer. So then we're like, well, why don't we just, you know, somehow feed it back?

You know, and this was the be the first agentic idea we had was get an answer, but before you get to the person, like quickly tell the thing to say, no, no, no, no, this is not right. Try again. And in that little loop, you were like is basic reflection agent is like um that's sort of step one. And then also in the world of uh data extraction, we started to realize that when things got too complex.

Let's say you need like 500 fields from a document that was 300 pages. At some point, the AI can't keep track of all of it to answer in one pass. So you're like, okay, well, why don't you just start with the first five, now answer the next five, now it's the next five. And then see, and then we started to hard code some of these things in our system. And then we realized that at some point you need to intelligently group these.

So for instance, data extraction, you'd be like the first five questions be like, Well, you know, who are the parties of this contract? And they'd be like, Oh, there's two. And then later on it's like, where are the addresses of those parties? And it'd be like, here's the three addresses. And you're like,

Whoa, they don't match. Like and then that will happen all the time. So then we were like, okay, we gotta group these ahead of time. And then and then and then so that kind of thing became itself an intelligence step. And then you realize that at some point that when you think of things as a state diagram with intelligence

AI models entering every state and then also transitioning into states and figuring out when it's done. That is a very powerful idea. Like you start to think, wow, I can do all of that. And then have it output an answer and then have that answer then be used by another set of agents that goes. So it's almost like you just start to envision this like very long, complicated like set of state diagrams when every little node is as powerful as the best model. And that's an incredibly powerful.

And then that became the word agentic. We had been building and using this term agentic our agent for a while. And in our first version, agents were actually quite basic. Like they would kind of have one step that they would do. So we upgraded our platform to not only allow you to customize the agents of the model and instructions, but then also customize

And th think of these as like Langgraph style little nodes that have uh state transitions and edges, like sort of computer science E view of this. Um and then that became very powerful. Now the interesting challenge here is that like

You have to actually think this way. So so like um like before you can use it appropriate. Like so we still to this day, like it's very common for engineers, even people on the in the I team to be like, Okay, I'm gonna solve this problem, I'm gonna do this, I'm gonna build this new microservice, I'm gonna go power this thing and I'm gonna distribute a system and then and you're like, Oh, or Ask a little agent to do it a step two of what something does. And then they and then they're like

That's not what we do here. Like this is this is not and then you're like, No, no, but if if you did that, it would work. And they're like, Well, let me prove to you how wrong it is. And then after a little while they're like,

Huh, this is you know for the class of problems it worked for. It was like this worked really well. And it's just like interesting mind shift of it's like a whole new paradigm of of programming. So th so uh context engineering, which is what I would call this this concept, is like really powerful. I am a big fan of context engineering as a new whole new paradigm of what people should be doing to get agents to do what you want them to.

There's been like coded ways of people sharing that challenge. It like to friction. It's like, oh, you know, we have these seasoned engineers that, you know, are resistant to adopting. And I think it's like not that case. That's not it. But really, it seems more like it's this mental model of like you're asking people to shift the mental model. So there's this pattern that you have to kind of disrupt. How do you help drive that shift?

So this is kind of funny. Like even in the days of prompt engineering, like do you know, like for a while, the best prompt and engineers at Bach.

were actually our CEO. And then I would I would say I was up up the thed there too. Like there was a the couple of engineers who worked on it. And then and then also like Aaron was quite good at prompt engineer. He could get AI to do many things for him because he kept trying. And interestingly, like early days and even if you're an engineer out there or if you have an engineer team like

I would I would not encourage this behavior where they consider talking to AI or making it do whatever. That's beneath me. Like and it was funny because we would also like be almost like so I'd be working on this prompt and doing all stuff and then and then and then I'd talk to some like, you know, staff engineer, senior staff, and they're like, mm.

Like I let me find somebody to do that. And I'm like, uh I'll I'll do it for you. There's like w like it's some for some reason it was okay for the CEO or CTO to do it, but it wasn't like okay for like a senior staff. It was there was like there were um in some cases like

Consider that it wasn't a hard enough problem for them and their skills should be used elsewhere, which in some cases are like they're experts in all sorts of things. But getting the context right and prompting these, these, these agents to work well is hard and it's incredibly valuable. And so um this is so when we uh now our principal architects uh on on this or are our our key engineers, like they're incredibly good at context.

And so we had to almost like break this mold to be like the most sophisticated people should be doing this. The most valuable people like should be doing it. And it's like part of our philosophy is also part of our platform. Right. And so um but if you don't do this, then no one will want to do this work they consider to be like you know, is um I can imagine like uh I I kinda remember a little bit the the early days of like HTML or JavaScript, people like, oh, those are like toy things like

the C plus plus language, that's where we use that's where very cool people are until you're like, Well, wait a minute, you can do a lot of great things in that and then that's where you should concentrate your efforts. So this idea I think is still an ongoing paradigm shift that um many companies haven't started thinking in this way, nor have some of the engineers like fully rotated over. Not not everybody, of course, but like but for people who work in

One more question about agents. So you you mentioned there's a long runway of what you can do and it like it can extend pretty far into into what's possible. I was I was reading something today, it came from like the Singularity University World and was talking about the the way we kind of measure models. is sort of reaching an upper limit. We have to kind of rethink how we measure success for these models.

One thing that Peter Diamandis kind of proposed was like, what if we measured it against like performing really complicated things like solving humanity's greatest challenges? Like put it against like solving poverty, put it against solving hunger. And I was like, oh, that's a really interesting way to measure kind of like, could something do that?

So when when you think about like the long tail of like extending of what's possible for for some of these agentic systems, is there like a futuristic case, like a picture in your mind that you consider to be like the extent of this?

So so first of all, like many of the benchmarks that we create, we have our own ego sets and we have these like challenge eagle sets and so on. In in in AI starts to come back with like a Oh, like on our basic set, there are like many models are getting in the nineties, you know, on our challenge set, there some of them are in the fifties, some of them are sixties, you know, percentage wise of this whole thing. Um, but interestingly, um, and this is this is really critical, is that like

Like if you ask a smart human to do it, they also start to score about the same. And so everybody's so used to this idea of being like, oh, the AI is only at sixty percent. Like, oh then you can't use it or like that's not enterprise ready or whatever. And then you start to realize that like some of the models are getting to the point where they're as good as people. And somehow as organizations we work together and we get things done.

You have to adjust your mental model to say, not only can AI do uh this, but can it do it basically as well as people?

In some kind of cases, it's like, you know, like there's there's a debate that goes on to figure out what the right answer is in some of these, like this is quasi risky. Like, well, for here's the pros and cons. Like that's not necessarily a uh like a ground truth answer that is is correct. But so then the thing that is is in the next step is for it to to do more, more complex things. And so in in in things that are hard for you to make an eval set.

And this is actually one of the challenges that that you have agentically is that like at some point if you need to sit down and make eval sets, which you should always do, but like it's like really hard to come out with examples because oftentimes people instruct and customize agents to do certain things.

You know, so so there's one company in the world who wants to do one thing, like how do you make an eval set out of that thing that you want them to do? The value going forward for agents and for some of these things will will kind of be benchmarks against Like the almost the way that we would think that like a person in an organization might be able to do something or team. And then and then can they either have an agent to do that or

even better is if you if you actually ask the person to do it by managing agents. So what can they do themselves versus what they can they do by instructing agents to do it. Like so let's say I'll just take an example of like, I want you to um maybe maybe create a blog post on the show, right? So um there's one way you could do it as a human, and then there's another way that you could do it um with a bunch of agents or or AA models or however form you want, then compare those two. Now

I don't know if you can get a great blog post that's in your mind with just an agent alone. You have to tell an agent what to do. And so that's the key, is that like like nobody's gonna make the the the engineering leadership podcast.

Agent that's predetermined those everything you want, but they can give you some like the ability for you to give it a few instructions on what you're looking for and then have it create an awesome blog post. That kind of thing to me will be the future where you're measuring what people can do versus what people can do with a. That's great. One element of this that I wanted to dive into was balancing sort of the speed of innovation with like security and responsibility, knowing that

Security and responsibility were like first principle for for development for for you all. Like talk to us a little bit about like what you consider when you're balancing like the speed of innovation versus incorporating security and responsibility. Like how how does that come in and balance and and tension work together?

The TLDR is like, don't ship anything to people that can be so dangerous that it will cause harm. It's like the there's plenty of analogies, but some people call it like uh like should you give give your customers like a loaded gun that is like they're not quite sure how it works? Like, yeah, like I I think you try to avoid that.

kind of thing very much. It'll come like boxwood. Now not to say that there's not a lot of great experimentation you can do and getting early feedback, but as a as a generic like you can't if you're gonna tell this vision that like, oh AI can do anything is so awesome, don't turn around and then like let it go crazy. And in a lot of in in in AI agents completely changed the the the whole scenario in my mind because the whole concept of tools.

Many types of tools are not only to like like investigate the environment, but then to change it. And then at that moment, you've got to be really rethinking everything because like if that tool can send an email, your agent might send an email.

If that tool can delete it can delete stuff like files and box or whatever else, then that f that agent, you should be prepared for it to do that. And the more complex your agent gets, the more likely it might be that it would do that. And and I it's almost like a human concept, is like. Like you might, if you gave everybody in your company the ability to to like push a button and delete all of your production systems, like probably that's gonna happen. Like, um it and and so.

So you you have to be very cautious on that. And so we are constantly trying to be at the forefront of of these things, but we will not do something that we we have a the direct knowledge to be like, oh, in this case the agent might accidentally do this and this will become a problem.

Data security, data leakage is just so ingrained into us is part of our promise, right? We can't do that. Now, it doesn't mean we can't release cool features that do things, but like often, like even to this day, like some of our customers are like, I why can't I do this thing in your system? It seems that easy. We're like, Oh, yeah, it's totally easy. Look, I can do it right here. I'm not gonna give this to you.

One of the examples we had was uh uh searching the internet. And so like searching the internet as an agent is a really useful thing. So our agents can search box and they can go through. We we we haven't yet finalized the securing them the search. Search the internet. And the reason why is because if you can have an agent go through and look at all your data and then turn around and make a query to the world, that query can leak data if you're not careful.

And in some like even a link can leak data when somebody clicks on it. And so you have to be very sure that it can't it that you're you have that problem solved and that you tell your customers about this and that you give them the option to turn off you're not fully solved before you allow something.

And that's just a simple example. There's a ton of agentic examples when they're using these tools where you have to be very cautious about like so we don't let our agents to do destructive actions today. And before we do, we'd have we would give them um sort of a set of guardrails and a set of like uh policies to be able to do these things. And then that's what we're working on. Um the going forward. And and this is the I think I hope everyone in the industry is working on this.

kind of stuff going forward. Do you have a a quick note on how to set those guardrails or like the key questions? Like'cause I'm thinking of like that specific moment where like an agent is taking action, like, and that's the thing to pay close attention to. Is there like a set of questions that you and the team ask?

yourselves to help like develop a a thing there. Yeah. What if it does something you really don't expect? Think of the worst c like basically think of the worst case scenario. Uh uh and then and then think about um like how would you you mitigate If the worst case scenario is bad, don't do it.

like, oh, I have a a rule I can put on it so that it will avoid that. Like it's allowed to send emails to three people, but I don't want to send two emails to to a hundred thousand people. Great, make a make an allow list of those three emails. Okay, done. And then also what do you do with people? Is it is a common question we asked. Because like sometimes when you have these agents.

you start to realize you have to almost treat them like as non-deterministic as people and we have processes and procedures for people. Uh so like the idea of requiring a human loop before you can change this feature, you have to have somebody else look at it. That's a common thing that we do for like really big changes. I think the agent do that too.

The agent wants to delete this database, uh the agent wants to change this file, the agent wants to do these things. Ask the person first about doing those those kinds of things. That's great. We kinda started off by talking about reimagining what's possible and the process of of setting a vision and then diving deep into agents and

How to reimagine your experiences with that and some really incredible practices for setting that up and intervening. So when you think about like enterprise technology transit, you're most excited about.

What do you think maybe are there are there some that you maybe think are overhyped or are some that you're really excited about uh evolving in the next five years? Like what are you most excited about and what do you think is overhyped? I'm I kind of worrying about Zerla, like uh like I'm gonna say age install.

And so like um like there's a a funny thing where is that the concept of agents is so powerful and I really believe that. And I believe that you can do so much with it as you think of these like a multi-agent system in in these like teams of agents doing these things. And then and so somebody who might come to me and be like,

Like I am so excited about this. I have this really complicated process and you told me that these agents just do all this great stuff. I want it to work. I want it to replace from end to end the like. My loan allucination process, my uh my uh process of the way that we do I don't know, insurance claims.

We want to do MA due diligence, like all these things that are like that's a really hard problem that is going to take us a while to get our agents to think this way and do this in a way that you'd ever want to utilize. Like, wait a minute, I thought AI was smart. Like, well, it is.

But like it in in the in the and then that you're like, Well, I thought it like is a when the better model comes out will work. You're like, Well, no, probably and and then and then and then there's this idea like, Well, what do you do if you got ten new PhD people in your company? They're smartest people you've ever met, but you want them to do these things. And they're like, Well

I gotta train them, I gotta do this all the stuff. I gotta do this and I gotta like teach them this tool. Like, and you're like, well, we probably need to do that with our agents now. Like uh like it's to get it to work in that process.

The you know, the the look on their faces like, I thought this was supposed to change everything immediately. Like it's supposed to be so easy. You know, it's not that easy. Okay, but I do believe that agents I'm in the future, I'm just expecting that we're all going to be like, I've got this job to do, and um I'm going to kick off an agent to do work.

work. You see some of the best developers when they're when they're developing these days, they're actually kicking off little sub-agents to do the work for them. Like and they have like they're almost managing a series of agents and coming back and double-checking their work.

And that that that sort of idea I think is really great where like it's it's I don't even think agents are going to replace big people's jobs. I think they're going to have like people are going to be kicking off these agents to do all sorts of work and that becomes like the same way a manager is one of the more important people in your company, like manager that team, manager statistics. Like everybody's gonna become a manager, like managing all these agents who do all this work.

Pro this is it makes me kind of think of, you know, some of the different conversations we'll have is like proxies for career growth is like how many how much headcount do you do you lead? Like and it's not the best measurement for career growth, but like proxy for career growth, like how many agents can you simultaneously manage or teams of agents?

I I've seen people who are really good at like getting agents to do things, uh really good at managing the the things that come in and out, very good at dividing up the tasks. And and even when you're building these multi-agent systems, like the number one interesting paradigm I think of is nothing to do with AI. It has to do with like how you organize people.

Like if you want people to do a good job of like, do you get them all in a room and then have them all debate? Uh, maybe. Uh, do you separate them out, give them a special task? Do you let them talk to each other? Like when you're setting up a multi-agent system or if you're using these agents when they go, like these are on your mind. I've seen agents get into arguments.

with each other and not accomplish anything. And then you give'em like the there's a funny thing in in when you're building this agencies is like one way to fix it is actually to give them a supervisor, like a legitimate you're a supervisor. You're here to like like it and and like and then that can fix problems. So then and then imagine you get all of these and then you say you're like, How am I gonna get all these things to work together? I've got Giant has to do these five agents.

I got these five other events here. You like I need a director. Like suddenly you're like re made an org chart to do this. And then at some point you realize that that's your job. You're like, I am the CEO of this company who's going to run these little agents. Like and and in and you're like, this is hard. I would not at all be surprised in the next three years if people are teaching um in schools like

how to uh instruct agents. Like it's almost like the way that we talk about prompt engineering today is it's like a A skill to have. I think prompt engineering is, you know, mostly in this old form, kind of like not as important because most models can kind of handle a lot of different things. But context engineering is critical. And then using the agents that other platforms make.

Is becomes like the same way like using Excel or using a PowerPoint well. Like it becomes like a skill that you have. You list on your SMA. I'm very good with X type of agents.

Love it. All right, we've got some rapid fire questions, Ben. If you're ready. Yeah. First one, what are you reading or listening to right now? I uh am interestingly listening to uh a a book that's uh a Harry Potter fanfiction that's called uh Harry Potter and the Methods of Rationality. Um interestingly it is written by Eli Izer uh Udkowski, who is a uh uh a famous AI uh philosopher. But this is this is a world of science.

And of high-end math and physics that are based on the world of Harry Potter. I'd recommend anybody who wants to hpmore.com. It's one of my favorite books. And um it is uh weirdly related to the AI world because of of the author. You were like the second person to recommend that within the last like two days to me. No way. Yeah, 100%. 100%. Somebody said literally said the same thing this weekend to me. I love it. Next question. What is a tool or methodology that's had a big impact on you?

This this idea of little state diagrams with uh powered by AI prompts and tools as a way to solve almost any problem, I continue to re reflect back on that is a very powerful idea. So I'm um sort of on round three of thinking that paradigm, that set of tools that like sort of there's different agent frameworks that do this, but the Landgraph is one of them. Like that style isn't

Do you have any like recommended resources for somebody who wants to like really deep dive into developing that mental model? Like is there is there something you might recommend to help kind of like encode that?

Um yes. Uh uh again, um there's multiple good uh approaches depending on who what level you do. But um uh there's a deep learning class so Andrew Ng uh has uh with the uh Lang Graft team. I think it's about a year old now. Um and it's like a relatively short, but a practical hands on uh set of

set of classes on make an agent, just make one from you know using Python. And then and then and then say, okay, and then now let's use a framework to make it and go through it all. And to me that was incredibly eye-opening because you start to realize

like what is a reflection agent? How does this work? And then also you start to realize like how it doesn't work if you're not if you're not careful and like and then just you think through the different aspects of how agent multi-agent systems, how nodes, how graphs, how this all comes together, uh

So that it's on deeplearning.ai somewhere. Next question. What is a trend you're seeing or following that's interesting or hasn't hit the mainstream yet? I know we've talked a lot about agents, but uh maybe there's a different slice we haven't covered.

The security of tool usage, which is agentically related, but but it it is um something that I think if if people haven't read the Anthropic Misalignment paper that came out, I think it was uh three weeks or four weeks ago, Anthropic Publishes paper about what can agents can do when they go wrong. Like uh this is is incredibly important. You start to realize How things can go very wrong, not just hypothetically, but like in practice, you can see it. You can download their code and run it.

Like there are many challenges of of AI, the things it can do, tool usage is how I'd phrase it, agentic tool usage, where we do not yet have the processes, the procedures, the security tools, the whipped people's enough knowledge so that it would prevent Uh uh at Box we're very worried about this. I'm I'd hope every company who d has a platform is worried about this. But if not, you you'll see some dangerous things that will happen.

That's a that's a great uh a great thing to call out. Last question, Ben. Is there a quote or a mantra you live by or a quote that's resonating with you right now? I'll see if I can if I I don't I I can't attribute it uh properly, but somebody said companies ARR is a trailing um indicator and a future indicator is the developer trend. Yeah, which is basically meaning that like the things that engineers are like kind of focused on at the moment should be an indicator for like sort of

what society is going to do. Now, in some ways, like there's plenty of reasons that wasn't true, but like maybe right now, like because the kind of people that maybe listen to your podcast here are thinking and worrying about these things that many other people are not thinking about. Like they they will be uh deducing and and and and making these trends happen that will then percolate through the rest of of society and the rest of companies.

It it really resonates. It's timely my my wife was just listening to a podcast with Pete Buddjec and he was talking about people are talking about AI, but people are not talking about it enough. And that's coming from like Pete Buttigieg, like like a totally different sector on another side.

And so I I think like that to me reflects like, you know, we have our own kind of communication chamber where we're hearing everybody, you know, talk about this. But like yeah, I think that's a that's a really powerful call out there.

Ben, this has been an absolute blast. Thank you so much for a deep dive and reimagining what's possible for a deep dive into agents and all of the incredible stories. We really appreciate it. Great being here. Thanks for having me. If you're listening to this and you're wondering, how can I connect with other engineering leaders in my city? Pull up your phone right now and go to elc.community, click our chapters page.

You can see that on the menu on the left. Find your local chapter and click join. We're hosting virtual and in-person events all the time. And this is the best way to help you get involved, expand your network in your city, and support your leadership and career growth. So pull up your phone, head to each other.

Dot community, join your local chapter and get involved. A huge thank you to all of our local leaders who make community happen. And thank you for listening to the Engineering Leadership Podcast.