LLMs are becoming more mature and accessible, and many teams are now integrating them into common business practices such as technical support bots, online real-time help, and other knowledge-based related tasks. However, the high cost of maintaining AI teams and operating AI pipelines is becoming apparent. Maxime Armstrong and Yuhan Luo are software engineers at Daxter, which is an open-source platform for orchestrating data and AI pipelines.
They join the show with Sean Falconer to talk about running cost-effective AI pipelines. This episode is hosted by Sean Falconer. Check the show notes for more information on Sean's work and where to find him. Hi, thanks for having us. Yeah, I'm excited to have you here. Why don't we start off with just a brief introduction? Who are you and what do you do? Yeah, my name is Yuhan. I joined Daxter for a few years ago, and now I lead developer success at Daxter.
My teams go with to ensure our users are happy with us and can get their job done effectively through the tools we provide. Before the answer, I am a tech-lea-yelp where I laugh part of the data platform team. I focused on product analytics and its automation. Awesome. Same question over to you, Max. Who are you? What do you do? Yeah, for sure. I'm Max. I'm a software engineer at Daxter.
So working with the developer success team with Yuhan, making sure we provide tools that help developers on board quickly with Daxter. Yeah, and previously I was at Retain AI in the region of San Francisco, where I was working on a customer success platform. I was a data engineer and software engineer over there. Excellent. Yeah, so we're going to be talking about AI and some stuff around cost optimization today.
I think my company is over the last year, so have started to move from just playing with Generate AI to actually building real AI-powered applications. One of the challenges they run into is simply around cost. It's expensive to train or run models. You can easily run out huge bills if you're not careful. Based on what you've seen in your experience, what exactly in the process of building an AI-powered application is actually expensive?
Is this purely the GPU cost or are there other factors that really lead to an explosion and cost? Yeah, that's a good question. I think training is, for sure, a good part of what is the cost in AI, but I think also running the models in terms of, for example, for ELLMs, in terms of the completions and asking all that energy or runtime from the model. In this example, I think we can talk about the observability. This is a key point when we talk about the cost in AI.
Like I said, training is one thing, but then after one, it's done, what do we do in the data pipeline and how can we make sure that we keep track of the costs? You want to do anything to add to that? Yeah, so I would second on what Max said, training and running basically.
In terms of running, it's one is actually running a job running all the moving pieces in a AI pipeline, but at the same time also making sure that you are actually aware of what's going on in pipeline because a lot of there's so many moving pieces. A lot of things could go wrong. So observability is what we've been pushing on in terms of pipelines. What scale does this start to become a problem? Are you when you're building sort of prototyping phase? Is everything manageable?
And then at some certain level things start to potentially become a problem that you need to really pay attention to this? Yeah, this is actually really a question. I actually think the short answer is whenever it's all manageable, the meaning like at a scale where you cannot control all those moving pieces. So I actually think this pattern is very consistent across the traditional software engineering or data engineering and also this new emerging AI domain.
People used to talk about like big data having this problem. Like when they talk about big data, there's like different kind of a visa. For example, like when it's reaching say a terabyte scale, this is curly, large scale, and it will cost problem because the big data is just unmanageable. But nowadays I actually believe the problem is more of in this like big complexity, which comes much earlier than simply growing the volume of the data before.
And with AI, I think it's getting even more complex and comes much earlier in the product development cycle. And as you mentioned, you know, for example, even when your application or your AM model are being developed vocally, you could be touching like so many moving pieces in different systems to say, you know, in this example, in a blog post that we wrote on the prototype that has a support bot via some, you know, LM APIs.
And you might call to say open APIs and you may collect customer data from various sources and then you may clean it. And in the end, you may actually also want to experiment it with different prompts to see the results. And all of these don't have to come with, you know, large volume. But then combining together becomes this complex problem. And in AI, this actually happens, you know, very, very early on.
Yeah, I think that's a really interesting point that you make about how early in the life cycle, this becomes a problem. Because like fun. And in the early stages of building some sort of web application or something like that. And I don't have any users today. Then probably like my relative costs are fairly low to spin up that application and start prototyping and testing and even getting some people like playing around with it.
But the amount of data that I actually need to feed into a model to be useful already supersedes probably the level of data that my simple web application and I kind of like testing is ever going to see for quite some time. So it's not about necessarily scale of users. You have essentially data scale problems from day one, even when you're in sort of that prototype. And actually, here's my hot tick. So I think AI pipelines are just data pipelines.
And all those prompt engineering or fine tuning or primary data engineering. Because exactly as you said, you have enormous amount of data since day one or even day zero. So at a very high level, it is collecting data from certain contacts, from different sources, cleaning, wrangling the data to be reusable and then feeding it back to the models or building tools on top of it. And all of these are data engineering. So in AI to answer the questions like at which scale, it will be a problem.
It's basically like at any scale, it can be a problem. And while we are in this like early in this AI subdomain, we believe that the principle still applies. And we think taking the learnings from the traditional general data or software engineering and apply here, you know, since day one would really be helpful. Yeah, that's interesting. And you mentioned this article that you wrote, which is about managing costs for AI pipelines using a combination of open AI, chain and DAGster.
I want to start there. So could you walk us through sort of the initial setup process for integrating those components and what were some of the things that you were trying to accomplish in the article that you wrote? I think in our example, we had three main steps. The first one was for sure how to handle the sources. So we have like a lot of data. We want to make sure we handle that properly. The second one was to handle Richter search.
So if we want to do some prompt engineering using the data sources, how can we achieve that in a way that it's reusable and that we can, you know, into data pipeline, achieve and use that properly. And then we want to for sure handle completions. In the example, we are using an LLM, we are using chat GPT and open AI. Yeah, GPT three, I would say. So yeah, I think these steps were all components in the data pipeline. So initially we want to handle the sources in DAGster.
We have that concept, which is the data assets or software defined assets. So basically an asset is a piece of data that we have in the pipeline so that we can materialize it and then after we can reuse it into the pipeline. So initially we wanted to represent the sources. So what we did was create a simple asset that would pull the data from GitHub in our case. We wanted to use the documentation that we have. And then from there, we went to creating a Richter search object.
So we use the FICE search index to leverage open AI and lynching. And so from there, it was really about making sure that when we do prompt engineering, we could use this vector search index to go with the completions and the prompt that we were receiving from users. So for example, if a user asked specifically about, let's say, our DBT integration, then we would provide that to the FICE search index. And then from there, we would help the prompt and the answer from the LLM.
So it was very like a simple pipeline, but you make sure all the components were there. Right. Yeah. So you built a rag pipeline using data sources. Primarily is it all GitHub, your own GitHub that you're pulling data from? Yes, and this example, yes, it is. And then DaGster is essentially pulling that in and then creating the embeddings and dumping those into a vector database, in this case, FICE index that you're using to augment the model when someone's creating a prompt.
That's kind of the general set up that we're talking about, right? Yes, exactly. What's the sort of size of the data in reference here? That's a good question. We have several pages that we're pulling. So basically, the sources here are all the pages or the guides that we use in our documentation website. So in terms of the number, I believe in terms of the tokens, it was several hundreds, hundreds, thousands of tokens. But in number of pages, I couldn't say exactly.
But yeah, I think when we pulled that in the data pipeline, eventually it became a problem in terms of how do we compute all that and make sure that the tokens are not, you know, when we train the embeddings, we don't want to retrain all the embeddings all the time when we update one piece of documentation. So that was one of the problems we addressed in that pipeline. But yeah. How does that work? How does the update to the index work? Yeah. So what we did was leveraging another Daxter concept.
So we used a partitions for that specifically. So what we wanted to do is imagine that you're updating one piece of documentation on your documentation website. And you want to make sure that your embeddings are up to date for when you provide that to your AI model. But you don't want to retrain all your embedding. So what we did was splitting the sources based on partitions. So we used static partitions in our case.
So what we did was, for example, in our documentation, we will have a section about integrations. We will have a section about our guides, our tutorials. So we leveraged the partitions for splitting the sources. And then what we did was, if we want to refresh the FICE search index, we just re-build it using what we had before. But we have one part of it based on the new sources.
So really, it was just to make sure that we reduced the cost because if we don't need to rebuild all the embeddings, we shouldn't have to do that. How is pre-processing and chunking of the text handled? In our case, it was very basic what we did. Basically, we split it, I think it was around something like a thousand tokens, the sources, and just we fed that to the FICE search index. I think in terms of pre-processing, we wanted to keep it simple.
Basically, I think in our use case, the documentation website is already formatted. It's not like social media posts that you want to make sure you remove like emojis and different things. So I think in our site was just a matter of feeding a correct amount of tokens when training the embeddings. So it's a very simple use case here where pre-processing is basically chunking by about like a thousand tokens when creating the embeddings.
Yeah, also to add on that, one is we build our documentation based on Markdown. So it's like we have a lot of source files, so those are easy. Another source is that we're using our GitHub and we also tried a little bit SAC. That's like we kind of prototype it in different iterations and the first version is like, let's just dump everything there. And then the cost was pretty high.
And then that we realized actually the highest quality sources we could use with our docs, which basically requires the shortest context window basically. And then it will go to the GitHub. And then in the end, after we kind of start to optimize the cost, we started to drop the SAC. Because we also did some manual work in part of our support. Like during support, we also surfaced out from SAC to GitHub. So it turns out to be like some SAC stuff for like kind of like a diminishing return per se.
So we ended up not using it. But that's also part of the journey. You know, we started really just like tried to dump everything and over time, over a few iterations, we optimized the cost. I see. And then is the cost in your case primarily like related to the input data volume? Is a correlation essentially between input data volume and overall costs? So if you reduce input data volume, then your cost goes down. So I think there's two things.
One is yes, definitely the input data volume and the other is also like how much like fine tuning you would do. Like for example, as Max said, we ended up partitioned stuff by like basically the feature category, a doc category. And in there, we realized the fine tuning becomes a lot lighter. Because they are like within certain contexts, they just not have to you know, provide a ton of context to the model every time.
And I guess the reason why we're so like, erring on cost efficiency is one is like, we definitely care about that. The other is like, we kind of want to try it as a prototype and show it to like our users who's writing AI pipelines to like basically show them and demonstrate how they can build a cost effective pipeline and then not using this as like really simple example. So what were some of the things that you like learned around the cost optimization?
Like what were some of the tweaks that you had to make from the original besides reducing the input data set, but what were some of the other things that you learned to help you control costs? Yeah, I think the main piece of optimization in the specific data pipeline was really about you know, splitting the sources, making sure that we don't rebuild everything. In an initial prototype, we were just you know, pulling all the data sources and we were rebuilding the embeddings every time.
So that was like for sure, the biggest optimization that we did specifically in the end, we also want to make sure that we reuse the same you know, tools often in the data pipeline where we can and we don't rebuild different you know. So for example, if we have like different completion assets in our pipeline, we want to make sure in our case, we use the same search index.
So that would be one thing, but I think in terms of like the completions and the way we are optimizing prompt engineering, we leverage actually the Daxor OpenAI integration that we implemented first in which we captured a metadata, the usage metadata on OpenAI side and we were able to make sure that we would have a look and observe the cost.
I think in our case, it was mostly about the data sources, we want to make sure we're not rebuilding everything, but after in terms of completions, you know, I think it's about what is the template you're using, are you providing enough context to your prompt when you're providing it to the OpenAI model and then from there, can we reduce that also? If you're providing too many examples, maybe it's not relevant.
So using the observability tools that we have like insights in Daxor Plus and just you know making sure what are the results, the completions good enough if we're using less examples and it's yes, could we adjust maybe the prompt template and go from there?
A slow QA process is bottleneck you or your software engineering team and you're releasing slower because of it, you need to check out qa wolf dot com slash s e d q a wolf gets your engineering teams to 80% automated and then test coverage and helps them ship two
times faster by reducing q a cycles from hours to minutes with over 105 star reviews on g2 and customer testimonials from sales off, drata, auto trader and many more, you're in good hands, go to qa wolf dot com slash s e d to see if they can help you squash the q a bottleneck. Yes, can you walk me through like that process like how are you making decisions about, you know, what is enough in the prompt and how does the observability tool help you make those decisions? Yeah, it's a good question.
I think so like I said before, in this example, we are using Daxor Plus insights, which is a tool where you see several metrics in your Daxor Pipeline and when using the Daxor Open AI integration, the open AI client will capture the metadata usage and then we will see it over there.
And after I think it's a matter of just testing your pipeline, making sure that you have like a data set that you can test against, making sure that your completions are accurate based on what is the initial question coming from the user. Then after I think it's a matter of comparing exactly like if I go to insights, I could create two different completion assets and these completion assets, one could be using like one shot inference and the other one could use multiple shots inference.
And then from there, we compare like the two sets, the two sets of completions and against a test data set like this is mostly how we use that. And then we can see like what is the difference in terms of like accuracy that we will have in the completions based on the number of examples we've provided to the problems.
So I think it's a matter of traditional AI or machine learning testing, just making sure what is your accuracy based on the data set and then from there also use the observability tools to see what are the costs for that example. Are you measuring the accuracy against the test set? I think in this example, we could use different tools specifically.
I think we will look at what is the content and what is the answer like in the completion, like is the maybe the topics are related to the original question. But for sure, I think for this example, we build the data set in terms of like what would be in the documentation. So we test against like what is the proper answer in documentation. Yeah, I would say it's more like quality on. So is it the manual test then? Yes, well, two folds to the manual test, one as internally before rolling it out.
Well, before putting in front of users, we do like some spot check, but at the same time we also put in front of users and when user sentiments, when we know the results good. And then in terms of like how do you handle errors or anomalies during the pipeline execution as you were starting to ingest these different sources? Did you run in any issues around like essentially parsing and chunking and preprocessing or did everything work? Absolutely. That's like the life of data engineer, right?
You always got errors or like anomalies that you don't even control. So that's actually a big reason why we started building this pipeline with the answer. It's like you kind of got the native, you know, the locks from each wrong. It gives you just like a very comprehensive on computer locks, goes into every single wrong. And you also break it down to like multiple sources, you show a basic attack, like a graph that have all the sources coming into one, you know, operation and stuff like that.
So whenever a box is read, we know what's happening and then we can go there. So that's like troubleshooting stuff and usually it's like troubleshooting the like my user error. Another interesting thing when we go on prod is like there is also a non-metetaction, right? There's also like unexpected upstream errors which don't necessarily like do anything with us or sometimes don't even necessarily need us to fail the wrongs.
And in Daxler we have this concept of like data quality track and it's called Daxler tracks. So like the idea of track is like you don't necessarily need to fail a pipeline because something may just be like unexpected like some different format but we still want the pipeline to, you know, wrong. So like we'll just say tracks files, but wrongseed you need to pick stretch tension to that, you know, and then we go into and go through the whole troubleshooting cycle.
Yeah, because those are also potential like cost centers, right? Like if you have a lot of errors and anomalies that you have to deal with or your pipelines like breaking all the time just for sort of normal engineering things, then you have to put a person on that stuff that's like. Yeah, totally. Yeah. Like when people talk about cost efficiency, cost optimization, we just talk about, you know, the bills are a fine as you receive, but actually it's also like a lot of, you know, human capital.
Yeah. Go into that. Especially right now where all these tools are quite new and a lot of people are just kind of figuring out you're cobbling together a combination often of like stuff that you built, some open source tooling that you've pulled from somewhere and maybe some, you know, existing proprietary technology that you're paying for. Right.
So that's why all we always felt like, you know, oxygen should not be, or like a control plan should not be the last piece you think of actually like we do the opposite. It's like the day one we think of bringing up tool that can manage a complexity. Therefore, you know, whenever something fails, we know exactly what's wrong. Instead of, you know, going through like tons of different sources and waste like days on troubleshooting all those.
I guess another thing I want to echo what Max said is about the insights. So insights is actually essentially this like you know, element in our on clouding sense and it actually shows you like charts over time. So you know, tracks failure, some cost tracking that if certainly there is a dip or certain there is a bump that we can easily capture those and those also made our life easier. How did you manage like the model deployment and versioning? Yeah, that's also a good question.
We are all kind of declarative. So everything are defining code. So you know, they're essentially all just versioned out of box through get. And another thing we really liked ourselves and as the speech or call branch appointments. So basically it's like we said it so every get how PR goes around to a branch, of course. And then every branch gets pushed to a kind of sandboxing like environment to doctor plus. So then before you push to prod.
So the idea is like you go from local and then you go to branch appointments and that's the basically the staging environment, but it's an informal environment. So it's like when you're you could think it as a sandbox. So it's like you can do whatever with that and it does not touch your prod environment. And it gives you basically this like integration or a small task. A small small tasks workflow. So before you hit go live go prod you are actually confident that the model is deployed.
You know, properly and automobing pieces are like brain. Is there a way to roll that back if you have to? Right. Yeah. You know, I'll just get comments. You just revert to commit and things would be back. No, I meant the model itself. Oh yeah. We currently do not have the model versioning. We do save the model as like like the intermediate step automatically. So whatever you want to get the previous data is somewhere in our s3 bucket. Okay. Got it. I see.
We're going to take a quick break to talk about BitOrden, our sponsor of today's episode. Are you among the 65% of developers who still hard code secrets and source code? Storing machine and infrastructure secrets in code, unencrypted ENV files or messaging apps can make your business more vulnerable to leak secrets and data breaches. BitWord and Secrets Manager offers a super straightforward solution to this problem.
It prevents secrets leaks by making it easy to manage and deploy machine and infrastructure secrets all from one secure location. BitWord is unique because it's open source and to end encrypted and can be easily deployed into your existing environments with a robust CLI SDKs and out of the box integrations like Kubernetes GitHub and Ansible. Start a free trial today at bitworn.com slash secrets.
And then one of the things that you you started off talking about was how you had this, you were hot take about how AI pipelines are not that much different than like a traditional data engineering pipeline.
But are there unique challenges that come from building these kind of, you know, either it's a, you know, Ragnap model or pipeline or Gen AI model pipeline, whatever it is, are there unique parts of that that you've encountered that are different than maybe what you've traditionally encountered in building a, you know, a data engineering pipeline that's going to drop data down into like a warehouse or something like that. That's a really good question.
I was actually almost on the parser's similar, but the interesting part that's like the learning curve and like the familiarity with all sorts of like, like different tools. Max can probably speak more about it, but to my experience is like, actually, you know, we also used to mention when building this in Lenschen really helped us making like model comparison, you know, like switching between model and also like the learning barrier for like switching between models.
So I was like, that's like a bit component to it because like in traditional data engineering, it's usually like if you're using snowflake, you're using snowflake, you know, and single is just like all the same. Whereas for those like newly AI tools all kind of perform differently. So we have to learn every single stuff. And Lenschen has a component here really helped us to make things like modular and composable and you know, the switch costs becomes pretty low. Yeah, it's interesting.
Like if you compare, you know, what you need to do, let's say you're building some AI application, you're going to use a model underneath the hood to, you know, do some AI magic. Like there's pretty big variance in terms of the quality of responses for the specific use case, which is very different than like you're hitting a database and you're running a SQL query.
Like maybe there's some performance differences at certain scale and stuff like that, but it's still the same SQL statements going to generate the same response in data is there. And you kind of poke around and say like the execution plan is just like it's all kind of deterministic. We're asking it's such a black box. So that's one deterministic. Yeah. Do you see much in gains in terms of cost by changing models? Actually, yes.
So originally we actually had this prototype really early on when the GPT-3 came out. And when we were freshest pipeline with our open AI integration, the GPT-4 came out. And around the time, after we released this plot post, the four all came out. So every time we would just try one, two, a parameter and change and see the cost. So the cost from three to four went dramatically down. And it's also interesting that the kind of quality of sentiment analysis also went up.
So all those AI companies are definitely doing stuff and it seems really promising to me in both the cost of fixing way and also making our job easier. You may not know this, but I'm curious to hear your thoughts on. So I could run essentially, you know, take an open source model and run it on my own instances versus calling something like open AI's APIs where they're going to run the stuff as a managed service behind the scenes. Do you think that there's cost savings to be gained?
I mean, obviously I'm taking on a lot of responsibility to try to manage the infrastructure to run the open source model and scale it. But in the early maybe pros typing phase where performance is less of an issue, is that a way to potentially keep costs low while I'm going through the prototyping phase? Meaning like open source models? Yeah, essentially run an open source model on my own like GPU cluster or even CPU cluster.
I guess it's going to be slow, but I could do it in order to just like test and iterate. And then once I get to a place where I'm happy, you know, then I can do something that's more scalable where the cost goes up. That's exactly why I mentioned them. Then chances like it's just like one or two prime changing and go to. Right, right, right. Yeah. So was that an iteration that you went through? I think we probably have compared Loma, I forget. But like we did compare like a few other ones.
So it's not like a initial product high phase. It's not like cost was not a big concern. So like we just like went straight to the AI. In terms of your experience like building some of these AI powered applications, you know, we talked a little bit about how the tool chain is a little bit. And certainly immature in comparison to other, even though like modern data stack or something like that, we're cobbling a bunch of tools.
We already see the biggest gaps right now based on your experience where that need to be addressed for people to be able to build more sort of production level AI applications. To me, I would say it's the matter of tracking. So it's like I don't necessarily want to treat autos models as, you know, black box. For example, the very first step forward in the past year was like this usage tracking or being exposed more metadata about each API cost.
So then this gives us basically this like platform who wants to interact with with those models to have the leverage say we let the model users to handle the business logic themselves. But let us take care of those like pipeline or API calls like metadata. And what's lacking in my mind is like trying to expose more, you know, for those problems to explore more like interesting metadata. Any thoughts on that, Max? Yeah, I would second you on on this.
I think one of the technical challenges we had when building that pipeline specifically was to make sure that when using Lanching, we would be able to capture the open AI metadata. So I think when using new tools like all of these are very powerful, we just want to make sure we can use them together properly, efficiently. And then we can keep track of the cost.
I think in this use case, we created beforehand the DAX or open AI integration in which we did kind of like a wrapper around the open AI client enabling us to capture the metadata. So basically the challenge for us was to make sure we could use that with Lanching. It was a good thing because Lanching was really to use that specifically.
But then it was a matter of like making sure how can I use like one of these tools and make sure I can bring it in my data pipeline and leverage as much as I can because in production, I want to make sure I can keep track of what's going on. So yeah, so in our case, we were able to plug the open AI client that we implemented the wrapper and was good to go with them.
But yeah, I think making sure we can leverage these tools, quite new tools and make sure we can bring them with the observability what that we want for a production data pipeline. Digital marketers, this one's for you. I've got 30 seconds to tell you about WIC Studio, the web platform for agencies and enterprises. So here are a few things you can do in 30 seconds or less when you manage projects on WIC Studio.
Work in sync with your team on one canvas, reuse templates, widgets and sections across sites, create a client kit for seamless handovers, and leverage best in class SEO defaults across all your WIC sites. Times up but the list keeps going. Step into WIC Studio to see more. We're going on the vector side, so you're using fast index for your embedding index.
But is there a trade-off with that of relying on that versus relying on something like a more fully-fledged vector database or operating with such a small enough data set that there isn't necessarily a visible performance improvement? Yeah, I think it would be the latter for this data pipeline. I think, like I said before, it was kind of like a few hundred thousands tokens for this example. So I think it was good enough to just go with FICE, it was fast enough.
And quite frankly, the cost of just going with something else was maybe too big for this use case. So going with that solution was just the best to do. But I agreed that if the input data was much greater, it would be interesting to test different approaches in terms of the embedding and how can we make the most of different approaches and how can we use the costs for that? What's next with this project? Is this kind of wrapped up? Or are you going to be continued to build and iterate on this?
Yeah, the project itself is wrapped up, and we do want to expand, you know, as we write more features. So we kind of use this as an example pipeline to demonstrate a dog food. The auto capabilities that we have with other AI tools.
So later on, we want to, you know, want to leverage our new data catalog to add more storage sources, like trying to add Slack in and also our support tickets and also build it on top of like the data freshness and reliability to deal with like bad or like spammy sources. So these are the things.
But you know, at the same time on we're also exploring, we're actually using, you know, other like AI support tools at the same time because they do better job there than they ask making it as like our part time projects. How do you see like the integration of, you know, dagster with Langchain and OpenAI sort of continuing to evolve? Yeah. So as I mentioned, I think the composability is actually a really big key part.
We on the extra side, we aim to be this composable single pane of glass for all your data. And why I cannot really speak deeply for an engine, but given my experience, I think it really makes the process of building AI apps, mostly via those like modular components. So when I went in with the engine similar to the engine, I think composability and it boils down to like one is like incremental adoption.
So it's like if we can move certain pipe certain stuff to and it's not, we don't want a belief we end, you know, other tools like first of all, like we are a huge believer in like incremental value delivery and we don't want our tool to be adopted as all or nothing. We will meet users where they are. So for example, like if in the future those tools does have a capability to like execute a model or like, you know, wrong stuff.
Then we would instead of executing instead of all in dagster, we would want to offer like this term called observe, which means like you can listen to those activities in external platforms. So that, you know, for example, dagster would know the option dependencies, for example, in the engine or other platform, but still be able to manage the things that depend on within the answer, but depend on external sources. But as in time, you know, we don't have to necessarily kick off those jobs.
And that's like a big component on when we think of integration. The other components are the second one would be like, and this phrase caught like learn once, write anywhere. So Lenzhen actually does a really good job as like it makes switching model really easy and we want to do it the same way. It's like sometimes users may want different tools in the same category or even switch between them. And we want our integration with AI to be like that.
We want our pan to be consistent enough to lower those switch barriers basically essentially like get users to gather job down quickly enough without learning, you know, new things. What needs to happen in order for that to occur? So part of it will be like in our integration abstraction, we would build like the same pan and same abstraction for different tools within the same category. Throughout the past year, we wrote out this tool called this like patent or concept code pipes.
That's essentially a tool, a protocol built on top of our like very basic building blocks. And basically it's like, if you want to integrate with external execution platform, that is the way to go. And then to our users, the APIs will just look the same. Great. So let's start to wrap up, is there anything else that you'd like to share? Let's see. There's an interesting thought recently in like data in AI that I may want to share.
So you know, we recently rolled out this blog post of called Rise of the Medium Code. So you know, AI is a big topic and it will continue to be in a future. And many people think that AI will completely kind of eliminate those like human, written software creation. But in our lens, and I guess this also kind of comes back to this whole integration. Sorry. So it's like we would love that humans are still remaining in the driver's seat when it comes to building software.
And we caught this kind of emerging class medium code. So we believe those authors are essentially the ones who own the business logics. The LMS and AI's cannot really author those projects like unsupervised. And we believe these jobs are not replaceable, which is also a key reason why we want our users to focus on like writing the business topics and leave all those added, you know, metadata moving pieces for us to handle.
I thought there's like some interesting conversation going around after we kind of start to think and explore that part. Yeah. I mean, I think that it's difficult to make almost any definitive statement about where things are going to be in the future right now because things are moving so quickly. And we always tend to sort of us, I think, underestimate in the long term and, you know, overestimate in the short term.
I think like one of the interesting things that's happening right now is if you had showed most tools that existed in the Gen A, I space to anybody five years ago, they'd be like blown away by the magic of it. But after just a year of chat, GPT, we're all like disgruntled and think that it's like terrible and consistently like criticize the output that comes from and forget the like utility and value that it's generated. Right. Yeah, I was about to say like five years is a huge honor statement.
It's like a year. Yeah. Absolutely. Well, Johan and Max, thanks so much for being here. This was great. Very enjoyable. It was great to learn from your firsthand experience, you know, how some of the things that you pulled away from this project of building, you know, AI pipeline and some of the things that you were able to do around optimizing costs. Thanks so much. Yeah. Thanks for having us. Let's be here.