Louis Knight-Webb from Bloop.ai - the YC startup turning COBOL into Java - podcast episode cover

Louis Knight-Webb from Bloop.ai - the YC startup turning COBOL into Java

Jan 02, 202546 minEp. 117
--:--
--:--
Listen in podcast apps:

Episode description

Louis Knight-Webb is the CEO and co-founder of Bloop.

Bloop helps with modernizing legacy software, particularly focusing on COBOL and mainframes.

This episode is brought to you by WorkOS. If you're thinking about selling to enterprise customers, WorkOS can help you add enterprise features like Single Sign On and audit logs.

Takeaways:
- Mainframes and COBOL are still foundational in many industries.
- Bloop started with a focus on code search but evolved to address legacy code modernization.
- The transition from COBOL to Java is a significant challenge for many enterprises.
- Innovative approaches are needed to effectively translate legacy code.
- Ensuring code quality during migration is crucial to avoid operational disruptions.
- AI can enhance the code translation process but has limitations with legacy languages.

Links:
- Louis Knight-Webb
- Bloop 

Chapters:
00:00 The Legacy of Mainframes and COBOL
03:05 The Evolution of Bloop and Code Search
05:58 Challenges in Modernizing Legacy Code
08:48 Navigating the Enterprise Code Landscape
12:11 The Transition from COBOL to Java
15:05 Innovative Approaches to Code Translation
18:02 Ensuring Code Quality and Functionality
20:56 The Future of Development and AI Integration
23:52 Building Relationships in the Enterprise Space
26:45 The Long-Term Vision for Legacy Code Modernization

Transcript

Jack BridgerJack Bridger

Know what a mainframe is? Or have you ever written COBOL? Maybe not.

Louis Knight-Webb

Everything that exists around software development has a clone in the mainframe space. It's like Mario and Wario, you know. Okay. It's a parallel universe.

Jack BridgerJack Bridger

We're all really excited about new technology like Bun and Svelte, but a lot of the software that dictates our lives like airline bookings and bank transactions actually runs on mainframes using COBOL. And my friend, Louis, is the founder of Bloop, which is a startup that is completely dedicated to modernizing all this legacy software that no one understands how to build anymore. I feel like the best one's often, like, where they're like, should I say this? Yeah. Alright.

Let me show you this or whatever. I don't think

Louis Knight-Webb

I should pull this idea up.

Jack BridgerJack Bridger

But Yeah. Yeah. They do it anyway. Yeah. Yeah. You know, something's good. Something good's gonna come from that.

Louis Knight-Webb

Yeah. What

Jack BridgerJack Bridger

what was the best talk ever, like, in terms of the ratings? Do you have do you know?

Louis Knight-Webb

Oh, I love them all. Okay.

Jack BridgerJack Bridger

No. Not your favorite.

Louis Knight-Webb

Yeah. Fine.

Jack BridgerJack Bridger

Statistically because you you just said you have a quantitative method.

Louis Knight-Webb

I mean, firstly, I don't actually know because I haven't I haven't really been looking out for that. I'd say, you know, things like, you know, former founders, so Guy, Pujani, who, you know, and, Jonas Templestein, we've had previously as well. So, you know, Jonas previously cofounded Monzo, and Guy obviously previously cofounded Snyk. So those kinds of talks, I think, where they span not just the tech, but also entrepreneurialism, Being in London is a is a common theme as well, you know, that gets people excited. And I think just generally, you know, it's like a huge vote of confidence when you're you know, most of the people probably graduated in the last 5 years that attend these events, and they turn up and they see somebody who has built a company worth 1,000,000,000 of dollars basically doing a similar thing to them and is huge validation.

And, you know, people really open up as well. I think we do have some some pretty good discussions. And so those probably get the highest ratings of all former founders.

Jack BridgerJack Bridger

Yeah. Yeah. Because and I guess it's just the the least BS, I guess, as well. It's just like the if you're a founder, you can say whatever you want. Yeah.

Louis Knight-Webb

So They're rich. Yeah. Like, they can do what they want. Yeah. They can say what they want, and, they don't have to worry too much in that environment either because nothing's recorded. Everybody's an engineer. Like, nobody's trying to do a gotcha or anything like that. It's, yeah, good discussion.

Jack BridgerJack Bridger

Yeah. And maybe that's a good segue to talk about that you are the founder of Blue. And to illustrate that you can do what you want, you have brought me a bottle of Yeah. Dorset knob, which is displayed for people that, are on the video.

Louis Knight-Webb

Well, we're meant to do this in Dorset. So in the absence of doing this in Dorset, we've got a slice of Dorset in South London today with us.

Jack BridgerJack Bridger

Well, what is Dorset? Where is Dorset for for our American and and UK business?

Louis Knight-Webb

Dorset Dorset is is is is, what is it, a county? It's county, isn't

Jack BridgerJack Bridger

it? County.

Louis Knight-Webb

County in the south of England, basically stretches along a piece of, coastline called the Jurassic Coast, which is very famous for fossils. And, they found lots of interesting sea monsters there recently. They, you know, won recently in a in this really cool BBC documentary that they called the Sea Rex, which I highly recommend going and seeing Oh, I see. Down in Cambridge in the museum there. And then it goes up a bit, but I love Dorset because, I quite like sailing, which is a kind of odd pastime and, the waters of of of Dorset have lots of nice little natural harbors, which are not too deep and you can, you know, go sailing on a small boat and capsize and stand up and you don't drown, which is nice.

Jack BridgerJack Bridger

Yeah. As Paul Harbor is like the second is it like the second biggest?

Louis Knight-Webb

It's not the biggest or the 2nd biggest natural harbor in the world. So there's a lot of dinghy sailing, a lot of like kite boarding, things like that there. Yeah. We were supposed

Jack BridgerJack Bridger

to be wearing sailor hats today, but Next time. Yeah. Same times. And Bloop is, doing something very different now to what it was when you started out.

Louis Knight-Webb

Yeah. Well, we started in a very different world in 2021. You know, there was no GitHub Copilot. I think, you know, the biggest open source model at the time was GPT 2. And, you know, our you know, I initially meeting my cofounder, Gabriel, who's a machine learning researcher by background.

My background is more as a kinda classic software developer building, like, websites, mobile apps, things like that, was how do we solve the problem of code understanding? And at the time, you know, you had to, like, raw dog Versus code. Right? You're, like, typing. Yeah.

Yeah. Everything you wanted, you copy and paste from Stack Overflow or you type. Yeah. Difficult to remember that world now. So, you know, the the biggest way to accelerate that workflow for us was to create a really good search engine for code.

So we downloaded GitHub. I think I don't think I I don't think they sue people that say that. So we downloaded GitHub. We chunked it up into I think it's like 256 token chunks. We trained a, we fine tuned a a small embedding model on code, and the result was this Versus code extension where you could type in anything like, how do I make Stripe payment?

And it would find you the most, you know, relevant code from GitHub, Asanshi to do that. And with that idea, we, we actually formed the company, part our first engineers, raised the pre seed, got into Y Combinator. And about 1 month into Y Combinator, GitHub Copilot came out. It was just I mean, it's just a better product ultimately, serving, users with open source code cause it had just been trained on all the world's open source code and, you know, you could personalize what you, you know, what you wanted, what what it would produce, to a much greater degree than what we could produce, which was search. And so the results were static.

They weren't generated by a model and they were much more difficult to personalize. But we kinda persisted with this idea for a bit and we, moved from open source code search into enterprise code search. So how do you help engineers within extremely large companies with huge code bases talk to each other and kind of share code in a better way? So if I'm an engineer at a huge company, I can type, how do we do payments? And it'll show me, oh, well, you use Worldpay, and here's the last senior engineer to implement that, and here's, like, the documentation they produce, all the rest of it.

So kind of permutation on it. And, you know, that was relevant because, obviously, in an enterprise setting, the code base is updating very rapidly. And so, you know, training, or fine tuning models on enterprise code is actually really inefficient. It's really expensive to do, and you need like a huge pipeline training pipeline to do that. So, using a search based approach, you know, made much more sense for that use case.

And that kind of marked the next year of the company's life, I'd say. So this is, you know, maybe a few months out of YC until maybe like a year after that. And, we we built a great product. We moved from the IDE out into a kinda standalone application where people could do code search. And I think, you know, right around the kind of the chat gpt moment, which is probably 6 months later, 9 months later, we started seeing a lot of competition in this space.

And I think, you know, there's maybe something natural about code search that if you're a developer and, like, suddenly something like the GPT 3.5 API comes out, like, what's the first thing you try and build? That's basically code search. So we had, we had, like, you know, 3 competitors probably in in Y Combinator alone, within a space of 2 batches as a bunch of open source tools. At the same time, GitHub Copilot eventually moved from just co completion into being in the sidebar as well with the chat experience, which is similar to what we were doing. And this was a problem in commercial discussions.

Right? We're talking to enterprises. We made a few sales, but in in some really significant sales that would have, you know, been really meaningful revenue for us, we just had consistent feedback that, you know, we didn't really have enough differentiation versus what they were seeing from GitHub and the bundle. Right? Yeah.

Because, you know, GitHub is not just code search. It's 40 other products in the bundle. Yeah. And if you're trying to sell, just one of those features, even if you believe that, your feature is is better than what they've done, it's it's quite difficult to cut through. Yeah.

And at the same time you know, and this is a challenge. Some people have have, done a really good job of this, but it is hard to prove, the productivity benefits of a Copilot. There are there are companies that, you know, do a great job of this. I think we maybe didn't do a great job of this. And so, you know, in articulating that value, we just maybe didn't have enough in the way of hard concrete evidence that if somebody bought code search, they would immediately save, you know, x dollars, in a way where I think if you look at the GitHub marketing page today, there's like all kinds of like white papers and research, things like that that they've invested in to really communicate the value of those tools.

So that's the world up until the kind of the modern blue. So

Jack BridgerJack Bridger

This episode is brought to you by WorkOS. If you're building a dev tool, at some point, your customers are gonna start asking you for enterprise features. WorkOS offers you single sign on, skin provisioning, and audit logs out the box. WorkOS is trusted by Perplexity and Vercel as well as Workbrew, a home brew management startup that I recently interviewed. I just told Mike that WorkOS is the sponsor and this is what Mike said.

Mike McQuaid

Yeah. So WorkOS isn't paying me any money for this. I I pay WorkOS money for this, but WorkOS is like one of the best developer tools I've, like, ever used. It's it's the documentation and the experience with building with them is so so good. Like, I initially was almost like, okay.

This seems expensive, but then I built an integration with them in about 20 minutes. So I had spent 2 days banging my head off the wall trying to build it directly with Okta. And then with WorkOS, I then have, like, many, many SSO providers, like, support instead of just one. So yeah. Like, for me, WorkOS is one of the nicest developer experiences I've encountered in the last, like, 5 years probably.

And it's so surprising because a bunch of the developer team are Execute Hub and therefore are very good at the job.

Jack BridgerJack Bridger

Go to workos.com to learn more. That I think, like, that Microsoft, it seems like they have really tried to create the, you know, the Office Suite for developers and and you're competing against that. And if they have it I mean, even we saw with, like, Office, it's like the Slack challenges that they've had even after all of that adoption that, like, Teams is really taking a lot of their, I think, a lot of their their share.

Louis Knight-Webb

Yeah. 100%. And, you know, distribution is really key. You know, they're coming out of the gate. It doesn't matter if they they ship a feature a year later because they've already got the list of customers and the renegotiation, you know, they can throw that extra feature into the bundle.

Right? Yeah. So we're always gonna have to reach, you know, a kind of escape velocity or die with that kind of product. And, I think, you know, we could have continued down that path, but at the end of the day, we started having some really positive discussions around legacy code with some of the customers that we'd been talking to for the original code search product. And, you know, it wasn't like we wanted to shut down code search and we started looking for something else.

It was like we saw another opportunity. We did the equation and we said, well, that's a more exciting timeline. I think we'll jump to that actually.

Jack BridgerJack Bridger

Yeah. And what was like the 1st week of, you know, jumping to that as you put up?

Louis Knight-Webb

The fur well, it was it was kind of a few conversations and then a reflection on those. So I think the very first thing we learned about was, spending some time in in San Francisco and going to some dev events and just bumping into, some really, you know, high up kind of tech leadership people in in very big branding companies and listening to them just talk about the challenge of of legacy code. And the first couple of conversations were legacy in the context of JavaScript. Like, they were considering JavaScript and I think Flow.

Jack BridgerJack Bridger

Oh, the Yeah. TypeScript.

Louis Knight-Webb

Exactly. It's not enough. To be their legacy and what they wanted to move to was TypeScript in their case. Yeah. And they were forecasting it was gonna take, you know, a decade of developer time to port from flow to TypeScript because the type system cannot be mapped 1 to 1 between all of the different flow types.

Jack BridgerJack Bridger

When you say, sorry, a decade, do you mean like

Louis Knight-Webb

One developer for 1 developer. Working solidly for 10 years.

Jack BridgerJack Bridger

For 10 developers for 1 year sort of thing.

Louis Knight-Webb

The same thing. Yeah.

Jack BridgerJack Bridger

Yeah. Okay. But they they were actually thinking time wise. It's literally not gonna be done until, like No.

Louis Knight-Webb

I think they would have got a squad on it. But in terms of cost, it would have been about 10 years of a developer's time to solve that problem because a human needed to, you know, manually reason about, well, what do we do about this flow type? What's the correct translation into TypeScript? So we thought about modern language translation for a little bit, and we started talking about this in the context of some of the customer calls we're still having about code search. We're dropping in well, you know, what about, like, legacy code modernization?

And at some point, somebody turned around to us and said, oh, cool. Yeah. We got like a 1,000,000,000 lines of COBOL in production. Like, can it do that? And we're like, what's to be honest with you, like, what's what's this COBOL thing?

Kinda heard of it, and, you know, maybe everybody's got a kinda older relative who's who who worked on it or something like that. But it was it was it was a completely new environment because the mainframe and, you know, for those in the audience that don't know, you know, the mainframe, historically was this kind of workhorse of of an architecture. Originally, you know, the very early, you know, business computation was done on mainframes, think like fifties, sixties with languages like COBOL, which, originally were designed to be very readable. You know, you read a COBOL file and it's very different from reading a a modern, programming language because it's it's laid out almost like a kind of technical, English, or like maybe like a a kind of an extension of a maths equation in a way. Mhmm.

But, you know, a lot of these companies have been rolling this stuff into production for decades. And so, you know, people just come along and

Mike McQuaid

they build and they build and they build and, you

Louis Knight-Webb

know, you wake up one day and you've got a 1,000,000,000 lines of cobalt in production. And it's so hard for those companies to move away from that because they've just accumulated it. And, there aren't really, you know, a lot of good options. I mean, using humans take a very long time to convert. Right?

You have to read the legacy code and then manually write it up into Java, you know, whatever you're you're moving to. People have been trying that recently, but augmenting themselves with things things like GitHub Copilot, ChatGPT. Mhmm. But the measured gains of that are, you know, kind of 20%. So, you know, you're only accelerating that process a little bit.

The other options are kinda traditional approaches that don't use any AI like code transpiration. Yeah. So these are taking your COBOL and doing a line by line conversion into a modern language like Java.

Jack BridgerJack Bridger

Which will just create absolute carnage.

Louis Knight-Webb

Well, yes. I mean, I don't know if there's any examples of, like, machine generated code that people like, but I assure you people do not like machine generated Java. It reads like, you know, and and there's huge differences between these languages. And so, you know, you're not taking full advantage of what the Java language can do, can, you know, represent things properly as, you know, all your entities as nice objects. You're not necessarily representing COBOL objects in clean Java objects because the, the the types are different in both of those languages.

The standard library in COBOL is is quite small compared to Java. So for example, if you want to format a date in COBOL, you have to write a program to format a date. Whereas in Java, you just use one line. Right? Java standard date formatter.

So, you know, these are all opportunities that that we kind of, identified. You know, can we do something that's faster than humans and produces code that's better than, automatic transpilation. Mhmm. And, you know, this is something that there's a few different approaches to, to solve. You know, the the most obvious one is, like, what happens if you just copy some COBOL, put it into chat gpt, and say, generate Java.

Jack BridgerJack Bridger

Yeah.

Louis Knight-Webb

And just copy and paste the the output back. Right? Now the problem with this and the problem with any task in involving COBOL is that there's basically no COBOL in open source. And if you think about how the very biggest models have been trained, they've been trained on GitHub and open source. And that's why they're really good at modern programming languages because there's so much Java and JavaScript and whatever in open source.

So what that means is that the same task performed on COBOL and Java will be much less accurate in COBOL than it is in Java. Mhmm. So this this approach was immediately written off, right from the outset by us. And we we actually ran some really interesting experiments to try and generate synthetic COBOL and fine tune, like, the kind of the big open source models on more COBOL and things like that to try and boost their performance. But, you know, there's just like, what we're able to achieve is like a drop in the ocean from what is what is needed to truly solve the problem using that approach.

So, so the system we ended up building was was kind of something that morphs, the old approaches with, with something new. So we we opted to go down the route of actually statically transpiling COBOL into Java, kinda similar to what I was talking about before, those machine generated code systems. Yeah. But we optimize the transpiler to produce code that is kind of chunked and structured in a way where it's quite easy for us to then, run an LLM over the Java, the machine generated Java

Jack BridgerJack Bridger

To make it.

Louis Knight-Webb

And refactor

Jack BridgerJack Bridger

it Readable.

Louis Knight-Webb

The readable, modern, performant, all the rest of it. So, you know, there's 2 halves to our system. We've got transpiler, and we've got, agentic refactoring.

Jack BridgerJack Bridger

And does it go like in is it like in waves where you run like, okay. Let's use the let's do the date for like, you were mentioning about, like, using the standard libraries and stuff. Okay. I'm guessing you're just doing it, like, let's do that, and then let's do this other thing, and then let's do this other thing.

Louis Knight-Webb

Yeah. So well, so, you know, these code bases are enormous, like millions of lines. And, you know, the context of, the best models will be, you know, a 100 k tokens. We've done quite a lot of work to understand how much of that context is actually usable. And so the the reality is, like, you know, at 30 k tokens in the prompt, you start to see a lot of performance degradation.

And in terms of output, you know, these models realistically can only output a few 100 lines of coverage before they start spewing garbage. So, you know, we gotta work around that. That's kind of the big limitation, and that means that we have to adopt an incremental approach. Right? Which is, you know, how do we make many smaller targeted refactors to this code base Yeah.

And kinda merge these different changes in a way that, you know, the the the end product is the the modern code base. And we the the structure is a bit like that, like what you described. So, you know, we have a kind of specialized refactoring pass for dealing with the standard library problem. Mhmm. So, you know, large language models, very good at spotting patterns. They can spot these 300 lines of Java that are kind of machine generated looking, actually just format a date

Jack BridgerJack Bridger

Yeah.

Louis Knight-Webb

And just replace that with Yeah. You know, the the standard date full matter. But, you know, there's so many permutations. Right? Because every implementation of formatted date in COBOL has been done by hand by a COBOL engineer. Yeah. So every code base has got a different version of formatting a date. Yeah. So this is something That's what. That the LLMs are really good at.

It's like taking, you know, a hundred examples with site permutations of the same thing as botting that can be, you know, all distilled to the same ultimate, Java standard library function.

Jack BridgerJack Bridger

And how does it work if there's kind of like sort of like Jank in the code where it's like they're using a different date format there. They're using a different date format there, and maybe they shouldn't be. It's like, do you sort of correct it or or is it like you kind of, it's the same. It's gonna have the same problems. But

Louis Knight-Webb

Yeah. So duplication is a huge one, and we do a lot of deduplication on the code basis. You you can literally reduce these code bases, you know, like, at least by, you know, 50%, in terms of the the size of what you started with. A lot of the type so, you know, literally these code bases have been started in the seventies

Mike McQuaid

a lot of the time Yeah.

Louis Knight-Webb

And developers developing them today are not the same that originally started off. And actually, they're often in maintenance mode. So often there's nobody on the team who knows the code base in a lot of detail. Mhmm. So if they're implementing any changes, there is a bias to write something new as opposed to reusing an existing program or module. So we do this is something that is really common as we see, lots of duplication of of functionality across the code base.

Jack BridgerJack Bridger

Mhmm. That makes sense. And I think you told me that the the people that do understand it are, like, quite expensive.

Louis Knight-Webb

So yeah. So well, okay. So we could talk about drivers a bit. So the talent is the the reason that everything, you know, everything kinda stems from. So, most of your listeners probably will not be seriously considering COBOL as a career path.

Jack BridgerJack Bridger

Yeah.

Louis Knight-Webb

And that's, you know, that's true of of most graduates joining the industry today. And so what that means is many more people retire every year with COBOL skills than join the market. And that means if you're, operating COBOL application and you need to make changes because a lot of these businesses are regulated and regulation forces changes. Or if you're in a traditional industry that is facing competition from I know you're a bank and then there's neobanks popping up, you are stuffed because you can't get enough engineering resource to make those changes. All you've got is enough engineering resource to keep the lights on.

So what are you gonna do? Just kind of roll over and let the competition ship, you know, features that potentially are encouraging your customers to to leave, or are you going to get into a, a framework and a, you know, an architecture and, that allows you to be really agile and and move quickly? So it's, you know, there is a a cost to talent because as the number of developer shrinks, obviously, the the cost of each developer is gonna go up. But the bigger cost is the is the competitive disadvantage of being stuck on a platform that doesn't really allow you to move very quickly.

Jack BridgerJack Bridger

Yeah. And do they so I I I wanna ask, like, more about the actual, like, getting going to market, but one last question on the like, do they run the Java on the mainframe once they've changed it, or are they also, like, moving?

Louis Knight-Webb

Yeah. So this is kind of a hot topic at the moment, which is like running Java on the mainframe. And for many years, you know, you could run Java on the mainframe, but it was like super old versions of Java. And and quite recently, they brought, you know, the newer versions of of Java onto the mainframe, to encourage more people to run Java on the mainframe.

Jack BridgerJack Bridger

Yeah.

Louis Knight-Webb

But I have really yet to find a single customer to reach out to us that is interested in running Java on a mainframe. I mean, the way the way I think about this, you take the the circle of people who understand mainframe, you take the circle of people who understand Java, and, like, you know, the intersection of those two things is even smaller than either.

Jack BridgerJack Bridger

Okay.

Louis Knight-Webb

So, you know, if you're trying to solve a talent problem, ultimately, you're not solving a talent problem by running JavaScript. Also real

Jack BridgerJack Bridger

new question. I actually don't think I really understand what a mainframe is. Yeah. What what is actually what is a mainframe?

Louis Knight-Webb

Well, it's a it's it's a it's an architecture. It's a it's a computing platform that is, you know, distinct from, you know, what we know, in terms of, like, x 86 ox architecture or anything like that. So it's got its own, chips. You know, it's got its own operating systems. It's got its own languages, IDEs, protocols for connecting to screens.

Everything that exists in the world as we know it around software development Mhmm. Has a clone in the mainframe space. Okay. It's like Mario and Wario. You

Jack BridgerJack Bridger

know? Okay. It's a parallel Yeah. Sort of universe.

Louis Knight-Webb

Exactly. A parallel universe.

Jack BridgerJack Bridger

Okay. That makes more sense. Okay. And so what what is it looking like when you go and talk to companies? Like, what does it look like to start working with them?

Louis Knight-Webb

So typically, people come to us today when they know they are moving their code from Cobalt to Java. Yeah. So we don't spend too much time persuading them to do that because there's enough people for us to kinda work with at the moment who are already somewhere in that process.

Jack BridgerJack Bridger

Yeah.

Louis Knight-Webb

We try and get, like, a chunk of their code to production as quickly as possible. Mhmm. So these code base is pretty big, and the key is to answer as many hard questions as you can as as early as possible because there's a risk if you try and take the entire code base from a to b in one step. You'll just die from, you know, overcomplication and then these things, you know, these huge projects can can run out of control. So we we take a small chunk of code and we, isolate it from the rest of the system, and we move it into Java, and then we work with the client to deploy it into, the cloud.

Mhmm. And we try and choose a a cluster that is kinda representative of what we'll see in the larger system. So, you know, where it touches things outside of the COBOL, like, I don't know, you know, writing a file or Yeah. Making a network request or accessing a database, things like that. We try and make sure that we're Proof.

Yeah. Exactly. Exactly. Because otherwise, you know, you end up at the end of that pilot project, and the customer is like, well, how long is it gonna take us to do the rest of it? Mhmm.

And you're like, well, I don't know. So, you know, the the pilot is really an opportunity to properly spec the rest of the the project. And the other reason why it's important to do that is because often the customer has no idea what they're running either. They've got like one engineer maybe sometimes who knows COBOL who keeps the lights on, makes very occasional small changes or fixes. And so it's very difficult to kind of have a conversation about, you know, how long is this gonna take?

How much is it gonna cost? So what we do is we just kinda put this into a fixed price package that we ensure the customer against. Well, okay. Well, if it takes us longer, it takes us longer. We'll eat it. And that allows them to engage with us. We just get in there. We do the analysis to find the cluster to, to convert the code, and we do all of our learning during the pilot, basically.

Jack BridgerJack Bridger

Okay. That that makes sense. And so you're basically just offering them you're gonna be able to convert this. You you're gonna do something. Or or the I guess at the beginning, they're kind of taking a bet that this is probably gonna work, and it's we're only betting this amount. Yeah. Yeah. And if it works, great, because we can actually get this done. Yeah. But worst case is like this amount of money.

Louis Knight-Webb

Yeah. Exactly. And for an enterprise, you know, it's like it's peanuts for them to run an experiment. You know, we have yet to see somebody not go on from a pilot to do a full project. They tend to be quite successful, and in comparison, it's a much quicker and often cheaper alternative to

Mike McQuaid

what

Louis Knight-Webb

they would otherwise consider doing with humans Yeah. Or, you know, the, you know, and it's it's kind of not really comparable to automatic translation because, you know, people don't want code that they can't read at the end because they're maintainable.

Jack BridgerJack Bridger

Yeah. I'm just imagining, like, how do they know if it was, like, successful? Because I I guess you mentioned, like, they don't even necessarily know what this code is doing.

Louis Knight-Webb

Yeah. So we typically define a kind of evaluation matrix ahead of time. So we say to the client, what is important to you in terms of code quality? And they can tell us, well, it's really important that, you know, the, we're using this particular open source library. Right? Because they will have existing Java developers and Java applications, and, you know, they can personalize the output to their coding style.

Jack BridgerJack Bridger

Okay. Cool.

Louis Knight-Webb

And we can say, okay. Well, how important is it that each database table is represented cleanly as a, you know, as an entity and, you know, or they can say, oh, well, it's really important that we end up in Java Spring Boot. Mhmm. Something like that, like a particular framework. And so, you know, the the the the key to, to to having a a happy customer at the end is not to do the project, show them the code, and then be like, are you happy?

The the key is to say, before you start the work, what would make you very happy? And then for them to really granularly break that down, like, to write down what happiness is.

Jack BridgerJack Bridger

And then

Louis Knight-Webb

at the end, it's it's kind of like, you know, it's like a math question. It's like yes, no, yes, no.

Jack BridgerJack Bridger

Yeah. Okay. That makes sense. And then but do they have, like, tests and stuff? Do they usually do they typically have tests written in Cobalt?

Louis Knight-Webb

No. No? Okay. Well, we haven't seen one yet. Okay.

Jack BridgerJack Bridger

But because I get what you're saying about, like, the evaluation of, like, if the the code quality and that, like, you know, it uses libraries they want to use and stuff. But Yeah. How do they know if it actually, like, does the same thing that the Cobalt does?

Louis Knight-Webb

Yeah. So that's, a huge topic of interest to us, obviously, is making sure that when we're moving these very critical applications for the biggest companies that we don't break anything along the way. And there's plenty of examples of companies that have actually received pretty substantial fines for getting this wrong Yeah. In the process of migrating their tech.

Jack BridgerJack Bridger

Because we could be talking about finance, like banks and stuff like that.

Louis Knight-Webb

Yeah. 100%. Like, you know, the regulator sees that suddenly, you switch over your system and half your customers can't log in to their online banking account the next day, you're gonna get fined 1,000,000 of of dollars. So it it's it's it's really important that we demonstrate, that the code performs the same after conversion as it does before. So, you know, how do we do this if there's no tests to begin with?

And by the way, the reason why there's no test is because often these programs have been in production for so long that they're just kind of assumed to be working. And people are quite unafraid to, you know, quite afraid to to touch them.

Jack BridgerJack Bridger

Yeah. Yeah.

Louis Knight-Webb

So what you know, we got a mixture of techniques. I mean, you know, technically, we're we're using a lot of the the classic techniques for, generating tests, but, you know, where the customer can provide us with data, that's very helpful because we can use historic inputs and outputs to create Okay. Cool. For the code when it's a batch system. For online systems, which are kinda more like I mean, when you go to, check-in at the desk Yeah.

You know, for your flight and, you know, there's a kind of green text screen that somebody, some agent is looking at, that's that's what we call an online program, and we can record sessions of that Yeah. In order to generate, tests as well, and use a bit of robotic process automation as well, to kind of automate some of those flows. And then there's new techniques as well. So we're actually using language models to generate tests on top of the Java. So once we we kinda do the first step transpiration, into Java, we can then bolster the test that we produced originally with additional LLM generated tests.

Mhmm. And that's, really important because, you know, when we come to do the refactoring, which does use large language models under the hood, you know, obviously, large language models, make mistakes. Yeah. And so they will cause unit test fail failures at some point. So it's important that we have as many tests covering as many cases as possible. Yeah.

Jack BridgerJack Bridger

That makes sense. This seems very it's like it's very interesting. It's like it seems so much harder in some ways, but then probably you have the benefit is that not many people would be mad enough to try and Yeah. Help solve such a hard problem.

Louis Knight-Webb

Yes. Yeah. So it is it is a funny one. It's it's kinda mad. It's also slightly detached from, I think, the the crazy hype of, like, coding agent stuff at the moment in a funny way.

So none of the customers that come to us, considering working with, any of the other, like, you know, coding, generic coding agents that are available. And it's because they can't because, of the the fundamental problem that, you know, there isn't really COBOL in the training data at these models, and they're not very good at at COBOL. So using a pure LN based approach doesn't work. So you have to be if you wanna enter this space, you have to be prepared to build a transpiler, and that's a huge undertaking. There's so many hedge cases.

The documentation sucks. There's different versions of COBOL. You know, I'm obviously very excited about it, but I feel quite secure for the moment anyway that, you know, you know, we we have a niche that is quite well insulated from the the wider, excitement around agents because of that limitation that is very, you know, specific to legacy code.

Jack BridgerJack Bridger

Yeah. And what what is the vision as well? Because I I guess you people ask you this. It's like, once you've once you've fixed all the COBOL in the world.

Louis Knight-Webb

Well, then we convert all their Java to somebody else. Right?

Jack BridgerJack Bridger

Okay. Yeah. Yeah. Yeah. That that makes sense. There's always gonna be people that are, like, modernizing.

Louis Knight-Webb

There's a lot of COBOL. I mean, that's the first thing. Right? There's, like, billions and billions of lines of COBOL out there. So it'll take us a very long time to convert all of that. And at the same time, there's, you know, successive waves of technology. I mean, flow was new not that long ago.

Jack BridgerJack Bridger

Yeah. That's true.

Louis Knight-Webb

Yeah. Now it's something that somebody's trying to buy a solution to convert away from. So Yeah. You know, these things, I I don't think, you know, I don't think I see programming languages, converging on a single one anytime soon. So as long as that remains true, there's always gonna be, you know, work to be done.

We're also you know, there's a whole, like, maintenance life cycle around these applications. The work doesn't stop once we've converted it. They have to continue to be updated and, move on to new versions of Java. There's opportunities to, build new functionality as well. So I I'd see COBOL to Java, the actual translation as, like, a really killer wedge into the enterprise, And that's our way to break in to prove to enormous companies that are very risk averse that we can solve really difficult problems

Mike McQuaid

Yeah. And then build

Louis Knight-Webb

on that relationship to see what else, you know, what other problems we can solve.

Jack BridgerJack Bridger

That makes sense. So it's not actually just like a one like, it's not just like, we're gonna migrate this thing and then that's done. It's like a it the it's a start. It's the beginning of a relationship.

Louis Knight-Webb

Yeah. Definitely. Even though today, like, our only product is

Jack BridgerJack Bridger

the code

Louis Knight-Webb

translation part. Yeah. But it's, you know, it's valuable enough that we can build a business just out of that. Mhmm. And the rest of, you know, the the other products that we can build, you know, this doesn't have to turn into, like, product 1 of 40 products that Bloop offers for this to be a big opportunity.

Like, there is so much legacy code out there. It makes sense. But that in itself is a hard problem, but I think we'd be crazy not to leverage those relationships and that reputation that we're building with the enterprise

Jack BridgerJack Bridger

Yeah.

Louis Knight-Webb

To not, you know, then help solve other problems

Jack BridgerJack Bridger

Yeah.

Louis Knight-Webb

That they also face.

Jack BridgerJack Bridger

Yeah. That makes sense. Maybe my question is kind of, like, naive in a sense that it's like when you hit just because you aren't faced with like, I am not faced with, like, COBOL every day and, like, mainframes. It's, like, easy to, like, compartmentalize something that, like, you're gonna be able to just, like, fix all of this Cobalt and, like, you know, and then it's it's done. So that that makes a lot of sense.

I wanted to ask you as so I think we're coming towards the end. But, like, I wanted to ask you because you see a lot of AI, stuff in particular. I know you've even written, like, your own, like, benchmarking, your needle in the haystack.

Louis Knight-Webb

Need multiple needles in the haystack. Needles in the haystack. Sorry.

Jack BridgerJack Bridger

Your multiple needles in haystack, benchmark. What dev tools are you excited about at the moment besides bloop? Dev tools am I excited about? Especially, I mean, AI ones. They've gotta be AI. Yeah. Yeah. Yeah. For for you.

Louis Knight-Webb

I think okay. This is gonna be a really self serving answer. Basically, I think dev tools are best at tasks where there is it is it is possible to deterministically evaluate the result. So what I mean by that is where you could basically run a language model infinitely and test its output deterministically to tell whether it has succeeded or failed. Yeah.

Because the real limitation of a lot of the, the the kind of the LN based products that I see on the market today that a human is actually still in the loop of them. Yeah. So a classic example is, like, you use ChatGPT to generate a website, you know, generate HTML for website, yada yada yada. And then, you know, how do you know if it's done the right thing? Well, a human has to sit there and actually evaluate.

Mhmm. You know, is that, is that button in the right place, the right color? And you know what? The the human is the slow part. And so you can't really run like these long chain, chained, you know, calls to LMs because, you know, they're constantly being interrupted by by human doing something.

So, you know, there's kind of a wave of of products now. I mean, it's difficult to be specific because very few of these are actually on the market, but, you know, maybe maybe if I can take your question and twist it into, you know, like, where are things gonna go after the Yeah. What's the next big thing that we're gonna see in the, I don't know, 12 month time horizon maybe that I'm most excited about. It's the ability to kind of spin up an L. M.

To go off and do some coding task where it can mark its own homework, and it can deterministically say when it's finished or not. Because those you'll be able to run for much longer than anything where a human is in the loop. Yeah.

Jack BridgerJack Bridger

And I guess, like, the final question maybe is like, do you think that what what do you think development is gonna be like in, say, 10 years' time?

Louis Knight-Webb

10 years' time. You know, it's funny. I think we wrote a our first pitch deck, we had this kind of, you know, like, evolution of mankind type Yeah. Photo with eventually writing code using natural language somewhere up right at the end at 10 years, and it's happened in 2 years. So, I mean, my record for predicting what's gonna happen in 10 years is is very poor and is probably an underestimate is what I've learned.

But, yeah, I mean, I think I think there will be 2 economies. Right? There will be the world in which new things are built and start ups and individuals, you know, will operate, and that could be anything. I mean, really, it could be, you know, very high level languages, could be things that are enabled through voice. You know, if we figure out like a computer brain interface, then you don't even have to be limited by the words per minute that you can type at.

I mean, there's so, you know, there's so much opportunity there. But then there's the other economy which is like all the software that has been built to date, which still needs to be understood and maintained and and the rest of it. And, you know, I think there's a risk that, you know, we're not gonna be able to take advantage of all of the crazy stuff that's gonna happen in the new economy with all that old stuff. Yeah. And so, you know, I guess that's like a very convenient way of, saying that, you know, that's the best you know, that's the the the reason why we need to be converting all this old code.

Jack BridgerJack Bridger

Yeah. I thought it was coming back to that. It was like it's the analogy of, like, there's all this cloud computing stuff. Yeah. But everyone's still the COBOL is locking people into Exactly. On the mainframes. It's

Louis Knight-Webb

The world's gonna be crazy in 10 years time. The the the ways in which, you know, software is written, you know, is gonna be so much more efficient. But if you're stuck, COBOL. So this is the good to action.

Jack BridgerJack Bridger

You're gonna be feeling like a Dorset knob.

Louis Knight-Webb

You're gonna be feeling like a Dorset knob. So full circle.

Jack BridgerJack Bridger

Louie, thank you very much. I think that was, that was great fun, and very, very interesting. Where can people learn more about if someone's got a load of cobalt cobalt, how can they reach out to you?

Louis Knight-Webb

Yeah. I'd be amazed if you're like the the I don't I don't know. Yeah. I don't wanna undersell the podcast. Maybe you've got, like, a huge cobalt fan base. If you have

Jack BridgerJack Bridger

a friend who's struggling with a cobalt problem.

Louis Knight-Webb

If you have, like, a great aunt or uncle who he's, like, working with COBOL and really wants to retire, at bloop.ai, please, you know, reach out to us. We're happy to have a chat. And, obviously, you know, we see a lot of people going through these migrations, and we could tell you about what we've seen that works and, what doesn't work. And, you know, often, it's not a case of actually jumping right into a modernization project. It's just building a relationship, understanding, you know, what AI really brings to the modernization space.

And, you know, who knows, in a few years' time, when the, you know, the the business decision or whatever is made to actually move away from the mainframe once and for all, you will remember that conversation that we had and, have that in your back pocket to accelerate your organization. So

Jack BridgerJack Bridger

yeah. Amazing. Well, thanks, Louie, and thanks, everyone, for listening.

Louis Knight-Webb

Thanks so much.

Transcript source: Provided by creator in RSS feed: download file