AI in Software Development with Roy Derks

Hi everyone, my name is Patrick Akio, and if you're interested in AI and software development, this episode is for you. We go over the core concepts and evolution of prompt engineering, rag retrieval, augmented generation, as well as GPT agents and how you as software engineer can leverage all of that and more. Joining me today, Friends of the show, Roy Dereks, entrepreneur at heart and always building products for developers. So enjoy.

One of my recent thoughts was more so on the full stack

development. And also with a lot of questions that I get is if you were to start your career all over again and either be self-taught or kind of transition into this career of software engineering, how would you start? Would you go kind of full stack from the start? Would you focus on front end or back end? Or would you try and kind of jump on this AI bandwagon and try and develop more so AI specific applications? How? How would you start from the

start again? That's also a very good question that I'd like to answer with. It depends, but I'm not going to do it this time. But now. So for me, I'm self-taught. I always had, for me always programming started with a use case. So I wanted to build something and that's why I learned programming. So I guess that's an important

starting point for myself. But maybe for others, if you're going to university or to high school and you know, I want to build stuff, I don't necessarily know what I want to build, then it will be good to find a starting point that's maybe either full stack or front end or back end. And there it really comes. Well, I would say start with a

full stack project. It would probably be a good thing because then you build a front end, you build a back end, and you're going to find out like what excites you most so you can continue from there or maybe go back end or go front end. You don't think it's a lot kind of to start off with full stack immediately? Well, it also depends what's your starting point. There is a, there are a lot of these JavaScript courses that will teach you basic JavaScript. So that's probably a better

starting point. But if you look at the world of where our self thought developers at least are coming from, they go to YouTube, they go to random websites, find tutorials and Start learning. So if you're on that sort of bandwagon and you watch a video like a four hour live coding video, and I have some friends who are these YouTube influencers and they do these 4 hour live stream builds and ready to take you from zero to a full stack app.

Insane. Yeah. And those are pretty pretty much focused on new developers. There's some JavaScript in there, but they'll walk you through it. So that might be a good starting point, like going from there. But it also means you need to be somewhat of a self starter if you need someone to hold your hand. And well, it might also be good, but there's some self learning there as well. Yeah. And then for AII, think it's

completely different subject. So a lot of the things I do now are like helping developers to build AI applications. But I wouldn't say that will be a starting point. If you have a use case and you say I might want to become an AI engineer and I want to build some app that does what Jet GPT does, then starting with building a full stack app could be a better starting point than starting to build AI apps.

Because there are so many paradigm you need to understand before you can actually get started. Yeah, interesting. I was wondering because for me,

I'm always kind of similar to you. I built something when I have a use case. And for me it's easy because I I'm not an entrepreneur. I'm in an organization and especially within a consultancy, I come into a project and there's already either a predefined goal or a kind of early and defining goals. But then I have use cases, right? I don't have to come up with my own use cases. I can and there's probably adjacent to whatever we're

trying to do in any case. But to come up with use cases from scratch would be way harder. And then when I'm looking at the technologies, I would always do kind of what is required and go extra based on interests. But I've noticed myself not going very, very deep. For example, I can use LLMS and applications, but I I don't have this urge to figure out how LLMS work under the hood. And I think I can still be productive and effective without

that knowledge specifically. But I've also talked to other engineers and they want to know everything there is to know before they kind of build on use cases or they actually want to understand something completely before they use it. And I'm not like that. How's that for you? Yeah. So for me, I usually start with a use case. So I try to build something, I try to get it work and then once it works, I'm going to figure out how it actually works. There are people that.

So if you take the use case of putting together something from IKEA, I will take the manual. I would look at the bits and pieces and I will just start doing it. And then some way I figure out, oh, I did something wrong or I need to go back or this wasn't supposed to go there. And then there's other people, which I I watched like putting

together some from IKEA. They look at the manual, they go through it a couple of times, they inspect all the different parts there are, then they're going to try and see like where would it fit and only then they will start putting it together. So those are probably the people that want to know how LLMS actually work before they even start to do something. For me, it's building something and then figuring out how to break it or how to iterate or how to make it better.

So yeah, generally speaking, I don't really care about how something works. I just want to make sure it works and then once it works and it fits my use case, I'm going to try and specialize in this area and figure out how to improve. Yeah, I've, I've been involved

most of my career in projects that have already gone, let's say from zero to one are already live and I'm there to make it more effective or build on top of it with features. There's one project where I actually worked from zero to 1, and I feel like that experience is actually quite different, which I didn't expect initially. You're actually starting out from scratch and you have a lot of options and it's really hard if you haven't done it multiple times before.

I feel like to not fall into this analysis paralysis, especially when it's more so for a bigger organization where people love coming up with solutions and future use cases which are not really valid yet and mostly assumptions. In any case. How do you also, from your own experience, kind of scope it down and make it tangible within this use case and not meander into a different path? Yeah, that's also a very good question.

So I'm used to either starting my own startup or work at startups and help them get somewhere. And usually there's a lot of choices you need to make and more often they're made by a small group of developers or maybe if it's a really small start up by ACEO or by ACEO. And they're going to tell you, we need to go this direction and we're going to use these technologies because these are the technologies we are familiar with.

And it's probably fine. I mean, if there is someone in your organization that will help you to sort of figure out what are the five key ingredients you need in order to build a successful application within your own own boundaries, that can already help. So I'd say always try and find someone that sort of has opinions about what you should or shouldn't use because someone needs to be accountable at some

point. If you go with a certain technology and then five months into the project you're going to figure out it has all these limitations, then well, it could be good, could be bad. I mean, you probably have learned a lot, but if you're on a consultancy project and you need to deliver something, then it might not be the best thing. If you're a startup, you can actually make these mistakes and you'll learn and get better and you improve.

But if you work on projects for clients and it will be harder to to do this. So myself, I don't really like to work in a project where all the boundaries are already been set, because then it's just work. You're executing on whatever is in there. There might be small improvements you can make, but it's not what excites me and I can easily see.

I mean, I know a lot of developers around me, they get excited by taking something really small and making it even smaller or faster or optimizing, optimizing it. For me, it's never really the case because I also always want to build for the end user. I want to make sure the end user has a better experience. If I would optimize one tiny thing and the end user doesn't see it, to me it doesn't feel like I've done something. Yeah.

No fulfillment. Yeah, I mean you might have 200 lines of code edit or deleted or whatever, but the end user still sees the same website so or same application. So I don't really I don't see the benefit in that. If you are in a growing organization, if there are requirements from clients, then you need to do these kind of things. If you work in start-ups, which is there for my preference, you don't have to care about this because the only thing you care about is the end user.

Are they using your product? Are they buying your product? Are they talking to friends or colleagues about your project? So yeah, that's maybe not really the answer to your question, but. It is.

I, I mean, it made me reflect on where I've usually had most fun. And I think actually going from zero to 1 to a point where a developer can be productive for me, that is still a bit of a pain because I'm tying things together. I'm setting everything up and trying to make sure that this way of working within the new stack or whatever stack we're building is going to be optimized for.

And I've really enjoyed coming into a project where that's been thought of and I can still obviously tweak with past experiences, but I'm productive kind of from the get go. And then there's still a lot of stuff to do. I've never been in that spot where you're talking about where probably you do a lot of AB testing and you do minor changes with big impact just by virtue of the skill that you have as

organization. But yeah, as you lay it out, I'm, I'm not sure if I would enjoy that as much. I definitely like kind of this AB testing approach and building a proof of concept really quickly testing it, and if it doesn't work, just throwing it away. I have no problem throwing stuff away. But yeah, if that's the only thing you do, you do kind of go into this micro optimization and you have to find fulfilment in that, otherwise it's going to be tough.

With regards to AI use cases that you've seen, where do you think this is going? Because so far from what I've seen, there's a lot of manual work that can be replaced. People that analyse PDF documents and put in information into a system for example as one of the use cases or having kind of this chat bot or guide experience where people explain either language or concept within a overarching tool. What are some of the use cases you've seen?

So I think the use case you describe are a lot of things. You see, it's people trying to automate workflows, which could be, like you said, someone generating PDFs or analyzing them and putting them into a different system. It could also be extending existing capabilities. So if you look at, I think Microsoft, Google, and even we had IBM, we had our big developer conferences or product conferences this week or last week.

Yeah, every product that was announced somewhere has some new AI capability where probably not much is happening in terms of what the LLM is doing. It's probably pretty, pretty straightforward in most use cases, but everyone is trying to put AI into somewhere. Yeah, I always like to look at memes, and there is all these memes about product managers telling their team to do something with AI but not really knowing what. So those are interesting use cases. And I think it's also, if you

look at start-ups that apply to Y Combinator, like two years ago, all of them were like jet GPT wrappers, meaning that the only thing they would do is giving you some assistance where you can ask a question and then it will probably just use a model from open AI to answer your question and it doesn't do anything more. So when you look at these LLMSA, lot of the start-ups, you see, a lot of the use cases, you see they are doing prompt engineering.

And probably everyone knows the term prompt engineering. So this is where you sort of rephrase the question you asked to the LM in order to get better results. And you can do a lot of things in there. You can set the temperature, like how creative should it get, or you can give specific instructions. You can give examples of things you want to see, say if you want to get results in a specific order or in a specific format. You can ask all these different questions.

And it's very basic prompt engineering, which all the LMS are able to use. So that's one thing, and that's what most of the companies are doing today and it's probably good enough for most use cases. You don't think that's a bad thing? That it's just a wrapper around something existing. Well, it's getting worse because people get higher expectations. I mean, when JGPT first started, we get mediocre results and we were like, oh wow, we get

mediocre results today. You're saying it's these are really bad results because another LM is able to give me better results or I'm using this tool which is a stand alone software service I need to purchase. It does better and it probably does better because they do slightly more than just interacting with the LLM based on prompt engineering. They make it context aware and

there's many ways to make LLMS context aware. So this is where a lot of things are shifting today and there's quite a lot of things happening there already. You maybe heard the term rag rag. So this is retrieval augmented generation and this is where you would include a database and usually these are factor databases that use semantic search. So semantic search isn't new, it's something we've been using all along. Like most search engines will

use semantic search. It is where you put in a couple of words and then it's going to find matches to those words based on a score. So this is semantic search. It doesn't just do a filtering of the database like give me everything that includes this word in the beginning or the end. Is can do way more, but it isn't smart. But it is able to give you relevant entries from a database.

So with RAG, what you would do, you would take your documents or your data, whatever however it's structured, you would split these into smaller pieces of text that you can insert into a factor database. And then you would have a factor database that might have a million lines of or a million rows actually, because it's a database with pieces of your

data. And then whenever you create your prompt that goes to the LLM, first you're going to retrieve some data from the factor database that's relevant context. And thereby you can extend the knowledge of the LLM from all the basic data that's trained on with your own company specific data. And then this is like very simple RAG because you give this context and then there's a million patterns within RAG that evolves different names and different orders.

And sometimes you want to re rank the data that's coming from the factor database or you want to do keyboard matching there as well instead of just semantic search. There's a million different variations you can do there. And this is only just within the Rack concept. There's more concepts within AI that get even more complex. So this is really where it's going, but it doesn't necessarily mean that you need to do all these things in order

to be effective. I, I always like those, those projects where you would see that someone really made was really thinking about how to structure a project. You probably have seen those as well. There's like multiple layers with an F in directory. And then if you look at reactor front end app, the components are splitted into really small components that all are reusable and have testing. This is how you can set up a front end project. Doesn't mean you have to do it.

Also build from the project that has 20 files, 20 files of 1000 lines of code. It might not be the best way, but it might get the job done. So it's the same in this AI

space where you can do LMS with prompt engineering. You can look at RAG where you make LMS context aware and then there's a new pattern which we can talk about as well, which I think is more of the future than than RAG itself. What's this new pattern you're talking about? So this is really what we call agents. You might have seen this. I think Jet GPT or at least open AI in our platform have introduced some way to start

building agents. So agents are slightly different from LLMS with prompt engineering or LLMS that are context aware using things like RAG. With agents, you would have an LLM that has a set of tools or skills. So one of those tools might be and if you're using Jet GPT, you'll notice one of those tools might be search the Internet. Or one of those tools might be make a call to my Salesforce or my CRM or my SAP or whatever you're using to store customer data.

Make a call to those services and extract data. If you look at the agents, you'll have an LM that has this set of tools that are ways to reach the outside world. And even REC could be a tool in this. So RAC could be a tool like go to my factor database and collect results based on all those millions of PDFs that I uploaded there. So this agent is able to look at all the tools and skills. There's sort of two words for IT. People either like to call it

tools or skills. And it's also the nice thing about AI. We're coming up with words for things and one company might think one word is better than the other. And then it always takes some time in order to figure out what conclusion will be and what will be the exact phrasing and terminology. So you will have an agent which has a set of tools or skills at

its disposal. And then you ask a question like, hey, can you get me all the packages from can you get me all the orders and tricking numbers for today? So maybe you're an e-commerce company, you want to extract like all this information because you want to know how many products that we sell and shipped. So the LLM wouldn't get this question. And then based on your question, it will look at its skills and

tools. One of it might be go to my postal provider and find out what packages we ship today. Another might be go to my database and figure out what orders have been placed. And then it also needs to take this information and put it in a coherent way. So it will come up with a nicely format answer to your question. So agents can do slightly more than RAC, which just depends on

a factor database. They're able to come up with a plan to collect all the data that needs to answer your question and then structure this in a good way. Because sometimes it's sequential.

So it first needs to go to System A and then to System B, and then the other time it might need to go to System B first before it goes to system A. So instead of having to go all of this by yourself, where you need to say go to system A, collect this, go to system B, collect this, which we do as programmers, we try to write these sequences of things

because we know the sequence. With the agents, you don't necessarily know the sequence, or you don't need to know the sequence, because the agent is able to figure out just by looking at natural language, which tools or skills that you use to answer your question. Interesting.

I mean, if it's, if I look at how it's grown, it started indeed with these more so command prompts and engineers would use it to make themselves more effective, right? To test assumptions more quickly, to gather more information on what needed to be done sometimes to generate a right answer as well and then implement it. And then now that it's moving, it's more context aware. Plus it can actually do part of the integration work on its own even without kind of the steps in between.

Sequentially. You just say this is what I need the orders of today and then from which system and then the slicing of today. It would handle it by itself. Which means either engineers are going to be more productive or how I see engineering nowadays is a lot of work is actually

integration work. There's not too much business logic here and there, and even if there is business logic, people are very mindful of it. It's usually very well tested and kind of isolated in one place within an application. Now that the integration work is also then kind of done more so automagically in some cases. How do you see the role of engineers changing with the kind of component of AI more and

more? Yes, if I look at myself I I use things like GitHub Copilot or like coding assistance in general. They've for example helped me to generate tests from my code. These aren't the best tests but they will do things that would took me 1020 or 30 minutes to do it myself. Like a boilerplate. Yeah, like a bottle plate or can actually look at your function and come up with some testing scenarios.

So let's say you can get a 80% testing coverage by having Copilot generate this for you in a minute instead of you having to write this in an hour. Yeah, that's pretty good. That's pretty good. So if you look at coding in general, it's part of it is just work. It's you know what you need to do, you can do it. It's just going to take you time could be hours, could be days, could be weeks. You know you can do it. There's no new new aspect to it's just execution.

And then there is maybe, let's say it's 20% of your job is actually thinking through things, coming up with a new solution to a problem or coming up with some integration between different components or different libraries that doesn't exist yet. And I'm saying 20% of your job might be like this. I hope it's a bit more, but let's say you're delivering a consultancy project for a client, then probably it's like 20 or 10%. If you were going to start up, it might be 80%.

And only 20% is work work. So let's assume this 20% of your job is the only thing you need to do, and the other 80% can be automated. This can mean two things. It either means we need less developers because a lot of work can be automated. It could also mean that we only have time to do the 20% in 20% of our time. But what if you have 100% to do? This 20% could also mean if we get better solutions, we get more creative integrations between different libraries or different systems.

So I don't think we need less developers, but developers that we have. They will spend more of their time on creating innovation rather than just doing the work that needed to be done. But there was no other way than to just do it manually. Yeah, I like that thought and I

think I'm hopeful that it's going to go that way. But also in, in reality, sometimes where times are tough, people look at costs and if we have people that all of a sudden get more productive, more effective, but then there's still the same amount of people, either we keep those same amount of people or we cut costs. And then sometimes layoffs happen. I don't know if you see that nowadays. I don't think AI plays a big role in kind of the layoff sequence that we've seen so far.

But organizations are definitely cutting out the the extra fat, as they call it, and try and be more lean. And definitely also engineering is sometimes impacted. It's very hard to see, OK, where does that come from? And I think if your organization is huge, it's also like what we discussed before, it's really hard to find things that actually impact an organization with either its mission or its values and its goals. So then, yeah, you might find yourself working on this micro component.

And then the organization just decides, OK, this is not impactful anymore. And then the layoff happens. I wonder if it's going to happen more and more than as people get more productive with this role of AI. And also an interesting thought, which I'm curious to hear your opinion on, is kind of this education gap, right? Because you and I, we've played around with software for a long, long time, then AI came so we can leverage previous

experience. For example, I had my partner, she was trying to figure out something to do with JavaScript, and she had GPT generated something. But she's like, yeah, it's not working. And I don't know why it's not working. I don't know which component. I don't know what it does. It's trying to explain it to me. I don't really want to know in the 1st place. Can you help? And then, yeah, I, I did figure it out, but that's basically

based on previous experience. Plus I know exactly what to then change and where it is. But new people will not have that same amount of experience. They will have to find a different way to get kind of this gut feeling of where things are wrong or where the right kind of building blocks fit in together. It's gonna be more difficult, I think. Yeah. So it's two things. It's the the first thing you said about AI maybe taking away jobs.

I always feel AI is kind of cheap right now because all the different providers, they're trying to get companies to use their models. And at some point the price will increase and companies will only start to automate away things when it gets less expensive to automate. And I think it will be a while before most development jobs can be automated away. Maybe some of the more administrative jobs can be automated. But if you, if you look at, and I like this analogy of like

shrimps. So shrimps need to be peeled and there are some machines that can do it, but the machines are really expensive. That's why a lot of companies, they will ship their frozen shrimps to like low cost countries to get them peeled and then ship them back. Even though there are machines, this is still going on because it's cheaper to have people do it. It could be that those people are underpaid than they probably are, but still people are less expensive than machines in that

sense. For AII think it will be like that for a while as well. So the bigger companies might be able to automate and have less people because they have AI, but the smaller companies for them it's probably cheaper to still have people do it. So there will be, it will take some time before we get to a point where machines will take over because they get less expensive where it is something

you see in different sectors. So that's maybe a positive point, like make sure you stay relevant and it will be hard to automate your job. But even though if it's easy to automate your job, then I also wonder like how excited do people get by this job that's so easily being automated? Yeah, it should be kind of maybe more mundane and repetitive in in any case then. Yeah, but also, I mean, you need to, you need to work in order to get money, in order to survive.

So in that sense, I'm OK with the fact that some people work because they need to work rather than they want to work. And then probably they have a ton of hobbies that excite them. So either way, it will work out fine, but it will take some time before those more creative jobs or even your hobbies could be automated. And then on the education part,

I, there's some things we're implementing. I see different clients implementing for AI because in AI there's always the issue of hallucination. Like you said, you ask a question about JavaScript and you wanted to generate some function that you maybe want to try out on your machine, then the AI probably doesn't know what machine you have. It doesn't know what MPM libraries you've installed, if you even have Node or JavaScript

installed on your machine. So it's going to come up with a solution based on its training data. And there are some things in AI that we like to call guardrails. And with guardrails, you, for example, can see the level of hallucination. So the AI itself, even though it doesn't tell you, it calculates some score based on house, how sure it is that it gives you the right solution. And then there's also something called groundness.

And this means it will look at its relevant training data or the relevant context it has, and then figure out how much of its answer is based on existing data and how much of its answer is based on the larger set of data in total. And with these kind of things, if your partner would ask the same question and we'll get like a low groundness score, it can actually ask the AI like, hey, I'm getting a very low score. Can you maybe try again? Or can you simplify things?

And if you look at agents, if you will work with an agent, they'll probably see like there's a low ground score. I need to figure out if I can get more relevant data from the Internet, if I can go to a different source to collect more data, or if I should actually tell the user like hey, this won't be the best. Or it could actually validate the code for the end user and say I've generated this code for

you. It doesn't work, so I'm going to generate a new function for you that might does work, because now I'm going to execute it in my own sort of JavaScript runtime. Yeah. So the interpretation is then also handled by the agent rather than the person that is prompting. Yes, interesting. Yeah. But then still, don't you think there's going to be a knowledge gap at some point? Because the even if a prompt is

then interpreted and gives you an answer that's similar or just can't figure out the answer and there is an answer, at some point, you have to be able to still figure it out yourself. And if we get to a point where let's say more repetitive or mundane tasks, let's call them simpler tasks, maybe would be automated, then those are usually the tasks that more junior people would start off on and learn actually and get experience to become and grow more senior in whatever they're

doing. If we don't want to have those tasks anymore. I, I don't know what junior engineers are going to do. I feel like the role is going to change in what they experience and how they're going to be productive, but I don't I don't see it yet how it's going to

happen. Yeah, I guess if I think back, so I mentored quite a lot of junior developers like either when they were an intern, my start up because, well, you're a start up. So interns are a good way to have more people help you or even as a startup, you probably end up hiring junior people because they have lower salary expectations. And but then there's also the trade off. They need more help. They need training wheels. They need to be mentored. They need to be taught how to do things.

And my thinking there is in a world where AI is going to automate parts of our job, AI can be some sort of a mentor for these developers and these developers are able to use AI. So let's suppose you're on VS Code, then you're trying to create a function and then you have a coding assistant and it's going to give you some boiler plate for the function. It means the junior developer is able to move faster because they get a boiler plate, but it also means they have to think less

about how to create a function. So like you said, there might be a new sort of group of developers coming up that doesn't know how to create a function because it's already created for them. They might get lazy. They might, well, maybe they don't get sort of these naming syntax burnt into their minds or have the muscle memory to do this over and over again. But I get these training wheels that will be generated for them and probably a large part of the

code can be generated for them. So they're more directing the code, like maybe changing the order or maybe getting better at asking questions to the AI in order to do this. And this will probably go on for like 5 or 10 years. But then after that it might be indeed that we only spend our brains or time in our brains for like the more complicated issues and then there will be some barrier to entry for for junior people. Yeah, that's what I think as

well. It's something you mentioned with regards to let's say code generation kind of triggered me and made me think of there are some platforms that just their business model is code automation or code generation. And then even beyond, we always have low code applications or

tools similar to that. I feel like with the role that AI is, is now having as well as growing towards, I'd be very fearful if I was in that market, if my business model was cogeneration and all of a sudden agents come out and they generate the code, interpret kind of mistakes and then execute on top of that as well. I'm wondering how many, let's say code generation, business models or actually start-ups we're going to see. And then to even go beyond that,

same for low code. Basically low code's whole thing is you don't really have to understand exactly what's going on. We have these integrations and building blocks and you can kind of clip them together and their main thing is speed. Maybe if your main thing is speed and kind of as a software business model, it's going to be hard to compete. What are your thoughts on that? So it's also one of the fun aspects of running a start up.

If you build a company and you're going to expect to keep the same company in the same concepts and the same sort of products that you're offering for the next 5 or 10 years, then you probably won't exist anymore at some point because you need to keep pivoting. You need to keep adding new products, improving, making sure you're building stuff people like. So if you're building a start up and you're relying on no code generation, you can also use AI

yourself. So maybe you can make, you can generate better codes, you can bring over your own, come up with your own AI assistance. You can do all these things in a sphere of AI that already fit what you've been doing before, just make it better or faster or more efficient. If you're competing with Jet GPT or other Jet assistants, then you're probably miles ahead at this point. So the only thing is you need to make sure you keep a hat.

So you keep improving, you keep using all these different things and make it better and run faster. I don't know about yourself, but if you ever tried to create social media posts in Jet GPT, they're pretty generic. And if you scroll through LinkedIn or Twitter, you can find all those generated generated posts.

There are some start-ups that help you to become better at writing using AI. They will still generate those posts for you, but it will also generate some sort of knowledge base based on your previous posts, based on posts that are doing well. Put all this data together and then it's a bit like the rag example I gave.

Put this in a factor database, maybe create a specific database for you as a person, and then it can generate new posts based on your previous posts that fit your own tone of voice. And this is something ChatGPT could do but doesn't focus on at this point. If you ever started doing this, you can probably excel for maybe 1-2 years. And at some point, some AI assistant, some generally available large AI assistant will be able to do this in a similar way or potentially better.

So maybe you need to start building agents at some point and then go on to the next pattern and the next pattern. And if you're doing this, you'll probably be ahead of those general jet assistants for a long while. If you don't, then you're just in here to make quick money. Then you're probably fine for the next one or two years, and then these bigger assistants will just take over whatever you've been doing. Interesting.

They don't have to run in parallel, but you can integrate with whatever is coming up. I really like the example that you gave of this. That's a social media post because yeah, I have experimented with it. I have trouble sometimes writing nowadays. I, I don't really look too much into what I'm writing and I'm writing more to engage or also to gauge. But yeah, if someone would, if I would have a tool that would be more personalized to my writing style, I would definitely not

have to correct as much. And nowadays I don't use it because it's too generic. What I get back, I would rather write myself. In the beginning I did and I was like, OK, so I'm correcting here and there, but then I lose. I feel my authenticity. And then, as you said, you're kind of ahead of the curve if you have that, because ChatGPT and main models are not really focusing on that personalization aspect specifically for the domain of writing social posts.

But yeah, how long is that going to last then? I think it will be there for some while. So there. It's also in the word right? It's a large language model. It's built for generic use cases. It isn't optimized for doing specific tasks. And this is different from the models we saw before. So I did a machine learning course in 2015. It's like a three day course and I learned to do some Python. I learned to build my own model. It was specifically trained to do something.

So large language models are trained to do everything. And something we're seeing now is, and this is a different pattern, besides RAG and agents, which are focused on making LLMS perform better for your use case, you could also fine tune or train a new model based on previous training data or based on different models. So another pattern we're seeing is companies taking a large language model and then fine tuning it with specific data. So that's another approach you

can take. But I can easily see large language models being sort of the the Holy Bible for for AI and then people creating their own testaments based on whatever is in there. Interesting.

Yeah, I mean, data has always been even before AI, how a lot of companies make money, right, Because they can teach a lot about a person with regards to what they like or what they might like or what they would like as well with regards to cross and upsell. And everything is very

commercial in that aspect. And then especially in Europe, we now have these guidelines, GDPR, you have to have data processing agreements and you can't actually put your data somewhere on a cloud that lives outside of Europe. And there's these more so guardrails with AI. Do you think we're going into the same route? And would that then kind of limit innovation? Because I feel like the more data the models have also about

a person, the better. For example a personalized experience can be. Yeah, yeah. I think for personalized data, always use something like RAG or agents where you make LMS context aware. They don't necessarily train the LMS on your personal data because I don't think the LM in general will get smarter if it

knows your address, for example. But it might get smarter if it gets access to your e-mail history because you might have been discussing things or concepts, or it would get smarter from those things, which could also be anonymized. You don't necessarily need to know your name or your address for that, but it might be able to benefit from knowing your way of thinking. If you look at what a lot of the bigger companies are doing today, they're selling off their

data to these providers of LLMS. So Reddit or Slack, they're all selling off their user data to train these different models, which might make the model smarter and act more like humans, but it, it wasn't really benefit from a lot of this data. So there's also the thinking of what data should I give it in order to make it better and what data is only relevant at a certain point in time. So I'm a bit worried about this. If you look at Slack for example, they said you need to

opt out by sending an e-mail. You opt in by default. Yeah. So I wonder, like, did their terms of service already allow for this? Have we all been sleeping when we went through these? Like all the bigger companies, they're using Slack, so they must have reviewed their terms and conditions and they agreed and now there's this new set of better needs to opt out.

So I'm really wondering what's going on there with all the legal departments in corporate because they need to opt out and somewhere they probably implicitly already agreed with sharing your data. Yeah, I mean the terms and agreements are so huge. And when Slack came out with that like it was everywhere and was actually quite a shock,

right. Because if you have to opt in by default, then already, I mean, from my side, I would assume that OK, they're going to start using this and I can opt in to have them use this. But indeed, if it's an opt out, then yeah, they're already using it because then the terms of service, indeed if they haven't changed, they accommodate for a kind of auto magically that's. Yeah, so that that kind of amazed me. I mean, for Reddit, that's fine.

Most of the people are on Reddit already on some sort of anonymized account name and they will share personal. They've already, they don't really share personal data. They share experiences or have questions and those need answering. But yeah, it's very interesting to see where this will go and what sort of guardrails we will get, not only for figuring out hallucination or how good the answer is, but also like what data is shared and what data can

be generated by by LMS. Yeah, yeah, absolutely. I am very curious though, and I'm very excited to see where it's going. Also in seeing new use cases pop up or seeing new implementations here and there and wondering, OK, is this how they've used it or how have they done it under the hood? I'm expecting more conference stocks as well in sharing knowledge. So this is going to be a lot of fun and I've really enjoyed this

conversation, man. It was very, very knowledgeable. Lots of learnings in there. Is there still anything you want to share before we round off? Yeah. So the final thing I want to share and it sort of touches upon your conference stocks thing. So I like to speak at conferences and I like to talk to developers, front end developers, web developers. And the biggest thing I wanted to to leave behind today also is like AI is just a concept. As a developer, you still need

to build the app. You need to make calls to databases, make calls to API, deploy your code, write tests or generate tests. And a lot of things you can do as a developer with AI just rely on an API call. So all the big LM providers have AP is, you can just call their AP is and access LMS. You don't need to be running Python codes. If you're a JavaScript developer, you can just call an API.

And probably more of these concepts we discussed like RAG or agents, they will become available either as API calls or through libraries such as length chain. So we didn't really have time to discuss length chain, but it's a library that sort of abstracts away all these different LLM providers. So you need to build it once and then you can connect to different LLM providers. So these are real really good things to think about. With Google Link Chain, you can probably find some nice

tutorials to get started. So in short, every developer can build AI applications because a lot of things are still the same. The only thing you need to know is how to interact with these different LLM providers. Yeah. And the accessibility kind of barrier is decreasing. It's getting more and more accessible, which also is really good for the future. Yeah, definitely. So LMS are just an API call away. Yeah, I like to say. Yeah, for sure. Cool. Then I'm going to round it off here.

Thank you so much for listening. I'm going to put all Roy's socials in the description below. Check them out, let them know you came from our show. And with that being said, thanks again for listening. We'll see you on the next one.