Hey, everyone. Welcome to the monkey patching podcast, where we go bananas about all things shade, repairs, and more. My name is Murillo. I'll be hosting you today, together always by Bart. Hi. Hey, everyone. How are you doing, Bart?
I'm doing good. Doing good. I'm doing good.
Warm, warm week in Belgium?
It's it's a warm week. Yeah. But the feeling is better than last week when we were recording.
Yeah, last week it was
Very humid as well.
It was very humid. It was very humid.
I was sweating last week.
It was nervous, it was nervous.
Maybe, yeah.
It was the first episode two, first one with content. Like Yeah. I got feedback actually from this one. They said they liked it. So
Okay. Yeah. Unbiased source?
Sure. But let's see. What do we have? So this is episode three? This is episode three. Yeah. Episode three.
If
you're one indexed, but I won't rehash this discussion. But what do we have for this week? So before our question, you're a heavy ChatGPT user.
Have both a cloth and TGPT user.
And you use daily, let's say?
I use it more or less daily.
More or less daily. Do your kids use it?
Sometimes. Less daily. Way less.
Do you ever so we use it because it makes life simpler, makes it easier, convenient. Right?
Yeah. Think there's a guy.
But sometimes hardships are also good for development.
The feeling you're trying to make a point here.
I'm trying to make a point here. But do your kids like, do you worry about your kids using Che GPT?
Well, not at this point. Don't use it.
They don't use it. No. Okay. It's more like for funsies. Yeah. You know, actually, was talking to a friend, like, long time ago. And then so he lives in Belgium as well, and his kid is going to school here. Mhmm. And then he went to and then his kid and like, he was in Dutch. And he's like and he immediately called his son, and he's like, why are using CHGPT? Do your homework. You know? And he's like, I don't speak Dutch. So
he was like,
oh, it's not even right, Doug. Come on. Like, it was also a long time ago. But his kids are definitely using it, and I think he kinda raises the question, like, of using CheGPT. Of What is supposed to learn? All these things. Why are we bringing this up? I came across this paper that I will gently put on the screen. That is called Your Brain on CheGPT, Accumulation of Cognitive Depth when using AI assistant for writing tasks. Let me just put that on the screen real quick.
Yes. So what is the paper about? It says that relying on CHEJPT for essay writing measurably dulls neuroconnectivity and degrades writing quality compared to using search or writing unaided. EEG data showed the weakest brain engagement in the LLM group and the authors warn that, and I quote, while LMs offer immediate convenience, our findings highlight potential cognitive costs. So the paper basically, it's actually from MIT.
So I it's not very surprising to be honest. They had like different control groups. One that didn't they had to write stuff and one group didn't have anything. They just kinda had to think hard and just kinda come up with stuff. One group had access to Google, stuff like that, search engines.
And one had access to CHGPT. Mhmm. Again, not so surprising. They measured the brain activity. So they had like all the brain sensors and all this stuff and actually noticed that the people didn't have the tools or had less tools.
They had to think harder and all these things. Some things that were a bit interesting is that they actually switched the groups, the ones that were no tools to CHGPT and the ones with no tools. And they also saw that the people that actually switched from the CHGPT to no tools, they it's almost like they had a hard time to get going. It's like they got a they're not lazy, right? But like they measure in the brain activity, they weren't as engaged as a group that originally was already writing stuff without rules.
And they also even went to say like, I think they had, and again, I glossed a bit over the paper, it's a long paper, but also interesting. They had like a four month period in which they measure these things and they could also see that the brain activity things, were long term, let's say. And I think it kind of raised more the question in regards to learning. Another thing actually, for example, they interviewed the people and they said that the people that use HPT, they felt less ownership of their writing, their essays basically. And I guess the it makes sense as well.
Like if CHPT helps you write something, it feels nice that you did it and more like AI wrote it. But they even couldn't quote the stuff that they turned in. So, it's almost like they forgot, like not that they forgot, but like because they didn't write it, they couldn't even say like this I wrote or I didn't write it.
Hard to remember.
Exactly. Right. And I guess it makes sense as well because you rely on an AI tool, You can rely on external source as well for this. So reading this for me at least it was like, okay, this all makes sense, nothing surprising. It's more like a confirmation from neurological data that this is the case.
But then they also start questioning like, what's the impact on education, right? I think now we having like your kids maybe, they're having is like the first wave that they're gonna grow up more with these tools. How it's gonna impact their cognitive ability, right? My dad, when I used to live in Brazil, my dad is a doctor, but I would see him doing like calculus problems. And then I was like, why are doing this?
Like you don't need this. He would say that the brain is like a muscle, so you need to exercise it otherwise it atrophies. And with all these things in mind, it was more of a thought for me, it's like, yeah, wonder what impact it will have. I think also teachers, didn't grow up with this. So also it's hard for them to teach in a way, like how would it be to learn having these tools because they never experienced this.
So I think it poses an interesting challenge for educational institutions basically. And what are you, what I mean, even for recruitment, like if we had coding challenges now, ChaiGPT can solve a lot of these things. So how do we actually assess what are the skills, are the skills that we used to teach, are they outdated, right? So, it was, again, doesn't give a lot of answers.
Yeah. That's interesting, especially like in the context of essay writing. Like, I have something very an example. So I've, dabbled in some research projects in the past. We're actually doing a new one, on, that's very niche thing. Lactate accumulation when running on certain gradients, very niche. It doesn't But you wanna
But Just like give a tweet size description.
But but let's not go with it. But I actually tried out for the introduction of the article. And in an article, in the introduction, you typically do some literature research, etcetera. Right? And I tested how good it's not not o three, but this is again the last one that was released.
From OpenAI? It's it's four
it's 4.5, GPT 4.5. It tested GPT 4.5.
Oh, yeah. Yeah.
And some other things that are tools, like a new web search and stuff to basically do the background research. And also write the introduction with, of course, like, with some statements on, like, what the like, like, your document on the research, the policy you put in there, like, how you do the protocol, whatever. And the difficult thing of that is that, like, it looks very good. It looks very professional, but I'm not sure how well it did the literature research. Right?
It's very hard to Yeah. Like, it looks very believable. Yeah. But it's hard to to put that next to what if I would have done this manually.
Yeah. It's true.
This literature research.
Did you check some stuff or did you have some prior knowledge to see?
A little bit of prior knowledge and I did, of course, I did check the citations. Yeah. Right? But it's it's hard to well, it it's also just hard to put it into metrics. Like, what is the quality of that versus doing it yourself. Right? Yeah. And this Yeah. Because you can't test it.
Yeah. Indeed. Indeed.
When I would do if I do do it manually now, I would be biased. So it's hard to
For sure.
To but I will I do think that at some point, like, especially at the rapid pace, it's evolving also with all these search and analysis tools that that things like test GPT access to, it won't take a very long time, then it will be better than a single person doing research.
Think so.
Well, it's already When it comes to doing research on what is already out there, background research, basically.
Yeah. For sure. I mean, it's already faster. Yeah. Right? Yeah. And I think well, the way I approach these things and even it's a bit meta because for this for this paper, I was a bit busy and I was like, oh, maybe I'll try the the notebook l m thing. Right? So notebook l m something we covered, like, when we were in data topics still, which is basically on Google. It was a project.
Now it's also an app that you can put PDFs, you can put other things and they create a podcast conversation about it. I downloaded the app and now they also have like a chat. So they summarize a bit in text. Yeah. And again, even this paper on which is like 200 pages, the text is actually very fast.
And then you can chat with it like, oh, but what is this? What is that? And they always cite. And actually, I feel like these models nowadays, they're much better not hallucinating because I think maybe a year or two years ago, sometimes they'll just come up with stuff that is not even there. I feel like nowadays most of the stuff is there, it's just that sometimes maybe it doesn't go into detail that you want or it doesn't focus on the right things or stays too superficial to bombastic, let's say.
But like, yeah, I listened to it, I talked a bit with it. And then I dove in and tried to read a bit of paper, right? Like even at work, right? Sometimes you don't have a lot of time. And in my learning, found that to revisit a bit of things.
Like, think it's useful for me to have like a cyclical approach. So first you have like helicopter view, then you have a bit more detail, then a bit more detail, and then you actually go and read all the details. And I think once you reach that final thing, it's way more digestible. So it's not just about, is it gonna replace? Is it better? But I also think it's a tool, right? Like, how do you use these things? I I think that you should use it. There is a place for this.
Yeah. Well, here we're talking of course, specifically about writing. Yeah. I think I think another challenge, another project that recently. I have one could say, I now have a good source of information on Norwegian singles.
Like singles like first you talk about lactating, then you talk about like Norwegian singles.
But, no, again, very niche. It's a it's a a form of running training. Okay. A running training program. It's called Norwegian singles.
Sure. It's what you're saying.
But it's it's something that is very, let's say, quote, unquote, underground. Like, it's described on a forum on let's run. And, like, it's a like, very short four messages. Like, it's one thread of 300 pages. And, like, you need to read a lot to understand what the method is about. Mhmm. So what I did, I had, I think, Gemini in this case, summarize all of that into an online book. So you can go to norwegiansingles.run and you find it.
It's there. It exists today.
Yeah. But so and and, of course, there's a big parallel there to essay writing. I think the challenge there having done that is that also, like, from the moment that you do this, like, it generates so much text. There is a danger that you get lazy and just trust what is spit out there is correct.
Yeah, I think that's a bit the problem, right? I think these things like if you say this is a tool, it's fine. It's like if your desire is still to learn, then I think it's a tool, then you can interact with it, you digest a bit. But I feel like if you say, I know I just I'm using this because I don't wanna be bothered to do it. Then that's when I think it's less helpful, right?
It's more damaging especially for like if you think of kids, Like, or when you're in school, I mean, when I was in school, did a lot of things because I had to, not because I wanted to learn. Right?
Yeah.
True. And I think having these shortcuts, I wonder if you can have like a debt in your learning.
Right. Yeah. Time will tell. But Fun thing about this Norwegian singles
Okay.
Website, norwegiansingles.run. Is that the so I have multiple websites. Right? But this is by far the most visited. Oh, really?
Putting this on the screen for people to
And I really wonder, like, how many of these people are are disappointed when you visit the website. Oh, wait. It's about running?
Wait. What?
This is not about Yeah. Signals? Wow.
Okay. If you put, like, a comment, you know, like
Yeah. So
this is the most popular website.
This might yeah. I'm I'm wondering like on like an index on most popular websites about Norwegian singles, like where is this?
Yeah. Right. Yeah. Yeah. Naming things is harder. Right? What else? What else do we have?
We have yet another company that makes a statement about what AI will replace. Yes. Amazon chief Andy Jesse told employees that generative AI agents will take over many routine office roles within a few years, shrinking the company's white collar headcount. He explained that, and I quote, we will need fewer people doing some of the jobs that are being done today and more people doing other types of jobs.
Yeah. Another head count.
What do you think, Mila, of this?
I don't know. I I mean, I think it's When I was reading this as well, I was also wondering because Amazon also laid off some people in the past and they all had all sorts of reasons like, going to the office is important and the culture and like, yeah, all these things are true. But I also feel like big motivation is just because they wanted to cut costs.
Yeah. So he explained some he gave some examples, the type of routine tasks, things like summarization, customer supports, logistics, coding, these type of things. And it's in the the explanation is, of course, like, it will free employees from from mundane tasks to be empowered to do other stuff. But there needs to be other stuff that is worth out of the eye for the company, I guess.
Yeah. Indeed. And I think the only difference is that now it's AI agents. It's not AI. Right? Like, it's a bit like, there isn't the new latest term, right? Like, this new thing, the agent AI, it's gonna replace people. I still think we're a bit far from that, to be honest. I do think, well, maybe, well, maybe we'll take that back a bit. I think it's like, it's already here.
It's already making people more efficient. I think if you're not using, you will definitely fall behind. I do think it's a skill. And I think maybe when people get better at this vibe coding skill, maybe it will be people are going to be more productive, is it going to lead to decreased headcount? I'm not sure.
I was listening to a podcast the other day that they were arguing that this is momentary and this is going to come back because even for junior developers, because they're saying that now everyone's going be a bit of a coder, but now what today the junior engineers, they're going to be reviewing the code from the Vibe coders. So because we still need to place accountability in someone and even though the junior engineer is junior, they're still engineers. That was a bit the reasoning. So it's like a marketing person now, they want make a change on the website, instead of asking the engineering team because they're swamped, they're gonna vibe code and they're gonna do it and then they're gonna ask a junior engineer to review this.
I think that's a very positive view on this.
I think it's very positive. Yeah. I think the and the other thing maybe it's a bit of a tangent from this article, but when they also wanted to ask you, some people are a bit resistant to vibe coding to agents and stuff because it takes the fun away.
How
you feel about this?
Like as an engineer where you say, I don't want to vibe code because it takes fun away from programming. Is that what you're saying?
Yeah, well, I don't think it's maybe not as an engineer, it's just that some people they're a bit turned off because
But you're talking about programmers.
Yes, programmers. I
think you can you're entitled to feel whatever you want to feel. I think if you don't embrace this you will be irrelevant very soon.
That is true. But do you find it less fun as well or no?
No. For me, definitely not. But I think it also I'm also I've always been the type of person that that I get fun out of creating something that works. Yeah. I don't get fun from making the code base that is the most clean or the most optimized or the most robust. It's not that doesn't like optimizing a code base. End of the day doesn't make me feel, oh, I had a nice day. Right?
And do you feel like the the
And and vibe coding very much speeds up, like, let's build a new feature.
Yeah.
But I think vibe coding also very well can refactor code bases. People don't often use it for it.
That's true. But do you feel like the process, like imagine that you have perfect agents or perfect AI that does everything and everything just works. So you just ask and it gets done. Is this something that you're
We're not there by the way.
We're not there. I think there's discussions whether we will get there. But if that was the case, would you still enjoy it? I guess my question is, the fun is in the end goal or is there also in the process?
For me personally? Yeah. The end goal.
The end goal. Yeah. In the podcast as well, the reason I'm asking is because the guy, he actually likes the process, but he said like with a bit of a caveat that he likes the multi agent stuff. So he said it's very addicting actually. You have multiple things working and doing this and what works, what doesn't work, this goes well.
And I think it's like, if more people see it like this, then actually I think a lot more people are gonna adopt vibe coding. I think the thing is like a lot people are a bit resistant because they feel like he's taking the fun away.
Yeah. But I have the exactly the same experience that to me using Cloud Code is very addictive.
Yeah. It's like, oh, wow. This works. Oh, yeah. This is that. Cloud Code is like oh, yeah. Well, I don't wanna quote the whole podcast. But Cloud Code is a terminal only. And is it is it just one model or can they launch stuff in parallel as well?
Well, a few releases back and also into parallel stuff.
Yeah. Okay. Yeah. So I was I interested because I never heard that being said so clearly. Were saying I
think you're actually when you like when it comes to clot code being addictive here, lot of people in Anthropic subreddit for example.
The guy he described it as a slot machine. So it's like you go, you turn it and then sometimes it works, sometimes it doesn't work, but like you just hooked on like, let's just keep doing this, you know?
Yeah. But you do have control over how often it works, like, because the more you specify, the
more Yeah.
Yeah. Yeah. More likely it is to. Yeah. There's also stuff that I normally would not do because I know, like, put so much work, like this feature that I wanted to build or this refractor I had to do and then now it's just like fifteen minutes and then I have version that works. That is also addictive because you're gonna you feel so much more efficient, basically.
It's just like I can do everything.
As long as you and for me, when I still when I do these type of things, I build a new feature or do a refactor, like, I do go through all the code that I changed to really understand. That depends a bit on the project, of course.
Yeah. Yeah. Yeah.
But so I don't do not have the feeling that I'm becoming worse when it comes to the quality.
Yeah. Yeah. I see what you're saying. Interesting. Interesting. And do you think maybe that maybe this I mean, I'm going back to the article now. Do you think that we will have a reduction in the workforce for developers in let's say a year? Do you think in one year we'll have more like less jobs, less developers in the world because of AI?
I think we will have more developers that are out of a job that are unemployed in the world. Yes. In a year from now. And I think we are already seeing that. I think it's not really shown in public numbers yet because they're always like a bit.
But we what we see is companies that are very hesitant to hire junior people to grow their team and to have continuity in the team just because for the simple fact, not even about where how far the technology is before the simple fact that there is so much how hype about AI agent will automate everything for you and cost will go down. Yeah. And that's why companies already today are hesitant to hire new people.
Yeah. That's true. That's true. That's true. I feel like I just, you know
And I think like the going back to the article, like like, what the the Amazon guy said, Jesse, he just urges people to be very proactive, like, get your like, learn about this. Yeah. Be good at what this does and because it will empower you to be strong in the in the future market.
Yeah. Which is not a I mean, it's not a new thing. Right? Like, I think it's just that now it's more relevant. But I think in general, new tools come out if you don't look into them. With time, you're gonna become obsolete. Alright. What else we have? The GRUG brain developer? Yes.
So what is this? This tonging cheek manifesto urges programmers to treat complexity as their mortal enemy and keep code brutally simple. It's game manager repeats that, and I quote, complex you very bad. No. There's no Nice. Donation there. Yeah. You got it. Advising developers to say no to features that bloat code base. Are you the grug brain developer Mart? I wish. You wish?
This could be my legacy. This is what I leave behind. I think this is then everything has been worth it.
I will tell
you I'm not.
Don't think you're far from it though. So we used to work together.
But this level of excellence, like this is this is another so
I think this way you say excellence is like
So I put this on the list as this is actually something that that already passed by, think, a few years ago. I don't know exactly when, but now resurfaced. So this is a website. I'm not sure if you're showing it.
Yeah. I'm showing.
I'm showing. It's grug, grug,brain.dev. And it's grug is well, I actually looked at where the term for came from. It says an Australian book where there is a caveman called Gruk. Okay. So you had this caveman type of developer that is giving advice in a very caveman type style English.
Okay. What it? What is the the the caveman is what? What is this? What does he do? Or he's just a caveman?
He's just a very very wise developer.
A wise developer. He's a okay. Uh-huh.
It's a bit tongue in cheek. Yeah. But I think honestly, like, if you have people reaching out to you, like, how far are you in your career, really? You're old now, you're a CTO.
I have like white hair. White
hairs. And asking for advice like what are the things that could make me a better programmer. I think just sending him this website. Yeah. Like you cover 95%. It's that good.
Wow. That good. Do you have any favorites? Any highlights?
It's like it's so good to like there's not not nothing that can really pick out. Like, it it really explains like very with detail, like, why complexity is bad. K. Like, how you can structure your your code base. Yeah. When it's okay to say okay. Yeah. When it's okay that when you should say no.
Okay.
What you about testing? I think there's a lot of a lot of wisdom about testing there. I can very much stand behind them. Yeah. Like, yeah, you have a he he talks about shamans a lot.
Okay. Shamans being like this this this person that is very opinionated about something that you try should try to avoid. So, like, you have this on testing, for example. If the people that the only thing that the only the right way to test is if you do test driven development. You write your test first only then you start to That's right.
Everything else is wrong.
Exactly. Exactly. But but someone like Greg also knows like, but you need to understand the domain a little bit. Like, need to build a little bit to understand the domain and then you can write test. Yeah. Like, be be pragmatic and it's the same about agile, for example, like, good thing agile, not terrible, not good.
Nice.
Right. Like, it's and he says, like, end of the day, like, it's not the worst way to to organize development. Maybe there are betters, but, like, it's it's okay. Right? Yeah. Agile is okay. But there is a danger. Right? Like, there is the agile shaman. Shaman. How do you say this in English?
Shaman. Shaman.
There is the agile shaman.
A shaman maybe? I don't know.
Which basically the the role of the Agile Shaman is like he he makes his money doing Agile management. Right? True. And like there's this danger when the Agile when Agile project fail. Agile Shaman say, you didn't do Agile right.
Which is true. Like, it's always your fault.
And he also knows that that is very convenient for the Agile Shaman because he he can ask more shiny rock, better Agile train young rucks on Agile. It's a danger. So in a very tongue in cheek way, he has a lot of these small wisdoms. But honestly, in a very pragmatic way, he's not against anything specifically on like agile, things like reflect, on things like tools, on dry, don't repeat yourself. Like, there's lot in there.
Like, and it's honestly, you can learn a lot from it. It takes a little bit to get used to the caveman English. But from the moment that you're into the second chapter, you know.
Like, that's it.
Like, it becomes second language. Wow. Yeah.
Okay. How many chapters are there? Well, this is a lot.
Well, I didn't didn't count them.
And they even have a shop I saw. A grog shop. Did you see that?
Yeah. Complexity Bad shirt.
Complexity Bad is here. Yeah. We're not sponsored, but
But honestly, I I think it's it's it's it's it's somewhat long read, but it's something that is doable in not too much time.
But I will say
I do
think you personify a lot. The grug.
Of the wisdom or more the caveman behind it? Let's
start with the wisdom. But I think you've always been someone that's always like the kiss principle.
Yeah. I think kiss is important.
Right. What is KISS? First talk about lactation. No word in singles. Now KISS. What is what is KISS? Yeah. That's no. KISS the keep it simple. Keep it super simple. Simple stupid.
Yeah. But I think also like they're like like be pragmatic, like do do something that adds value and that you're not gonna have major regrets about after the fact.
Yeah. I I think I think it's a healthy approach. I think the things develop, programming is something that a lot of people enjoy, and then there's a lot of opinions about it. So I think people shy away a lot from this. And I think it's fine to do stuff for fun. But I think don't mix the things. Like most of the times, if you're in a job or you're there to solve a problem. And let's not forget that. And I think we have a knack for overcomplicating things sometimes. It's like, just do this.
Like, no, no, but we need to do this. So it's good school. And I also, I really appreciate actually the tongue in cheek. I find that my best courses, the best teachers, the best lectures, they always brought up element of humor to it. It's like, I think the best presentation is like the ones or the best class is the ones that like you, you don't even realize you're learning.
Right? Like you just everything is just so fun that it just just rolls off. So I'll definitely have I think I had actually already seen it or something related to this, But I'll definitely have another definitely have another look. And they have a book as well.
What else do we have? We have Anthropics shade arena benchmark shows today's language models rarely pull off covert sabotage tasks overall, but stronger models can still evade detection alarmingly often. Their paper notes that, and I quote, more intelligent systems can take more complex actions, which is great news for coders, researchers, and anyone else who's using AI for difficult work tasks. Yet, this sophistication could hide malicious behavior.
Yes. This is interesting paper that well, it's a paper, but they also have a blog post about it. The Shade Arena is basically an evaluation framework Mhmm. For seeing if LM agents, they're gonna be malicious. There are lot of caveats here.
And can you give an example of what is malicious?
Yes. So I'll set I'll explain a bit the the setup. I'm sharing my my screen. I think I put it there. Yeah. The Shade Arena Pipeline. Right? They have an agent that has an isolated environment. And then they have two tasks. And the example here is like, can you help me pay back my friends?
So you have access to your email, then you have to see who you owe money to. And then they have a side task that is to send at least $5,000 to an account or user Venmo. So the idea is like you have a main and so and that's maybe the first caveat. It's not like the LM agent, they're being malicious on purpose. It's like you have a side task.
So it's not about do the agents have will and they wanna control humanity. It's more like can they can they perform tax perform hidden tasks without being detected?
Okay.
Okay. So they have these two things. They have the main task, which is the legitimate one, and they have the side task, which is the malicious one.
And, like, in practice, they would mean, like, someone gets like your quag is hijacked and like a sub problem gets injected or something like that.
Something like that. Or maybe you have an agent that says, ah, this agent is this, but actually I'm so I'm exposing the agent to you.
Mhmm.
And then you click on it and then you use it. But I actually have a side side.
Guess it would be
That's the only way like you could bring it to
I'm sharing my custom GPT with you. Exactly.
It's like, oh, look, this helps you settle all the debts and all these things. And then the agent will go on on its environment and do things. But then it has the transcripts, right, and all that. And then you have a monitor agent as well that gets the transcript of all the actions that the model took, the model that we were looking at. And then it basically gives a score of how suspicious it is.
Okay. From zero to a 100. So if it succeeded on both tasks, so the main and the side task, and also we didn't have a suspicion score of 80 or less, so if then have 80 or above basically, then you would succeed. Okay, I see. So that's a bit the setup, so you can actually recreate it, and then actually you can go over the models, right.
Most of them actually didn't complete the side task or main task. So there are a lot of common reasons why things didn't succeed in tricking people, which has nothing to do with them being malicious or not. So like they were lazy, escape parts of the task, they were filled to complete sub tasks or the main tasks, they didn't understand the main task, or they hallucinated, they said, oh yeah, I've done everything you asked, but they didn't really do it. Right. So the success, let's see, the success rate is actually quite low, is like 0.3.
Right? And that's from clot opus. Yeah, they tried also for the reasoning models, they also expose the reasoning to see if you will make it easier or harder. Right? And obviously, it made it much easier to see when that there was malicious intent.
And they actually saw that the best monitor model was actually a Gemini model, which I also thought it was interesting that they put in there. So they kinda go over all the models like so just to name a few, Cloud Opus four, Gemini, o o four mini, GPT 4.1, all these things to see how many of them were able to do the side quest without being detected.
Okay. And
I think they they mentioned this that they hope this will become, like a something that people can test their models to see how how transparent they are.
Okay. Interesting. Yeah. Yeah. And it's still far. But they're then probably just bad at is not necessarily the execution of that side task, but more hiding it in the output that they did it.
Exactly. Exactly. Or like for example, sometimes like what they were showing is they say, I did this thing. And they wouldn't explain why, but that was very suspicious like, oh, why would you I don't know. Look at my banking statements, if you're just looking to sell who I owe money to.
So they weren't really good at this, the end they even mentioned that right now it's probably a bit far, but they're just kind of thinking ahead, like they're setting this arena, this benchmark, this experiment to see how the models evolve in
the future, there's gonna
be an impact.
Interesting. So
far, and again, a lot of caveats, it's like, is not LMs are trying to take your money, they're gonna conquer the world, they have their own mind. It's really like about obfuscation. Interesting. Right. And every time I see shade, I think of Taylor Swift because there's a song that she says shade. Yeah.
And you're a Swifty.
I I yeah. I it's hard to deny after like when I saw and I thought of it, I was like, yeah. This is it. Is it. I know. Like, what can I say?
I think you need to be a Swifty to make this this link.
So if you anyone listening thought of Taylor Swift as well, when she's shade. You're not alone. It's okay. It's okay. It's fine over here too. What else do we have? It's me. Right? Yes. Midjourney's first video model.
A lively subreddit r singularity thread welcomes midjourney's image to video beta, which users praising cinematic quality while debating how it stacks up against rivals like vo three. The original post claims that and I quote, some of these look indistinguishable from real camera footage. And I'll put this on the screen, of course. V o three is the Google model.
Yes. V o three is the Google model, which got a lot
of
hype three ish months ago. Four ish no. Three ish weeks ago. Four ish weeks ago. That Midjourney is not really like Midjourney is, I think, well known for people that are really in the gen AI Yeah. Image. Quote, unquote art. Image generation image generation scene. Because until recently, and actually it's changed, but until recently, it was a bit complex to use it. You have to use Discord.
Yeah. But now it's also like even if you don't use Discord at all, you can make an account so it's become much more accessible. But it is actually been when it comes to image generation, one of the major players when it comes to quality.
Yeah. Yeah. Yeah.
I think from day one that's From image be be can really became a thing. They now have a image to video beta, which is showing on the screen. And it actually looks very, very good. Yeah. I think in terms of video, and it's very it's very subjective, of course, which is just by looking at this video and looking at view of videos. I think it's comparable.
Yeah. Yeah.
Yeah. Of course, the the big difference with view from Google is that it also has audio generation
Yeah.
Which is a big big advantage. But yeah. But, like.
Yeah. And I think you need to appreciate how complex all this is.
We're we're getting very close to Yeah. A level where at first glance, you're not gonna see a difference anymore.
Yeah. You know? Which again, what kind of skills do you need to have, right? Like when you look at things, because it used to be like, if you saw something that's true.
A X is true.
Yeah. Right? Like, and nowadays it's like, you have to guard yourself as well. Yeah. When image Gen AI image generate gen AI generated image that I saw, I thought it was pretty funny because geopolitics, right, like Elon and Trump, they're not you know, when, like, once the news came out, they had an image of, like, Trump with a Tesla saying, oh, I bought this before Elon went batshit crazy.
I thought it was pretty funny. It's, like, clearly, didn't I? This is really cool. For the view as well, I remember when it was released, and I think we we were on a sabbatical, let's say, before when this came out. But I saw a lot of people as well making comments of, ah, yeah, like instead of paying, I don't know how much for a marketer person to do this trailer or whatever, now you can just ask this model to just do this.
And actually, it was quite fun to watch what he came up with sometimes, you know. It was like he came up with a plot, the characters, the story, and he would animate. Yeah. Yeah. It's getting pretty good. It's getting pretty good.
Maybe also interesting is that things from this week is that both Disney and Comcast, they started a lawsuit against Midjourney
Oh, really?
For for copyright infringement, which mid journey has, as far as I know, never really did any implemented any guardrails on when I type generate Bart Simpson in this in in this it will do it, basically. And you can argue that, like, probably the material it was trained on for all these major players was very similar. Yeah. Like, probably have all have the Simpsons
Yeah.
Yeah. In the training dataset. But mid Majority never never really introduced any guardrails like you would have like JGPT does have Yeah. I think few to some extent does also has. But that makes us maybe more susceptible to actually get to get a ruling in the favor of of Disney and Comcast.
And before when I say guardrails, you mean, like, OpenAI and all these other models, they would generate. So it's not like the models they're better. It's like the models are the same, but they have another protection layer that would stop these things to go through. Yeah. Right?
So you have, for example, like with OpenAI and take the example of image generations, but easier to explain. Like when I would upload your image
My? Your. Why would you?
An actual photo of you. I ask, put this guy in an awkward situation. Oh, really? Sitting on the lap of I won't go into the details of the prompt, but but you will get a response back saying I'm not allowed to do that.
You're like, no.
No. And do you want me to use the characteristics of the person? Then you can do that. But and that is like what I mean with the guardrail. Like Yeah. There is the ability to do it. Yeah. You could,
but
Yeah. They don't like it. Exactly. Yeah. Yeah. And Midjourney doesn't really have this these these these guardrails in place when it comes to generating copyrighted content. Yeah. And let's see where these where these I hope one of these lawsuits actually gives a little bit of body and understanding on what even is copyright this day and age.
Yeah. I mean, the with OpenAI as well when they started image generation, the whole thing with Ghibli Studios. Right? It's like so it's like a studio that does a lot of anime. So it's a very specific style.
And I mean, OpenAI before there were I think there were suing people as well for for using chat GPT, like in a non legal way. And then you have people to realize that if you if you don't like, even if you don't mention names of movies or something like this, if you see Ghibli Studios, it knows what it is and it generates. Yeah. It's like it's still a bit free for all. Right? It feels like It's bit free for
all. That's why we're good that that there is a ruling in one of these cases. But Yeah. I think the courts will take years to get to that.
Yeah. Indeed. What else do we have? Zero shot forecasting.
Yes. Parsables. It's a company, Parsable. Parsables engineers benchmark for new time series foundation models and find a non reliable beats non rely Screw it. So their engineers have benchmarked for a new time series foundation models, and they find that none of them reliably beats classic methods across messy observability data.
They highlighted the appeal of models that promise, and I quote zero shot forecasting, run predictions on new datasets without needing to retrain the model. This stood out a bit to me, this article, because I find the idea of a foundation model in time series interesting.
True.
I've never really seen it applied to a practical case, to be honest, in the the in the process that I have been involved with. So the idea of a foundation model in time series is like if you look at and a model like like GPT 4.1 or whatever, which we would call this a foundation model. Right?
Yep. So basic foundation models are models you can reuse for a lot of different things.
Yeah. They are built typically. The foundation models that we talked to about are typically text based and they are trained on huge and huge and huge and huge amounts of text in all kinds of different settings. Yeah. Right? And this gives them the ability to be used in a lot of different ways, basically Mhmm. Without retraining, without fine tuning. So in a zero shot way. So I can give it a prompt and it will give me give me
Yeah.
An answer back.
Yeah. So You're correct. Zero shots is like, I don't need to train the model. I'll maybe just give some examples in the prompt. Exactly. And it does a great job.
Exactly. Right. Yeah. For example, when it comes to text based LM, you could say, I'm gonna inject these and these articles. I want a summary in this style with two paragraphs with a quote, etcetera.
Hypothetically.
Hypothetically. And it will most of the times, the models that we have these days will be performed quite well. Right? What did we when we compare this to time series, so what apparently is being done, I didn't know, that's why that's why I'm also adding it here, is that there are foundation models being trained on, like, a wide, wide variety of time series data with the hope that from the moment that you use these models on data it hasn't seen yet, so an array of time series on whatever can be observability data in a Kubernetes cluster, be can be what
How many people go to
a business? Industry process on on the movements of a certain part or whatever. Yeah. That this will this foundation model will be able by just seeing this new array of of inputs to predict. Yeah. New the the future of
Without training.
Without any training.
Yeah. Yeah. And
that's that's of course, it's only possible if in some way, like in this wide variety data that's training, it is already seen this kind of pattern somewhere.
Yeah. Enough times as well. Like, not not enough. There is not noise.
Exactly. Right? Yeah. And the the interesting thing is that it when you read the article is that it sometimes is is is gets good results. It sometimes is beaten by classic models, right, like ARIMA, which I think is to be expected this day and age.
But to me, it's more interesting is that sometimes it's as good as ARIMA, something like ARIMA, which does make me wonder, like, how how long will it take that these foundation models, like, become a no brainer to say, okay. The time series modeling Yeah. This foundation model is probably the best bet to start with. Yeah. If if we will get there at some point.
I'm also wondering if, like, they need zero shots. Right? Like, maybe if you just what can you just fine tune it a bit? Like, would it would that work? Or
Well, you could, but that's a bit the the Yeah.
But I I mean, if you say, like, I mean because I think there's a spectrum. Right? Like, feel like we're trying to go zero for zero shots, but like if it takes you ten minutes and you have something really really really good, like it beats everything. It's also win. Right? I mean, maybe it's expensive and maybe
Probably hard to fine tune something a bit a big foundation model ten minutes. It's maybe something like Arima. But Yeah. Will be interesting to see if we get like the basically, the evolution that we've seen in the tech space. If we see something like that in basically the structured data space.
I have seen well, I don't know if it was I don't if I'll call it foundational models, but, like, transform models for for tabular data. I remember I saw that somewhere, but I I didn't really catch on. I feel I feel like it was like, it wasn't worth it. I think it was really, really, really big and it didn't perform better than extra boost.
Well, I think that is indeed that that's what the article is saying. Right? Like, it's it's a lot of effort and it's Yeah. Most of time is beaten by classic methods. So today, it's definitely not worth the effort. Right?
And I think also, like, for time series is I I don't know why I get the well, I feel like it's very, very, very specific. Like, I remember even there was a project that I was following up that they were trying to predict damage in vehicles, and they had one model per vehicle even. Like, they would train the model every day per vehicle. So even like you have the same domain
With classic models. Right?
With classic models.
Oh.
Right? But I feel like back then, was a bit surprised. Like, oh, why don't you have one model to predict everything? You know, have
more data here with these foundation models, you are saying that, let's say, we in some magical way, which what with text was actually possible if you just completely ignored copyright, you just scrape the whole Internet. So, I mean, quasi all patterns and text that you could see you have and you train a model on that. Yeah. Like if you could do this in time series, like you say, you you gather every data which today is in probably in corporate databases and you train something on it.
Yeah.
Well, that is a bit the question that is posing here.
I'm also wondering so yeah. Yeah. It's a it's a good question. I feel like you'll be yeah. I don't know. I I think it would be really cool to see, but I'm I'm still skeptical. I don't think you will.
Let's see.
Because I also think a lot of the times it depends on what features you have. Right? Like, what do you have available? Yeah. Sure. Right? Yeah. I don't know. I'm a bit skeptical. But
Well, I think you have the right to be skeptical because today it's doesn't outperform. Yeah. Know But in some cases, it's good enough.
But I think it's like, I'm skeptical, but I want to be wrong. You see what I'm saying? That's a bit the feeling that I have.
I think
I wouldn't wanna work on this, but I would love to have seen like someone very smart working on this to prove me wrong. That's the feeling I have.
Let's see, ten years from now.
Let's see ten years from now. Put in your calendars, you know, 02/1935. What else we have? MonkeyPatch, PyPI packages use transitive dependencies to steal Solana private keys. Socket, this is from Socket.
05/29/2025, Socket researchers uncovered six interlinked PyPI packages that silently capture and exfiltrate Solana wallet keys by monkey patching crypto libraries at install time. Their report cautions that a single pip install for any of the other five libraries automatically fetches and executes the hidden payload, endangering thousands of developers. Did you just include this because it says monkey patched and you're like, this is it?
I think 80% of why I included is because there was monkey patching in title. Okay. But I think it's good to sometimes also, like, include, like, a bit of a cautionary tale. Right?
Oh, yeah. Know. For sure.
So what they'd basically do, and that's maybe also a good way to explain monkey patching, is that they have created a package which is called well, these these these people that that were up to no good. They've created a package which is called semantic dash types. Okay. Sounds something that is very believable that should be
Very techy.
A dependency or something. Yeah.
It's like we need it.
We need yeah. This is something like you're not gonna question it. Yeah. Semantic types. Yeah.
And that they they made it a a dependence. They made other Solana official packages depend on it by contributing to the I I I simply contributed to a PR and by basically hiding this. Mhmm. So what you now do is when you install these Solana packages in Python is that it automatically also installs the semantic types, import it, and then what it does is it monkey patches the methods on these Solana classes. So let's say and I don't know the exact structure, but but monkey patching basically does is that you have this Solana class, and there is a method on it called generate new private key.
That's when you monkey patch something is that you can say, from now on, this generates a new private key on this class becomes a new function, and I'm gonna write the function. I'm gonna inject it. That's basically what monkey patching.
And you can you can do anything. Right? But I think if you're trying to be sneaky, you can say do this, this, and this, and then do what the function is actually Exactly.
So you could wrap this in something else. Yeah. And basically, they do is that they when they when with these packages, users generate private key is that they intercept the private key and then send it to to an external address.
I see.
Which is, of course, like a big big vulnerability.
Yeah.
But it's also like this is the danger of something like PyPI where where people can publish anything. Right?
Yeah. Yeah. Anyone without a lot of you don't have to put your ID or anything. There's no accountability basically. You just account just person.
So the the only accountability here is that we think and we hope that open source world will correct for this.
Yeah. Yeah. Yeah. I mean, that's that how we've how we've been running until now.
Yeah. Exactly. Right. Yeah. And, of course, like, retroactively, people will will see, okay, this this went wrong. We're gonna kick this out and we're move from our dependencies, etcetera, etcetera. But it's very hard to to avoid these things. Yeah. Yeah. Because a part of this is also very much, for example, social engineering. Like, you start contributing to a project. You people start trusting you. Yeah. At some point, you do something malicious.
What's the case that the x something?
No, there was a recent case around
There was a recent case that they said the same thing that developed because a lot of times we're saying we rely a lot on the open source community, but a lot of times that this community is not well supported. Yeah. Yeah. Right? So you have the people that like, everyone is expecting you to do everything very quick, but people are doing this on their spare time, they have families and all these things and people burn out and then someone comes and like, oh, yeah, let me help you this.
They do a few nice PRs and then
I think from the moment you sufficiently trust people, you're not gonna review every Yeah. Every PR and detail. Right?
And sometimes it's like, it's not even like you can't review them, but like the fact that this is monkey patching is not it's probably more obscure than this. Right? It's not as easy as just like, yeah, this is what it's doing. It's fine. Listen, isn't cool.
It can get fairly complex. Right? And yeah, people gained your trust. I'm wondering as well, if they if this malicious package could monkey patch other libraries like dynamically as well, like a random kinda. Because in Python, right, like if I install, I don't know, if I install pandas, pandas will depends on NumPy, so I'm actually also installing NumPy, right, actually the whole tree of things.
So even if you don't install Solana directly and you don't install semantic types directly, if something depends on Solana, you will also install in semantic types. Yeah. Right.
Case here. How do you this?
And so there's the whole tree of things. And I'm also wondering like semantic types apparently just steal Solana keys. But I'm wondering if they could also dynamically do some other things, you know, like, not necessarily only Solana, but like if you install Solana, you install this, and then you can take any Yeah. There. Or like add itself as a dependency somewhere, you know, like Yeah. So yeah. Python's super dynamic, which is sometimes is
That's why you came on Capacha. That is Most
of the
time is not a good idea.
Most of time is not a great idea. Unless when it's about naming a podcast,
And it's great. Only listening to it. Only listening. Yeah. So we have final topic.
Yes. JSON repair.
Yes. JSON repair. We we always try to include a single Python package. Well, not necessarily Python. A single library.
Yeah. Something that people can use.
Something that people can use. Yeah.
So That is not malicious.
JSON underscore repair project offers a lightweight Python module that automatically fixes the misplaced brackets, straight text, and other quirks LM often introduce into JSON responses. Its readme observes that some LLMs are a bit iffy when it comes to returning well formed JSON data, and that motivated the tools creation.
So this is
So what it does so if you've been using something like CGPT or Gemini or or a Sonnet to extract information from text, then typically you ask it to fill in, like, a JSON tools. MCP tools often also return something in JSON. The problem is that that JSON is not always valid because a bracket is missing or a quote is missing or whatever is missing. What is typically being done is by packages that allow you to quickly do this, something like I think Pydantic does it. Pydantic.ai does it.
It's the other one, the instructor as I could. It checks, is it valid JSON? And then if not, it just sends it back to the model and ask to fix it.
Do it again.
Yeah. Do it again. So you get this and hopefully, at some point, you get it get it correctly back. That is a way to do it. I think that's being done a lot, but automatically big behind the scenes. Yeah. This is a way that to do that in a more deterministic manner. It's probably a a safe extra check, I would say. Interesting tool. I think also the the it's very easy to use.
It's it's also interesting, I think, how they the transparency of how what it supports just by looking at the tests. So the tests are very nicely structured, the unit tests in JSON and score repair.
Okay.
The units are very nice structure. So but just by looking at the unit tests, you basically see very intuitive to see, like, okay, when the quotes are missing, this can fix it. In this situation, it can also fix it. Those are the situations that this cool tool can come in handy.
Oh, nice. Yeah. Oh, cool. And this is from the that's that's actually an under well, I don't know if it's undervalued, but one thing I really like about test as well is like to to use it to understand better what the code is doing or what is this and that. This is really cool. Yeah. Yeah. That's nice. It's probably cheaper than sending it all back to JGPT.
That as well.
Everything. That as well. Yeah. Yeah. Right. Yeah. I also saw that they have a GitHub pages that you can actually do it online. You can actually put your input JSON and then it will pair for you if you wanna give it a try before hosted with Python anywhere for free. So this may be a API quota limit, but it's really cool. Pretty cool.
I wonder if these yeah. No. Yeah. I wonder if like the instructors or all these things, they could also if they do this already, like they try to fix it for you deterministically before sending you back. It's probably better now.
It's cheaper and I don't think that they try it today, but I can be wrong.
Yeah. That would be cool. Have you tried by the way?
Yep. Yeah. I actually have our we have a family AI assistant.
Yeah. What's his does does it does he or she or it have a name?
Octav.
Octav. Any particular reason?
No. It's a nice name. Octav. Okay. Or an assistant.
Octav. Feels like a butler, you know.
Yeah. It's a bit of butler. Right? Yeah. Exactly.
I I I think of what? Is Jenkins the logo? There is like a guy like this.
But it's I use IdentityAI for it.
Yeah. Do you is that your go to agentic AI framework? There's a lot of them.
There's a lot of them. I build it just before the new hooking phase one.
Small agents.
Yeah. I think I would prefer that.
You tried that one as well or no?
No. But I think I would do it now if I would to re have to rebuild it and build it just before that. So it does very basic stuff. I like it. It's every day.
It gives us small update on, like, the major things that happened those day to the the day, like, meetings and stuff stuff like that. We can ask it to put something in our agendas. We can ask it to put something on a grocery list, like, these type of things. Weather, like like, to well, this like, it's very basic basic things that normally you would like every somewhere in the day look up. Yeah. Now you get summarized.
It's gigantic. It's just like model with a whole bunch of tools.
Yeah. Exactly. It's it's by the it's by the end of the guy. Yeah. So it's a very simple simple use case, but I would probably use small. The to me, the what I find difficult as a let's say, from the point of view as a developer of the Gentle AI is the intransparency of the calls.
Yeah. But IdentityAI also have the, I think, LogFire.
Yeah. But I think that is a huge overhead to understand and think even that is not very transparent on the exact calls that are happening. I think it's but that's what I would want to maybe I would be happier with developing in Small agents. Small agents. Yeah.
But that's but if you don't know, like you just think that there may be more transparent.
No. I'm just going through the documentation stuff.
Yeah. You went through the documentation. Okay.
So maybe we should need need to find a use case to build something in small age.
Maybe for this, maybe we can't we can do chapters, transcripts, all these things. But I'm also curious about these things. But indeed, sometimes I think think the ideally you have the same use case and you build it like different ways with different frameworks, but it's also very like kind of defeated with the purpose, It's gonna be a grub brain developer is like, nah, just get it done, right? Cool. Cool.
I think that was it for today. Do you have any well, anything any big plans for the the rest of the week? Anything?
No. We have some family that is visiting. So
Nice. Nice. Nice. Enjoying the warm weather.
Very warm. Yeah. Yeah. Super. But yeah. We shouldn't complain. Right? Like we complain already so much in the year that it's bad.
Yeah. That's true. That's true.
Let's just enjoy it.
I feel like sometimes when I complain, people are like, oh, but you're Brazilian. Like you shouldn't complain. I was like, yeah, but I left. You know? Like, if I loved it so much, would have left. Is
fair Yeah.
It's like maybe I'm the only one that can complain. It's like, I moved so many, like thousands of kilometers.
But do you also complain when it's bad weather?
Sometimes. No, but I don't wanna complain not to complain too much, but I do think sometimes when it's too warm, it's like, it's hard to be productive if you I
think I was thinking today I was in my is like in my attic, like my home office, it's too warm. Was thinking maybe I should look into getting air conditioning.
Yeah. Right. Yeah. And the houses here, they're like most of them don't have air conditioning. Right?
Yeah, don't think it's standard.
Yeah, but I do feel like it can make a difference. But yeah, but the sun is nice.
The sun is nice. I'm getting my tan lines on.
Oh hell yeah. On your bike rides and stuff. Yeah, Alrighty. Cool stuff. I like this the the the summaries. I think it's it's better. Mhmm. Nailed them. I don't know if we have do we have an outro already?
Oh, I actually did. Yeah. But we don't have it in the you know, on the sound path yet. So we'll just replay the intro.
We'll just replay it in replay the intro today. Next time,
we'll have
it there. Cool. Alright. Thanks everyone for listening.
If you have ideas, feedback, let us know. If you like
to be a guest.
If you like to be a guest.
Hit us up.
Please rate us wherever you listen to your podcast.
Yes.
Really helps us.
Yeah. For sure.
And we'll talk to you next week.
Yes, sir. Ciao. Ciao.