How Deepfakes are Evolving (And What You NEED to Know)

With Jenny I, it's become easier and easier to create videos, very cool, but also fake videos and deepfakes. Today I'm joined by deepfake expert, CEO and Co founder of Duckduckgoos, AI Pariah Lotfi. Believing in what you're perceiving. That's the company motto. And she shares many stories of what can happen to businesses, but also to you and me and things that have happened to businesses and to people like you and me. I love this conversation so enjoy.

I was on the way to the office and I was on Reddit because that's what I do till I relax and stuff.

And I saw this video and it was like the seagull on a car dashboard and the guy throws a fry, but obviously the fry is like behind the window screen. And then all of a sudden the seagull bursts through and grabs the fry bursts. And I'm like, what the hell did I just watch? And go in the comments section If you're like, well, this is AI

generated obviously. And then other people said the beginning of the video is not even is that's real and all of a sudden switches to AI. That's also what caught me because I was like, the hell is this video? And then it burst through the window. I'm like, Oh my God, yeah, it's getting very subtle right now. That's true to a point where it's ridiculous.

Yeah, yeah, yeah. They are mixed in reality with fakeness using AI, Yeah. And that's very powerful actually, because indeed, the first thing that you see is like something real, something familiar to you. So you would believe it. And then all of a sudden changes and the chances that you would for example also believe that

part right now is higher. And that's also why we say the the basically the difference between what is real and what is fake is getting everyday like more and more fake basically. Can you also do like if you have a video, can you do part of a video frame and then fake that part as well? Yeah, what happens right now also with these very accessible large language models like ChatGPT and the others is that you can upload one single frame picture, OK.

And it's going to make it fully like a moving basic D video for you based on what you have been uploading. I've seen a couple of examples of you what what you mentioned basically just yet, but that was with more like in a setup of like politicians, they were all sitting Trump, Biden, etcetera. The the first part was real and then in the second part, all of a sudden they started fighting. Like, you know, they started

basically fighting. And I was like, wow, what didn't, what did I just watch what had happened here? And that's basically also how they play around with such technology to not only make funny and nice things out of it, but also really fraudulent and with bad purposes, with bad intentions basically. And that's basically also where

this dangerous part comes in. I'm not sure if you have seen it, but there was an interview of Sam Altman, I think yesterday or the day before, where he also mentioned it's unbelievable that these days we still rely on facial biometrics, voice biometrics to, for example, do very high security jobs such as logging into your bank account, transferring money. And also for normal communication ways that we have like we FaceTime the whole time basically, right?

We use Zoom, we use Teams. And also in those scenarios, it's very likely that someone using only open source technology, you know, can impersonate someone else. And this can happen in a business setup or context, but it can also happen to a mother who receives a call from their child saying, mom, I got kidnapped. You should pay this amount to this person or this bank account, and then they will let me go.

And you can imagine with that, you know, emotional state of mind what you would do. Of course you would. Maybe, I think a high chance that you would do what they asked for. And this is becoming our reality. This is already the case. Like the you can already do stuff like that, yeah. Yeah, yeah, it's unbelievable. But you can already do stuff

like that also on a big scale. So the difference when it comes to so generative AI based image and video manipulation and generation compared to traditional ways of doing it because it's it's been there always, right. We have some pictures of Stalin basically with his friends and then all of a sudden the friend was gone from the picture because they were not friends anymore. Yeah, removed from the history. So those were the, for example, the first types of image manipulation.

It's always been there. The difference right now is that it is so much more scalable and so much easier for everyone to be able to make it. We have seen cases where mostly also in the US, but also in the Netherlands, you would receive a call or a voice note on WhatsApp from someone, a loved one, and then they claim things like I just mentioned. I received also a call lately from it was, it was clearly an AI voice. I received this call. It was, it looked like a normal number.

So I was like, like, let me just answer. And then it said in Dutch, Hey, I think it also knew my name. That was really surprising. Hey Pariya, just wanted to know if you're interested in buying houses and real estate in Almira. I was like, wow. And the goal of the of the basically voice behind the phone was to make an appointment with me, which I don't didn't understand because it was like an AI voice clearly. And then making appointment with me.

I'm like, why is that? Yeah, what's the purpose of it? Or maybe they just wanted to hear my voice. That's also possible. Oh really? Because that's that's also really dangerous because right nowadays they only need like maybe 3-4 seconds of someone's voice to be able to generate a perfectly realistic sounding clone of it. So this podcast. It's a horrible for me, it's a horrible for you. That's true, yeah. Yeah, I think.

About Nah, but like for me, I don't feel like those types of technologies are very accessible, at least for me. I tried to play around with like video editing software. I I don't edit the podcast anymore. I used to so I was still playing around with it and I saw that the software that I use had this AI functionality, seamless transition and we use Gen. AI to do so It's like, OK, I have my podcast, I'll just cut and exactly the same frame.

I'll put a little bit after and then I'll just have Jenny I in the middle. That's what I thought. And it's just me reacting like this. And also I go like I basically blurb out in a weird way and I was like, what is this? And then it just snaps back to the frame. That's real life.

So it didn't look great. I, I could clearly see if something's wrong, but then I do see examples online that it's like very, very specific and I have to double check and I have to because I mainly see these on like Twitter and Reddit and more community based things. I have to go to the comments section and other people have to educate me what I'm seeing. Yeah, that's scary. That's scary.

Yeah, exactly. Yeah, and you're working on software to signal that to at least see what is real and what

is fake? Yeah, that's exactly what we do. So we basically use AI to catch AI in that sense at the the Goose, we develop explainable but also generalizable AI technologies that would help human beings, companies, citizens, everyone basically to be able to distinguish what is fake from what is real when it when it comes to the digital world. How do you do that? Like can I say me with the AI to actually create this? Can I also have the same AI say as this real or fake?

Because I've tried that, it's not as good. You have trained. Your own systems. No, I haven't trained my like I, I, for example, I received on Twitter sponsorships opportunities. I wrote a LinkedIn post about that. It got pretty close actually, because they can now generate a website and there it's like, oh, you have to do this digital signature. And it downloaded something on my computer. And I was like, oh, that's,

that's a red flag. And then I thought, OK, everything in this conversation, there were many signals of red flags. But because the companies were super cool and the people had had Twitter accounts which were verified for years with post history and I could see they have LinkedIn accounts and they're actually people that work there that I thought it was real. I want to tell myself it was real.

But in the end, when it came to a digital signature, I'm not going to download a program from some. If I look at the domain, a weird sub domain that is like a tool that I've used in the past, but it's not really the real thing. Like those are all signals to me. And then I asked because one of them said, OK, you have to sign this NDA. It wasn't a here's the contract. Let's sign it was your first half to sign this NDA? I got the NDA and the first thing I did was I added it to my

ChatGPT at the time. And I was like, is this real fake? And they were like, well, there's no, not many reasons to think this is like fake. So it could be real. I'm like one of the reasons and they laid out the things that are excluded or like omitted that for me made total sense. How long is the NDA valid for which specific things? Does it make sense, yes or no? Some legal things that were missing.

So I was like, OK, like all those signals and even my AI, which was likely the same AI that they used to create an NDA in the 1st place, yeah, Said OK, it could be real, yeah. Yeah, yeah. OK, that's that's neat. And I won't understand what you're saying. I think it sounds almost also like organized crime people setting up websites and, you know, trying to get others to communicate with or do something else with them.

When it comes to large language models like Chi GPT being able to distinguish what is fake, we have seen indeed also research being done regarding that. The only difference is that these models, they are general Transformers, let's say and they are really good in pattern recognition when it comes to

text. But when it comes to videos, you know, pictures and even like NDA's that you mentioned, specifically asking them is this fake or real might not be that effective because they are not really meant for that purpose. And that's the difference between, for example, your question that you asked to chat versus our systems. We have been out there for five years almost right now. And that was also really preach IGPD era before it got really popular and everyone started using it.

And what we have done throughout these years is basically training our very specific systems for very specific purposes of catching different types of deepfakes with the biggest basically also uniqueness to be able to explain as to why a video, a picture or audio is seen as a deepfake. You can explain why. We can explain what kind of reasoning the neural network is following to come up with that

conclusion. Interesting. And AI being explainable is not really common. Of course, it's one of the biggest challenges in this world right now. Since AI is by design, like kind of a black box, like magic. Exactly. We don't know what's happening. Also at the the Coos, we're trying to kind of understand what are the reasonings behind this trace of thought. And we visually also show that

to the end user. So when you have, for example, a face swab or this bird that you mentioned going through a window getting this fry, we could for example make our users see which parts of this video have been manipulated specifically and which type of defect generation is behind this manipulation or creation being generation basically. So in that sense, we are really specialized in basically

detection of deepfakes. And with deepfakes, we basically mean every type of audio visual media that has either been manipulated or completely generated using AI technologies in comparison to ChatGPT, which is more, I'm so sorry. No problem. Okay, yeah, Which is more like a general type of AI for general usage scenarios. And when you said it could kind of explain which type of deepfakes have been applied to what you're seeing. Are you then saying audio video as type?

Or even within video there's subtypes. Within video. Even so, because we use this comparison like defect generation techniques just pop up like mushrooms from the ground like everywhere on a daily basis. You have 10s of new defect generation techniques only from the scientific world basically

coming. And then let alone what happens if people who are also maybe part of organized crime groups have bad intentions to use the effects to maybe open bank accounts or scam companies, scam individuals. So in that sense, it's very important for us to know how a certain deepfake has been made to also work on our threat intelligence part. So when it comes to, for

example, a single frame simple image that can be a deepfake, but it's a different type of deepfake. Then you have a face swap deepfake. So you can right now take a picture of me and put your face on my face. Sometimes I kind of know blended sorry and the other type would be for example generating a completely non existing human in that sense not a picture anymore, but something that has been created by a neural

network. So we can for example tell that this image has been fully synthesized or this image is partly real, partly face swap or partly lip sync. When it comes to videos to make someone say something that they never have said, that's basically usually a lip sync deep fake video where you don't have mostly this part of the face manipulated to make you could synchronize to the certain audio that you want to put on the video itself.

Fascinating, how much education do you need to do to your customers or organizations in general with regards to this? Because I can imagine a big difference between kind of pre Gen. AI era and now. Because I saw this video just online so I am aware of the possibilities or at least I have a lot of assumptions in what is possible and what is not. But also, I'm in the tech scene.

I don't know many organizations that are not, but let's say more traditional banks, they do have IT capabilities and they have a very low risk appetite. So maybe they are more aware. But companies that don't have that, they can be like, well, why would it hit us? Like what would be interesting for us in this aspect? Yeah, yeah. So once we started five years ago, we basically had to explain to everyone, like 99% of our interactions.

We had to explain or start by explaining what a deepfake actually is. Right now there is much more awareness regarding what generative AI is, what deepfakes is thanks to HIGPTH programs and and tools. Basically, we still need to touch upon every relevant use case for the person or the company that we are talking to that has approached us to make them understand to what extent they can already be exposed to these deepfake based or Gen. AI based fraud types. And it starts with like

individuals getting, you know, a deep fake call from their loved ones, but also a teenager being put in an 18 plus non consensual video up to market manipulations, fake fake news that we all know about. And even like stealing money

from banks up to 1,000,000. Aside from the ones that have made it to the headlines, we have seen several cases like even Dutch small medium enterprise companies, they're also getting targeted by a voice that calls them that sounds like their colleague asking for either information or money. And this basically showcases that the scale that is happening right now already is a real threat.

It's not theoretical anymore. We see it also happening a lot in flows that you have lots of volume of volumes of data. As I said, just with with the Sam Altman example, he just said, you know, why are you still using face for her for authentication and identification? It's still a very secure way, it seems like, but it is not because deep fix Gen. AI are basically able and capable of mimicking all these, you know, human like characteristics that we have in a video or in a picture.

And what happens usually, of course, you use your, you know, face, your voice to be able to open a bank account along with your passport picture. And when it comes to, for example, also banks and neo banks, they have on a daily basis thousands of new clients that want to, you know, open a new bank account and maybe like 10s of thousands that want to log into their account as well. And they use facial biometrics.

So in that sense, we also see a lot of deepfakes coming in to basically be able to make an account based on a non traceable identity using like a picture of a non existing person along with a fake passport. And then they use that to be able to open a bank account. And it doesn't stop there.

Once they find a method, these people with bad intentions to basically open that account, they go ahead and open maybe 10s of or hundreds of them and they eventually use it for really bad purposes such as money laundering. And then from there on also supporting organized crime even more. And that's one of the flows that we see lots of defects basically up to 10% on a daily basis in peak times coming up. And that's only one use case, let's say with remote identification and

authentication. We also had cases in which people like individual people, maybe also more vulnerable groups of society calling us, saying I just heard you talking on the radio. I have a girlfriend online, never seen her. Oh no. But I send money to her because she. She needs it. She needs it. Yeah, of course. But yeah, it got me thinking,

can you please check it for me? And of course, we went along, we checked and we said please don't send money again to this person because you know, it's she's, she's a deep fake, which is. This use case you understand because they reached out to you. But when it comes to the other use cases, the more let's say Baker skill fraud, how do you know these things already exist? Like how do you get your information?

Because if I'm a bank and this happens, I wouldn't want to publish this because I'll become a target. And it's kind of a shame thing. And you know, these cases exist in the first place. Yeah. So that's basically by applying technology. And also it's all starts first with a simple question.

Am I exposed already? And that's a very powerful question, I would say, because if you'd go ahead and say, you know, doesn't happen to me, it's still a technology that needs to mature 1st and then maybe it'll happen to me. And once it happens, then I will take action. We can also do that. But it's like I would say it's not really a smart way of dealing with new types of fraud specifically in this age of AI and everything automated. Yeah, you want to prevent. Exactly.

Yeah, you want to prevent monitor mostly as well, because cybersecurity on it's own is really, it's, it's like it's a cat and mouse game. You have these fraud, these fraudsters, they're also usually very clever people, also

collaborative. And what's surprising to me because one of the most prominent reasons that we see so many defects in banking flows, for example, is that on the dark net, but also on some other places and online sources, you have groups of people basically brainstorming together what

types of defects, what types of material and input material is useful to be able to spoof a banks identity verification system. And they basically they. Discuss this, you can see this. On the darkness, if you know which channels. You know where to look. And once they also find this way, they don't stop by only using it themselves, but they go ahead and monetize their knowledge. Oh shows a business. Yeah, it's a real. And when I heard this, I was like, it was a couple of years ago.

I was like, yeah, I wouldn't buy deepfakes from these criminal people in the darknet, right, Because you can give them your money, even though it's. A scamming other people. Exactly. They can scam it and you know, they can get away with it. And I cannot go to the police, no customer service, but apparently they do have a customer service. This doesn't make any sense. Yeah, it's, it's real. It's like for real.

I I talked to an expert who is following also along what we do and, and he was like, yeah, if the deepfakes don't work for your purpose, so you cannot open this bank account, you can go back to them and they will give you a new one or they will give your money back. Man, that's good customer service. They they know their business. So, yeah, but that's basically knowing what's happening on the other side, being the criminal people side.

And then, yeah, basically trying to answer this simple question of to what extent are we already exposed? What should we protect ourselves from? And once you do that, there is already technology that you can put in place, like the one that we develop, but also other things, of course, other methods that you can apply and to basically train your, you know, stuff, train your own mindset regarding this new reality that we have right now, the fake

reality. And yeah, and, and that sense when you start monitoring, then you see things happening and then you can also, once you do that, you can preventively stop things from happening. Instead of letting these people make this bank account 1st and then letting them put their money on their black money usually and wait for anti money laundering systems to get triggered. And then you're going to off board them. You also do a lot of investigations before.

And once this start happening on on scale, you also have big problems with, of course, legislation as a bank, taking that as an example. Yeah, and and really high fines basically. So in that sense, your reputation is also in danger, your own. Yeah, systems and the flows are in danger. It's a very interesting and also very king, king. It's very dark because we heard that it's happening in South America a lot as well in Europe too.

But in South America, they are really creative when it comes to using deep fakes and the cartels and mafia groups they're basically use it for. So once they have this, this golden way of opening accounts based on non traceable DPEC identities, they make hundreds of them. And then every account they will put a small amount of their black money, which is basically millions or billions even to make sure that the AML systems are not getting triggered. And it looks like a normal bank account.

It looks like normal bit of a couple of thousands of money being put on it and then they use it for human trafficking purposes, for supporting terrorism purposes, which is of course, really, yeah, a really bad thing, I would say. Yeah. It's like, I agree. It has become way more accessible than it used to be. Yeah. And I think if I were to find and like create a video that looks real, I could probably achieve it like as a means to an end. I could do that. And then it's the looking from

the other side. How do you prevent that? Because accessibility is huge now. Everyone can do things and people with questionable morals will do questionable things. And then you're there. You already mentioned there's stuff being discussed in the dark web, so you also talk to experts and you can also do kind of your own investigation. Is that usually how you think of the perspective that these people have? People with questionable morals. The fraudsters.

The people that are scam scamming other people. Yeah, I'm not sure what they're thinking. It's basically, yeah, it's another way, of course utilizing technology to for your own purposes, which might seem good to you, but of course the person that got scammed, they are not happy for sure. So in that sense they are. Technology is open source and it's basically a neutral thing. So you have a lot of deep fake tools out there, applications that you can just download. You can use it for funny

purposes, but you can also try to use the same tool or try to use the basically the tool maybe in a modified way for your own bad purposes. So about that, we cannot do anything I would say drastically effective. We cannot really prevent. That was actually one of the aspects that a couple of years ago the Dutch government contacted us about saying we are investigating ways to be able to prevent defect technology online.

And my Co founder and I were like, yeah, even though you've if you would do that, how are you going to enforce that? You know, how are you going to ban all those applications, all those websites, all those open source code that is being put on GitHub? So it's basically not doable. And when it comes to specific dangerous use cases, of course there are there are things that you can do.

And also, Patrick, one, one of the things that we see is that it's still a new phenomenon and people are trying to kind of figure out how to deal with it. But also sometimes, actually, usually we see companies that want to take measures. They're like, yeah, we actually want to do it, but there's no legislation yet that kind of, you know, motivates us more to be able to go ahead with, for example, putting technology in place to be able to detect deepfakes.

On the other side, legislation is really slow when it comes to coming with new aspects for, you know, protection regarding new technologies like deepfakes. And then in the meantime, we have normal citizens. So not the private company world anymore, but normal citizens also suffering once a teenage girl, you know, gets deep faked in front of her whole school and then they eventually end up depressed and Oh yeah,

humiliated basically. And the question that since is where do you start investigating this problem? And I would say us having be having been in this space for five years, it's actually a problem that you would need to collaborate a lot on together to first of all identify the scope in each vertical in each place of the society. And then see how together with of course, education and technology used in the right place, we can come up with a new framework to deal with this new fake reality.

As I just said, I was also seeing some deep fakes lately with all these wars going on, you know, around the world. One of the things of course, that we need to also ensure is the integrity of news and media, right?

So we see videos, you know, coming on coming from Iran, where I'm also originally from, by the way, with this war that was there, the 12 day, the 12 day war or, or Gaza or African countries that are also in war. And then you see, like I saw literally a couple of deep fakes being published from these bombings in Tehran, in Iran, only because it was the deep fake basically was supporting the framing of this news

channel. And it got me really thinking like if these news channels that all have their own framings, I was following different channels when this 12th day war was ongoing between Iran and Israel because I wanted to get a lot of information. I still have a lot of family living in Iran. They're all good, by the way,

for now. I saw clearly the difference between a lot of media channels that I was following and then them using a couple of them, two of them using deepfakes on their social media and under other channels to basically support their own point without questioning even, is it real or is it not? I'm not sure if they made deep fix themselves, maybe not, but they published it and they do have a big audience. And that got me really thinking that. Yeah, it's basically weaponized.

You know, it's, it's they can weaponize it to basically also achieve bigger purposes to. Yeah, mislead people. Yeah. Spread less information. Go back to like proper propaganda. Exactly. Yeah. And this, this got me thinking like if you don't do anything properly about it, soon our reality will become everything that we see digitally is fake unless the otherwise has been proven. The the, the contradictory has been proven. And then that will be it's not

fake, it's real. But again, that's also a complicated question. So how would you prove if something is real or not? Yeah, this is getting really scary. Like pre Internet era, you had the newspaper and there wasn't as much global knowledge. Whatever was in your newspaper was the truth. And then it's very easy to have one source of information and then have propaganda there as well. Then it's decentralized because with the Internet, I can go on Twitter and I see people posting things.

I'm like, well, that's a different lens, right? And those people are more, maybe more accurate and maybe more there. And now with general availability of Jeep deep fakes and it being more accessible, there's going to be a lot of noise out there. Like worse than the truth. I think it's going to be

incredibly scary. If you're working on cybersecurity, It's not a domain I'm I'm very much involved in nor aware of. But for any technology that you use that is kind of preventative, these fraudsters are like your audience. And I feel like you need to know your audience to be able to reenact a scenario.

If you're developing a piece of software, you can't really user test with fraudsters. It's like, yeah, you have to know your audience. How do you develop something that is preventative in nature like this? Yeah, that's that's a great question. At Doctor Goose, we do have a lot of red teaming activity. So it's basically us trying to think as these criminals and. Is that what red teaming is thinking? Yeah, OK. Yeah. And then you kind of try to attack, you know, the defence

systems based on that. And then we have also our blue team that tries to, you know, it's it's basically this kind of mouse game. Gotcha. Yeah. So that helps a lot because we really need to understand the mindset of the people who have intentions to attack, you know, some system or some person, some company. Otherwise you just blindly develop technology. And then once you put it in production, you'll see, oh, it's not working. And of course, a lot of monitoring, as I said at the

beginning, that would help too. Because this threat intelligence that you're gaining right now comes partly from indeed these conversations that we see happening in the dark net and these great customer services that they have.

But also in production environments, once we are monitoring data and indeed trying to identify defect types that are being used to make these defects that we are seeing in production, that helps a lot in basically trying to adapt our strategy as well when it comes to designing our defenses. Yeah. Which topics do you focus on? Because you mentioned a lot of things, right? It can go from large scale fraud in banking. And I've also seen many posts online because getting a job in

tech is now harder. People are being laid off, people that are graduates are finding it harder to get their first job. Might even automate something with AI to apply to jobs and like fake a voice thing and get through a first screening call. I feel like all of it's fraud.

That's. Happening already. That's happening already as well there. Was last year, I think, or the year before even this American cybersecurity company, you know, before they published A blog written by the CEO saying that they mistakenly on boarded a North Korean spy as a new remote employee. Yeah, big whoops. Yeah. Yeah, exactly. And it's really nice that they're being open about it. And they also, right before this person was about to do these fraudulent activities, they

caught the person. So that was good. But in the whole process, the person used a deep fake face, apparently to be in those Zoom calls or Teams calls. And I don't know which platform because it's possible with all of those platforms. Yeah, it doesn't matter. I even have a deep fake mask. And. Yeah, I don't have it with me right now, but we wait. What does that mean? It's basically if I would put you in a Teams call right now, you can get my face and then.

Oh, really? Can we do that after I I would love to see that. Yeah, 'cause I'm gonna call, I I need to call a colleague. We can even like if if we take your face right now from this video recording on YouTube later and we can also train a model that is able to mimic your face in a Zoom or Teams call, OK. So I might do, I might do a separate clip, yeah. So look at the camera as well for 10. Seconds setting up, yeah.

Wow. So yeah, it's happening already in terms of remote job applications. There is even like a whole North Korean, I think group of people doing this, attacking American companies. And in the case of no, before, they only found about it once they sent them, send this person the hardware of their company so he could start working. And then they saw that he is trying to inject malware basically into the system and that that's the point where they blocked it.

So yeah, it happens already a lot. And if you have like a company that is sensitive to lose information like ASMR or other companies, let's say, and there is like a shortage of stuff as well, so you would tend to maybe remotely hire people, then yeah, there is a good chance that you have a deep. And also to answer your question, where do we focus on right now? It's for us, it's mostly the fintech and the financial use cases along with video conferencing as well.

So and, and that's mostly also for companies that are required and also desirable to have high security measures basically in place. On the other side, if there are requests such as this person that I mentioned calling us saying I have a girlfriend, can you tell me if she's real or not? We also do that. We have like a specific team taking a look at those requests

as well. The ultimate goal that Duck Duck Goose has is to become and in three to five years become a platform that everyone, not only banks, but everyone can use basically to go there to see if the video, the picture, the audio that they are seeing or hearing digitally is real or

not. And that also includes Citizens, yeah. Yeah, I feel like everyone now in especially in tech, in kind of the onboarding process is dealing with some form of AI indeed. It could be someone that's not really the real person. It could be someone whose voice is different, could be someone that's not even that same person and just another person doing the onboarding thing, which is not even really AI.

It could be something that someone uses with regards to kind of asking a question real quick to AI and then reading from a screen or a script or something like that. I feel like onboarding is right now already being kind of compromised. Let's say XEBIA, we have a eight hour, 8 hour assessment. It used to be 8 hours. Likely with AI, it's no longer. And we were like, well, should we prevent people from using AI because we do use AI to be more

productive? Yeah. So we were like, okay, we will allow people to use AI, but we will kind of frame our questions and then that interview round with the assumption that people have used AI. And if they use AI with a certain technology, we already have pre prompted kind of our prompts. So we can say tailor it to this technology and we can then compare outputs because then we can see if our AI has this, this and this, it takes us X amount of hours.

This person has delivered this in eight hours. What are the differences? And then we specifically can ask questions on those, but that's because we're aware of this. We're already changing our onboarding. I feel like if you're not aware then even kind of changing your onboarding is not there. Yeah, yeah, that's that, that's very true. I would say trying to yeah, ban AI usage or tell people you're not allowed to use it.

Also, even like students at universities or or in I I have a cousin, she's 11 even she uses she lives in Paris, she uses chachi PT for her homework. So I would say it's, it's like maybe the time that these digital calculators, we could also say to people like don't use it. It's I would say it's a tool that we can use to become more

efficient. Right now, if you use chat or cloud or perplexity, it cannot write a good quality report for you on a specific topic, being cybersecurity when it comes to Gen. AI. But you can use it as something much faster than Google to come up with relevant sources when it comes to perplexity. For example, to have your, you know, database of, you know, papers knowledge that you want to gather.

And then based on that, you can maybe even ask ChatGPT to summarize it for you because I, I think ChatGPT is better in summarizing that than perplexity. You can use those insights to make your own work faster, but it wouldn't really. I went with you right now with this stage of, you know, that the quality that they have trust in AI to write an essay for me or write a code for me.

Basically right now we do use a lot of AI for more like indeed what you said, junior level of programming, cloud mostly it's like I think €160.00 a month for the professional version and you have like a kind of a junior, you know, AI assistant there. The code is suitable for, for example, front end.

But if you want to use it for the back end, you shouldn't actually unless you have a senior person, for example, like we have in the team that looks really at the code to see, you know, if everything is good and OK and scalable for later. If you want to change things, is it in good quality and changeable later and then deploy that code. So it's kind of basically there to make things faster, but definitely not to replace, I would say for now.

On the other hand, I also heard, for example, the BIG4 have always been really targeting fresh graduates and even like interns, graduate interns fresh out of the university right now. Apparently what I read is that in the UK they completely stopped doing it because of AI. They're like junior work can be done by AI. It sounds scary. It's also maybe for us as human beings, like, oh, my job is not necessarily, you know, needed anymore.

On the other hand, I think it's an opportunity for us to become better in other things. I mean, the same thing also happened in like 1800s, right? With the industry revolution. You had machines, they were faster and it was like the people are kind of only serving these machines. But then we got the chance to become better in technology,

right? We had then we started making these tools and, and right now look, we are where we are and without those machines, it wouldn't maybe have been possible because we would still be doing, you know, a lot of job with our hands and, and, and not not even out of my. So I compare it to the same basically what happened also 200 years ago, but then the scale is much faster and it's that's maybe why people are scared. Yeah, I can see that. Yeah. I mean, I think so with digital,

we had digital natives. And I feel like with generative AI, you can have people that are very well equipped that are not afraid to try out new tooling that can be very effective maybe from a learning and education standpoint and can get up to a certain level that they need to be effective very, very quickly. Or they maybe don't have the same depth, but they can go really wide and they know a lot about a lot of topics.

I feel like now more than ever, the way to distinguish yourself if you're starting out, if you're early in career, is to indeed find your skill set, find what you're really good at, and leverage whatever tooling is out there to distinguish yourself from others. It's just, yeah, there's a lot of people and the technology is ever accessible. But I do think we need to create software and all companies will still live in a digital era to create things. And likely, this is kind of my hope.

A lot of companies are kind of creating the same thing, slightly different and making money with it.

I hope it also enables people to create new things and unique things and innovate. Yeah, yeah. Then I think companies will be unstoppable. They're just developing things. You have anthropic open air, all of them. Google, Microsoft, then the next big thing that the Western world is talking about is artificial general intelligence. Of course, that's like artificial intelligence that is not only statistical models, you know, but human. Like, I'm not sure what you

think about that. But yeah, I would say that that could be that's something that personally scares me. So if you have, I saw a video on LinkedIn, I think it was yesterday or day before there was this robot in a Chinese factory who was able to replace his own battery. Oh, OK. And it was. It was. Really like a self surgery, yeah. Yeah. And like, it's maybe like a first mini step in basically preventing yourself from shutting down because of battery

shortage. So you replace your own battery. And then so AGI, if possible, ever combined with these robotics. That scares me personally. Yeah. What do you think? I mean, so in a vacuum, I'm a very logical, objective person. I don't think we know enough about our own intelligence and like the human brain, the perception of the world, the perception of interactions, what thought actually is, where does that live? I mean, we know it from a brain perspective. What does it actually mean?

What happens when we die and stuff like that? We are so unaware. Like even how the body works. The fact that we have placebo is incredibly interesting. So then how can we, the fact that we don't know everything, create something that will kind of supersede intelligence? I don't know, like in my head that doesn't add up. I know right now a lot of models are just prediction models. It's like a huge decision tree. I cannot even comprehend the

decision trees. And sure, it might be really good at math and I I'm really bad at that level of math and maybe that's sad, but that's also what computers are good at. Like it makes sense. It's very black and white to a certain degree. And yet prediction models are still prediction. That's why we have now non deterministic code and non non deterministic features because we don't know what's going to come out. We can't predict what we have a high likelihood, but that's Jen

AI from me right now. So I don't see it evolving to that unless proven otherwise. Yeah, exactly. What about you? I I think I mostly agree with you. I was thinking about this myself as well. The, the like, AG is everywhere. Everyone. I was like, OK, but have you ever defined what intelligence actually means? That you're creating the artificial version of it?

The human brain is so complex. I used to listen a lot to Andrew Huberman. And he also, he was, he's a neuroscientist trying to make information about our brain accessible to everyone by explaining it in his podcast in a more simple way. But like, neuroscientists like himself and also including Andrew Huberman are already saying we, we know only so little about the human brain and indeed, per SE, but a lot of other things as well.

So yeah. I I don't know how and when we will be able to develop something that is really human like, but if it happens in theory, yeah, I'm not sure if you'll see it. Maybe. Maybe after our lifetimes. I don't know. Yeah, it could be. This could also age very poorly. It's like in a few months it's like, oh, we're actually quite close. Yeah, well, who knows? That could be that's that's also one of the aspects of this modern world. It's really unpredictable.

So yeah. Yeah, with things that move super quickly, Google can release something which all of a sudden creates a new version of generative AI in videos, looks super realistic. Things are moving quite well and quite fast in a high quality manner. How do you as a company kind of try and manoeuvre all this thing pops up? Do we actually do something for this? Do we need to focus on what we were doing? Do we really need to account for this now? How do you make those priority

calls? That's that's a great question. So also things like AGI, agentic AI, like everywhere at every event that we go, it's either about Gen. AI, defects, AGI or yeah, agentic AI sometimes also about stable coin. But it's like, I'm like, is this like only hype? Why is everyone talking about it? And I would say personally, I think right now there are every setup that I see out there also in the Netherlands is about something agentic basically. And it could also be because of the high.

But I would say maybe in five years, 5, five years time, maybe like 60% of these HNDK applications won't be there because they started right now. And then there on we'll see how the usage is not really that optimized and and let's you

know, let's not do it anymore. On the other hand, I heard in countries like China, they are much more instead of trying out these high or or maybe glamorized terminology such as agentic AI or AGI, they mostly are focusing on which parts of our society and of our normal daily works can we improve using automation, using indeed prediction models instead of, you know, going with the hype. That's also more the mindset that I personally have together with our team as well.

So we are right now, as I said, DPLAY technology generate technology is evolving really fast. And the video that you mentioned just yet with this bird example is maybe that's something that we should be able to catch in five years. But right now there is not really any harmful activity behind it. So in that sense, we also distinguish a lot of context saying OK in in that sense, there's a new type of deep fake. Should we be able to catch it or not? Is it harmful? First of all?

Is it something that is bad for our clients and for our basically people that are trusting Dr. Goose to catch their fraudulent AI based activities? And that's one of the questions that we have basically go back to a lot when it comes to all the other yeah words. Yeah, Thank you. No, yeah, it kind of indeed I get this question a lot like do do you, are you scared for indeed Google developing something?

I'm not sure. Yeah, if they do it, that's like kind of a maybe validation that even more validation that what we are doing is really useful at the the goose. And I would also be like it. It's kind of a strategy. Of course you can also, you know, merge with these companies and do things on a bigger scale. Right now there are lots of investment in the counterpart of what we are doing being the

generation part. And the detection also of course, needs to be there to make sure that these generation techniques that are out there and everyone can use them are not really used for, yeah, bad purposes. Yeah. I mean, from a personal sense, you work on this 4-5 years ago, right? No one was talking about it. There were probably some videos that all of a sudden proved that oh, this is a thing. But now with Jen AI, things have accelerated.

As you mentioned, investments in the fact that we are able to do this and that it's accessible that I only have to swipe my credit card and I could do something similar is insane. Prevention needs to kind of also have the same acceleration and you were already there and now you're in the middle of it. How does that feel like? Is the team super excited to work on these things? You don't need to educate customers as much. It's completely different. Yeah, exactly. It's it's, yeah, it's, it's

really nice, I would say. So once we started that, we started that the Cusoversa at the university and of course, it was like starting part time really only. And it's for example, also caused me to be able to get my you know, my my school paper later, for example. But if I would go back, I would choose it like 100 times again

and again and again. It's it's such an incredible journey being able to have a team of really ambitious people together with your Co founders, with the other team members working on something so innovative, so challenging as

well. Because deep detection, I would say is maybe one of the most challenging problems that we have right now in the world of AI, computer vision, etcetera, cybersecurity and being able to work on that, being able to see that your product is being used by banks basically and you give them value. It's really, yeah, it's really a next level feeling for me, I would say. So I'm really grateful that we decided to start Duck Duck Goose like five years ago from University Project.

And yeah, I, I sometimes also go still to Tudelph University of Technology to talk to students. And yeah, I always would say if you have this ambition in you that you would like to create something for yourself, just go for it. And you can't even start during your student time. I was lately at a talk where also parents were there. So I was like, OK, shall I say this because parents usually don't like their children. I was like, OK, I will say it. Yeah. So there are, I think, 200

parents plus 200 children. To like start a company and don't like don't worry about school too much. You said that. Kind of a bit. That's quite funny, is. It parents. Sorry bud. Yeah. Yeah, I'm gonna say this. Yeah, Yeah. So I would say it's really incredible and there are of course, ups and downs. It's difficult, it's challenging, but I sometimes or usually enjoy solving those challenges. Like there was a time two years ago, three years ago that everything was quite stable.

You know, things went OK. It was good it. Was before Gen. AI. Yeah, exactly. Yeah, yeah. Before things exploded. Before things started exploding exactly and I was like you know, I would like to have, you know, a problem, I would like to solve a problem. So I was like OK, universe give me a problem and then I got 2 problems, two big problems to solve. But even like solving those big problems was I really enjoy when after that I felt so empowered together with the team.

And you can see sometimes also, of course you get tired and we are all humans. But then you have your team that I've also had days that I was like, oh, I don't know, it's I'm tired, just tired, you know, just getting tired. But then I was like, yeah, I will just continue only for our team. We have these amazing people in the team, my amazing Co founders. So we'll do it, we'll do it together. So it's also this feeling of connectedness, you know, being

in this together, being united. I love that a lot. I also love how full circle it went with you then educating kind of your journey to other people. You mentioned in there that this part of cybersecurity, detecting fraud in this manner with regards to defects is also becoming super challenging. And I only have assumptions because I can see there's a lot of investment here. From my human eye. I can barely, I, I can see some

distinguishments. But yeah, I go to a comment section. I, I use the community to figure out what is real and what is not real, which is not really great signal, which means that I can see from my perspective that I can be more and more challenging. But what type of problem is that? Is that the current models that do detection versus generation are not as accurate? Is it a data problem?

I don't even know if you need let's say sample data or like data to go off of. Is there not enough data? What type of problem is it? It's a combination of all the things you mentioned. It's, it's all about data as well. I I don't believe more data will solve all the problems that you have. Definitely not. It will also maybe even buy your bias, your systems in a certain way. The the challenge comes with first of all, all these different deepfake generation types out there, these methods

and techniques. The second one is also once a fraudster has really its intention of doing something, they won't stop with using only one method. For example, the insurance use case. There is also like it's really even in the Netherlands, it's booming. It was on the news I think recently that someone managed to get 150,000 tons of EUR basically because he was able to show that he couldn't go on his 30K vacation. It was from 5 or 6 year

insurances. And then he had this Jenny, I made a doctor proof that he's, he has been sick so he couldn't go there for him to cancel. So insurance pay my money back. And of course, you go ahead and generate this type of, you know, proof with maybe ChatGPT or some other tools that you have to help yourself based on open source things. But then you won't stop there. You won't. You will also maybe use like a signature to put it there.

So you will go ahead and do other types of modifications as well. So that's one of the challenges, for example. The other one is what happens if, for example, when it comes to the news and media use case, what happens if a video that has been made gets forwarded on WhatsApp and other social media channels many times causing it to have a like a worse resolution. And in that sense, the image is then or the video is not of a

good quality anymore. And when the quality drops, specifically if you have like dark videos with no light or small faces in it, it becomes challenging to detect them. So those are purposefully, I think, activities that people can do to make it more challenging to be able to take to detect, detect a deep fake in that sense. And there are also, of course, adversarial attacks happening.

So that means that basically if they know how to fool your system, so if they somehow have an idea of how deep fake detection works, they can go ahead and basically use your own strategy against you to put it there to fool you to think something is a deep fake or not a deep fake basically. So it's, it's complicated. Data is there to help, but it's also not everything I would say. So it's a combination of threat intelligence, seeing what is needed, really focusing.

Also as a start up, once we start it, you need to focus. There needs to be a focus. Right now we're skating, we have worldwide clients from Singapore to Brazil to US in Europe, of

course, as well. So being this company in deep tech detection, one of the industry leaders, it's also again requires, you know, focusing on a in a balanced way because we want to scale up, of course, also when it comes to developing, developing our own platform, being able to also detect those videos of those birds you mentioned. But then again, it needs focus along the along along the way as well. So the milestones need to be

focused. And that's how I would say maybe the biggest challenge that that we have that goes also maybe for every start about. Their Yeah, yeah, I can see that. I mean, my assumption is that your software is slightly different, right, because yours needs to have a certain level of accuracy. If you label something as deep fake and it's not a deep fake, then it's even worse reputationally. So you can't put out a half baked product.

And then you also have a very complex domain in that if you do reconnaissance and you go on the dark web, I don't know, I'm not I all my knowledge of like police and what is legal illegal, a lot of it comes from TV. So I don't know how much of that is legal or illegal. You know, you may be on the borderline there. So it's also very hard to actually like if I were you, I cannot, my assumption is go and ask for a fake bank account.

That doesn't make any sense. So I would never be able to experience the real good customer service that some fraudulent people offer. But then you also don't know what type of new attacks are out there. But you do need to know that because you're creating preventative self. So it's like very challenging in many aspects. That's true, yeah. But it also, I think, makes it a very interesting playing field to play out, yeah. Yeah, exactly.

It's really interesting. It's about again, monitoring, you know, threat intelligence, following different sources to know indeed what is the newest that they use and also making our systems explainable. So when it comes to really sensitive use cases, we do have, for example, clients in Singapore as well and those are defence agencies. There are cases that are quite highly sensitive. So in that sense, we don't tell them, oh, just trust our technology. We are honest with them.

We say this is the accuracy that we can offer, 95%, let's say this is the recall and the false rejection rate. Also important to know of course. And if the case is highly sensitive, we can help you with basically manual expert analysis as well based on, yeah, our forensic basically expertise and knowledge to be able to validate the output of our technology. So that works for use cases that are highly sensitive and volume wise not so big.

So occasional use cases that can happen when it comes to, for example, banks flows or social media content, that's of course about huge volumes, but the chance and the risk appetite there is also of course different than the one, for example, the defence agency in Singapore has. Yeah, I can see that. If there's a person that's responsible more from an organization standpoint, what is some low hanging fruit that organizations can use or adopt?

Have some preventative measures with regards to this. So when it comes to Gen. AI based fraud, our technology for example to give to give basically an impression of it is really easy to be deployed really easily to be basically put there to monitor your existing data, production data. And that already is a great step in knowing to what scope you're already dealing with the problem. It could be there at a big scale. It could also be not there yet or only a couple of them, but we

see still sold. First we had to educate around what is a deep fake and now we still do a lot of education regarding basically what is the scope of the problem that you're dealing with right now because of Gen. AI based fraud. And that's the question that still a lot of companies haven't found an answer to. Yeah. Yeah.

What about the consumer side? So me, you already mentioned that some of my 2 factor authentication which I thought was really good, Face ID, voice ID in some cases are compromised and I understand that. Now what can? What can consumers do? Or what do they need to pay attention to? Yeah, that's a that's a great

question. So consumers are, I would say when it comes to your bank or your Apple ID, iPhone, iPhone, it's I would say it's the responsibility of the bank or the company behind it to make it secure. Of course, consumers are attacked by different types of defects, like the ones on social media, the ones that you can, yeah, you can expect a call from a loved one. And then again, one of the challenges that I mentioned along this conversation was we don't know who's responsible for

what. So I would say when it comes to banks, they are definitely responsible to not to let a deep fake fool their systems. But when it comes to, for example, getting this call through WhatsApp, then I would say Mehta is also responsible. But then Mehta can say no, the person who received the call should be aware and not accept that. And then you go to the police saying I got scammed. The police says, yeah, how should, how are we supposed to find this person?

They are professional criminals. And so in that sense, there is still not really, I would say clear image, clear strategy for the for the society, where to start basically coming up with a solution for this big problem. Yeah, also feels like kind of a gap in the market, right, Because right now maybe consumers are not really aware of fraudulent behavior or compromises or risks that they're there. They're also not like targets necessarily, maybe on a more

smaller scale. But then when big problems happen, if there is a piece of software that all of a sudden says, OK, now that's our responsibility and we'll make sure that when you use that, that's not going to happen, then that's all of a sudden like a USB. That would differentiate if we're talking about a chat or a voice or video calling service, something from WhatsApp where Meta says, OK, it's just your own responsibility.

Yeah, that's very interesting. It's really interesting because I think it happened quite recently, a couple of months ago that I think Facebook had something that you they they would require you to say if your content that you would upload is Gen. AI or not. Yeah. YouTube also does that. Yeah, but I think Facebook has removed it.

I was like, wow, this is and it specifically happened after the new administration in the US. And then I also get questions like like, do you have any interest and you know, or any benefit by letting deep fix on your platform that can mislead people and send the funny ones? They are, they're nice. I also watched them. There's hilarious ones. Yeah, yeah, yeah. But the yeah, but, but the other ones then like the really fraudulent ones.

So in that sense, I also don't know what we did these these big tech companies have, as you know, as their strategy when it comes to fake content. Yeah, I have no clue. I mean, fake content is super cool. I saw this. I think it must have been like 5 seconds. Someone made it and it wasn't even for that company. It was like an IKEA ad where there's a big box and all of a sudden there's an empty room and it explodes and there's IKEA furniture everywhere.

Super cool, super great marketing for IKEA. This person was not affiliated with IKEA. Made it with Google VO and like spent X amount of money and then that's it. Video blew up. Yeah, super great. It's it's really nice it. Becomes really scary if like those super cool things, yeah, happen for fraudulent things, yeah. And sadly, that's reality. Yeah, I think I even McDonald's had themselves made a fully AI generated ad not from a marketing. Perspective.

Yeah, that that's true, yeah. You also see like a new type of. Of course, influencers are relatively a new type of industry and job, but there are now also AI, fully AI influencers. Then you would see like Swedish looking girl. Let's say. They also say like they have a personality, like I'm Diana from, I don't know, Stockholm. And then they go around the world. They have nice clothes and it's

basically like a teenager in the room behind this Diana. That's it. And it's it's becoming really prominent as though they have millions of followers on on TikTok, on Instagram. And yeah, it's nice to see. But again, like, personally, I'm questioning what is the purpose behind, you know, following such an account or even making this account. They could be making money. I understand. But like following an AI account which has nice pictures could also be nice to see.

I don't know but I was questioning that a bit. I understand that. Like for me I I never was interested in. Like Instagram and stuff like that. But the fact that you're following, let's say an AI person versus a person that you know exists, but you don't really know who that person is. You've just seen them online. Like there's a, it's quite close to each other. I would say one just lives in the real world and one doesn't, but they do the same thing.

Yeah, I think there was one that that's a real really. Sad example, there was one teenager who thought that he fell in love with such an AI person and I think it was also like maybe a HHPT ish AI that would talk to talk to this person to this teenager. He was 14 years old, I think. And eventually he committed suicide because he wanted to go to his beloved girlfriend, AI girlfriend. And the mother was of, of course, devastated.

And she was like, you know, Cha Chibi did this to my son because the AI would say, come to me, you know, I'm here. And the the person just, yeah, yeah, there's no possibility. That's really sad. Yeah, yeah. It's like it's, I also feel like with consumers and right now this is not a problem. But if I were to go and I'm I'm victim of fraud, if I go to the police, I don't know if they're well equipped. To help me, I don't know what cybersecurity capabilities are already.

Now I feel like we have too many people, too many issues for already the current police system. Add something like a digital layer on top of it. I would love to call like a cyber police hotline but it's not there. I think we need something. If this becomes a bigger problem and like more towards the consumer rather than the people that create the apps for consumers, then we need something like that. With regards to the fence. Exactly. The police are specifically in that last.

They are super. Busy when it comes to cyber criminality and you know, cyber attacks. I don't know what they're doing right now because they, they could do takedown requests. There are also specific hotlines for that to do a takedown request when it comes to, for example, 18 plus videos of someone, not not consensually, but of what happens after, because the video can also be already circulating in social media channels and maybe even

WhatsApp chats or telecom chats. It's, it's indeed like a big skill of it's a quite fast speed of sharing material, even if, if it's a deep fake. And then at that point I'm not training it. What the police can do, I don't know. Have you? Because I'm not aware of any of these cases. But my assumption is usually it's someone close to a person

right? If there's something that's in fake of a specific person, it might be someone in a close circle of friends or family that just doesn't like this person or something happened. Yeah. Is that usually the case? That's true if it's a non

celebrity or non pep. Then politically exposed person, then it's a need either a classmate that doesn't like this person or they think that they're funny or also like people who have ended their relationships, their romantic relationships have ended and right now they're enemies. And then the the deep fake mostly also like the girl, but also the boy can get deep fake. And it's also used sometimes for blackmailing purposes, which is also really bad. Of course. It's quite messed up.

Yeah, yeah, I must say I have. Learned a lot. I I knew there was depth to this, but there's much more depth to this than I think. Thank you so much for coming on and sharing. But yeah, this was a a lovely conversation. Thank you for having me. Yeah, I'm going to round it off here. Normally cameras on this. Level we switched. Thank you so much for listening. Leave a like if you like the episode, let us know in the comment section what you thought and we'll see you on the next one.