From Indias largest newsroom, I'm Arun George and this is the Times of India podcast. We're just two phases into the long national elections in India and two Bollywood actors have already filed complaints over fake videos featuring them being shared on social media. Aamir Khan and Ranveer Singh file Police Complaints over videos that was circulated on social media. They claim they were campaigning for a political party. Celebrate Kare Hamari.
Dukhi Hui Jeevan. We done this episode at the beginning of this year about deep fake videos, and we're bringing back that episode today. We've spoken with Karen Ribello, who's a deputy editor with fact checking organization Boom Live, about why it's so hard to tackle AI generated fake videos and other forms of disinformation on the Internet. We'd ask Karen what kind of misinformation is most commonly found in India.
India is a country where you see a lot of misinformation purely in terms of the volumes of misinformation, it is a lot like on an average we do about four to six fact checks every day. And right now I think in the country there may be 15 to 20 fact checkers as well and they do similar volumes as well. So purely in terms of like, the amount of misinformation is
enormous. What we've generally seen over the past five to six years, we had a big problem and I would say we still have a big problem with something called as cheap fakes. So cheap fakes are something where you know, it's manipulation which doesn't involve very high technology As such I crop a video or I speed up the playback speed of a video or I, you know, morph a photo using Photoshop or something like that. Those sort of things would be called as cheap fakes.
Now that doesn't involve AI. What we have seen over the past one year I'd say is that we are seeing an increasing amount of AI based misinformation and this trend sort of started I think around mid last year. So in May last year, you had that image of wrestlers protesting and they were detained inside a police van during a protest. And there was an image which went viral, which showed them smiling while they were sitting inside that van.
And the narrative around that photo was, look, these guys are doing it just for a few laughs or that they're not serious about this protest. We can see that here through social media posts from netizens like this one, which reads that the photo is part of a a drama, part of a toolkit in order to break our country. And of course later on we found out it was edited or doctored with an app called Face App, which is an AI based editing app that happened in in May last
year. And since then there has been an increase in the sort of AI based misinformation, especially that has been catered to a local audience in India. The deepfakes that we used to see were mostly Joe Biden or Zelensky, you know, something related to the Russia, Ukraine war. What we started seeing last year is like deepfakes in the Indian context and this sort of increase. We saw a lot of deepfake pornographic videos involving Indian actresses.
There were always pictures that were circulated on platforms like X. And then of course, the Rashmika Mandana incident blew up. I did a story this month that during the Madhya Pradesh elections. So the Madhya Pradesh elections happened in October, November. We found that a lot of videos were doctored with AI voice
cloning. So you had people make voice clones of Shivraj Singh Chauhan and Kamal Nath and overlay them onto real videos of these politicians where you know, you have them saying completely fabricated statements, things that they've never said. And those videos went viral. Just to see how, how effective and how convincing it is, I went and created 2 audio clones of of those two leaders and it was terrifying because it was so real.
It sounded just like them. Me barbar Karahu Kamal Nath ko roko Congress agai to Sabko Pandora surupe mahine or path surupe. Me gas Denne lagegi, Chodo ugly Barbie hamara jetna namkin ho jayega. And you could never tell that a machine had created that or any sort of AI. So in terms of the use of, you know, AI to spread misinformation, India has already seen it last year itself in the context of an election. So we already have precedence in terms of the general election.
We can expect to see AI based misinformation. What changes when it's Artificial intelligence based? You know, these AI voice clones are also being used to scam a lot of people and to create videos where they're impersonating the voices of actors. And they're either, you know, promoting some betting apps or some fraudulent investment scheme or like diabetes medicines. And all of these videos are like
sort of rampant. They are almost everywhere on Facebook, on Instagram and stuff like that. So now I think what you're seeing is now the first iteration of how these things are playing out. And the reason why I guess a lot of people and a lot of fact checkers and researchers and technologists are worried is because of the level of sophistication and and the scale that this sort of technology offers to bad actors.
So it's very easy to do something and you know just amplify it on a very large scale and it's and it's a lot more difficult to catch. I would say the the reason why this thing here is problematic is also because the tools that we currently have to catch and detect these things are not very reliable, especially with AI, audio, voice clones. Just by hearing it the average person or even a fact checker trained fact checker, you would not be able to spot the
difference. And plus I guess the human brain has evolved in such a way that it overcompensates for, you know, discrepancies in audio. So if I tell you that I'm playing a voice of so and so person, and even if it has a few glitches, your brain will still make that leap to try to make it sound like the person I've just told you so. In that sense, AI voice clones are far more dangerous than AI based images or videos because it is harder to catch.
Karen, could you talk a bit about the misinformation sort of industry in India, especially given India is both a massive creator as well as a massive consumer of misinformation? The kind of misinformation that we see in India is, is a bit unique in the sense that the two big baskets if I have to categorize misinformation in the country, it would be religion and politics. And now increasingly the the separation between the two is blurring.
So that is something we don't see in a lot of other countries where you have so many claims around religion that is like a unique feature that we see in India. But generally I would say, you know the the sort of misinformation that we see tends to come from the right wing purely on the basis of the fact that there is a nationalistic government in power and there is a tide of majoritarianism that
is sweeping the country. So the sort of narrative that accompanies it will also see a lot of misinformation. That is not to say that there's no misinformation coming from the left, but it's the scale and the reach are not comparable at all. It's almost like negligible majority of it is coming from the right wing ecosystem and that ecosystem has a lot of different motivations for creating misinformation. So one is that it's very lucrative in India to create misinformation.
You know, if you are a YouTube or a content creator, then then sort of jumping onto this bandwagon, you can get a lot of traction and make a lot of money. And that is what we tend to see. Right earlier it was Facebook pages and accounts. Now it's shifted to YouTube videos and influencers and and that sort of ecosystem is financially very rewarding. Then you also have a whole set of people who ideologically are aligned to this Causeway.
So they may not be doing it for any financial gain as such. They're doing it just because they 100% believe that what they're saying is right and so they do it. So these are the sort of people that we see as well. And another unique thing I suppose in India and why we do have a lot of misinformation is that around, you know 2015, 2016 you had a lot of people come online for the first time with mobile Internet rates falling through the floor.
So then you have a whole section which you know is not that I would say savvy when it comes to on just in terms of dealing with online misinformation. So there are number of factors that have sort of created this very unique situation that we find ourselves in right now. In terms of India's handling of misinformation, for one, how has it been so far? And you know, we now have two laws coming in.
You've got these sort of public announcements against deepfakes and AI generated imagery and voice cloning. How much do you see these things? Solving it in the short term at least? Yeah, I'm not too optimistic that the laws are going to be of much help. I think they're going to complicate things further, especially if they are not, if they don't take all stakeholders on board and their opinions.
So in my opinion, generally laws have not been very effective when it comes to curbing misinformation. You know, a few countries around the world have have sort of tried it, but it tends to lean towards, you know, either censoring content or crushing dissent. We have to be very careful plus the nature of this this problem is so complex that you can't like have A1 size fits all solution. So to be honest I'm not very
optimistic. There are some low hanging fruit which is easy to do. One is the election Commission should should come out with some sort of guidelines that prevent the use of generative AI and deepfakes, especially in in any sort of campaign, purely because we don't know the extent to how
this tech can be abused. We have examples most recently in Bangladesh where you had deep fakes of candidates and their voices were cloned and it was overlaid onto a video S it appeared like the candidates saying that they are dropping out of the elections and asking people not to vote for them. So that kind of thing is terrifying when it comes to deep fake pornography and CSAM, which is child sexual abuse material. There is no sort of difference of opinion from anyone that this
is bad for for everyone, right? Everybody's on the same page that this stuff is harmful and it should be taken down. That should be addressed on war footing. From my own experience, I'll tell you that I have reported several handles on Elon Musk's X platform for tweeting and posting explicit deepfake pornographic material, and the reply that I've gotten from X has been that this does not violate our community standards and it has happened on more than
one occasion. Platforms also have wake up to this and and do more and not just pay lip service to this problem. The path ahead looks very difficult. How effective are these platform based initiatives so far in terms of curbing the spread of misinformation? So right now you have a very unique situation where these platforms are almost imploding like X. It's become this hub of conspiracy theories and misinformation. And we saw this during the Israel Hamas war where you had
misinformation on steroids. And it was like a dam breaking in terms just in terms of the volume of and the amount of misinformation and all of it was playing out on X. So X went and changed its blue tick verification system where it allowed basically anyone to pay $8 and buy that blue tick mark. And at the same time it completely decimated its trust and safety teams. Facebook as well, you know, laid off a lot of people in in those
teams. So now what you're seeing are like the consequences of those actions. It brings me to the point that, you know, there is a structural and fundamental problem with the way these platforms are built. They are built in a way that it fosters polarization, that the most divisive content and conversation and material out
there gets traction. I also see there's a lot of conflict of interest in the sense that a lot of these and it's not just the platforms, I guess it's all big tech is that on one hand they are saying that you know they're worried about AI based misinformation and all of that. But on the other hand they are coming out with products and technology and features that enable you to to use AI.
So let's say a Facebook comes out with a feature that enables AI editing where you can change the background of a photo, you can insert something that was never there. You can in fact make a whole video with things that were never in there. Now they use polar bears and puppy dogs in in examples for, you know, these products. But that is not what it's going to be used for, right? People are going to use it with real life instances of people
and do all sorts of harm. So this conflict of interest where you know on one hand you're saying oh we are doing everything we can and we're using our tech and we're partnering with fact checkers and and all of that. And then on the other hand, you are building these products which again fuel the same problem that we are fighting. Karen Ribello says that these tools that allow the cloning of face and voice should be more restricted than they are.
She points out that while they may be created for entertainment, it doesn't take very long for them to be misused. First of all, I don't think this technology should be widely available. Like what is the point of allowing someone to clone somebody else's voice? And if you are doing it, then you need to build detectors within your product that are free and not subscription based where anybody can use it.
These image generation websites, they say they have safeguards, but if you are determined to sort of bypass those rules, you can find a way to beat the system. And we've seen that, right. You have examples of AI based images used in the Israel Hamas war. I think they need to really clarify what their sort of approach is to tackling this problem because as of now, I think all their efforts are like coming to naughty.
But with a lot of these AI tools, The thing is also they're seen as tools to foster creativity, a sort of leap forward in terms of the human imagination, right? Would say markers like we had, say, a reverse image search earlier or something like that. Do markers like that actually help detect a fake? Do people actually invest their
time in finding out it's fake? So from our experience, I mean, I I don't think people spend that much time and this argument of these tools have been built for enhancing creativity or productivity. It sort of reminds me of the conversation that happened when social media platforms were new around 15 or 16 years ago. Social media platforms came into, you know, existence with people promising us the idea of
building communities. Now how much of of community building has happened over social media than actual destruction is a conversation for another time. You know, we are making the same mistakes with generative AI because the problems that all these social media platforms have faced. I see all of the same problems with generative AI as well. When it's in its first few iterations, right now is the time to talk about it and fix these problems before it gets worse.
A lot of the debate around the curbing of misinformation has come to center around artificial intelligence based tools that are used to create misinformation. What's spoken of less now is that platforms like X, YouTube, and WhatsApp are still very active sources to spread misinformation in spite of all the measures that have been taken over the years. Karen Rubello explains the things that platforms can do as a sort of quick fix to curb the
spread of misinformation. There have been studies which show that, you know, when people come across information that has been flagged by the platform as misleading, whether it's on X or on Facebook or anything, there is a sort of slowing down effect that happens in terms of its velocity. We can't just say, you know, if whoever wants to forward it will still forward it. But yeah, it makes a difference in terms of slowing down its distribution for sure.
In addition, steps like. Enforcing some sort of monetary penalty or something. So, you know, making sure that pages that or accounts that share this information can't earn a lot by sharing misinformation, those steps help as well because what we saw during the Israel Hamas war is that a lot of the misinformation came from verified blue tick
accounts. Now a policy that says, you know, they will not be able to monetize if their content has been marked or flagged, that certainly helps because it incentivizes the creator of misinformation. So small steps like that definitely play a huge role. There's been a lot written about the work of a fact checker and the sort of thanklessness that it is.
What has it been like for you to work on curbing the spread of misinformation, especially given, I mean, even AWF report kind of points out that you're literally sitting in the hub of it in some ways. Yeah. So I was thinking about it the other day, and I figured that I've spent almost like half my career in this space, which is a
long time, so to say. Misinformation has definitely evolved in the sense that it's become more pervasive, it's become more sophisticated and it's definitely more influential in terms of its effects that we see. Initially that I used to think that only negative news events leads to to a lot of misinformation and and that still holds true to a large extent.
But what we're seeing right now is that, you know, almost every newsworthy event is sort of accompanied by misinformation in India. So it's almost become par for the course. And yeah, it's an uphill battle and and personally for me it's been hard but completely worth it. I wouldn't change it any other way. Today's episode was produced by Jayaraj Singh and Sahil Gupta. For a daily spotlight on people, ideas and stories that matter, subscribe to us.
We're available on TOI plus Spotify, Apple, Google Podcasts and all other platforms of your choice. For any new steps, e-mail us at toipodcast at Timesinternet in.