Shilling Attacks on Recommender Systems

Exploring the

Welcome to another installment of Data Skeptic Recommender Systems. For this episode, or really for just the intro here, let's put on what the cybersecurity community calls our black hats. Forget about being a good guy. How can we manipulate recommender systems? What vulnerabilities do they have that we can take advantage of? Well a popular one is called a shilling attack.

A shilling attack is when a malicious user, probably one person really, creates multiple profiles, and then they start interacting with recommender systems in a way where they intend to manipulate its outputs. Usually that goal is to promote a specific item. Imagine you're in a ska band, you want to promote your ska band on a site like Bandcamp or something like that, maybe Spotify, where your first thing is to go and promote your own group, you know?

Whether it's upvoting or liking or favoriting, do all those things to your specific piece of content or artist or whatever, but you can take that a step further. Look at those items you're competing with and go and give them a down vote. Now if one person does that, it kind of nets out in the noise. But if a single user can puppeteer a bunch of accounts, perhaps they can influence these networks.

Perhaps they can influence the output of the recommender system. Our guest today Aditya is a senior machine learning engineer at Walmart, with a strong academic background researching these shilling attacks and other malicious strategies. In this interview, Aditi walks us through how collaborative filtering works. As you should know by now, that's one of the core algorithms common in recommender systems.

And as we understand that algorithm, we can start to understand the different types of shilling attacks that it might be vulnerable to. Yet like anything it's a cat and mouse game, just as a bad actor can try and take on these tasks Research approaches can be used to detect them. We'll get into all that more in this interview.

Hi, I'm Arthity Jachani. I'm a senior machine learning engineer currently working at Walmart. So I work for the Walmart search team. I did my master from UC Berkeley. And the undergrad research that we are going to be talking about today is from my undergrad, which is S P I T. And can you share a few details on what you were studying in school? I obviously machine learning, but any specialty or focus within that?

So at Berkeley I did my master's in computer science, but with a specialization in data science. And in SPIT it was just, you know, bachelors in computer science and I was, you know, trying to figure out, okay, where do my interests really lie, right? And I think this particular paper that we're going to be talking about today.

I think it shaped, you know, my interests and this is one of those defining moments where I was like, Okay, no, uh this is something that I would not mind spending the rest of my, you know, life working on. So yeah, it was definitely that moment when I decided that okay, you know, I want to get into machine learning and search and recommendation seems to be something that I'm very interested in. What made that specifically an interesting problem worthy of study?

Of course there are a lot of places where machine learning can be applied. You could apply machine learning in something as specific as energy grid optimization. Right? But recommendations in general, it's something that Each and every person, regardless of whether he or she is an ML or not, has interacted with, right? So it's very intuitive to think about, okay, you know.

Now that I'm watching these movies on Netflix, what other movies is Netflix going to recommend me? Right. If I'm buying a product on, let's say, Walmart or Amazon, you know, it's seeing that, okay, as you buy more and more products, how these companies kind of learn about you and then try to show you the most relevant products. So it's very interesting.

Well I have a somewhat similar academic background to you, and I was never really exposed to recommender systems. I think I got a good education, but I didn't have that elective or that topic. What first got you exposed to it? My professor at SPIT, she was doing research and recommendation systems.

So she was the one, you know, Kirin Gavande is the name, and she was the one who kind of introduced us to uh recommendation systems and what kind of problems are there in recommendation systems. And then once we started exploring There's this very famous challenge called Netflix price challenge and recommendations, right? It it started in mid two thousands and they had like a million dollar price.

So I started reading about that and the more I read about, you know, the vast number of problems that are there in recommendations, the more I got interested in it.

And I know one of the big methodologies in recommender systems is collaborative filtering, which I guess we're gonna talk about as we get into the shilling attacks I I wanted to get to, but for listeners who maybe have just a passive familiarity, what is collaborative filtering? Right. So I would say collaborative filtering was one of the oldest and first methods of you know how you can do recommendations. And I would say it's also one of the most intuitive ones.

So basically what you do in collaborative filtering is, you know, you have a user item ratings matrix, right? You know that, okay, these are the users and they have rated these items either explicitly or implicitly. And then what you're trying to do is okay, if you're doing user user collaborative filtering, what you're trying to figure out is.

There are these users who are probably who probably have similar tastes to let's say our source user and they have seen these movies or these items, right? So because they are similar to each other, maybe this user would also be interested in those ideas. Right. So you're trying to build that understanding on let's say if two users are similar to each other and a user has interacted with some items, maybe the other user would also want to interact with.

Right. So that's the core of user-user collaborative filtering. And then in item-item collaborative filtering, it's doing the exact same thing, but from items point of view. Right. So I would say the most Intuitive way to think about it is in Amazon you have if you bought this item, you would also be interested in these items, right? Users who bought this also bought that.

Or let's say if you are buying a TV, maybe you would also be interested in buying home theater. So it's about creating this similarity matrix, but between items.

Well, this seems like a really intuitive approach. I can understand why people would pick it up and why it would give good results, but what could go wrong? Specifically for user-based collaborative filtering, right? Usually what happens is...

I mean think about it. You have millions of users, but then you probably have like hundreds of millions of items, right? So and at any given point, one particular user would have interacted with very few items, right? Think about how many items did you buy. uh on Amazon or on Walmart in the last one year and then compare it with how many total number of items would be present in their catalog, right?

So the signal is very sparse. To begin with, that's a big problem that okay, you don't really have a lot of data, especially for user-user collab filtering. And then you could also expose yourself to problems such as shilling attacks that we'll get into, where essentially what people would do is okay.

Now that they know that user-user collaborative filtering is a very common method that companies could be using, you could create fake profiles, you could try to become rate most popular items similarly as genuine users. And then you could try to either promote your own item or nuke your competitor's item. And then because of this, uh, you know, even genuine items would kind of start feeling that impact on recommendations, right?

So you also expose yourself to such problems when you start using collaborative filtering. So that's one particular type of shilling attack that promotion, I guess in practical terms, let's say we were doing music recommendation and I wanted to promote my friend's band, I guess would I like big acts like Taylor Swift and the Beatles and then like my friend's band and have enough accounts that people would start to associate those three together. Is that the basic idea?

What you have talked about is I would say segmented attack where essentially what you are trying to do is Okay, you know that these particular items are very popular. These particular music albums are very popular and they are similar to the genre of your friend's music, right? So essentially first you listen, you know, you create profiles and listen to these music so that you know you kind of have co occurrence with a lot of genuine users.

And then you also start to, you know, highly rate your friend's music, uh listen to it more, give it a lot of more plays so that it shows up as a top recommendation. And then it will start getting recommended to people who have also listened to Taylor Sweat, for example. Well in a case like that you're assuming my friend's band is not particularly popular it's

Uh a real David and Goliath. You have a giant with millions or more downloads and someone with maybe less than a thousand. Could there be a better strategy to it? Should I maybe find like a a regionally popular band but not too popular? would I have more success shilling through a a strategic approach like that? So there's two things, right? There's selected items and there's target items.

So selected items are the ones that are already hugely popular and you know you are listening to them or rating them highly so that essentially you kind of have some connections with genuine users. And then, you know, you kind of try to push your own target item. I'm assuming your point was what if your target item is also popular?

Right, I can make a limited number of fake accounts, let's say five hundred. Uh maybe that's a drop in the bucket for a a top forty artist or something, but it could sway a ratio if it's a smaller group. So for selected items, right? Let's say if Taylor Swift's album already has like millions of listeners, right? Your goal while creating those 500 profiles and listening to Taylor Swift's music is not so that you can boost Taylor Swift's music, right? It's to kind of

create connections with other millions of listeners who are already actively listening to Taylor Swift's music and actually like it. Right. And now you start promoting your own or your friend's music, which is of course, you know Uh naturally you would only have incentive to do that. But you know, to use a shilling profile when

your French music is already not so popular, right? And you're trying to get it more viral, right? So your French music doesn't have so many listeners. But now that these profiles have listened to Taylor Swift's music and they feel like genuine profiles, these fake profiles will start listening to your friend's music as well. And the idea is that now because of this association,

Okay recommendation system would think that okay, you know, these profiles also listen to friends uh you know your friends music. And because they are similar to these other genuine profiles, let me also start recommending this uh you know friends music to all these genuine profiles. So I think that's what we call the segmented attack. Are there other vulnerabilities here we should be worried about?

Yeah, absolutely. So all of these shilling attacks essentially depend on how much information do you have about the system, right? So let's say if you know nothing, absolutely nothing, right? So let's say there are no explicit ratings, you know, the way you have for IMDP or something of that sort. And all you know is that okay, on an average the rating for all of the items is let's say three on a scale of one to five, right?

So if you have absolutely no knowledge apart from, you know, overall average distribution. What you could do is just when you create these attacker profiles, you also have filler items where essentially what you're trying to do is create rate these filler items to kind of you know go undetected in the system. For these filler items, right, all you could do is just give a random rating between one to five. Let's say three is the average, you give it three, right?

So this could be a random attack. So this is the least, I would say, powerful shilling attack, but then because you know you don't really have a lot of information about the system. Then you have average attack where essentially let's say you know that for each filler item, what is the average rating that that particular filler item gets? And then you particular you try to give that average rating. So for example

Let's say you're trying to attack a movie recommendation system. You already know that, okay, you know, Harry Potter, let's say, has IMDB of eight or whatever, right? And it's a popular movie. So you give it a higher rating. When you are saying that you like the movie from one to five you give it a higher rating. So it's a better attack than random.

But it also requires you to have the average rating of each filler item. And then an interesting attack is bandwagon attack, where instead of you know targeting a specific segment. You select items which are very popular, right? You already know that okay, these are popular items. They have been uh you know interacted with by a lot of genuine users. So you start rating those popular items highly, right? Because you know they are popular, by default know that they must be highly rated.

So you just start rating them highly to kind of, you know, get into to create connections with genuine users. So these are like the different kind of shilling attacks.

Do you have a sense of how many fake profiles one would need to make an impact? I would say depends on what kind of system it is. For user-user collaborative filtering, right? You don't really need to create too many profiles, right? It could be a very small set because of how user-user collaborative filtering works.

For one particular user, you don't have a lot of other users who would have seen or you know interacted with the exact same items as this particular user has. Right? So it's a very small subset. And because of that, let's say you create like five hundred shilling profiles or even lesser, right? Depending on how big your system is, it's very easy to kind of break into that subset.

And that's why user-user collaborative filtering is essentially a lot more prone to these attacks. But instead of that, if you think about item-item collaborative filtering, right? At any given point, let's say a user maybe interacts with 10 items. But if you think from the items point of view, so many users would interact with that particular item. So the signal is a lot stronger. And because of that, what happens is you would essentially have to, you know, create a lot more shelling profiles.

You would have to rate a lot more items to kind of create the same impact that you would want, let's say, on a user user CR. So it would be a lot more expensive to, you know, attack a system which is based on item based collaborative filtering, for example.

We've talked about how this can be done against music systems and movie systems and probably just about any system that uses user user collaborative filtering. With this risk out there, it begs the question, can you detect this? Yeah, absolutely. As we have talked about these shilling attacks, right? One thing that you would have noticed is how shilling attackers behave.

So essentially what they are trying to do is they are trying to first highly rate popular items or let's say, you know, give average rating to a lot of items. And then specifically they try to either push their own target item or nuke, let's say you know, arrivals item, right? So this behavior is kind of

different from what a genuine user would do, right? So essentially you kind of bank on this. One way to do it is to figure out, okay, how many other user profiles does this particular profile have a high correlation with? Right. How similar is this profile to a lot of other profiles? Because essentially that would be the first goal of a shelling attacker, right? To become similar to a lot of profiles to kind of amplify the impact of their attack.

And then you look for behavior with You know, let's say for this particular target item, your other genuine users would rate it between a scale of one to five with no particular focus on let's say one particular rating, right? But your shilling attacker would either always rate that particular set of items highly or very poor. So that is kind of what you bank on. So uh some way that you kind of do shilling attack detection is either you use PCA.

where you are trying to kind of you know represent all of these users into a smaller, you know, lower dimensional latent space. Usually genuine users kind of fall in the same cluster. But users with shilling attacker profiles would have a different distribution, right? Because they have different goals. So because of that, uh they kind of tend to fall out of this cluster. So that's one way which is used for attack detection.

And of course it's a cat and mouse game, right? The more you improve how to detect shilling attacker profiles, the more shilling attackers kind of try to improvise. Although we are talking about collaborative filtering as of now. Collaborative filtering is a pretty old method. Now you have these m much more advanced recommendation systems. So why are we talking about shilling attacks now?

Stilling attacks have also evolved in that way. So let's say earlier you would just have fake ratings, recommendation systems kind of evolved to also include other signals such as reviews, right? But shilling attackers now also create fake reviews to kind of promote or nuke the item. So it's also that where The recommendation systems are trying to use very different kind of signals to create a more robust recommendation.

But then sharing attackers are also trying to, you know, kind of mimic that approach so that they are harder to detect. And as long as there is some benefit for the malicious attackers to gain from it, right? They would always try to do it. So if you think about it, I think there are I was reading a recent report by World Economic Forum where they mentioned that around four percent of reviews uh today are fake.

And you would think that four percent is a small number, but it kind of translates to around eight hundred billion dollars annual sales just in the US alone. So it's a very big market. And because It's so beneficial, right? Let's say you think about Yelp or you think about Amazon, giving fake reviews, good reviews for your own item would make such a big monetary difference, right? So there's a lot of advantage to gain for these shilling attackers.

So it's kind of okay for them to put a lot more effort and investment into it. Before, let's say if they just try to predict average ratings, now they are using LLMs to generate fake review. So they have also evolved to kind of, you know, ensure that they still end up getting that monetary gain and that benefit from doing these shows.

Delete Me makes it easy, quick, and safe to remove your personal data online at a time when surveillance and data breaches are common enough to make everyone vulnerable. Want an easier way to deal with data breaches? Get Delete Me. The fact is we're all at risk. How many times have you gotten an email or a letter saying your data has been breached? It's unsettling. But the good news is delete me can help.

Delete Me does all the hard work of wiping you and your family's personal information from Databroker websites. As someone with an active online presence, privacy is really important to me. I've been shocked at how much of my personal information was floating around on data broker sites. Since using Delete Me, I received detailed reports showing exactly what they found and removed, giving me peace of mind knowing my digital footprint is being minimized.

Take control of your data and keep your private life private by signing up for Delete Me, now at a special discount for our listeners. Get 20% off your DeleteMe plan when you go to joindeleteme.com slash data and use the promo code data at checkout. The only way you get the twenty percent off is to go to joindeleteme.com slash data in your code data at checkout. That's join delete me dot com slash data code.

Well I'm thinking about your methodology and how you're describing detecting them and one thing that occurred to me is you often have people with similar tastes, like let's stick to the movie domain. people that like campy B movie horror films, right? It's sort of a small genre, but loyal fans that love that kind of stuff. How do you tell the difference between those people who might all see all of the main movies and someone doing a shilling attack?

Yeah, absolutely. So essentially what you're talking about is we don't want to have a lot of false positives when we are detecting these shilling profiles. A lot of companies would think that it's kind of okay to maybe end up having a few shilling attackers in their system, then you know, weeding out genuine users and, you know, creating that mistrust for their company, right?

So the overall idea is to kind of avoid these false positives. So essentially what you do in those cases is you try to come up with heuristics. on how you know you are going to assume whether a profile is sh a shilling attack or not. And I'll give you more context on that. So essentially let's say we have decided

That this is the correlation threshold above which we are going to consider that these two users are similar. And this is the profile number of profiles threshold where we will consider that: okay, let's say if This particular user is similar to hundred other users. So this hundred number, this number of profiles threshold.

is another metric on the basis of which you will decide whether the profile is a shilling attacker or not. So you tune these metrics, right? You d tune this parameters on at what point, at what correlation threshold do you assume whether these profiles are similar. And at what number of profile threshold do you start assuming that this profile is a potential shilling attacker profile? Right? So that's one way.

And the other way is it can be a tiered approach, right? Essentially you shouldn't just do that, okay, you know, any profile which is uh meeting these particular thresholds. is by default a shilling attacker profile and will you know directly remove this account, right? Usually what companies do is they have a tiered approach where first they kind of get a potential subset of profiles.

uh which are kind of tagged as suspicious profiles. And then you kind of pass it through multiple either different models. where your models in the latter stage could be more advanced, right? Because essentially, you know how it works in search and recommendation systems where you have a retrieval stage and then you have a ranking stage.

Where first you are going to get a simp use simple heuristics and try to get a big potential subset. And then within that you use more modified and you know more advanced models. to finally decide whether this is actually a positive or not, whether it is an a shilling attacker profile or not. So you can use that tiered approach, maybe have a human in the loop, then decide finally whether it is a shilling attacker profile or not.

Well one very popular data set you could look at for benchmarking some of your techniques is the MovieLens database. Could you comment on uh your experience with that or any other ways that you looked at it? Yeah, so you know, when I started I was first thinking about, okay, maybe we should use the Netflix price tiling data set, but it's a pretty big data set. And at that time, the idea was to kind of first focus on okay, how do we design recommendation systems?

How do we attack these systems and then how do we detect these attacks, right? So it was not mainly on the dataset size, but of course the numbers would get more and more reliable as you kind of increase the dataset size. So group lens, movie lens is, I would say, one of the most popular data sets for recommendation problems. Group Lens has multiple tiers of datasets. So the smallest dataset I believe is the hundredk ratings dataset, which is what we use.

Right. So it has around thousand users and around seventeen hundred items and overall around hundred K ratings for this user cross items, right? So this is the data set that we used and I I believe it's something that was created by University of Minnesota. So a huge thank you to them for creating this. So that was the dataset that we used and it's very clean and it has a lot more information than just user item ratings, right? So it also has

information about demographics and all of that. And because it was uh as far as I remember, it was created purely from for academia and by users who volunteered to you know be part of that uh data set collection. So you don't have those privacy concerns or came up with Netflix, right? Where users didn't really choose to be part of that data set, choose to be part of that challenge, right? That's I would say also a positive point for the data set.

Well, given that policy of data collection, I wouldn't expect there to be any shilling attacks in the MovieLens data set. So what is there to detect? Yeah, that's a good point. That's why when we created the dataset. We injected shilling profiles first. So we created a synthetic data set where essentially what we tried to do was depending on what each kind of attack would be, right? So let's say if you are thinking about average attack.

You would know the average rating of those filler items, right? So we synthetically generated essentially, okay, we know that for these 10 items, these are the average ratings. And we would, as a shilling attacker, we would start rating these. items that way. So based on what each kind of attack was, we tried to mimic that particular attack into group lens, uh into our movie lens data set.

And then we created a separate attack detection model where we were trying to detect each of these attacks. Where of course, you know, you wouldn't know what kind of attack was done on the model. It's a black box for you and then you are just trying to detect attacker profiles. I don't think a lot of companies would be willing to provide this data, right? Because it's so sensitive, right? No company would willingly provide this data on okay, you know, how many fake profiles were

created uh for our company. How did we detect them? Because if you tell how you are detecting them, it could give NH to the attackers, right? The attackers would try to then game that. So it is also a very sins sensitive field because of which a lot of times you would see that all the papers on shilling attacks are mostly from academia, right? Because companies for natural reason do not want to kind of रिलीज देटा पुबलेक वे

If you're injecting some shilling attacks, if you put just one in, maybe that would go under the radar, could you talk a little bit about the degree to which you have to synthesize this for it to be detectable and also how you detect it? Yeah, absolutely. So we did one of the parameters apart from your profile threshold, which is the number of attacker profiles and correlation threshold. We also had this parameter on attack size, right? In terms of okay, for the same number of shilling profiles.

How many let's say popular items would you have to rate in order to you know kind of create impact? How many items otherwise let's say if you are creating a segmented attack? How many other items that belong to the same segment would you have to kind of rate in order to create this attack detection? And then we kind of plotted a graph on okay, as you increase the attack threshold. what happens to your overall detection. So it is true

that as you kind of increase the attack size, it also becomes easier to detect these attacks. But it's balanced. If you just create one profile, you are not going to have any impact on your items, right? Just creating one fake profile and highly rating your target item is not going to make any tent at all. So just creating one profile is not very useful for the attacker.

And they kind of try to create this balance where you want to have as many attacker profiles and as big of an attack as you can while flying under the radar. So it's a pretty hard job. But that's the whole cat and mouse game again. As you go through that principal components approach we discussed, how I guess separated does the data become? Is it obvious that you have shilling attacks, or could it just be one user or a few users that have very similar tastes?

From what we saw, it's more obvious when let's say the attacks are let's say random attack or average attack because then attackers don't really have a lot of information. and then for a lot of those items, they are rating it exactly the same way. Let's say for random attacks, all the filler items would have Somewhat the average distribution rating, right? So in such cases, when you apply PCA, it's pretty easy to detect such attack.

But then for attacks with higher knowledge, right, it gets difficult to reliably find such profiles. because a lot of your genuine profiles would also behave in the same way. And then there are other ways in which attackers kind of try to go undetected. For example, The whole idea of this attack detection and PCA is to find attacker profiles by deciding that, okay, how are they behaving differently from our genuine users?

So that is the part that then shelling attackers play on that maybe let's say instead of always highly rating your target item, let's say you know always giving it the perfect rating of five, maybe you vary between three to five. for your target item so you're trying to mimic a genuine user to kind of go and detect it maybe when you want to duke an item you're going to always give it the worst rating possible you just give it lower rating so there are these other ways

Or maybe you add noise in your filler items. Maybe you don't always give it an average rating. You give it an average rating with some standard deviation. So there are these other ways to kind of try to behave as genuinely as possible to kind of go and detect it.

Is there anywhere listeners could follow you online as well? Yeah, absolutely. They could uh reach out to me on LinkedIn, uh my handle is just my first and last name, other Tijachani. Yeah, if at any point there are people who would kind of want to connect with me, maybe in person, to talk about the search and recommendations problem or shilling attacks in general.

then I would be coming to I C D M conference this year. I'm organizing a multimodal search and recommendations workshop, which is something that, you know, I have been For some time, I have organized workshops at CIK and CIR. So if they want to talk to me about these things, they could come there and meet me in person.

And is there anything you can comment on about how this may or may not relate to your uh day job and uh whether or not you get to do fun stuff like this at uh your uh corporate gig? Absolutely. So like I said, right, I work at Walmart Surge. So I work on ranking, which is related to recommendations, but it's a slightly different problem.

where essentially for recommendations you don't have an explicit query. But for search ranking, you customer has explicitly searched for something and then you are trying to figure out, okay, what items to show to that customer. So it's still an extremely interesting problem. The stakes are even higher because for recommendations, for example, if you don't show the most relevant items to the user, the user won't take offense.

Let's say if the user explicitly searches for something and you still don't show relevant products to the customer, then it would be a problem. So that's something that I currently work on. But my current focus of work is not on these attacks per se or you know on chilling attacks or something of that sort. It's more on this is the customer's query, how do we ensure that we are showing the most relevant products to the customer, something that they would like to purchase, and so on and so forth.

Thank you so much for taking the time to come on and share your work. Absolutely. Yeah, it was great talking to you.