The Problems of Regulating Algorithms are Solvable

They're pieces of computer code that shape many aspects of our digital lives. They're closely guarded trade secrets and powerful tools, and they're regularly blamed for amplifying cultural and political divisions. We often hear technologists say we couldn't have known, and the idea that they haven't turned those lenses on questions impacting the common good. It's a scandal if they haven't ask the question. It's a scandal if they've asked it

and they're not telling us what they found. This is a fourth chapter in our Solvable series examining solutions for America's polarization problem. Today we're talking about social media algorithms and how to deal with them. You can think of social media companies as fancy restaurants. The cooks behind the most successful one often don't want to reveal their recipes, but customers have a right to know what they're eating.

It turns out we've been down this road before. The Good Housekeeping labs started just around the turn of the century. People were concerned about what was in their food what was in others. This was before the creation of the FDA, and so people subscribed to Good Housekeeping. Those labs would test common products and tell people if they were safe or not. Ultimately, the federal government stepped in to regulate

food safety, including disclosure requirements around ingredients, nutrients, and calories. Similarly, establishing algorithmic safety and accountability will take a variety of players. I want to live in a world where digital power is both guided by evidence and accountable to the public. Nathan Mathias teaches at Cornell University and leads the Citizens and Technology Lab. The problems of regulating algorithms are solvable. My co host an Apple Bomb, spoke with Nathan Mathias.

Here's their conversation. So Nathan, tell me what's an algorithm. Algorithms can be thought of as a recipe, a series of steps often programmed into a computer that determine how a machine behaves. But the challenge, as any cook often finds, is that when you put them out into the world, especially something of sufficient complexity, they often behave in ways

that are different from what we expect. Can you just take a minute to explain how that's problematic and why why should we care that algorithms are deciding which piece of content you see on Facebook or which video you're being recommended on YouTube. Algorithms happen at all levels, from exactly how the electrons go from one point to another on the Internet to the much more high level things

that we think about in our direct experience. For example, an algorithm determines what your email imbacts decides is spam, an algorithm on Twitter decides which faces to show when it's displaying a photo. And algorithms also and critically make decisions about what information to prioritize when showing us feeds on Facebook, on Twitter, when determining which adds we see which adds we don't, and those are often some of the uses of algorithms that people worry about in society

and policy circles. YouTube makes a recommendation system to help us find the videos we like, and suddenly we're worrying about recommendations of extremism. Microsoft makes a fun chat assistant that will have interesting conversations with you, and now we're worrying about it learning racism and hatred. So we find that although we have the simple building blocks of an algorithm that an engineer can imagine, they often grow to

be something larger than we might omit initially imagine. I've written and others have written about the problem of algorithms on Facebook favoritizing or preferring content and posts that are emotional,

that are negative, that are divisive. You know, there's been an argument that that's one of the reasons why we have so much division and polarization in our societies, that we are being fed more and more excitable and angry content because the algorithm tests and guesses that that's what we're going to want to see or anyway that's what's going to keep us online or keep us using Facebook.

Is it accurate? Is that how they work? We do live in a world where many of the systems that determined what we see and give our attention to our learning from our behavior, our preferences, and from the collective behavior of many others, some of who aren't paying. Some of them have motivated, coordinated campaigns to influence the algorithms,

and they're adapting in real time. And so because we've never really faced a situation like this at such scale, people have a lot of concerns about how those algorithms are behaving and what they're doing to society. One of the fundamental challenges that I think scientists are still wrestling with is this challenge of influence. Typically, if a car crashes because there's say a faulty drive train, we can point to the engineering and say there's a problem with

this system. With these adaptive systems, they're reacting and learning and responding to human society and human behavior. And we're still developing the scientific tools to understand what it means to have those feedback loops. And in the meantime, we have to live in a world where these things have very real power. If the Facebook algorithm is designed to keep all of us on Facebook as long as possible, who's able to watch that, who's able to control it,

who's following the science? Almost no one is in a position right now to regulate and manage those algorithms. For example, in February, Facebook announced that they would be reducing the political content appearing in people's news feeds in several countries. We don't really know the details of what they're doing. We also have evidence, because they say they're doing tests, that they're not necessarily sure themselves what the impact is

going to be. When you think about who currently has some power to shape what algorithms do, I think there are some people at different levels of society who have a little bit of influence. We've seen, for example, European regulators step in around antitrust around what kinds of products get promoted by search engines, for example, So governments have

been doing a little bit. We definitely have companies themselves are being seen as almost government like and having policy teams, so they're trying to understand how their own systems work and manage them in some way without the rest of us having that much transparency into their values or goals or even their results. And then in some areas there are other actors who have power to manage and govern

algorithms in a constraint way. So if you've ever been a Facebook group administrator, for example, or you know someone who's a Reddit moderator. They have a little bit of an ability to tweak what gets promoted or how they given algorithm works, even though they don't have a lot of visibility into the underlying code or necessarily the power to tell a big company to change what they do.

A couple of weeks ago, I had reason to talk to a Facebook spokesman, and the topic was the experiments that Facebook does with its algorithms, the way in which they test different things. As you say, they try and use more or less political content. You know. Actually, after the events on January sixth at the Capitol, they came up with a way of moderating the news feed so that there wouldn't be so much disinformation in it. But of course, as you say, they don't publish the results

of these experiments or of these changes. One of the solutions that I know that you have suggested is that there should be outside moderators, or there should be citizens scientists who are studying these algorithms, you know, either with the cooperation of Facebook and Google or maybe not with their cooperation. What step one The place to start is

often with your own experience. I'll tell you a story just to illustrate this about six years or ago, I was approached by a group of women who faced online harassment, threats of violence, and other kinds of risks. For them. The first step was to acknowledge that it was a problem and to find other people who had the same problem.

They were able to realize that they had common needs and common goals, and they actually came up with a way to record their experiences, both the kinds of harassment that they were facing, and also to record how Twitter did or often didn't handle their reports. That was the point actually that they then reached out to me and said, this is clearly a systemic problem. We've all experienced it,

we want to see change. We know that better understanding data and science will help us think of better solutions and also, if necessary, to create pressure for those solutions. That was a great moment then for me and the team of researchers I led to develop a methodology and analyze the data they were collecting, and that report that

we ended up creating has been influential in industry. It's helped law enforcement understand how to better support people who experience online harassment, and it's also been useful in policy debates in this country about online harassment. We're at a moment where we're still building the lines of communication and the idea of citizen science as a mode of understanding and improving our digital lives. So at this stage, I think the best first step is really to find other

people who care about the thing you care about. So we need to identify the problems that have to be studied, and we need the labs where they can be studied. That's the first step. Absolutely, there's another important step at

the ecosystem level. There's a funding challenge. Most of the search that goes into funding that goes into social computing comes from the tech industry, like hundreds of millions of dollars, and if you look at the money that comes into industry independent research, it's a tiny drop in the bucket. So as policymakers debate ideas like taxing tech companies, I could imagine they're being funding within that for industry independent

accountability research. We're also finding ourselves having to invent new funding models for this kind of research as well. And then presumably at some point, some regulatory mechanism that makes sure that the Internet platforms will work with you and we'll listen to you exactly. So I think we're seeing more and more researchers in this space say that we're going to need some kind of regulation to provide protections and support for independent research to go on even when

companies find it uncomfortable. One of the crises, you know at the moment in American life is the fact that a part of the country now lives in a completely alternative universe from the rest of the country. And we all saw on January the six that there are people who are so convinced that Donald Trump won the election that they were willing to attack the capital and even murder policemen and other in an attempt to disrupt Congress's work and prevent the naming of verifying of Joe Biden

as president. How do you relate that to this problem of algorithms. I mean, if we had if we could solve the algorithm problem, if we if we were able to structure algorithms so that they favored civic discourse and civil conversation instead of promoting division and anger, could that help us heal this deep divide, this epistemological divide whereby

we all live in different realities. You know, we know that when crowds of people get involved in stuff that doesn't necessarily mean that the outcome is good or better. So why should we be so sure that citizen participation in the regulation of the internet will give us good regulation. It's important to differentiate between who's making decisions and who's

producing evidence. Evidence is something that you can put into the conversation about what to do, and so long as that evidence is produced in a reliable way, it has value to bring to the conversation. So your feeling is that this is a question. It's not just important for

I don't know the future of social media. It's really the question it's important for democracy giving that power, giving some of that oversight ability to citizen scientists, to outside groups, maybe to some government ombudsman, maybe to some regulators, that

this would democratize that power that social media companies have. Yeah, one of my personal heroes in the social sciences is Kurt Lewin, one of the founders of social psychology, who himself barely escaped Nazi Germany with his life and went on to influence so much in science and society. And he had this great quote which says, it's essential that a democratic commonwealth and its educational system apply the rational procedures of scientific investigation to its own processes of group living.

And Lewin believed that that needed to be done in a democratic way if we were going to maintain the values that we have as democratic societies. That it wasn't just enough to do research that supported democracy. You needed the research itself to be democratic in nature. And I think in an era where so much of what we do is influenced by design and algorithms, that reality is

clearer than it even was in Lewin's time. There's a long tradition of citizens, scientists, and outsiders working outside the government are sometimes in tandem with the government in order to push regulation or particular direction. Do you see yourself belonging to that tradition and can you describe it a little bit? You're asking me a question about something that

I absolutely love and obsessed by, so question. Yeah, you know, I grew up, you know, in the United States as a Guatemalan American, with this sense that science was this tool of like powerful people in institutions that didn't always include or pay attention to the general public or the marginalized as anything other than research participants like you can be a subject in the research and we will call

you a subject. But when I was a graduate at the MIT Media Lab, I started to learn about this amazing tradition of citizen science in different places and times over the last really two hundred years. In the mid nineteenth century, there was a group of people who went around London and bought bread from different shops and used this new idea of a microscope to count what was

actually in the bread and found widespread food adulteration. This set of studies ended up helping launch the trajectory of what is now the Lancet, one of the premier medical journals in the world. Another example I really love is the story of the Good Housekeeping Labs, which was started just around the turn of the century. People were concerned about what was in their food, what was in other products.

This was before the creation of they DA. There really wasn't that much regulation of what went into the mass

production ecosystem, and so people subscribed to Good Housekeeping. Those labs would test common products and tell people if they were safe or not and use the good Housekeeping seal of approval, and often in fact in the late nineteenth early twentieth century, because there was this convergence of the rising women's movement and a passion for science, you would have women's organizations actually leading a lot of citizen science efforts.

And then later on when the US established the FDA, it was actually the scientists from the Good Housekeeping Lab that built up the FDA's initial scientific capacities and leading us to where we are today, where we have more organized and supported regimes of testing and science and regulation.

So when you think that's how algorithm regulation or social media regulation could evolve with teams of citizen scientists like the people at your lab, or is the idea that eventually this is something the government would do or is this something that will some other kind of civic body will do. Do you have a kind of trajectory of how this could work in the long term. In the short term, citizen science and work from the outside is

a necessity. We're currently at a moment where if you want to look at what tech companies are doing from the inside, you have to sign these NDAs, you have to do work that they feel comfortable with. And like many other citizen scientists in other domains, we find ourselves inventing methodologies to answer urging questions that people need to understand now, and I think, you know, we have a small but growing number of institutions that are starting to

do that work. The Barkup Consumer Reports Digital Labs has been building a team that are initiatives like Joey boil and Wine's Algorithmic Justice League that all do work of this kind. In the longer term, I would love to see a healthy ecosystem. I draw a lot of inspiration from the work of Eleanor Ostrom, the Nobel Prize winning political scientist who wrote about how you incorporate science into

complex governance scenarios where you have competing interests. I think we're likely, I hope, in the long term, to get to a point where companies are going to be more transparent. They're going to actually publish their protocols and research on the issues we care about, and that's going to be an important part. I think we really desperately need more government supported efforts, and I'll leave it to the policy

makers to figure out what that actually looks like. And I think will continue to see citizen scientists trying to make sense of and improve their own contexts and environment, just like we have in the arena of environmental protection, consumer protection. Those are all mature ecosystems where you have science happening from different perspectives and different points in the ecosystem. Right, So it's not just government scientists. They're also independent scientists.

And there's the Sierra Club, and they're individuals and they're you know, so there there are lots of different perspectives on the same environmental problem. And you imagine that that would eventually be possible in monitoring and regulating the social media companies too, exactly, and in democracy, we hope that having multiple perspectives helps us get to a better solution. At least that's that's the vision of democracy I want. I want to cling to in how I imagine the work.

And so I think we need that for governing social media, for governing the role of digital technologies in our lives, and we have a lot of work to build up the industry independent part of that ecosystem. Nathan, I know that you started your education in the humanities and you moved later on to technology. Can you tell me a little bit about how that happened? How does an English major become part of this other world? When I was a teenager, I had this amazing opportunity to meet with

and talk to a local computer science professor. I was really passionate about the arts. I was really passionate about computing, and he said, computing as a lens on the world. If you really care about understanding technology, you need to understand society. You need to pay close attention to the world around you, because computing without that has no heart, it has no moral compass. With his encouragement, I felt empowered and prompted to spend my undergraduate time reading literature,

studying the humanities, asking myself the big questions. Was really during my second undergraduate degree, when I was a student at Cambridge University that I started to ask questions about literature and what we read, and its impact on democracy, its impact and connections to psychology. I realized that not only were we collecting massive amounts of data about human experience and behavior that could help us answer some of

those questions. I also realized that those enduring questions about what it means to live well together in society that we've been asking as long as we've had written records are incredibly important to the present time. And that's what led me to actually go back to grad school and study those questions further. And those aren't questions that are

normally asked in Silicon Valley, presumably. And I don't know if I can speak for all of Silicon Valley, but I do think that I think we often hear technologists say we couldn't have known, and I can't really tell whether that's true or whether it's a rhetorical line to take, because the reality is that companies have built some of the world's most sophisticated social scientific research endeavors in the history of humanity, and the idea that they haven't turned

those lenses on questions impacting the common good is just unimaginably astonishing. That it's a scandal if they haven't ask the question. It's a scandal if they've asked it and they're not telling us what they found. I want to live in a world where digital power is both guided by evidence and accountable to the public, and so I'm very dissatisfied when people tell me they haven't asked the

question before. Nathan, what are a few things that you could ask our podcast listeners to do to help solve this problem themselves? So are there books you think they should read? Are there, you know things they should watch to get a better understanding these ideas or their organizations. You can suggest they be involved with things they can do. Yeah. First, there are some organizations that are building up this kind

of work. You can join, subscribe, or give to organizations like the Markup, like Consumer Reports, the Algorithmic Justice League, or the Citizens and Technology Lab, which I lead. In addition to that look out for opportunities to participate in research, kat Lab will be announcing some new studies later this year. Many other researchers, some of whom I've mentioned, will announce public calls asking people sign up and help us measure

or test a new idea. For example, the Mozilla Foundation, who run the Firefox browser, have a volunteer program for people to sign up and collectively monitor what kinds of recommendations YouTube is making about the role of that algorithm in our lives. That was Nathan Matthias, who leads Cornell University Citizens and Technology Lab. Will include links to his suggestions for ways that you can get involved with evaluating

algorithms and improving the social media ecosystem. This is the last episode of our mini series about dealing with the problem of political polarization. I'd urge you to go back and listen to previous episodes if Eli pariser with former President Juan Manuel Santos of Columbia and of course my co host Anna Applebaum, who you've just been hearing from. When you listen to them, I think you'll come away with an understanding that polarization doesn't have to keep getting worse.

It's not a one way street, and there are societies we can point to where it has gotten better. But to diminish polarization, we need to address factors propelling it in technology, media and politics. Next week I'm Solvable. We'll talk with Catherine Coleman Flowers. She's the founder and director of the Center for Rural Enterprise and Environmental Justice. We'll discuss how poor sanitation in America is solvable. Yes, it's still a problem here in the United States. I hope

you'll join us. Solvable Senior producer is Jocelyn Frank. Research and booking by Lisa Dunn. Managing producer is Katherine Girardou. Mia Lobell is the executive producer of Pushkin Podcast. Solvable is a production of Pushkin Industries. If you like the show, please remember to share, rate, and review us. It really helps to get the word out. You can find Pushkin podcasts wherever you listen, including on the iHeartRadio app and Apple Podcasts. I'm Jacob Weisberg.