[MUSIC]
Welcome to Abstracts, a Microsoft Research Podcast that puts the spotlight on world-class research—in brief. I'm Amber Tingle. In this series, members of the research community at Microsoft give us a quick snapshot—or a podcast abstract—of their new and noteworthy papers.
[MUSIC FADES]
Our guests today are Shrey Jain and Zoë Hitzig. Shrey is a product manager at Microsoft, and Zoë is a research scientist at OpenAI. They are two of the corresponding authors on a new paper, “Personhood credentials: Artificial intelligence and the value of privacy-preserving tools to distinguish who is real online.” This exploratory research comprises multidisciplinary collaborators
from across industry, academia, and the civil sector. The paper is available now on arXiv. Shrey and Zoë, thank you so much for joining us, and welcome back to the Microsoft Research Podcast.
Thank you. We're happy to be back. ZOË HITZIG: Thanks so much.
Shrey, let's start with a brief overview of your paper. Why is this research important, and why do you think this is something we should all know about?
Malicious actors have been exploiting anonymity as a way to deceive others online. And historically, deception has been viewed as this unfortunate but necessary cost as a way to preserve the internet's commitment to privacy and unrestricted access to information. And today, AI is changing the way we should think about malicious actors' ability to be successful
in those attacks. It makes it easier to create content that is indistinguishable from human-created content, and it is possible to do so in a way that is only getting cheaper and more accessible. And so this paper aims to address a countermeasure to protect against AI-powered deception at scale while also protecting privacy. And I think the reason why people should care about this problem is for two reasons. One is it can very soon become very
logistically annoying to deal with these various different types of scams that can occur. I think we've all been susceptible to different types of attacks or scams that, you know, people have had. But now these scams are going to become much more persuasive and effective. And so for various different recovery purposes, it can become very challenging to get access back to your accounts or rebuild your reputation that someone may damage online. But more importantly, there's
also very dangerous things that can happen. Kids might not be safe online anymore. Or our ability to communicate online for democratic processes. A lot of the way in which we shape political views today happens online. And that's also at risk. And in response to that, we propose in this paper a solution titled personhood credentials. Personhood credentials enable people to prove that they are in fact a real person without revealing anything more about themselves online.
Zoë, walk us through what's already been done in this field, and what's your unique contribution to the literature here?
I see us as intervening on two separate bodies of work. And part of what we're doing in this paper is bringing together those two bodies of work. There's been absolutely amazing work for decades in cryptography and in security. And what cryptographers have been able to do is to figure out protocols that allow people to prove very specific claims about themselves without revealing
their full identity. So when you think about walking into a bar and the bartender asks you to prove that you're over 21—or over 18, depending on where you are—you typically have to show your full driver's license. And now that's revealing a lot of information. It says, you know, where you live, whether you're an organ donor. It's revealing a lot of information to that bartender. And online,
we don't know what different service providers are storing about us. So, you know, the bartender might not really care where we live or whether we're an organ donor. But when we're signing up for digital services and we have to show a highly revealing credential like a driver's license just to get access to something, we're giving over too much information in some sense. And so this
one body of literature that we're really drawing on is a literature in cryptography. The idea that I was talking about there, where you can prove privately just isolated claims about yourself, that's an idea called an anonymous credential. It allows you to be anonymous with respect to some kind of service provider while still proving a limited claim about yourself, like “I am over 18,” or in the case of personhood credentials, you prove, “I am a person.” So that's
all one body of literature. Then there's this huge other body of literature and set of conversations happening in policy circles right now around what to do about AI. Huge questions abounding. Shrey and I have written a prior paper called “Contextual Confidence and Generative AI,” which we talked about on this podcast, as well, and in that paper, we offered a framework for thinking about the specific ways that generative AI, sort of, threatens the foundations of
our modes of communication online. And we outlined about 16 different solutions that could help us to solve the coming problems that generative AI might bring to our online ecosystems. And what we decided to do in this paper was focus on a set of solutions that we thought are not getting enough attention in those AI and AI policy circles. And so part of what this paper is doing is bringing together these ideas from this long body of work in cryptography into those conversations.
I'd like to know more about your methodology, Shrey. How did your team go about conducting this research?
So we had a wide range of collaborators from industry, academia, the civil sector who work on topics of digital identity, privacy, advocacy, security, and AI policy which came together to think about, what is the clearest way in which we can explain what we believe is a countermeasure that can protect against AI-powered deception that, from a technological point of view, there's already a large body of work that we can reference but from a “how this can
be implemented.” Discussing the tradeoffs that various different types of academics and industry leaders are thinking about. Can we communicate that very clearly? And so the methodology here was really about bringing together a wide range of collaborators to really bridge these two bodies of work together and communicate it clearly—not just the technical solutions but also the tradeoffs.
So, Zoë, what are the major findings here, and how are they presented in the paper?
I am an economist by training. Economists love to talk about tradeoffs. You know, when you have some of this, it means you have a little bit less of that. It's kind of like the whole business of economics. And a key finding of the paper, as I see it, is that we begin with what feels like a tradeoff, which is on the one hand, as Shrey was saying, we want to be able to be anonymous online because that has great benefits. It means we can speak truth
to power. It means we can protect civil liberties and invite everyone into online spaces. You know, privacy is a core feature of the internet. And at the same time, the, kind of, other side of the tradeoff that we're often presented is, well, if you want all that privacy and anonymity, it means that you can't have accountability. There's no way of tracking down the bad actors and making sure
that they don't do something bad again. And we're presented with this tradeoff between anonymity on the one hand and accountability on the other hand. All that is to say, a key finding of this paper, as I see it, is that personhood credentials and more generally this class of anonymous credentials that allow you to prove different pieces of your identity online without revealing your entire identity actually allow you to evade the tradeoff and allow you to, in some sense,
have your cake and eat it, too. What it allows us to do is to create some accountability, to put back some way of tracing people's digital activities to an accountable entity. What we also present in the paper are a number of different, sort of, key challenges that will have to be
taken into account in building any kind of system like this. But we present all of that, all of those challenges going forward, as potentially very worth grappling with because of the potential for this, sort of, idea to allow us to preserve the internet's commitment to privacy, free speech, and anonymity while also creating accountability for harm.
So Zoë mentioned some of these tradeoffs. Let's talk a little bit more about real-world impact, Shrey. Who benefits most from this work?
I think there's many different people that benefit. One is anyone who's communicating or doing anything online in that they can have more confidence in their interactions. And it, kind of, builds back on the paper that Zoë and I wrote last year on contextual confidence and generative AI, which is that we want to have confidence in our interactions, and in order to do that, one component is being able to identify who you're speaking with and also doing it in a
privacy-preserving way. And I think another person who benefits is policymakers. I think today, when we think about the language and technologies that are being promoted,
this complements a lot of the existing work that's being done on provenance and watermarking. And I think the ability for those individuals to be successful in their mission, which is creating a safer online space, this work can help guide these individuals to be more effective in their mission in that it highlights a technology that is not currently as discussed comparatively to these other solutions and complements them in order to protect online communication.
You know, social media is flooded with bots, and sometimes the problem with bots is that they're posting fake content, but other times, the problem with bots is that there are just so many of them and they're all retweeting each other and it's very hard to tell what's real. And so what a personhood credential can do is say, you know, maybe each person is only allowed to have five accounts on a particular social media platform.
So, Shrey, what's next on your research agenda? Are there lingering questions—I know there are—and key challenges here, and if so, how do you hope to answer them?
We believe we've aggregated a strong set of industry, academic, and, you know, civil sector collaborators, but we're only a small subset of the people who are going to be interacting with these systems. And so the first area of next steps is to gather feedback about the proposal of a solution that we've had and how can we improve that: are there tradeoffs that we're missing? Are there technical components that we weren't
thinking as deeply through? And I think there's a lot of narrow open questions that come out of this. For instance, how do personhood credentials relate to existing laws regarding identity theft or protection laws? In areas where service providers can't require government IDs, how does that apply to personhood credentials that rely on government IDs? I think that there's a lot of these open questions that we address in the paper that I think need more experimentation
and thinking through but also a lot of empirical work to be done. How do people react to personhood credentials, and does it actually enhance confidence in their interactions online? I think that there's a lot of open questions on the actual effectiveness of these tools. And so I think there's a large area of work to be done there, as well.
I've been thinking a lot about the early days of the internet. I wasn't around for that, but I know that every little decision that was made in a very short period of time had incredibly lasting consequences that we're still dealing with now. There's an enormous path dependence in every kind of technology. And I feel that right now, we're in that period of time, the small window where generative AI is this new thing to contend with, and it's uprooting many of
our assumptions about how our systems can work or should work. And I'm trying to think about how to set up those institutions, make these tiny decisions right so that in the future we have a digital architecture that's really serving the goals that we want it to serve. [MUSIC] TINGLE: Very thoughtful. With that, Shrey Jain, Zoë Hitzig, thank you so much for joining us today. Thank you so much, Amber.
And thanks to our listeners, as well. If you'd like to learn more about Shrey and Zoë's work on personhood credentials and advanced AI, you'll find a link to this paper at aka.ms/abstracts, or you can read it on arXiv. Thanks again for tuning in. I'm Amber Tingle, and we hope you'll join us next time on Abstracts.
[MUSIC FADES]