#216 - Practical Data Privacy: Enhancing Privacy and Security in Your Application - Katharine Jarmul

We all know that icky feeling when your privacy is violated, people feel like their phone is spying on them, and all these things are happening, right? A lot of times when we talk about privacy, there are a lot of technologies that could expose us to this privacy race. We heard a lot in news data breaches, data leaks, people's privacy.

Cambridge Analytica. Catherine Jarmel is the Principal data Scientist at ThoughtWorks and the author of Practical Data Privacy. If you don't know what data you have, if you don't know where it lives, then you kind of end up in a place where privacy is not possible. You don't know what you're going to use the data for. You're probably just amassing large cloud computing fees for no reason. Why can't we hold a higher bar for organizations to create secure and private by design

services? Maybe can help explain what is privacy by design? Privacy by design is actually a quite old concept. There's seven design principles. You see a lot of software thinking allowing user choice, allowing transparency, but also by default building things that respect the user. And a lot of people associate privacy with PII, personally identifiable information. Do you think that privacy relates just to PII or is it more than that? Privacy is a lot about having

the. Hey guys, welcome back to

another new episode of the Technical General Podcast. Today we are going to cover a topic that is becoming trendy in most parts of the world, data privacy. I have Catherine German here. She is the author of a book titled Practical Data Privacy, Enhancing Privacy and Security in Data. So Catherine has a lot of background expertise in machine learning and data scientists.

But today we'll try to cover the topics in more like journalist term and maybe we can discuss the importance of data privacy and maybe some potential risks that you need to be aware of whenever you build your projects or products. So, Catherine, welcome to the show. Thanks so much, Henry. I'm excited to be here.

Right, Catherine, I'd love to ask my guests to maybe share some turning points in your career that you think we all can learn from that? I think one of the turning points in my career, the early ones, is I started my job in technology as a data journalist, or what we would today called data journalists. Kind of like how do we use data and interactives to tell stories and to support investigative journalism. I was at the Washington Post

doing that. There was certainly a large turning point where I left kind of the field of media and journalism and I went into startups. And at the time I went into startups that were focused on media companies as a customer base. And that led me into what I would today call large scale natural language processing or very early models compared to

what we now call AI models. But still a lot of exposure to how do we do language processing as scale and a lot of exposure to thinking through things like parallel computation. Those were the days of Hadoop and all these other types of problems that came with how do we process large scale documents, stores and use them for some sort of other service. And then around 2014 is when I moved from Los Angeles, CA, where I was based to Berlin, Germany, where I now live since

then. And this was also the time when NLP, So the space of language processing was moving from more simple model designs or what we would today maybe call more simple model designs into deep learning, which basically fuels a lot of the architectures that we talked about today when we talk about AI. And I basically took a year off of working and just studied deep learning at that time to kind of update. I already knew a lot of the math behind it because thankfully I always loved math.

But to really upskill myself from how do we think of the problem from a statistical point of view? Statistical learning at the end of the day, deep learning is also statistical learning. But how do we move from that space to maybe a more kind of notion of applying concepts of calculus and linear algebra to a learning model? And that was a good idea because, yeah, kind of deep learning then ate most of the world of machine learning.

Then the final career change was maybe about 3 or 4 years after that of doing deep learning. I started thinking or became enmeshed in the problem of kind of how do we think about what we would today called trustworthy and responsible AI development? Then most of the time we use the concept of ethical machine learning, and in looking into that problem, I became very interested in data privacy and data security of machine learning systems and models themselves.

And that kind of fueled what eventually led to writing the book and obviously my current work, which is as a specialist in privacy and security machine learning models. Well, thank you for sharing your story. I find it quite interesting the journey that you had, right? And I think it's very unique that you took a career break simply to study deep learning and all that, right? So most people maybe took a break to do something else, you know, beyond work or some did go study.

But I think it's still pretty rare to go deep into a certain area and just to study and do maybe more things to get expert on that. This episode is brought to you by Swim dot IO and I'm excited to have its CTO and Co founder Omar Rosenbaum with me today to tell you more about SWIM. Hi Henry, very nice to meet you and thank you for having me. So tell us a little bit more, what is swim dot IO? At Swim, we want to help companies understand their code

bases. We combine static code analysis with generative AI to create comprehensive documents that help you navigate the code base. As an engineer myself, I wouldn't want them 10 years to spend so much time understanding existing code. I would want them to spend time creating and building new stuff. When you have code that has accumulated over decades, and especially in legacy languages that not many people are adapted nowadays, then the problem is

even bigger. Swim dot IO is specializing into helping mainframe developers to understand their code base. Why mainframes? We actually didn't start there. COBOL had been by some people obsolete for a few years, and I discovered that it's not really obsolete, not at all. There are more than 800 billion lines of COBOL code that are in production and they drive lots of the business in the world.

And we got more and more requests from customers to help them understand the legacy code visas that was written decades ago and got accumulated over a very long period of time. So from your customers so far, what are the some of the success stories that you can share? So we worked with an analyst who shared with us that it took them a year to document a single mainframe application, and using SWIM they were able to document a similar application in a matter of hours.

So saving that amount of time enables them to focus on other tasks. Thanks Amir for sharing with us about SWIM today. To learn more about SWIM, check out their website at swim dot IO.

So today's topic we are going to cover about data privacy. So I think in a lots of parts of the world, this topic or this kind of thing has become like the mainstream topics, right? We heard a lot in news, data breaches, you know, data leaks, people's privacy, you know, the Cambridge Analytica kind of thing, also as part of the privacy thing. So maybe if you can give us an overview first, what is the current landscape of the data privacy thing and what should people know about it? Yeah.

I mean, a lot of times I try to Orient people first with the idea of kind of personal or we can think of like sociological privacy because sometimes people hear data privacy and they just think, oh, that's for lawyers. Like that's those people are responsible. They stuttered privacy law. Those people are really, really important in the field of

privacy. I'm not trying to diminish the importance of law and regulation in the field of privacy, but I think we can all relate to privacy because each one of us probably has some sort of personal or what we might call individual understanding of our own privacy. And often that's informed by whatever cultural influences we

grew up around whatever society. Like where did privacy play or where did trust play in whatever societal bonds we kind of grew up with and we're socialized with or maybe even where we live now. So like, for example, now I live in Germany has a very different relationship with privacy than where I come from, California. And so we can also be influenced by kind of shifts or changes along our life that allow us to

see the problem differently. But essentially privacy is a lot about having the autonomy and the control to decide who I want to show up as and what I want to share with whom under what circumstances or contexts. And I think we all know that there's been times probably in our lives when information that we didn't want to share with a particular person or with groups of persons got out some which way, sometimes through technology, sometimes through a person, maybe both.

And we all know that icky feeling when your privacy is violated. And that's how it can also teach us how much it is about trust, how much it is about context, how much it is about us having a choice. And I think that you can see kind of the social constructs in a lot of the regulation that you then read. Because then of course, we have regulatory or what I would call like judicial legal understanding of privacy. And then we have a technical understanding of privacy.

And kind of the best part is when all of those work together, when the technology is reflecting not only what we legally need to do, but also what we socially and culturally understand, and knowing that that can shift depending on all sorts of different things down to the individual level. I like the three things that you mentioned, right? First, it may be relates to the context and society, right?

So privacy in one particular area might be different and maybe the norm of the culture in some other parts of the world. I also like the trusting because like most of the time when we use software, right, the concept of trust is a little bit abstract. It's vague, right? I know that I submit my details, but that's not necessary a trust being created or established when I use the software. And the last one is about choice, right?

I do have a choice to actually maybe take out my data, delete my data or whatever that is, right? So I think those are the key terms that I just picked from you what you explained just now. And a lot of people actually

associate privacy with PII, right? Personally identifiable information. Do you think that privacy relates just to PII or is it more than that? Yeah, I mean, PII is a great place to start. I don't wanna PII bash here, but I think PII is a very small view of data that we might think of as what I like to call sensitive data. Within the field of sensitive data, of course we have PII. That's things just for people who don't know that term.

That's things like e-mail addresses, birth dates, names, things that we would say might be unique to an individual and certainly in combination are unique to an individual. And then we might have what I tend to call person related data. Not everybody calls it that. Some people want to call personal data. Some people call it, yeah, data related to persons. Anyways, there's all sorts of terms for it, especially when

you get into the legal realm. But I think of person related data as like things like what I click on, what I buy can be person related, What I enter into search terms can be person related. Because when we think of all these things in combination with each other, probably when we combine them, those can also point to uniquely identifiable

characteristics of a person. And you already mentioned Cambridge Analytica. Some of the original research was focused on can we use things like Facebook likes or very short Facebook surveys or those combinations to decide how this person might vote, whether they're easily influenced in voting, and these types of characteristics which we probably would think of as sensitive.

And then we also have other sensitive data that isn't really person related that we might call proprietary or confidential data. And a lot of times we don't think of that when I think we think of privacy, because obviously it doesn't really have to do with privacy. And yet the types of protections that we use for privacy can also be applied to things like corporate secrets, other proprietary confidential information that we also don't want to share outside of a particular context.

And so these all kind of can fall under some idea of data that we might need to protect in a different way than other data. Nice. Thanks for all the classifications of different person related, right? It could be the sensitive, highly sensitive ones, it could be just the person related, it could be like confidential data. So I think that totally makes sense, right? So not necessarily just like person related, it could be entity related. And I think a lot of times when

we talk about privacy these days, there are a lot of technologies that could expose us to this privacy risk, right? I mean, just to mention a few things like social media, definitely. And then you have the web technologies like cookies, you know, when you browse a certain things, right? It tends to follow you from one page, one website to the other. You have the connected device. You know, we talk about, you know, this Alexa or, you know,

the Google Home thing, right? And lately also AI, right? So people just, you know, ask AI about certain stuff. Sometimes it could take your data away, maybe looking at those technologies. What will be your advice for people when they use those technologies, right? Because it's such a ubiquitous thing now. What should people care about in terms of their privacy? Yeah, that's, that's a hard question because obviously

everybody might be different. And again, like we said, like there's also different levels of trust that people might have with different services, right. So you might really, really like Copilot or whatever it is that you use everyday and you might decide, you know what the usefulness of this is enough for

whatever trade off that I have. At the end of the day, a lot of my work is focused around informing organizations what they should be doing to better protect the people's privacy that use their services and products.

And I think we run into this problem and I think this happens a lot also in information security or cybersecurity where we kind of decide that is a user problem and that like it's your job to figure out like what you think is the most secure e-mail service and what is like the best thing. And like, oh, you're bad if you use WhatsApp because Whatsapp's not secure, which is not necessarily true, right?

But the this kind of culture, I call it kind of like blaming the victim type of culture, where why is it that I, if I want to talk to a person and they only use one particular messaging application, why is it my job to make sure to try to inform people what is the most secure or not most secure? And this why can't we hold a higher bar for organizations to create secure and private by design services so that people can choose whatever one they like. If they like the colors, I don't care.

They like the interface, they like whatever product they like. If they want to buy a package deal from a certain cloud provider, If we're looking at privacy more holistically, it's less about like you must use this software. Then we need to hold software companies and product companies to a higher standard so that people can choose whatever it is they like, which I'm like very pro-choice on that.

One of obviously there's lots of good advice out there how to inform yourself, doing things like reading through the boring privacy policies and figuring out how to make it a fun game for yourself. There's plenty of good advice to inform yourself for how do you make those choices? But I think it almost focuses like the blame on the individual

who has to use things. And at the end of the day, I want people to, yeah, be able to connect and use technology in cool ways and not have this very privileged burden of reading through every single privacy notice to figure out which best aligns. And, oh, by the way, then it gets updated six months later. They got to read through and

compare all of them over again. Mozilla has a great resource, by the way, for people who might be looking for what criteria might I use called Privacy Not Included. And that reviews things like I think they did one on autos, like automobiles. I think they did one on like connect to devices, home assistance like you mentioned,

and a few other things. And then at least you can look like what criteria did they use and maybe start to build your own criteria to inform your choices should you have the time and energy to do so. And there's no shame if you don't because at the end of the day, we got to hold the orgs accountable. Right. So very interesting the way you explain that, right. So it used to be, you know, like it's a user's problem, right? So the one who is responsible to take care about the data that

you share. And I think we can read all this privacy policy and all that sometimes in most of in, I don't know most of it, but many software, but we don't have a choice. We simply just to have to Czech agree and you know, like, you know, whatever language that they provide, we just agree, right. Otherwise we can't use anything from the software at all.

So I think that's a very yeah, that's a very good segue now to actually move into towards the organizations or the teams who build the software and the products, right. So the first thing is I came from some, you know, belief, probably right last time, especially in the era of big data, we have to collect as many data as possible, you know, from the users so that we can derive insights, we can analyse things. And now this seems to be clashing, you know, with data

privacy. So in this era, these days, what would you advise people in terms of balance, right, about collecting as many data as possible versus, you know, treating privacy more seriously? I think First off, I want to say like, I'm not trying to get anybody to lose their job. So if any of the advice I gave you think would make you lose your job, don't do it right. So there's some mini disclaimer.

But I think I think as you mentioned, I think there's, there's starting to be I think a more popular shift into even just consumers, so to speak, the average person thinking more about privacy. Namely because I think this trend, as you mentioned, of the big data era of just collecting, you know, at will and doing all sorts of things that at the end of the day make people feel creeped out, you know, make people feel like their phone is spying on them and all these

things are happening, right? And in some cases that might be true, and in other cases it might be a fact that we have so much data that we can actually infer sensitive things about people without their knowledge and also without our intention, right? When we think about recommendation algorithm development, search response algorithm development, so search algorithms and content algorithms, when we think about a lot of this, there can be latent, what we call latent variables.

And what I mean by that is, you know, because you follow and you like these five things, we kind of have learned your gender or your age or your family status or these other things. We didn't explicitly try to learn them, but we kind of learned them by amassing, as you say, so much data and then putting an algorithm on top and not controlling for anything

like privacy. And then you get super creepy ads and you're like convinced your phone is spying on you and then you must hide it or you know what I mean? Like, I don't want people to be afraid of, I don't think any of us want people to be afraid of technology. I also don't personally want to be afraid to have a mobile phone. It's a nice thing to have.

And so when I think about like this shift that's happening, maybe one of the things that you can start talking about in an organization that may or may not already have a culture around thinking about privacy is how do we reason about the trust that

we're creating? And I think this is great for tech leads, as you aptly point out, and product people, because at the end of the day, when you're like a technical lead or a principal or a senior person in the type of architecture decisions, product decisions, data decisions that get made and you're a top product person, those people can have a real conversation about, do we want to talk with our customers about

this problem of privacy? Do we want to talk with our customers about what's the mental model that they think about how our service works? What parts of this is up for debate? What parts of this is not? Because there are like indeed businesses that by default, probably privacy is not going to play a huge role, right? But you can at least start that conversation.

And what we often talk about in the field of technical privacy is balancing between the most privacy we can offer, which would be collecting no data and utility, right? Is what we think about privacy, utility balance or trade-offs and utility. There we might think of as this old idea of we're going to collect every single mouse movement or single character you type in in the hopes of getting

some sort of insight. Which I would also argue as a data person, if you don't know what you're going to use the data for, you're probably just amassing large cloud computing fees for no reason. And so it's better to have smaller experiments with less data and then to grow it over time. Again, if you will lose your job for doing that, I'm sorry. And maybe you can find a new job one day. Right. So I think you mentioned something very important, right?

And many, many privacy experts also say the same thing, right? If you don't know what you collect that data for, then don't do it, right? Don't store it, don't do it, don't ask for it, right? So I think that's also very, very important, right, for everyone who builds software these days, right? Especially the technologies, right? And also when you build AI model, right? I mean, last time we also simply say for machine learning, it needs as many variables as much

as possible, right? So that it can train the model better. But again, like, if you don't know how you will want to use the data, maybe it's better not to collect them. And this comes back to this

thing called privacy by design. I think it's mentioned a lot of times, right? In security, there's a secure by design. In privacy now there's the privacy by design. So maybe it can help explain what is privacy by design and how we should use it. Yeah, Privacy by Design is actually like a quite old concept, which is pretty cool. I think it was developed first in the early 2000s by a researcher in Canada, Kabukin, I don't know if I pronounced the name right.

Anyways, and she developed these principles as part of research. I think she was working on as somebody that both understood some of the privacy regulatory aspects and then also the software aspects. And when you read the original privacy by design, I think there's seven design principles. You see a lot of software thinking this is like really pre data, pre algorithms as like a normal part of software design.

And a lot of it talks about the same types of themes we're talking about now, allowing user choice, allowing transparency, but also by default building things that respect the user, that respects the fact that privacy does have all these, you know, is informed by context. And so if I put something in a form, it doesn't necessarily mean I want that to be used to train an algorithm, right? That we understand like how the user is imagining what it is they're doing.

And that we're building technology that hopefully mirrors that or if not, makes it more obvious what is happening. And then security principles, so basic, not only application security, but also infrastructure security principles so that we don't end up exposing something that we've collected and trusts that we are even using responsibly. But then open the system up for potential attacks or other ways of exfiltrating that data and using it for another purpose.

And this is where I think we can take privacy by design. We can use that to architect and build our systems. And then we can also evolve that thinking for new things like building large scale data systems, algorithmic systems. And then we have a huge problem now around third party data systems of the, you know, which I don't think by any means

private by design. And so when we think of third party, it can either be services we're using and we are the intermediary then between our users who maybe we have a direct relationship with and direct trust with.

And then we want to use a third party system, which is fine as long as we're really clear and as long as we're actually reviewing the third party vendors that we use for things like privacy or the other way around where we're actually a third party vendor, we have 0 access to the users. And this is a lot of us that work in the B to B space.

We don't actually, I mean, usually the companies that we're interacting with, they might have customers or they might even have customers who have customers, right? And so how many layers until we get to a human who can make a choice about whether they want their data used that way or not.

And do we have a good review process to make sure that that kind of chain of trust is verified and that it's mentally sound both the way that we're dealing with these trust relationships and also how we're eventually communicating all the way down to a person who can say, you know what, I'd actually rather not if I could opt out. I'd rather use this service, but without the interface to ChatGPT, for example.

And I think we kind of see this in some of the new products that come out that there's a little bit more transparency of, hey, we built it this way, here's the services we're using. Do you want to use this particular feature or not? At least? I see that a lot in the way that Apple is choosing to design some pieces of Apple intelligence and so forth. Is trying to have that conversation in the open. Does it always go well?

Maybe not, but at least trying to start that conversation with users in the open about how they can better understand and control the way that the data flows through the systems. And I would call that like today's updated version of Thinking through Privacy by design. So after you mentioned about, you know, third parties and how data their data intermediaries or even people these days, we use a lot of SAS, right? I mean software product teams,

right? We use a lot of SAS, you know, maybe manage database or serverless database and things like that. So that kind of like implies the data can be stored in most of other places and you know, beyond our premise, right? So I think this also speaks

very, very true. The fact that sometimes if your organization grow really large, it's very difficult to actually know what data have you collected, where they reside, the retention, you know, who people get access to, right? So tell us about the importance of this, you know, data governance, because in many parts of conversation I was in about data privacy, I think the first step is always to know the inventory of your data, right? Govern that particular

inventory. So maybe tell us the importance of data governance. Yeah. And I think data governance is such a critical even outside of privacy. But you said it perfectly.

I mean, if you don't know what data you have, if you don't know where it lives, if you don't know what pathways it goes through, like when we talk about data engineering, if you don't know where it's being processed, how it's being processed, by whom along the way, then you kind of end up in a place where privacy is not possible to some degree, right?

Hopefully you've kind of, again, I'll re mention having strong practice of vendor review and vendor assessment because I think like thinking through risk always means can we create a reasonable way that we as an organization want to think about what third parties we work with, what intermediates we work with, what SAS as you point out we work with.

And there's no going to be no magical right choice, but there needs to be a commitment to some sort of choice and perhaps some thinking through of why it is that you choose one vendor over another that relates to principles like privacy and data security. But then beyond that, if we don't understand the data flows that are going through our systems, if we don't have data properly tagged, like some people might use tagging, other people use categorization, other

people use all sorts of things. But if we don't have a reasonable way of organizing the quote UN quote, data catalogs that we have and data stores that we have, then we also don't have any reasonable way to think through retention schedules.

What is a retention schedule? It means that we say that we collect data for a certain purpose for a given timeline. Maybe we collect purchase data for every customer, right? They go to our site, they buy something. Or maybe we hold other people's purchase date. Doesn't matter. The purchase data exists, right? And maybe we say we retain purchase data for as long as you're a customer or for up to two years after you close your account or something like this. Well, how do we even enforce

that? If we haven't bothered to like connect, then somebody has to write like some really nasty SQL query. It may or may not work if somebody did bad data entry. And this all relates to better data governance, which relates to things like data cataloging, but also things like thinking through data quality metrics and how do we ensure that we're actually getting value out of the data we collect. And what I will say is if you improve trust relationships with your customers, you will get

better data quality. Like that's by default proven time and time again. And so if you're just spamming, collecting whatever you can, especially through some sort of intermediary, you're probably getting pretty low quality data. If the signal that you're trying to measure is something like is this customer interested in ABC and you have a good relationship with a customer, then you could just directly ask them, right?

And then you have both high data quality of high trust, you've created a good privacy and security relationship with your customer, and you know that the data that you're storing is actually accurate. So I think in the end, these all end up supporting each other. And I just want to say it is not a trivial undertaking to do data governance at scale and most organizations are not doing it very well. They're just kind of starting right now, I think, for a lot of

organizations. So it's OK to decide, OK, We want our next few years to be a goal of having better data governance in which we can also have better privacy and data quality. Yeah. So I could imagine companies that have been around for years, right, having production data serving lots of customers. Definitely it will be a challenge, right, Especially, you know, these days also people work remotely.

They work, you know, using Internet, you know, something got downloaded from the system into the device, sent over to WhatsApp or something like that. Or you have data pipelines, you know, data warehouse where data moves from one side to the other. I think it's really, really challenging. I would, I would just empathize.

I also have this kind of problem as well to catalog knowing the lineage, right govern, whether it's sensitive, not sensitive parts of it, whether it can be shared with many others people or not. But I think it's definitely a challenge, but I think it will take some time. If we build awareness, I'm sure maybe we can improve the data governance that we have. So maybe let's go into the

techniques. How can we actually improve our privacy practice, right? I think the first that is always mentioned is about, you know, pseudonymization or anonymization or data masking, right? So tell us maybe for people who are not familiar with these terms, what are those techniques, right? And how can we apply that practically? Yeah. I mean, I want to add a mention here that is maybe kind of also helps bridge data governance to like actually implementing what we might call as like

mitigations or controls. And that is, these are not decisions to be made in a vacuum of like what controls get applied where or how do we decide what we want to do with what data? How do we even decide what sensitivity a data has? Hopefully your organization has the ability to create a data governance board or practice or even a data privacy practice where there could be multiple disciplines of people in the room.

Because I think you have to have people that are business informed that know what are the business goals of why we're trying to do this thing. Data informed, software and tech, tech informed, and then of course, regulatory informed and privacy informed. So you can also hire privacy professionals who are not legal experts, but instead focus on the topic of like privacy by design.

So if you have these people together, maybe also somebody from Infosec, you can start to say, you know what we think the biggest challenges for our data governance or our data privacy or our data security are all three is these top three, some new systems that we want to develop. So we're going to put that on a kind of our collective road map for the year and we're going to prioritize those.

And I think then you start thinking once you can understand what's the risk space that you have, So what are the biggest problems that you have with the data privacy, then you can start thinking about, oh, do we need to student mise? Do we need to anonymise? Do we need to run some other

type of setup? And then we get into the fun, fun bits, which I would all fall under kind of these types of privacy controls or start to go into what we would call privacy enhancing technologies, which are different technologies that we can use to help meet these goals of privacy at our organization. And most of us have seen pseudonymization at some point in time that can take many

forms. It can be simple masking, like maybe you've used one of the popular libraries is Microsoft Presidio to try to identify things like PII and mask them. Masking to take several forms. You probably already use some hashing mechanisms or something like this, maybe a one way hash or maybe a two way like a reversible hash. These can be ways to pseudomize information. You've probably already have already used redaction.

So just simply like removing certain types of sensitive data from let's say reporting tool or ABI dashboard to say, OK, now this is ready for organization wide consumption or this is ready for our marketing report or whatever it is.

So like the data is scheduled for release publicly or at least semi publicly within an org and any of these things are small privacy mitigations that you do. And then those can lead up to more serious privacy mitigations like anonymization or even thinking through things like local first data processing, distributed data or encrypted computation. So gets very space aged the further you want to go. And those only fit certain types of problems right.

So first you have to shape the problem so you can know as every developer knows it, not to say it. If you don't know what problem you're trying to do, then just putting a technology in it is unfortunately not going to solve the problem. Yeah, that's a very, very good advice, right. So I like the in the beginning you mentioned that it's not something one like one particular team or you know, person to decide, right? So it's like a multi departmental kind of effort,

right? And there are many departments that should be involved, right? You mentioned a few that are very important, right? It could be the privacy, you know, department, the technology, definitely the product, right? And maybe the legal side in four SEC, definitely data and security is kind of like interrelated with each other and there could be many other

parties, right? And especially in the organization, I think it's very important to have this so-called decision making process to actually identify whether something is sensitive or not sensitive.

So I think during the explanation just now, you mentioned this term that is quite trendy these days, privacy enhancing technologies. I think still tech we all love, you know, to know, like what are the technologies out there so that we can kind of like apply and try it. So what are some of the privacy enhancing technologies that are, you know, like quite modern these days that people are using

to solve this privacy thing? I think there's been some really cool developments, I would say, over the past 10 years. And I think the ones I'm most excited about are also the ones that show up in the book that I know you're now a bit familiar with, which is technologies like differential privacy, which is a way that we can reason technically about privacy.

I like to call it for people the idea of measurable privacy or rigorous privacy, because we're using a scientific way of thinking about or trying to shape the problem and then trying to decide that. And that works well for things like if you have to release data publicly, if you want to think through something like anonymization.

And it can work in many different types of use cases, but it can never work if you're trying to say customer A wants B. It has to work in some form of aggregation of an idea or a person. But of course, when we think of anonymization, we cannot ever say that we release something anonymized that could be tracked back to one individual, because then it's not anonymized, right,

By definition. So I think differential privacy is really cool technology, lots of different approaches, lots of different algorithms to think through, and also an increasing number of really cool open source libraries that allow you to think through differential privacy from a development context. Then probably the next most famous one of recent years has been Federated learning or

Federated analytics. Sometimes I like to call it more distributed learning, and it also relates to what I would call the field of local first software. I don't know if you've already had local first software experts on your show, but Martin Klippmann and who's famous for the book? Nice. Martin Klippmann's book is very famous. Sorry. Data in intensive applications. Something like that. Yes, yes, yes.

Thank you. And he and a group of folks that have kind of started this movement, I think it probably started before them, but have popularized this movement of local first data, which is more thinking through kind of exactly what we were talking about before, which is if we can now we have devices, we have edge devices, we also have edge compute like mobile devices. And certainly within whatever AWS cluster you run, you have plenty of compute.

So can we push processing as far down to the edge that we need? And therefore can we push the data as far down to the edge as we need? So this might mean what we call cross silo learning or cross silo analytics, which says you're a multinational company or you're in some sort of B to B situation and you allow every company to at least house their data within their own, you know, realm premise, so to speak.

So within their own boundaries of their cloud system or whatever it is that they're using. And they don't actually exchange data, they exchange some sort of analysis or output. So this could be machine learning processing. This could just be simple analytics, but we kind of try to push it there. Or it could go all the way down to I'm building a mobile application, I'm going to keep all of the data local and I'm only going to send certain artifacts.

Am I going to push that back up to some sort of centralized compute or some sort of redundancy compute, Right. And depending on how that works, you could even offer things like end to end encryption and then other types of things. Why would you go to this bother? Will you go to this bother? Because A, you're greatly reducing your exposure risk, right? You'd actually don't have centralized data. You just have some centralized

insights. So how attractive is it to try to hack your system much less so the less data that you have. And B, if you can get the same insights, you're also saving a huge amount on cloud compute because again, if you're like, if every ping, every two seconds I open it, you're just checking whether I'm still at my house. Like, cool.

I mean, not so cool. I don't think so, But you know, you're wasting a whole bunch of network compute and storage to just find out that, guess what, between the hours of, you know, 9:00 PM and whatever time in the morning, I'm at my house, right? When in aggregate analytics, he probably could have already figured out that is, you know, 80% of your user base or

something like this. I don't really know why you need to know that, but I just give you an example, a rough example of how differently we could engineer, architect our systems if we thought through again this question of what days you

actually need. And can I do that in some sort of privacy respecting manner so that most of the data stays far away from some sort of centralized setup and certainly far away from some sort of data sharing setup where companies are just exchanging sensitive data at scale.

And then that leads to the third type of privacy enhancing technology I'm personally really excited about, which is encrypted computation or people might have heard about that under terms like homomorphic encryption or multi secure multi party computation. And this essentially enables a whole bunch of use cases based on cryptography, which allows you to compute insights on data without decrypting it. Of course, then eventually somebody will decrypt it.

So the insight gets decrypted and used at some point in time, but the actual processing of the data can be done in an encrypted state. And that provides a quality that we like to call secrecy, which is different than privacy. But secrecy we can use to say, you know what, in our cloud compute we only ever computed in encrypted and then we move back to your device where you personally unencrypted it.

And so by the design that we have offered, we actually never saw the data in an unencrypted state except once it was on device or in a data sharing landscape. We actually only ever process data with another company by keeping that data encrypted so that neither company could learn anything other than the final released results. So any intermediary results, anything that we think might leak an individual level privacy, we can kind of cover up

with a secrecy blanket. And then at the end, when we decrypt and we pull off that secrecy blanket, we've already ensured that the end result has enough privacy that we can remove the secrecy. I hope that was understandable. You tell me. Well, I think definitely sounds really cool, all these advanced technologies, I may not be exposed to many of those technologies, but it sounds really promising, right? Especially I think the local first concept thing will be very important, right?

Especially if you don't want to get exposed to many sensitive data at your organization. Encryption, I think still kind of like the go to kind of like strategy for securing the data, right? It could be encryption at rest, it could be in transit, right? It could be just like what you mentioned, maybe just now homomorphic encryption and

things like that. These days cloud provider also come up with, I don't know, like secure chip, you know, where you can actually do more secure computing on the virtual machine. So definitely a space that maybe for those people who are interested, right, you can follow the technology's trends. So you mentioned about all this,

right? Definitely 1 aspect for engineers or product teams, it makes it really, really more complicated to actually, you know, come up with the solution, right? How can we actually, you know, talk to the stakeholders that we need to, you know, improve our solution by implementing all this complex thing that might increase the effort and also the amount of resources that we need, you know, with all these technologies, with the amount of development effort and things like that.

Yeah, yeah, yeah, yeah. One thing that I think that your podcast has done well so far is like talk about lean principles, talk about creating buy in overtime, developing like a practice and eventually developing platforms, right? Because I think that's kind of the evolution that you have to work with. If you're starting with even just basic data governance, you're not going to tomorrow deploy homomorphic encryption, like that's not going to happen

for you. So what I think you have to start with is like you have to start with some of the building blocks. But try, I think from a kind of a lean product thinking, try to think through what types of these more advanced technologies actually fit the problems that

we have. You know, maybe it is exploring local first development or something and then pulling in very small use cases, very small product features or maybe even new product launches and saying, you know what, we're going to cordon off an extra 20% of the planned development time to think through an experiment. Could we use this new technology to help expedite future products so that they are launched in a private by design or secure by default type of mentality?

And what you're doing there is a, you're giving people who are interested in the topic enough space so that they can actually start exploring, right? And as you know from any new development technology, new type of thinking around development, if you give people a little bit of space and they already have some interest or motivation, they can actually go pretty far. And you're also developing this culture of learning and experimentation, which we all need.

And then B, you're also like boxing it enough that it doesn't become a blocker so that there's some sort of backup technology that you've always used that's there and that you're kind of giving, you know, you're growing the experience and knowledge. And I think as soon as you start focusing on the process rather than the outcomes, then you're going to see the rewards happen over time. And you asked like, how do we explain to the business stakeholders?

Well, maybe you don't even yourself have to explain. It will become evident by allowing time in the process. For that, you can say as simple as we need to stay up to date. We need to modernize the way that we think about privacy. And I think this is going to pay off in the long run by allowing us to build platforms that allow us to do privacy at scale, which

is the end goal, right? But along that way, we're going to add some extra time and cushion so that we can build in these new exciting technologies that I promise you, you will be able to talk about in some press release one day, right? And show that we're forward thinking and show that we're kind of advancing this space. But to start out and allow that.

And I think the advocates that will come out of allowing some of your developers or your data people or other technologists at your org to learn more deeply about this stuff is going to pay off in and of itself. Those people will end up being the advocates that educate the other people in the org about how this stuff works. And they will find, I promise you, they will find clever ways of explaining it if you give them the time and space to be able to learn.

Yeah. So I think that's a very, you know, important advice, right? So start small, right? Focus on the process, not necessarily just the outcome. And I think everyone can build awareness, I think within the team, within the organization, right, about the privacy, right, treats privacy more seriously. Obviously, one easy way to

explain to stakeholders is about regulations, right? I'm sure in Europe it's easier to actually mention about privacy because there's the GDPR that is very strict, right? In some parts of the world, they are starting also to adopt these kind of stringent rules about personal data law, personal data protection and things like that. So let's go to that legal aspect. So what do you see the trends

these days? Especially what I know is that many countries now starting to come up with this kind of law, this kind of GDPR like regulations. So is this something that we are going to see going forward, like all countries will have their own kind of like regulations? And I think it will be, it will become very complicated if let's say you build a product that works in multiple jurisdiction, right? So maybe tell us a little bit your insights on this. Yeah.

I mean, I think it's already pretty complicated. It's probably gonna get more rather than less. I mean, that's why I think a lot of times when I talk about this, especially with high level stakeholders, like when we're talking with C level of some sort of multinational or company that wants to become multinational, we have to think about it as like future proofing the data strategy, right?

Because if we just kind of developed data or AI strategy within what we know today, we're not going to be prepared for the future. And even with what we know today is we have a very fragmented legal jurisdiction set up around almost everything data, but certainly data privacy. But that also applies to data security. It also applies to data governance protections that different organizations are looking at.

And one of the big trends that unfortunately I don't see going away anytime soon is the data sovereignty laws as well. And these data sovereignty laws are kind of around the jurisdictional control of the data. So for example, that data cannot leave a certain nation state or a group of nation states or that data cannot travel across these

untrusted nation states. And if you already work in the government or you work in government adjacent, you probably already know this pain and you're like, you're not saying anything new, Catherine. Well, now the pain is coming to you, whether or not you're government adjacent, whether or not you deal with kind of highly secure systems or critical infrastructure. I think it's kind of ballooning

out to other areas. And again, at the end of the day, a lot of this is a policy problem internally at an organization to decide, OK, what's our stance towards regulation? Where do we exist? Like what's important to us? And then obviously the legal experts, not the technology experts, get to decide, OK, here's our risk posture. Here's like the policies that we want to see related to what we know about the data, how the data stored, where it's stored, what is used for.

And then we kind of get to step in as technologists at kind of like that Olicy and principles level, I would say and say this is possible, this is not possible. This is like a good idea. This is not such a good idea because and then we can talk about things like architectures, clouds that we use, infrastructure questions alongside cool stuff like privacy technologies.

And I would say there's a non trivial number of legal professionals and privacy professionals that want to also learn more about technology or might even already know a lot about technology. And you if you build those bridges, you can have a really useful conversation around what

is this actually look like. And yeah, I think that can be probably the most healthy approach rather than just avoiding the topic because it's scary and you don't want to talk with the lawyers because they might say no. And then five years down the line they're like, why do we architect it this way? And there's like no answer other than we didn't have the conversation early enough.

Yeah, I think you mentioned something really, really, you know, very important, right to have now probably legal team, legal aspect when you build the products, right? When you architect the solution, when you architect your data storage and things like that, I think we are used to discuss about whenever we choose a cloud provider, right? It's the data center in the particular country. Now I think that's not enough, right?

Because the data transfer, you have to also look at like if let's say you use a SAS provider that resides in another country, another jurisdiction, maybe the data sovereignty law doesn't allow you to do that, right? And maybe you can have a trouble, right, when transferring data over. So I think a lot of countries also implement this differently. I think it's very hard to keep up.

And that's why probably the legal team is also still the best person to kind of like help advise the team to actually how to implement the better solution about this privacy.

So Catherine, we have spoken a lot about privacy technologies and all that. Is there something else that you think the listeners here should know about data privacy? I think lately I get a lot of questions around AI and data privacy. So I just wanted to note that there's a lot of new developments and thinking through, A, how do we build machine learning or AI systems that can utilize private data without problems?

But B, how do we, if we don't build the AI models ourselves, of which most organizations still don't at this point in time, how do we actually build kind of a protective blanket around these interfaces to, again, these like third party AI systems or AIAPIS? And I just want to suggest out there, obviously, I think my book is a great resource. I also have a newsletter on the topic.

But to just kind of start to inform yourself, because what I'm seeing in the circles that I'm in is that this is a growing trend and a growing problem. And I think that it will come. It will come for everyone. It will come for people that are not machine learning experts. Soon enough.

If it's not already a problem that you're facing where you're sitting there with some other team and you want to use a third party API and all of a sudden this questionnaire comes in from the legal department or how is it going to look? Or how do you actually build that with privacy or with

security, right? If you're using proprietary documents, let's say you want to build like a rag system and you want to have access to a bunch of sensitive documents that the company has, like how do we make sure that we build that

responsibly? So I just want to point to the fact that this is like a growing field of practice and you don't have to learn it like to the greatest depth, but to just kind of start to, when it pleases you, start to inform yourself lightly like, hmm, like what might that look like if we built this rag system with privacy or security? How could that look different? Do we have the capabilities to

do that now? If not, what capabilities would we need to grow and to just, you know, your tech leads, right to just start to allow that topic to percolate in your head a bit because I think it's a growing importance, I think in how we think about integrating machine learning into normal work flows. Yeah, I could imagine many teams now are kind of like scrambling to implement some sort of AI into their systems, right? And especially when you do that, sometimes we don't think further

ahead, right? Especially if you're dealing with, I don't know, like customer support, you know, customer confidential data. You think just by implementing AI it's going to make life easier, but you are not sure that they will expose certain things. Also, when you use some third party systems these days, you know, when, whenever these companies adopt AI, sometimes the choice by default is to actually enable AI training, right, using your data sometimes.

So be aware of that. And also like, I think those things need to be more transparent and explicit so that we can build trust with the third party, the systems that we use. So Catherine, I think it's been a great conversation. I learned a lot about data privacy, although it's kind of scares me a little bit on how to build a more private by design system. Unfortunately, we reached the end of our conversation.

So one thing I would like to ask you to end the conversation is what I call the three technical leadership system. So you can think of them just like an advice. So maybe if you can share your version to the listeners here. Yeah, I mean, I hope maybe I've sparked some people who might be interested in learning more about data privacy, data security topics. But the number one piece of advice is like, even if you're not inspired to learn.

I ask that you empower people on your team or in your organization to learn and that's because as you can probably tell from my advice today, the field is actually quite deep and very intensive and it's not necessarily a field that a lot of people specialize in because they studied it right.

So you going to want to open doors for people to grow and to learn and to foster kind of that ability as a specialized, my culture of learning and for you all as tech leads, like if it's coming from you, it's going to be much more powerful than like a junior who just joined the team and who's like, oh, wow, cool. Like I learned about differential privacy in college, because sometimes they do that Dow like I want to try out this thing.

Like if you can give that person space, if if you can kind of shield them and give them some ability to learn, they're going to level up the entire org eventually, right? And so certainly level up your team, but eventually this kind of spreads and creates these cultural changes that you need. So foster that learning, you know, support your people and learning. Create those spaces for learning. Try to give some back pressure to delivery deadlines to allow for some of that learning

cushion. Another thing that I want to share is I don't think all risk conversations have to be scary. I think it's totally natural. And like I, I empathize so much with it being scary because it is a big job and it is like a serious job to talk about the risk of people's data and that trust relationship. And yet at the same time, if we can normalize conversations around risk, if we can kind of allow for some of that space to have like scary topics, be part

of our Rd. mapping, be part of our planning, where we can actually kind of just regularly do privacy risk reviews, security risk reviews, auditing reviews. If we can kind of normalize that as a normal thing, like we do with like testing for bugs or figuring out if something is deployed properly, then we also make it like more tangible. And I think that it becomes less scary by default and we make it less uncertain. And I think most of the scariness comes from this uncertainty problem.

So to try to focus on that, right? And then finally, like that relates to both of these relate to like creating a team culture and hopefully an organization culture of psychological safety. And I think by knowing that we're not going to get everything 100% right all the time, we can foster both learning, we can foster risk

discussions. And we can certainly build much, much more private and secure systems by allowing us to have real conversations, by allowing us to like escalate problems when they exist, when we see them, and by empowering people to kind of have reporting up and down about what do we think about problems like privacy and security. So up the hierarchical chain of the Oregon down.

And by fostering this kind of safety conversation around these types of problems, I think we end up creating by default better software systems products in terms of privacy and security. Wow, very lovely. And you know, it all interrelated with each other, right? I especially love the second one. You know, not all risk needs to be scary, right? Sometimes it just because of the uncertainty, like we don't know about that particular risk.

But I think if you dive deep, if you learn right and you allow people to experiment and try to solve the problem, psychological safety here. So I think maybe we can all improve together. So thanks for that beautiful wisdom. So, Catherine, for people who love to talk more about privacy, is there a place where they can find you online or maybe resources where they can learn more about privacy? Yeah, I mean, I have a newsletter, it's called probably Private.

So if you're more of like a e-mail newsletter person, you can find me there. I'm starting kind of to produce also some YouTube videos. They're mainly around machine learning systems and privacy and machine learning systems. So you can find me there. My book is mainly written for data people, but I hope or I heard some feedback from also software and architects that it can be useful parts of my book. And then there's such an amazing wealth of resources online

around privacy. I would even like even just start with your cloud provider, like your main cloud provider or a few of your main service providers if you have major services and just start looking around of like what settings do they have around privacy? What do they make available to you? What knobs can you turn?

Even just starting there or even having like somebody on your team, like the art, the cloud architecture, cloud engineer on your team focus on that could end up paying dividends later of just starting to know, hey, it'd be really easy for us to turn on, you know, this or turn off this. And that would provide better privacy. So I think those are some easy ways to get started and make friends with privacy people at your org.

Be nice to them. Just reach out, set up a 1 to 115 minute coffee because they could be huge resources if you develop that or maybe you already have the relationship, but if you can develop that relationship, that can be a quick person that you can just double check your thinking with or think out loud with private, if they have a legal background, they're never going to say yes or no, which is fine.

That's what they're trained to do, good job for them, but they're going to give you advice, they're going to give you steering ideas, they're going to give you guiding questions is going to help you. So be friendly with them. Well, thanks for the plug. I'm sure if there are legal people listening to this, they will feel happy. So I think, yeah, make friends with the legal compliance team,

security team, right? So those people not there to make the job harder for you, but actually they help you and the organization to improve. So, Catherine, I love this conversation. Thank you so much for your time. I hope the listeners here learn a lot about the data privacy today. So thanks again. Thanks so much, Henry.