Hey, what's going on? Y'all? Welcome to another episode of Adventures in dev Ops and joining me in the studio today my new routine co host, Warren Parade. What's going on? Warren? Hey? Thanks? Well, you know, at some point that's going to get old and I'll be the old co host and we'll have to come up with a new tagline for me. I know, I was just thinking that and I was like, damn, I didn't come up with anything new to say, so I had to go with the old, stale one. I'll work on. I'll work on
that before next week's episode. I promise you didn't have enough homework to do. You've got one more thing now, right, right, It's good to keep busy. But also joining us today we have the co founder and CTO
A Hero, Jonathan Elder. Welcome Jonathan, thanks for having me. Hey, I'm excited to have you here, and I'm looking forward to this episode because I've got two security experts here in the studio with me, and I think security is one of those really really, it's a really odd specialty to have because it's one that I feel like from a business perspective, like businesses, let's just be honest. Here, businesses tend to prioritize customer facing features
and cheer about those much more than they do security work. But the end customers themselves, I don't think they ask for Like the end users of my applications don't ask for security related stuff. They just assume that I'm doing it. And so from a developer's perspective, it's really hard to feel like security gets the prioritization that it needs because everyone, I think everyone just assumes that you're doing that, because that's what you do as a good steward of your
code. And so I'm interested to talk about that and how y'all view that and get some actionable tips that help us all bring security up and like make it the first class citizen that it should be. So, with that rambling intro out of the way, Jonathan, how did you get to this point
where you're the co founder and CTO of a bureau? That's a long road, Like professionally it's been twenty years starting in the idea in Israel, but even before that, like there are many developers who are you know, tinkers to like pick apart things, figure out how it works, trying to reverse engineer stuff, whether it's you know, toasters or applications or back in the day like I don't know, Prince of Persia or whatever, trying to figure out how stuff works. So I think this was the start for me.
I've been working on many startups since then, quite a few, and it took me twenty years to or maybe twenty minus five years since we started a bureau to start as a founder with the down plotting my partner who's the CEO. After we work together on a totally different thing, which is also security, but in a different field altogether, because security is really really broad, you know. Yeah, for sure, I think it's one of those that
it's easy to to get specialized in a specialized specialty. When you talk about security. Yeah, there's there's a lot of niche like niches in in security now now people are saying cybers like. There are a lot of terms. There are people who are you know, working on ed R, people who are working like in pentests, people who are working on AppSec for thirty years. People. There are various fields that are everything is related, but there
are many specialties in the industry. So there are communalities like the cat and mouse game with like good people versus bad people. It happens a lot, and it's it's like it takes different shapes in the net work security domain where it done. And myself we worked back in the day in Microsoft and UH and in in other places, but also in apsec, and also in in
IoT and in very like diverse like sub cyber terms right on. And so with a hero, you're working to to provide visibility and and actionable context to the consumers of your platform. What does that what does that actually mean? What kind of things do you help help someone do or accomplish? Uh, there's a there's an evolution in AppSec, in application security. UH. It
started with like very like technology related products. There was at first there was you know, SASS finding vulnerabilities in your code and it and it evolved, and there there came os S security, s c A finding vulnerabilities in open source, and then software supply chain security that finding stuff like this is more develops proper right if the GitHub configuration allows you to do such and such as an example and various other fields, right, secret detection, et cetera.
So it all evolved and now it's like got a proper abbreviation, a SPM Application Security pasture management, which like takes everything into one single like UI or one single source of truth where you can see what goes on in your applications and it Uh, we've come a long way uh for Gartner to coin that term and for companies to understand that having seven different like specialized tools is not a good way to understand what goes on. There's a lot of reasons,
but I think that the main one is you don't have context. Right. You have thirty thousand alerts, twenty thousand came from that SAS tool, five thousand came from that OSS tool. And now what I have three people in my A team covering thirteen thousand developers, We have twelve hundred poor requests a day. What do I do so to answer your question like this is the context maybe to answer your question like we look at it in like three layers.
First is visibility, knowing what you have. I always say, you'll be amazed how developers, CTOs, CEO, CIOs, AppSec people they don't have a clue what they have. As long as you have more than maybe ten fifteen twenty developers, not to mention one thousand or thirty thousand, you don't know. I don't know which database is, what technology is, how many platforms for oh off you are using? How many languages are all this you can't fathom too much. So you can go from there to what APIs?
Do I have? Do I have graph quel even? I don't know? Does it go through some engeniques? What's the technology that says don't go, go, no go? I don't know? So first visibility, like what do I have? We call it an inventory? Right. The second phase is, as I said, you have thirty thousand alerts, findings, whatever. So prioritize. Tell me what should I focus on right now?
What will bring most value to my company? What will stop and attack, what will be worth my time, what will save us the most money, et cetera. So prioritization is the second thing after that remediation. And after remediation, like help me find who's the right person to talk to because I have again thirteen how much did I say, thirteen thousand developers? Who's the right person who can say this API should actually go to the internet and download
such and such other Who I can say should I go and ask? So this is remediation, and after that, like the advanced customers of a PIRO do more of managing and prevention for such things, so they employ workflows that automatically whenever a certain risk goes above a certain threshold, then they will start a certain trigger, maybe appentest, maybe a code review. So this is how we help people, like three people versus thirteen thousand people maybe try to
get a hold of application security right on. Yeah, So a big part of that is just, well, like you said, just set in context, like taking all of this information and figuring out what parts of it's relevant, which parts are urgent, and just trying to turn it into something actionable
so you don't feel like you're overwhelmed and destined to never succeed. Yeah, it's really like I think personally, one of the most frustrating things that I faced in that context is people saying shift left, shift left, which is like, it's a nice term, I get it, but if it says all the burden on me, the developer or the develops engineer, then someone else do the job, Well, why should I be the one to handle everything? Right? So shift left is really important, but you have to
do it right. If you have six thousand findings from your automat like automated certain process of static analysis, and you're shifting it left. You have to really chew it up and spit out only the three, the five, whatever actionable findings that I can handle. If you'll give me one hundred, it will be deprioritized in my I have features to develop. But if you'll say
these certain vulnerabilities, it might have false positives, it's fine. But if it's two out of three, I'm fine with it because I see the value. I can see I'm helping my company be more productive. But if these three a let's say vulnerabilities are in APIs that I know that are not in test code. They are in some application that is facing the internet, right, so it's exploitable. Perhaps maybe not, but it has a certain like it might be exploitable, and I know the code right, it's like people
from my team has written the code. And more than that, I'm a security champion, right because my team has one hundred developers. But the apps gave that task to me because they know that I'm like well versed in security stuff. Right. In that case, go on shift left. I will make it right. If you have a false positive, I'll help you be better. But I will fix it and I will accept the the two out
of three false positives. I got a hypothetical for you here, So like, I know that there are these other platforms out there that are compliance driven checklist aggregators right there for SoC two or ISO, and they collect a lot of things from individual sources within your your company, your engineering organization to help you achieve passing some sort of certification or regulation. But it feels like your company, uh actually focuses on the value of security there right like actually,
rather than some sort of checkbox approach. It really is these things are important to solve and not just to achieve a particular In my that's a great question. As my head is a CTO. I have to go through sock to type two. Of course I have to do that, and it's important too write right like show our customers that we take it seriously, even if some questions in those compliance frameworks are kind of mundane and seem like that they didn't
adapt to twenty twenty four maybe, but these two are important. So in general, we do have like a module for questionnaires. There are a lot by the way, for I'm not sure from the audience here, what's the average size of a company that they work for, from independent contractors to one
of thirteen thousand developers maybe or thirty thousand. But for the larger companies, they do have a lot of compliance going on, sock to type, sock to ISO, and also like they have all sorts of controls for you don't you can't change a certain area of the code base without answering like two hundred questions. So this is an area that we also help our customers and do
it smart. Like if we can automatically answer, as you said, or thirty seventy sixty percent of the questions because they are code related or developer related, we can easily answer that. But for the rest of the forty percent of the questions you can go and answer. Then you can automate the process.
I say, I mean, and I'm just really intrigued here, Like it sounds like you could almost utilize a pro to replace whatever automation platform you have for soalk two findings and utilize that to actually achieve it is that accurate well talk to like let's say it might require you to present that you have some sort of like a rule set of how you do back up and restore, so this is an area that we won't help you with that. You'll still you still also have to go through I don't know, some sort of
an audit vendor that will the stamp you have to do that. I won't tell. I won't save you from that one. But for the real answers like that are expensive in terms of time and money to together, right uh uh. They will ask you show me evidence that developers, maybe statistical evidence, but that developers are going through such and such process in order to to
trigger some uh some other let's say paintest or whatever. Okay, and all of these things, like many of these things we can derive from what we analyze. Because we analyze every commit in your guitar, bit bucket, git lab, whatever you use. Perforce. We analyze every commit, every ProQuest, every Gerra ticket, as a DevOps ticket, whatever. There's a lot of data there. And the last thing I want like our customers to do is go through tedious like uh like exports of things and go with their eyes
over a thousands of rows and figure out stuff by themselves. We want to help as much as we can, hope I answer the question, yeah, No, I mean you said something really interesting to me, and I think maybe interesting for the listeners as well as like where do you hook into the process? And I got this idea, maybe there's an integration that happens if I change some database through terror form that I'm already a learned that this could
have security applications. That is that accurate? Yes? Yeah, so so if you if you go let's say let's say that you're an advanced shop, right and you have everything via their form. Okay, so this is a good case. Uh we uh at at least hook up into your source control manager. Okay. If you're more advanced, we will hook into your gra maybe your Kubernetes environment in a w S or Google or asure. If you're more advanced than that, then to your API gateway, a PI security tool,
and and and maybe your security scanner. Maybe you have sneak, maybe you have menned, maybe you have like dozens of others. Right. Uh, And we take all that data and crunch it up together. But at the very minimum, we analyze your code. You can opt out, you
can choose what code to analyze and whatnot of it. But assuming that you have terror form, then yes we will alert if let's say a certain application shouldn't be connected to the internet, and maybe you hook it up into an ingress in GK whatever that is connected to the internet, then you can automate
that process and put a guardrail. Right. So I should have mentioned maybe before that we have like a concept that we called governance, which by default it comes with a certain rule set, but you're free to change, to delete rules, to add rules, to tailor it to your specific needs. But you can say, you can define what's to you, what's a material change, as we say, right, if someone added an API, it
might not be material to you. But if someone add then API that is connected to the internet and it is exposing a certain type of data, then maybe this is dangerous because whatever in your realm, this is what happens, right, And then you can say, let's define a workflow. If such and such happens, then open a ticket block. The PR don't allow a developer to change the terraform module X, y or z in such a way,
right, and so you can control that. Usually people start with understanding what they have and then they evolve, like their process evolves into putting more and more prevention guardrails like that one. I think automated feedback to the engineering teams when they're performing something that matches one of these governmance rules. I feel like there's going to be a bunch of platform develops related people out there that
are jumping for joy over this functionality. Honestly, yeah, this is as a developed like I have to people enjoy it, like people usually say, eat your own dog food. Right, Yeah, it's an important concept. So for us, it's like in the extreme, right, because our platform is for APSEC teams and developers. Developers are are a bunch of people who are really hard to please. Okay, let let's put it that way. And that again, like developers are used to tools like compilers or IDs where
they get I don't know, five nines of accuracy. Right, the compiler doesn't make mistakes. Usually it happens, but it's really rare. That's good point. Actually, usually it's your fault as a developer if you see an
error, it's not the compiler. So again, going back to the example from before, like going through six thousand alerts from a certain security platform is not acceptable for a developer, So we we really make an effort to cater the needs of developers where they get when we get to the developer like the develops as you said, the develops engineer who or who is jumping out of joy. We really we are working with our customers to make sure that the
process we will start with visibility. They understand, we understand what goes on. And then when they let's say, automate jolting with electricity, the developer who did such a bad thing, do it only when you're sure, right, or let's say more realistically, like opening a ticket for a certain team when such a thing happens, is only when they are certain right. There's always mistakes, It's fine, but we really advise customers to evolve into it.
And if you see that, you can not ask your develop steam to be always they will be alert. But don't go every day at five pm go through all pull requests and make sure there is no such and such right, don't do that. Automate that right. This is an easy thing, let's say so yeah, with care, but as much automation as we can
is really the ROI is really clear for the customer. Yeah, definitely, Yeah, And I think that's really a great approach as well, because by making sure that you're relaying the information to only the people who can actually do
something about it. You're going to minimize the tendency that we have to just turn off things that we we determined to be background noise, you know, like you like you mentioned you know, if you get ten thousand security alerts, we're going to turn it off as background noise, even though they might
not be. But if you provide specific information and and remediation steps for it, and it's like, oh, that's that's something I can do, And I think that actually lends a lot of long term stability to your overall security process for sure. Also, there's a I think something that that is people tend to consolidate certain tasks. Let's say there's a there's an app SEC team. Let's say I get a report every day every week with a list of
vulnerabilities. I might say, hey, John, you're the you're the remediation guy. H here's the list. Go every day again and again, go through it, make sure it's fine. You might stick some boxes. You'll say to the ISO person from the auditor, You'll say, I have a process. Here's John, here's my vulnerability person. I'm good, and you'll get You'll get the v there but like, at least in some cases, not all the time, but this in some cases, the shift left here
is really substantial. Right out of the six thousand developed developers. Sorry vulnerabilities, you might say, these four thousand are irrelevant because they are in tesco, they are in old repositories that are not deployed anywhere. Forget about it. Yeah and so and so, And you narrow it down and you get to the twenty seven actionable vulnerabilities that like tending to will bring impact to the company. And then you'll say, for each one of the twenty seven,
who's the right person? And here comes the concept that I like the most here, which is code ownership. In appare we have a collective code ownership, but when a certain problem arises, there's always the one person that knows about this thing the most. And it's not, you know, naively, the last person who touched that piece of code. It's more usually reality is
more complicated than that. So we make huge efforts in analyzing the behavior of every developer, what code they are touching in which way do they like. There's a difference between like the CEO doesn't know well enough and he does refractoring and he makes the code cleaner because he wants to the difference between that guy and someone else who went and replaced the authentication mechanism from user password to octa
integration datata. Right, there's a huge different difference between those two. I'm not saying the latter is better, but that person is way more capable in saying, hey, this vulnerability is real, it's actionable. This is the way to solve it. Maybe some other person will solve it, but this is the way, and this is how we can, you know, get
to do remediation much much faster and cheaper. That's super cool. So you're actually building like a a historical audit of the actions of the people on the platform, so you can identify what their expertise is and figure out who the right person is without asking, without doing like an ad channel and a SLAT group with thirteen thousand people in it. Mm hmmm, yeah exactly. And we have a thing where we I feel more like, let's say it's maybe
a bad word, like more privileged than my previous self. When we did in previous company, we did network security. We analyzed network traffic, so we connected to switch in the server farm and we tapped to the traffic and we analyzed it. But it started tequal zero. It started when we connected. Right now we are privileged is an app sec company where we connect like via API to let's say guitub and then we can analyze the entire history of
a Git repository from ten years ago. Right. So in that sense, we have from when we start as we start to have a lot of information about the code behavior, who's doing what, and we analyze the behavior ever developer and we say this subset is what we call a security champion, right. So some of our customers have actually using that. They have built like
a security champion program. So they have let's say three app sec people on the team, like who's in charge of the entire company, and they have one hundred and fifty development teams and they chose one or two people from every
division will be like their ambassadors into security. So when there's an issue with product X, they can say, hey, team, we've talked before, we've taught you about our processes, there's such and such problem, please advice, right, And he's one out of seventy people, so they don't need to go through proper channels with the management and and do like a like a heavy thing. They have people they can talk to and it's not bureaucratic.
It's data driven, which is really like it proves helpful in many cases, right, and that's super cool. One of the things you talked touched on a little bit earlier was talking about the context and the visibility of it, and you kind of mentioned that, you know, fixing these vulnerabilities gives you the ability to say, hey, we prevented or we blocked this before it
was exploited. How much value do you think there is in that by by showing the things the ability that you had to stop something before it happened and kind of elevate secure already work to that of like future requests because everyone gets excited over the new future requests. Do you think that showing the security vulnerabilities
you've re mediated helps bring bring some awareness to that? For sure? So I think coming back to something I think you said maybe before we started recording, security is a thing like everybody wants it to not exist until something hits a fan somewhere, right, and then oh my god, we had to do such and such before. I wish three months ago when we designed the
feature we would have thought about it. I wish we drink testing. We would go through the list of things that could go wrong and find out about it. So, but the reverse of it is everyone, everybody wants to celebrate new features and sell it to customers. So uh, it's a hard bargain sometimes to for security teams to prove their their worth to the organization because
nobody pays attention to quiet. Like the CSO can can come to a board meeting, or the APPSC engineer can come to a certain meeting all hands for the company and say, hey, for seven months no incident, everything is quiet on my front. They might say, okay, let's reduce the team right exactly. Yeah, life, life of security is really hard that way.
But I think I think that it's a game of numbers proving. Let's say again, when people are used to false positives, and it's true both for networks, security, intrusion alerts, whatever, DLP viruses, for everything including appsect. When people are used to false positives, they're they're they're not
paying attention, like everything is noise. We have a we have a concept of funnel where we are showing you have starting with thirty thousand alerts, and via various steps you go to a lower and lower and lower number until you get to a manageable amount of things that you can actually work on. And if this is the focus, and in each team, it's a little bit different. For some is like the is it connected to the is it exposed to the internet? Just no, it really reduces. For some everything is
exposed to the internet, So there are differences. But as long as the number goes down and and is it becomes manageable and real, then the teams pay more attention. When you come as an absect to the dev team, you're not wasting their time. They feel once they do like there's a I think there's people underestimate developers or develops engineers in the industry where some say that they don't care about security because it's someone else's concern. I don't think it's
true. Maybe that's just my opinion, but I don't think it's true. And and if if like apseck came to developers with a small number of real things, then the developers will be excited to deliver on it and they will feel that they're building a stronger product. By the way, the same thing is true also with performance and other things that are not like purely like features. Performances a feature. Security is a future. I really believe in it.
So I think as long as the numbers are are manageable, I think people will treat it the same way as features, and they can even celebrate it. Yeah, for sure. I think especially like if you are just trying to say, Okay, we got to get we got to get a grip on our security, and you're working with a code base that's been around,
you know, for a year or more. The first time you run that security scan, it's going to be overwhelming and that your first at least for me, my first instinct is to look at the findings and go, yeah, that's a problem for next sprint and then move on and next year I'll have the same conversation with myself. So I think there's really a lot of value in saying, Okay, let's just put this in context and here are the key things that you can focus on this brand that'll actually make a
difference. Yeah. I mean, you're really talking about curation there, and I think that's one of the things that a lot of tools in the Devil of Space overall get wrong. They think it's like a lack of information. They're all about exposing as many things as possible as if that's where the value
is. And I think the tool we haven't said but kept talking about here is sort of like depend upon on GitHub, which you know spams every single thing, and it's very difficult to know whether or not there's value in any one of those alerts. So realistically, you want it to be curated. That's where the value is, right, having not only information, but relevant
and useful and correct information, Right, that's prioritized. Yeah, exactly, like this is our this is our claim to fame actually, because we're saying like ESPM is the is the abbreviation, and we say deep SPM because we really believe that just aggregating alerts, putting putting them, displaying them in a nice table which streams nicely, uh, and putting nice graphs on top.
It's like, I don't want to sound disrespectful, but as you said, right now, we're putting context uh into into that data is really crucial, uh, in order to allow prioritization and remediation. So we when we say deep I'll give you an example, when we say deep uh we uh we analyze the code right, We analyze the gerry tickets, We read all the
fields. We analyze. We know what's an API. We know what's a data model, and if it's safe to the database, if it's exposed via that a PR or that I p I we know UH if a certain UH like code element is in a certain module, which might be a part of a repository. There might be a mono repo, might be a certain application that has seventeen different modules from six different repositories, some in guitub and some
in bit bucket, and some I don't know from somewhere else. So when we see a vulnerability, it might be a secret that we detected secret in code. It might be a vulnerability that came from check marks or sneak or someone else. We know a lot of context around it. Okay, So if it's part of an API, we know a lot about that API. If you connected to a COMI, then we can say, okay, that API is exposed in that cluster and is getting bombarded right now in production.
So we try as as more and more things get connected, we add more context into it. So it's really really important to inject that context into those lists of vulnerabilities or material changes, because asking the developer to do the work is not realistic. Right. We see we have customers with I think the most has a forty something thousand repositories. Oh wow, that's that's a that's a number that I don't think a person, a human being can fathom in
terms of like there's no intuition around it. So you have you have to have that knowledge modeled somewhere. I mean I do, I do have some intuition there that micro services went wrong somewhere, But you know from that, I mean you bring abouts an interesting aspects here, which is information about the likelihood of these vulnerabilities actually being a problem. And I think this is a difficult area to answer. You can look at some code, even with the
context, and know that there is a security problem there. Maybe it's open to ssr F, you know, service I Request forgeries or something on the UI side that's open to cross site scripting, but it's you know, I've been doing research and I just cannot find actual numbers on the number of say data breaches there are, because the actually detecting that it's happened and knowing that
a change that you've made is going to has actually stopped. One is really a challenge, and I assume that it's not even something that you would be
able to know about. Right. We do have some statistics for interesting, but I can say easily that a state in terms of like level of hacking might might uh utilize or take advantage of some some sort of nobility that could be found in code and might help you to find your way through after four months of an attack, like doing various things in order to get to a
certain crown jewel type of information. It happens, but many more times, ah, some some person has connected the wrong wires and and and opened the the I don't know, the some proxy server connected to the wrong the wrong network and like maybe it's via terraform or click ups or uh, maybe a certain a p I that is supposed to be under authorization is actually without because like a bug in the code, these type of things are not immune to
bugs in code. And you might have like a huge wall, a firewall and octa and whatever in front of your application that prevents the most brilliant hacker to pass through. But your API doesn't require authorization. You can just approach
it and inject things into a database or pulled data from the database. So I think what we see in various companies is evolving from the notion that we have X vulnerability is let's solve every vulnerability right, and evolving into let's do a threat model, analyze what I have in the most important applications or the
most sensitive applications, which is another area that we help with. But let's focus on those, and in those, let's understand what's the attack vector, and there we will say what types of actions we think we need to do. Maybe make sure there are no exposed secrets. Maybe make sure that the APIs don't do things that they shouldn't do. Maybe it's super duper critical. Let's make sure we have zero verun abilities and do everything we can, but
not for everything. So this is really important in terms of time and effort that the people put into it. Does that mean that you're actually also able to investigate the sort of transitive dependencies of source code pack venders that are being utilized? Yeah, actually yeah we do so, uh open sources. I'm assuming you're in open source, but not necessarily but yeah, I mean right, I mean they could be internal packages that another team has published or you
know, obviously code that comes from outside for sure. Yeah, okay, so uh yeah, we do. We have a distinction between external and internal packages, whether you publish it via an internal Nougat or Maven or something else inside your organization, or even if you have a package that you copy and paste it into some other area. I think, for me, the most the more interesting thing in terms of like developing a product is that you have
a certain module that is shared between various applications. Right So you have the classic example is an infrastructure package that has I don't know, utilities or ways to connect to you or central user service or something like that, and it's being used in like three different versions of that are being used in seventeen different applications. So what's the impact of Warren going and changing a certain threshold of
I don't know password, the strength in that package. What's the impact If it's on seventeen applications, it might be more substantial, right so, and it could go through more than one hop of analysis. So we're doing really interesting things in that area. And it's hard that computationally if you think about it. Yeah, for sure. So Warren, you're the CTO of a security off company. How do you like obviously security is like front of your
company. How do you approach this whenever you're building and deploying your product? Oh, definitely don't ask me that question. Sorry, Should we have the editor cut this part out? No, I mean it's totally fine. I mean realistically, we are definitely looking for products that, like you know, ton Scott here to review. I think a lot of the problem is that many products in the space fall into this sort of uncanny value where they're either
unnecessary alerts that don't really tell us anything. We've definitely gone through the snakes, sneaks and dependent bots code QLs of the world, and as well as look at the ones that review stuff for a Soalk two and ISO, but they're mostly checkbox driven security and don't really focus on actually finding real problems. So I think realistically it's on us to do this investigation most of the time manually, I mean obviously using the tools that are available to make these decisions.
Now, I think we have a lot of benefit over some other companies given the size that we are at much smaller, so it's not like we have forty seven thousand repositories that we've got I think the number is under a thousand, and most of them are not services. Yeah, well, I mean there's just an amount of overhead as far as cognitive overload that teams will have to deal through. So more repositories can only be managed by more people, and that breaks down as size increases. So I mean, realistically,
there are things that you just don't want to have to deal with. Now in my position, what I've really focused on is what are the platforms or tools that are out there that help eliminate a whole class of problems. So classes of problems can be eliminated by I mean me personally, you know, i'd say get away from Kubernetes, but really virtual machines in general. While it may seem like it's easy to spin them up, the number of security
concerns from my standpoint is just massive. There just patch schedules, et cetera. Even for things that are zero day, right, it's so much easier to delegate that to a potential cloud provider. How to get around those. Another thing that we're leading with I'm not sure when this is going to be available in production, is allowing our customers to sign the data that they're sending
to us. That means that we have a fidelity over you know, whether or not that data has actually been changed, not necessarily access, but specifically since we are in the AWE space, we do care about when it's been changed, and so there are things like this that we consider novel, but realistically, it's about finding those tools that help us elevate what we're utilizing to
another level. Yeah, and I think that's like a unique problem because you do have like a limited number of people on your team, but literally an unlimited number of security problems. You know, Yeah, for sure, you know that that's always going to be a trouble. I think I think this goes for everything. Focus on your core competency, whatever that is, and
find a way to eliminate or at least delegate those concerns elsewhere. So every time one of those comes up, find a company that helps you eliminate the broadest class of them, or you know, transition to a different stack or something like that where you don't have those issues, because you will never solve every problem. And I think often you may hear things like, well, we can't depend on another company offering a critical component for us, because what
happens if it's down, what happens if there's an issue there? I mean, like you should trust the security experts to get that right more so than I mean, there's no different between the set of engineers that they're hiring and the ones that you can hire. And they're probably focused on this, right. You know, we're focused on hiring people that do care about that security
aspect, just as you know Tom's focused on this as as well. I'm sure, right, you know, you hire people that understand this part of platform management and security to go forward, and you're not going to be able to do it as effectively unfortunately. Time and resources, Yeah, for sure, we at Apparel, we do have an infinite amount of time and resources
that that's their usp there right right there. You know you heard it there first, right, So a lot of times with sort of continuing on what you were saying there, Warren, you know, there's like a there's three models of solving a problem. There's do it yourself, there's tell get someone to tell you what to do, or there's just hire someone to do it
for you. And so, Jonathan, what are the what's the like entry level do it yourself version of handling this like obviously your apiro is like the Hey, we're gonna collect all the data and and put it in context and prioritize it for you. But like, what's the entry level version of that that probably most of us are actually trying to do but unsuccessfully. That would
tell us, man, there's got to be a better way. You mean, you mean, how how the company, like a software shop would do what we do manually like without yeah, uh so so like people more people essentially? Uh yeah, so so we see that, uh and I understand the like like when when you're facing a certain amount of work, then you can either uh focus the team on it and do it like one person for a small shop or one team for a larger shop that handles like the load
for everyone. And there's the companies that uh distribute to work with the load on the developers. I think the most important thing to understand to maybe a marker that might tell you that this thing got out of hand is if like hopefully you're asking your developers like how they are doing and if they they feel
that their job is going well. So if they say twenty percent, thirty percent, seventy percent of my time is going after things that I don't see any value coming out of this is a good marker that you might want to change the process. Either either use it like a platform, like a peer,
or maybe something else. Either either that or don't do it. Like sometimes it's better to take take the lead and not go through a certain process save like ten percent of your one hundred developers time, that's ten ten developers. Ten developers can bridge the gap and bring more either more value, more features, or more security. So either use the right tool for the job, or maybe do less of it of the of the noise. Sometimes it's
better then, like bringing everything to a halt. That's probably a really great
point there, actually using the right tool for the job. And I don't think that could be said enough because you know, to answer a little bit more of your question, Well, i've seen the let's build our own sort of internal strategy for automatically updating versions of packages for instance, right, And it's like, well, you're building this yourself, Like why are you building the tool versus using something that's way more intelligent at actually finding where the security
vulnerabilities are and focus on that one. And I think there's the other one I've seen, which I also am not a fan of is the replicate all the external packages in the world locally in your own package repository, because unless you have a team that is expert in reviewing that or a tool that's actually going to do it, you're not really doing anything other than having out of
date versions that are available. So you're actually almost hurting yourself more than you're helping by using one of these because it's not solving a problem of security, it's solving a problem of anything reliability or where the single point of failure is. Right. It's not a highly reliable package manager out in the world that has security teams dedicated to finding vulnerabilities. It's some tool off the shelf that
you're deployed in your own infrastructure which is out of date. Yeah. I think also this is a great example of sometimes not understanding the trade off usually most things, maybe in life, but in our field, there's a trade off. Like if you're saying, don't like I have a certain certain version of package X that is not vulnerable, then let's freeze it in time and
not never ever upgrade it because there might be a vulnerability. There's a trade of there, right, both functionally and maybe maybe it does have a vulnerability, but it's unknown. Now I'm not saying like for everything, newer is better, but there's always a trade off. So yeah, I think that they like what what you said is really really true. If you're doing it yourself and like inventing things that are not your core business, then be really
careful because you're probably not the best of breeding usually. No, I mean no, no, it's for true. I mean I think there's another problem there, which is you don't know whether or not you're doing it right. So you're going through the motions of things right, you know, you're even if maybe maybe updating the versions is the right thing, but it could just as easily be the case where it's not, and you're actually having a negative
impact overall. So you're not doing the research to understand what the best practice trends are, and you're using your own sort of heuristics from your own experiences, which they're still valid, right, but just in a small niche compared to what the collective wisdom says, and going about that, you know, in that way, which may lead you to, oh, yeah, we need to automatically roll all versions of all packages all the time, or we need to freeze all of them, and I think both of them are equally
problematic in some way. Yeah. One of the other things you know, you mentioned higher or use the people who are best at that. I think one of the things that I see common with that is whenever you go to like you say, Okay, I shouldn't be writing on authentication package myself. That's a solved problem that's been solved by people way smarter than me. So
you go out and you start looking at available packages. Like part of that research is understanding who the expert really is and digging in, like dig into their GitHub repo. Is it being updated frequently? How many committers are working on this? Because if you find an authentication package that was updated three years ago by one person and hasn't been touched since, that may not be an
improvement over your current authentication layer. Yeah. Sure, I mean i'd be curious what you know, Tom, actually if it has any information about this, Like are there heuristics about update frequency or the type of packages that lend themselves, especially in the open source world, to being more vulnerable or more secure? Yeah? So really interesting, especially right now, so in Apiro
you can see a lot of like interesting properties about open source packages. Okay, so I think will you mentioned some of the most important ones, like if it's maintained or not maintained. There are other things as well, but right now, we just published, like I think a week ago, like our research team found a lot of malicious code in many guitab repositories and some
of the guitub repositories. Like it made a lot of waves I heard about some of some of the guitub repositories are really things that you would like at first glance. You you wouldn't say there's something fishy here, right, A lot of maintainers, it's being maintained really st regularly. So I think not to be too cynical, but I don't think you can trust anything right pully
with you. Actually, I was just thinking about this, like whatever a metric you're going to use, just think about how it's going to be potentially
abused, right, you know, it's maintainers or updates or issues. You could easily clone an existing package out there on GitHub, have the same number of issues, copy everything one to one and then make just a small little malicious change in the package somewhere, you know, post install, or just for a production run time and also publish it to a new get MPM, you know, wherever, and you would never tell the difference unless someone actually
investigated that source code. So those sorts of heuristics and may not even be valuable. Yeah, yeah for sure. But you know, the same way is it's hard to get like a good grasp over forty thousand repositories in your organization. Yeah, which there's not a lot of eyes going through that code base, but in like let's say React, JS and Internet there's a lot
of people like going through. But even that repository is really large, and there are dark places within that repository that people like for years didn't pay attention to. So when you get big data, it's really like it's easier to to to hide in plain sight sometimes then if there's a two file repository and to take hold of that, so you really need the tools the machines to
do the work for you. No, I mean that that actually gives me some flashbacks to I think it was a few years ago there was a bunch of academic centers that were intentionally making malicious changes to like Linux Foundation source code for the kernel as well as other things just to prove that you could get malicious code into an open source project under all pretenses and so like that's pretty ridiculous. Yeah, yeah, I remember that. That was kind of wild.
Yeah, So nothing not things off the table when it comes to where problems could be. And you know, if you look at the lots of issues that show up in public repositories, like they're not necessarily people that know, you know, fundamentally every single aspect that's going on there, Uh, poll requests that get opened are you know you don't. No one has full understanding of how all the packages interact with each other, so you know,
who is even able to validate that? I think at this point you pretty much need to rely on some sort of tool to do this for you. Right on. Cool, So I want to ask both of you this question, because you both have strong security backgrounds. For someone who's just starting to put focus on their security footprint, what's your best piece of advice on where
to start start with? We'll start with Jonathan. Okay, So, without qualifying if it's a small company or a large company or like one developer, I think the most important thing to get a grasp on is like inventory, what I have. I think this is the fundamental part here. By the way, as a developer, you might want to reduce it as much as you can get rid of what you don't really need, and in large scales it's even more important. But knowing what you have, I think this is
the most fundamental thing to really really understand before I do anything advanced. Know what you have and plot twist. Apiro can help you with that, right, I wouldn't I wouldn't imagine saying anything that won't lead you to say that. All right, Warren, what's your thoughts? Yeah? I mean I think actually Anton mentioned this earlier and I know I've become a broken record for
this, but it's start with your threat model. If you build or approach any problem without first thinking about why you're doing it, not only can you what you're doing isn't going to nessly be useful or relevant. It can even be harmful as far as your overall approach goes. So you know concretely what like, what do we need to do, why are we doing it? And then start looking at okay, what are our options for implementing solutions?
Because otherwise, I mean, you'll end up with the six thousand dependent on alerts and not knowing what to do, and the result may be developing your own platform that automatically increments, you know, package versions, versus knowing that, hey, we're getting lots of spam on our endpoints in the form of
WP dash admin and we're not running any PHP stuff. You know, it's not a DUS attack, but it's someone's looking for an exposed WordPress instance or Mango dB or elastic search, and if you've got that exposed, then you're going to have a problem. So, as Youzon said, the the inventory of what your services are and how they're exposed to the Internet is equally valuable
for sure, right on excellent. Any any other closing or parting thoughts on security before we move on to picks, I'd say, alluding to a previous point that you made before, that security is a feature and and uh, if if you do the right thing, then then people will appreciate it.
If it's sometimes even more than doing the features. No, I mean, I love that approach actually, because often people see it as the like a cost center to prevent something and the best case scenario is that you nothing happens, and so the ROI you know, is a hard justifier, But I think the missing part there is that actually the way you've implemented security has a lot of value that for your product, for your end users, potentially a
better experience with s oh and log in is it is it defining factor for companies, especially enterprises, who want to be secure likewise goes having the latest functions and features from your dependencies, being able to utilize them and not be stuck on legacy versions for fear of a security issue that just on my gets pulled in. Like using a platform like a bureau you know, helps you get that, helps you achieve that without getting stuck in in fear land or
dedicating your resources in an area that's not your core competency. Yeah. I think that's like one of the common things that I see with teams or departments that are traditionally seen as cost centers is changing that perspective that you're not a cost center, you're just bad at marketing, you know, because there's like it there is like it's everything comes down to politics at some form or another, and so part of getting your team to not be is a cost center
is by marketing your team and showing hey, this is these are the security vulnerabilities that we closed down that we were exposed to? Or from a DevOps perspective, hey, here's how implementing this platform allowed the engineering teams to go from fifteen deployments today per day to fifty deployments per day in a safe and secure manner. So I think really learning to market your team helps give you mind share across the leaders of the company and brings additional focus. And with
additional focus, you get additional resources. I mean, last time we talked about product management, and I think this is closely connected to it. Like, you know, you can even start before that, right, Like, if you're going to be rolling out features or functionality or a platform to your organization, what do the other engineering teams actually think about? You know, do they care about even being secure? Because if they're not already on that,
then it's the same challenge as marketing to external customers. You what problems are they trying to solve? And if security is not part of it, then you're really starting with an education approach, right Like, here are the sorts of things that could happen because of how you set up your services and if the organization hasn't aligned incentives actually encourage them to be secure. Then it's going to be an uphill battle wait for them to get hacked and then go
oh oh okay, so now y'all are concerned about security. Oh. I mean there were definitely there were definitely some suspicious companies out there that were offering like DEDOS protection and as part of their actual operation, I mean they were they were hugely malicious to start out with, but uh, as part of their operations they would go out and actually just dds customers and be like, hey, you know you got d dos Because you don't, you're not using
our product. And I mean it's a little bit suspicious if after our security incident you immediately get someone showing up your door saying, you know, hey, we could help you solve that problem. Though, it's like the mafia style approach they send and then go so, hey, it looks like you need some protection. Right on. So let's do some picks as we round this episode out, Warren, I've been picking on you, so I won't
pick on you this episode. I'll actually start it off, and much to everyone who's a frequent listener, surprise, my pick this week is going to be platform con coming up this June the five day virtual conference on Platform Engineering, where they will have some folks talking about security topics too as they relate to platform engineering. And then my favorite part of the conference is at the end, I am going to be interviewing some of the speakers in a live
Q and a session to close out the conference. And as I mentioned last week, if they're specific people you want me to talk to, or specific questions you want me to ask, be sure and hit me up on Twitter or send me a DM or any other contact method and I will filter and aggregate those and get those questions to ask for you. So platform con dot com if you want to check that out, and that's my pick. So Warren, what have you got? Yeah, Actually, this is timely.
It's going to be security related. I just got back from a conference in Germany last week where I was actually talking about the security journey we took at authors and I got asked, you know, well, how do I get into security? How do I become an expert? Is it just the do I am? I either completely don't know anything or do I have to know
everything? And the truth is, you know, you just learn things over time from different sources and eventually, one day you get to stand up on stage and start giving a presentation about security, even though you believe you don't really know anything. But I will say that there are a couple of great newsletters out there that I read myself. They may be too technical for some others may be way beneath them, but I highly recommend the TLDR newsletters.
Specifically, they have one Security and there's also another one called the cloud Secklist, which is oriented towards cloud security infrastructure, and they've been great. They talking touching on a lot of different topics, and you can go deep with the links that are in there, and I can take you a long way, especially at the beginning of your security journey. Right on Awesome, how
did you talk? Go by the way? Well, you know, like usual, I'm sure I did terrible, but then everyone tells me how great it went, so I have to I'll have to watch the video once it comes out online, which will probably be posted on the author's blog somewhere Right on awesome. All right, Johnathan, what you got for us for picks?
Okay? So I had something else, But Warren, I think I can maybe connect to what you what you said because something that I feared for a long time happened at my family yesterday, where my son was a twelve year old came to me and said, Dad, I managed to hack the Google Dinosaur game and and made it such that I can never lose. It's not that advanced. It's I think it came to him through YouTube or Instagram or something like that where you just override the game over function or something like
that. She's really nice. Uh so, so it came to me as a shock, but uh it got me thinking and and I like alluding to when you said how to start in the security business. Maybe uh so. I think my piak is is when it comes to kids, maybe uh start, start from the fundamentals, not not. It might be cool to show your friends in class that you can hack a game, but I I since then, I since I tried to show him the fundamentals. What goes on there? What does it mean a function? What does it mean to change
it in run time? Like, uh, maybe you should prevent that maybe
not like like really have him understand what goes on there. I think it's really important for kids that can see YouTube or whatever TikTok, they're they're seeing really advanced things in life, not only in security or coding, but they're really pushed to go fast and like having them understand the fundamentals, understand for for a non native English speaker, understand English, understand math, understand the like the concept, and then go and hack the Pentagon or No, we're
not recommending that anyone does anything. I do not condone that sort of no. But that's a really great point actually, because there's security is so vague as an individual idea that really defining for yourself what it even means to go into security, right, Like are you interested in hacking things or even depending against attacks, which I think, you know, we talked a lot about here, but or our products that help you build up tools and whatnot.
You know, these are all different, fundamentally different areas for sure. Yeah, And I think there's like just going beyond that. I think there's huge understated value in just having a basic understanding of how code works in a modern technology life that we live in now, because once you understand what's going on under the hood, I think you get a lot of perspective in how you interact with different companies, you know, whether it's through their website or their
application or their call center or whatever. Like. If you have that basic fundamental understanding, I think you start to approach problems differently, even if you're not in a technical role. I think when you're talking to a sales rep of a customer support agent of a company and you get an error on your screen, that's as like you know, on unhandled exception, and you're trying to explain to them how their website is broken, right exactly, all right?
Cool, Jonathan, Thank you so much for being on the show. This has been incredibly insightful, so I really appreciate you taking the time to join us. Yeah, thank you so much for inviting me. Yeah. Yeah. Happy to have you back on at any point, so let us know if that's of interest to you. Qaarren, thank you again for joining me here and co host to miss with me, and to all the listeners, thank y'all for listening. Love having y'all out there, and we will see y'all next week.