Software Dependencies: Do you Know What’s Lurking in your Software? - DevOps 219

I'm also doing an accelerator and you can find more about that at dev influencers dot com. We have a special guest this week, and that is James Donahue. James, do you want to say hello and introduce yourself all the way from the Britain and BBC.

can I get it on the devop show? And initially I told her no. And then because because usually I see the article a name, I don't see the author name, and so you get on and I'm trying to make this connection, like what are.

the other code reviews were doing. So I just kind of want to give people a little bit of background. This does apply to what you're doing in DevOps, but to get us started, I was looking at this article and you kind of explain, Hey, look, you know we we had to go remote, and because of that, you know, we experience certain things, and so we started doing the

code reviews in a certain way. And I was wondering if you could tell that story, because I think this is something that people are really going to identify with that people are going to go, oh, yeah, we have that problem, and you know, maybe this will help some of them find that solution that they're looking for.

that application a couple of years ago. We had an old version that was a monolithic PHP applications service very well, but it was time to update the technology and we had lots of ideas new features we wanted to build. And what happened was we went in quite a short space of time from having about five developers working on a new renderer. We built a React based isomorphic single

page application to replace the old monolith. I think we had about five developers working on it initially and over the course of really just a few months we went from that to about thirty five developers. And also another thing happened was that we went from all being based in one office in London at Broadcasting House to being distributed across different cities in the UK and also in

other countries as well in Africa. We had some developers we were starting to have to get used to remote first development, and this is before the pandemic, this is before you know, the big changes of last year. We were having to adapt to scaling quite quickly and also being remote first all the time, and that's when we started to notice that something interesting was happening to code reviews. Some of our code views were taking out a bit

longer than we'd like. We noticed there were some problems where code reviews would spend a long time going on. There'd be a lot of comments on the pull requests, a lot of back and forth between different people. A lot of new engineers were contributing to the code base, and not all of them really knew how we did code reviews or how we approached them, and we felt that there was probably a lot of tacit assumptions in

the way we were doing code reviews. We also had a bit of feedback from some of our developers that they found the process quite confusing and quite stressful sometimes. And it was really at that point that we thought, Okay, we need to take a step back from this. We need to talk about what is this code review process for?

And so it's it's definitely interesting that, Yeah, all of a sudden you're having discussions about how you communicate in these ways. I'm a little curious before we dive into how to start approaching this though, or other panelists Jeffrey and Caleb, I mean, what, what what's been your experience with this kind of thing, going remote in this way, especially through COVID.

to throw a question back to James. I'm curious what you guys came up with, because when I read your article on Medium, I was thinking, Wow, you know you ask all these questions about like why are we doing code reviews? And I was thinking, you know what you said, it's all the above, and you're like, what fifteen or twenty different answers? What did you guys come up with? What is the reason, what's the motivation?

where we were asking what code reviews for? So you know, if you're if you're if you're discussing different approaches on a code review, that's a form of communication. If you're linking to other documentation or resources, that's communication. You know, you might be checking that work meets certain quality standards, or thinking about readability and maintainability, those kind of things.

that are of a security in nature. Right, how do we how do we try and find those underlying issues in code and sometimes just to go off on a tingent for a moment. I think that's interesting is that, you know, what, do a lot of reverse engineering of code to sort of understand, you know, where can somebody who's trying to do something nefarious, where can they sort of grab a toe hold and do something interesting, whether it's around access control, authentication, something like that, you know,

in the software base. And so I just sort of gravitate towards that idea that code reviews are always about,

you know, finding code defects. And it's interesting because I think there's a lot to what you're saying that it really should be a lot about just getting the whole team together and sort of communicating and you know, sort of more on getting them on the same page, because you know, at least generically speaking, I can see a lot of value in that and then doing you know, if you've got specific purposes like what I'm describing, then that might be a different use case, a different reason,

and maybe a different setting for running a code review outside of I think what you're describing. But I don't want to put words in your mouth, but I'm sort of just trying to get an old Sheldon in my own mind.

One of the things I mentioned in the article is there's a really interesting piece of academic research that we've done at Microsoft in I think twenty thirteen, where they examined hundreds of code reviews across large parts of their code base, and they analyzed each code of view in terms of what the actual outcome of that code review was.

They looked at every comment on the poor request and they categorized them all, and they basically figured out that the most common outcome of code reviews was basically people asking questions about something, other people explaining things, and that process of communication, learning about the code or discussing the code is the most common outcome. Whereas when they interviewed the developers and said what do you think is the most common outcome of code reviews, they said, oh, it's

all about finding defects. But there was a just found The funny effects is it does happen. Of course, it does happen. Sometimes people will find a bug in code just from looking at it visually, but there's actually a lot of other stuff going on. There's there's coaching and mentoring, there's you know, exchanging ideas and approaches.

and then we just we code all day. But then we pass off our work in a PR to somebody else who does a code review on it. And yeah, a lot of it is just we want somebody to look at it and give feedback on it.

But some of those are really kind of how do I put it, they're they're kind of cloaked to something else, you know, if it's a maintenance issue, and then we eventually clean up what's causing the maintenance issue, it's not necessarily a defect. It is a defect, but we don't think of it in the same way as we think of a bug, right because the code still works. It's just every time we go change it. We have to you know, jump through a whole bunch of hoops to

make sure we didn't break anything. It's a maintenance nightmare. And so you know, and I can kind of imagine the same thing with you know, whether it's infrastructure or chef recipes or you know, whatever you're dealing with, that same kind of thing. And so yeah, if somebody can pass along, hey, look we do it this way because we found that it saves us all these other hassles

in the back end. And what's funny is is sometimes you'll get the new person in my I have a fifteen year old who's playing video games now, so the total nube will come in, right, and they'll say, oh, well, why don't you try this other approach then? Right, that has all those benefits you just listed for why you're doing it this way, but also gives us this other benefit that I'm seeing we might be able to add.

And then all of a sudden, you have a better process, right, And by understanding and documenting and pushing and communicating, that opens up all these doors. And that's really yeah, that's what we're talking about with these code views. And that's back to the original idea. It's all of the above. It sounds like you have a process, you know, after reading through this article, you have a process for conducting these code reviews that make sure that all these things get communicated about.

Just for the reasons you've given that that quite often you get really interesting perspectives and questions and challengers from people who are new to develop new to engineering, new to this particular code base, and so in order to try and help facilitate that, we put some some prompts and suggestions into our code reviews guide, where we're saying, if you're a new developer, if you're need to this code base, here are some things you can try when

you're doing a code review. Here are some things you can ask Have you spotted something that looks a little bit inconsistent? I mean you could ask the author to explain that. Have you asked the author about if they've considered security, performance, accessibility, other kinds of non functional requirements. So a bit of a cheat sheet almost where you can say, you know, these are things we should be thinking about when we did code reviews. And it brings

more people in the team into that process, right. It means that more junior engineers or less experienced engineers can participate and actually give just as much value to that process as people who you know, have been working on the code for years.

And I don't know if this is a problem you guys have had the BBC, but I've seen teams in the past where junior developers very intimidated by the code review process, and you know, it's kind of seen as a gatekeeping role where they're checking in code and then senior developers are looking at it and kind of you know, going over it. So I really like the kind of idea of reframing it so that it's you know, code

reviews are everyone's responsibility. It's a collaborative team process, and it's more about the communication rather than the gatekeeping, even though you still get maybe some of those gatekeeping functions if you do need to have security reviews or have

senior developers reviewing code and looking over those things. Just by reframing it you get all the benefits and maybe a much better participation rate, particularly in a remote context where onboarding developers can be a little bit more intimidating, Especially for junior developers. You don't have that kind of face time. You know, it's a little hard build team cohesion remotely in those kind of things. Is that the correct kind of analysis of the direction you're going with.

at old levels to get involved in that process. And yeah, I think we were really reframing code of views a little bit and make it feel making it seem more like a collaborative process that is benefiting everybody in the team. I mean, I think we have to recognize as well that code views can be quite demanding, can be quite stressful in fact, for people of all levels of experience.

You know, having your work looked at by other people can be challenging sometimes, But at the same time, it's part of being a software engineer that you people are to look at your code and you have to be able to receive feedback about your code and give feedback on other's code and get comfortable with that process. So there is obviously still a gatekeeping function within code reviews, and there's an important role for that, but we wanted to try and promote the idea that the team owns

the code collectively. It's not like up to an individual person to decide whether something is okay or not.

He's a dream hire. Blah blah blah blah. And every time I do a code review, I feel like I have to live up to that, right, And so it feels that way for seniors sometimes too, right They feel like they have to be the one to catch it, they have to be the one to see it, they have to be the one to know the answer to that. And there are some parts of these applications that I just have not really done a whole lot with and I have to swallow my pride and take that too.

And so I think it's important for everybody at whatever level you're at, to just realize, look, you've got to you know, this is an opportunity for you to level up as much as it is for an opportunity for you to help somebody else level up.

we want to find code defects? Did you or did your team sort of decide or sort of come out of this and say okay, in order to sort of keep people focused on what we want them to be doing and not just sort of you know, getting them too stressed out about, you know, trying to find something that may not even be there, right, did your team sort of relook at well, maybe there's a better way for us to find you know, let's get people's mindset completely away from you know, I'm going to find something

wrong with somebody's code, right, and and and focus that on hey, maybe we'll do a better job, or maybe we'll sort of rethink how we formalize the QA function or you know, testing or something like this. This way, everyone understands that that's really where we do, that's where we find code efacts, right, and and to really focus people on the review process is not about that if it comes up great, if not, so that that's My first question is, you know, was there change in how you also.

difficult and stressful. We then had some feedback later on that things had improved and that there was there was a view that things were now getting easier for especially for new developers. That's kind of a very kind of

subjective thing. On the more sort of objective quantitive side, we could see we had some software that we were using to track the number of poor requests that were open, how long those poor requests had been in code review, how many could review cycles they've been through, and how many comments from the pull requests. And we saw that all of those measurements kind of moved in the direction

we wanted them to go in. So code was spending less time in code review, it was going through less revision cycles, and the result was we were actually merging and shipping that code faster. Now, the code view process wasn't the only thing that we changed during that time. I've got to say there was a lot of changes going on at the time. One of the other things we were working on at the same time we tried, Charles, you've spoken about mob programming or swarm programming.

down the burden of code reviews. I'm really interested to hear what other people think about that because we found that by doing more collaboration upfront before you get into the code review stage, by having more voices and more kind of perspectives heard during the initial implementation phase, quite often that the sort of formal okay, now let's do a code review part of this is actually tends to be a lot smoother because a lot of that communication

has already happened, a lot of possible objections or concerns have already been anticipated earlier on. So that was something else we were trying at the same time. And I think it's a complementary technique. It's not like you don't have to do just that or just change your code reviews process, that both things can kind of can kind

of help you achieve the same goal. But yeah, we have seen we have seen improvements both in terms of the rate in which we're able to deliver stuff and also subjective feedback from anonymous feedback from developers in the team that they were finding the process just easier and more enjoyable.

What about this? What about the other thing? And I do the same thing for them, and we just turn out way better code, you know, from the maintainability, from the reliability, from the performance, from pretty much any measurement you have for code. We just do, you know, And so they they make my code look better what I commit, and you know, all around vice versa. We do most of that over Microsoft Teams. I will not tell you how much I dislike Microsoft Teams, but it works. We

also use poll requests on GitHub. That's our primary way of doing that, and so all the feedback goes into the pole request and then when it gets merged, then you know, it's all still in GitHub if we ever have to go back and look at it. We don't have any real in house tools, and we don't track how long they sit in pole requests because honestly, it's abnormal for a pole request within our organization because we actually review code from the other projects that we're not

working on. Within the organization, we everybody reviews everybody's stuff, and so if you don't understand what's going on, in the other project.

as well. So yeah, you know a lot of the same approach. But I think we're also you're working for a fairly large company. I'm working for a large enterprise. I'm deliberately not saying where I work, but I work for a fairly large enterprise company here in the US.

comments on a pull request. And when we started to talk about code reviews and think about how we could do them, we realized there's so many other ways of doing code reviews. You know, the idea of code reviews long predates GitHub obviously, and often adding comments to a poor request on GitHub is the least efficient way of doing.

if you need that asynchronous communication. But there are you know, there are situations where the most efficient way of doing a code review would be to get on a team's call or a zoom call or something else with the other person, the author if you're the reviewer, and talk through their changes and try and understand their reasoning, the

thinking behind the change they made, talk about alternatives. Yes, you might want to then think about summarizing the results of those conversations in some forms so it can be persisted to the poor request or some other forms so that other people can can you know, see the outcome

of that discussion. But one of the things we try to encourage our engineers to do is if you're having a conversation with somebody on a pull request and you're getting a comment and sort of back and forth comment report, comment reply, that's usually a sign okay, maybe this would benefit from a call. Maybe we should talk about this face to face, because it's just, you know, it's just not the most efficient way of two people explaining their thinking.

You know, getting on a call is often going to save time and save misunderstandings as well.

pick up the phone and talk to somebody. Other people because of the way they work, because of their communication preferences, would rather just write a long description of their thoughts on a pull request. So what we wanted to do through this, through writing this guide to code reviews, was to sort of at least give people the opportunities to think about the alternatives and why there might sometimes be

better ways of communicating. So you know, we don't insist upon everybody going on a call all the time to discuss poor requests. It's sometimes it's just not practical, but it should always be there as a you know, as a kind of something that people can say, Okay, look, I think this might be a good idea. It's something we probably want to consider, and at least people are aware of the options.

author to clarify them. Sometimes we find that when people seek that clarification, the author might say actually, or you know, in a conversation talking about it, you might say, well, okay, if you're asking that question, then somebody else is going to think this in the future as well. So is there a documentation need here? Is the code readable enough? Do we need to put something? You know, kind of do we have to have some additional documentation or explanations

make it easier to understand? So, yeah, getting clarification is an important first one. And then I mean everything really kind of is all about asking questions. So you know, the cliche about there being no stupid questions. We try and kind of spell that out in our guide to code reviews. We really want people to ask questions on the assumption that if you're thinking it, if you're going to ask the questions, somebody else is probably thinking it too.

Somebody else is at least wondering the same thing. And as I already said, we try and encourage developers to think about things like non functional requirements quite often, things that might have been missed during the initial requirements analysis or during the implementation. You know, have we thought about performance on this? Are we load testing this? Have we

updated documentation? Is their missing test coverage? You know, a lot of these things that can be very easily even if we have the most perfect engineering process from start to when things can get missed. And sometimes asking those questions, even if the answer is just yes, we've already anticipated all of those things, it's useful to have somebody checking

those questions. And yeah, I mean that those are kind of those are the main areas really, and we do kind of try and encourage people to if they're adding comments on THEO pull requests to sort of offer links and examples as well. So if you're going to sort of say, you know, there's a better way of doing this, or there's an alternative approach to doing this, or there's a more I don't know, idiomatic way of doing this

in this particular language. You know, it really helps it to sort of give a bit more context than that, either to give an example to link to some relevant documentation or a blog post that explains the technique in more detail, and it really kind of gives the author or the person reading the comments of the opportunity to do a bit of research and read up and understand what it is that you're saying and save some of that kind of time with back and forth explanations.

and designing around that during the initial phases. And some of the things that you said kind of brought that to mind as far as pull requests go, where it's like instead of kind of pushing all of the review and the collaboration to the end, you know, kind of the swarm programming approach and things like that seem like it could kind of be used in a similar manner to actually bring some of the review process to the beginning of the design rather than waiting until something's already

implemented in order.

you know, ahead of time. My hope is that by improving our code view process and gradually pushing, as you say, more of this thinking earlier on in the process, it will gradually become more ingrained in the way we work, and we can and we can we can get better

at anticipating those things earlier on. But yeah, I mean, I think from my point of view, the truth is that some of those things will will still get missed, and we need the code of view stage to sort of really just double check that we've we've not neglected any of those areas.

in a remote context. What I've seen in some teams is developers will kind of get lost in whatever they're doing and they'll forget to communicate because, you know, communications, it doesn't come naturally at all developers, right might be a stereotype or personality type, but it is what it is.

code of view process to try and improve the earlier stages further left than our process for the for the next iteration. So when we have team retrospectives and things like that, we'll often try and talk about code reviews that were difficult or issues that only came out and code reviewed that really could have been discussed or you know, would have benefited from more communication or more collaboration earlier

on in the process. And I think that is actually quite a helpful technique, because sometimes I think there's a temptation you think, right, the code reviews finished, let's ship the code, let's forget about it and move on, And often you can actually look at the code of view and say, you know what, what have we learned from this that we could have, what insights we take away from this that we could actually apply earlier on in the development process next time we go through this cycle.

bad code review process? Maybe that's a better question. I don't know, are there are there reasons that you wouldn't want to do this or ways to avoid problems while you're trying to do this.

understanding of what they're for. So I think that's a useful exercise because you know, in the past we had a situation where, you know, maybe something would take quite a long time to get through a code review, and product managers or product owners might say, well, what's happening?

So I think reflecting on the process is a for us has been a really powerful way of of kind of getting more understanding of our process. But you know, code of views a it can be really complicated and difficult, and things do still go wrong. You know that there are situations where code views don't go as as we'd like, but it's something that we're constantly trying to work on. We didn't want this to be a thing where we just write a document about code reviews and then said, Okay,

we're done, We've solved the problem of code views. We're going to move on. We wanted it to be something that we could keep on building on. We put it in our one of our open source repos so that it's kind of it's fronts and center that people outside the organization can see it, and we keep on iterating on it and adding to it and changing it as we get more, as we learn more about what works

for us or what doesn't work for us. You know, anybody can do a poor request against it and say, hey, this technique really worked for me in when I was learning to do code reviews, and then it becomes something that can benefit other people in the future.

then do this, do and do this and do this. Right, How should they start implementing that if they haven't been doing that today.

in those resources is a good fit for them. So you might read I think it's gaily as post which was a big inspiration for me about code reviews, and you might say, well, that's actually a perfect description of high I want my code reviews in my team to work. So we're just going to say this is our process,

We're just going to follow this. For us, it was a little bit different because we felt that we had there was lots of specifics about how we worked in our organization, our specific technical stack, how our continuous integration, because the r delivery process has worked, how we used githubs specifically in our teams. We wanted to have something that was a bit more tailored to our process. So I would say decide if any of the existing online resources is a good fit for you.

for a few years. So it's a question of picking out the things that make the most sense to you and are most relevant to you, know, your team, the structure of your organization, the technology you're working on, and then have somewhere for developers to find that information so you know and get help. You can obviously have pull request ten plates you link to a document with these guidelines and bits of advice that you want them to follow.

And I think the other part of it is, as I said, just to just to keep on reviewing that process and reflecting on it in the future. You know, if you see how it goes and try it for a few sprints, or try it for a few months, and then have a look at the process again, see if it's working and see if it needs any changes. The other thing I would say is one thing that really helped us was about in terms of reducing the burden of code reviews was to try and automate anything that you can.

at those kind of code reviews. You know, if you've got linting tools available for whichever your whichever technical environment you're using, use those instead, you know, apply automatic formatting, to use automation so that when people are doing code views they can focus on the stuff that really matters. So yeah, and also, as we already said, I think pairing and mob programming can be a really good way of making code of views less difficult.

and it's out there. It's in one of our open source repos because the project that we developed it for is an open source project itself, so we wanted to put it in the read may say, look, this is how we do code reviews. If you're going to submit a poor request to this project, this is what you can expect is how we're going to approach our code reviews.

really is a pretty robust platform. I'm starting to look at it, starting to use it for just in my practic how I provide content. Sometimes it is course courses that are customed for specific clients. Sometimes it's just you know, sort of you know, content that I just want to provide to either you know, all my sort of audience

and prospects and clients and you know, just everyone. And it's a really like I said, what I've done so far is it's really cool way, really efficient way and cost efficient as well, just to get your content out there to the folks that.

shout out about dev Influencers. I'm not going to go into too many more details than I went into at the beginning, but you can get the podcast at dev Influencers dot com a's Dev Influencers with an s dot com slash podcast, you can also apply for the Accelerator Dev Influencers dot Com Slash A reply and I've been helping people get their podcasts started and build an audience so that they can build a career.

opening up. The mask mandate lifts for everything except schools, and I guess certain spaces with certain numbers of people where you can't social distance, But for everywhere else, the mass bandate lifts. In like three days, my wife and I actually went on a date and we went out to a movie. We were like two of the four people in the movie theater watching the movie, but we really really enjoyed it.

Bond movie. And I'm sitting there thinking to myself, you didn't watch the trailer, did you? Because the trailer it's very obvious it's a Cold War spy based on a true story kind of thing, right, and so it's going to be about tradecraft. It's not going to be about somebody jumping through windows and shooting guns. Sorry, but I really enjoy some of the historical contexts that some of these films have brought up.

that's how they knew about that. Oh, that's how they knew about this. So it was really really fascinating. But yeah, so he would just pass it off and this guy would just take it away, and I never knew what was in a lot of it, right, And anyway, it's a terrific film, and it's really fascinating because you're spending the whole times that they're going is he gonna get caught? And yeah, anyway, I'm not gonna spoil the end of it. I have no spoilers policy. If it's been out less

than a year. In this movie has been out less than a year, so go watch it. It's terrific. He does such a terrific job too, because you kind of see him in a lot of these other movies as kind of this svelt kind of action figure, and you know, he's been a at coub badge and so he's he's in shape, but he definitely pulls off playing the kind of normal, friendly, jovial salesman guy who you know isn't the action figure that's going to go in save the Day.

So I loved it, absolutely loved it. And so yeah, that's the courier if you're out looking for a movie to go watch.

to do with plant mL. For those who are thinking, oh, I don't want to do UML diagrams, it actually generates lots of different types of diagrams, not just UML diagrams, and we're using it quite a bit of my work for creating threat models for some of our services. One of the benefits of you plant YOUML rather than another diagramming tool is that you actually write the diagram in a textual format and then it generates the graphics from that for you, which means you can keep the source

of the diagram in version control. People can do pull requests against it and that kind of thing. Anyway, this website, real World plan YOUML, is really helpful because the syntax can sometimes be a little bit forbidding if you're not used to it, and it shows you some examples of what can be achieved with it and how you can achieve it. So that's been I've been copying quite a few things from that recently just to help with making my diagrams.