Developer productivity is not lines of code written. It's not number of commits. Its kind of irreducible to any sort of low-level metric like that. It really has to do with the ultimate problem. You're solving and the users that you're solving it. For. Hey, everyone.
My name is Henry Surya Barragan. And you're listening to the tekhelet Juno, the show will be bringing you the greatest technical leaders practitioners and thought leaders in the industry to discuss about their Journey ideas and practices that we all can learn and apply to build a highly performing technical team and to make an impact in your personal work. So let's dive into our Journal. Hey everyone, very excited to be back here again with another new episode of the tekhelet journal
podcast. Thank you for tuning in and spending your time with me today, listening to this episode. If you're new, to the podcast know that technology, you know, is available for you to subscribe on major podcast apps such as Spotify, a podcast Google podcast, YouTube, and many others. Also, please check out and follow technology, you know, social media channels on LinkedIn, Twitter and Instagram.
Everyday, you will find words of wisdom from the latest podcast episode and I share them on those channels to give us some inspiration and motivation for us to get better each day. And if you'd like to make some contribution to the show and support the creation of this podcast, please consider joining as a patron by visiting technology. No dot f /, Patron. I highly appreciate any kind of support and your contribution would help me to add sustainably
producing this show every week. For today's episode. I am happy to share. My conversation with Bianca, you beiong is the CTO and co-founder of sauce graph, a developer tools company that brings universal code search capability for developers in open source and every software organization including leading companies like uber, Dropbox, Yelp, PayPal, and
cloudflare. In this episode, beiong, shared with me his story on why he started building Source graph, and the kind of problems that he's trying to solve it learning from his Has experience working at Google and experiencing firsthand a great developer experience using Google internal tool called code search. He shared with me, his perspective about developers productivity and how we should
go about measuring. It, including some words of caution for measuring productivity, using proxy metrics, such as lines of code, number of comets, or pull requests Etc. We then discuss the rationale for universal code search and why he thinks there's a massive. Need for such developer tool to increase developers productivity and especially to cope in the current ERA of big code.
A new kind of challenge that every software team and company has to deal with related to the ever increasing size and complexity of the code bases that are typical organization has to maintain towards the end being shared, how individuals can improve their personal developer productivity, and what the future state of developer tools would look like Duo. So listen and check out some of the source craft cool use cases that Beyond shared with me. Based on the feedback given by
his customers. Those use cases, are pretty cool to me and you may understand why universal code search. Might just be a tool that you need to increase your developers
productivity. I enjoyed my conversation with be young and if you also enjoy listening to this episode, consider helping the show by living it a rating review or comment on your podcast app or social media channels, those reviews and Comments are one of the best ways to get this podcast to reach more listeners and hopefully they can also benefit from the contents in this podcast. So, let's get this episode started right after a sponsor
message. Are you looking for a new cool swag tekhelet Journal. Now offers you some swags that you can purchase online. These wax are printed on demand based on your preference and will be delivered safely to you all over the world where shipping is available. Check out all the cool swag is available by visiting. Eating technology. No dot, f / shop, and don't forget to break yourself. Once you receive any of those tracks. Hey everyone, welcome to another
new episode of the package. You know, today. I have a special guest with me. His name is being Lou, he is the co-founder and CTO of sauce graph. So, if you haven't heard about Source, graph is actually a tool for developers to do code searching. And basically, helping the developers in order to be more productive with their code. I'll let beiong to A mobile Source graph later. So being welcome to the show. Hey, Henry. Thanks for having me on.
So, beyond before we start, maybe let me hear from you your introduction. What's your background? What's your highlights selling points in your career for us to hear about? Yeah, definitely. So I've been programming since maybe like Middle High School. I think I got into it first on, like a TI-83, graphing calculator. There's like a dial, like, the basic that you could code in that kind of got me into.
It is really fun. I was always kind of like a And science nerd in my elementary middle school days and this kind of fit right in with that. After I got to college. I took cs101 and fell in love with it and things unfurled from there. I guess from there.
I ended up doing this internship at Google during my college Years. I just mentioned that because that was where I first got exposed to this concept of code search, which is a thing that started me down this path, that ultimately culminated in the starting Source graph with Quinn and building developer tools, but that was cool because it gave me a bunch of Experience playing around with like different developer tools that Google had built in internally,
one of which was with this Google code search thing. After I left Google and work at other companies. I kind of realize that code search was not the standard thing. And so it always felt like this piece of the toolkit that was missing. That was the case at palantir. I joined palantir shortly after graduating college and worked on the commercial side of the company or building Big Data analysis, tools for large Banks,
and financial institutions. And that's where I got to work closely with Quinn. My co-founder for the first. Time we were on this kind of like small and startup within the startup team, trying to tackle the data related problems of these large Banks Quinn and I were both programmers by background. Both had this kind of like side hobby of trying out different tools, different editors different command line utilities.
And we were always talking about how much of our time was spent reading through existing code and making sense of that. And also, this like gap between. When we first got into programming. I think a lot of people get into and fall in love with it with that sense of being able to create something. Sit down type away your desk and then couple hours later or maybe a couple days later. However, long the coding session is you walk away with something
that you built and created. That is almost like this living thing that does what you want contrast that to like the experience of typical professional software developer, which we're getting more and more acquainted with. And they're like, man, we spend very little of our times doing that and way too much of our times, just trying to make sense
of the existing code base. We saw also that pain reflected over in the software engineers at our Customers that were working with, that was the kernel or the initial conversation that ultimately grew into Source graph. So wanting to get back to that awesome flow states. That every developer loves that inspired us to get into programming. Make that more accessible to day-to-day developers by helping them understand their existing code. So we end up starting Source
graph from there. At this point. Probably the majority of my professional career has been working at source craft. So we started the company back in mid 2013 and now it's seven going on eight years later. Thanks for sharing your story. Definitely. It's interesting because you bumped into this opportunity. Also, by chance, seeing how Google does the internal tooling for developers. You mentioned a lot of times about developers productivity. Maybe let's start by the
definition. What do you mean by developers productivity? Because these days, there are so many things about productivity, right? Everyone is into productivity by the, your to-do list, your calendar, whatever apps that you use. What do you mean exactly by developers productivity? Is it any different than any kind of productivity out there? Yeah, that's a great question. Developer productivity is a notoriously difficult thing to Define.
So I will try to come at it from maybe a couple different angles and see if that can help hone in on roughly what it means. First off. I just want to say developer productivity is not lines of code written. It's not number of commits. Its kind of irreducible to any sort of low-level metric like that. Because if you think about it, at the end of the day, what we do as developers is, we're really creating new technology every single feature or product. That we work on.
Is this new thing. It's never existed before our job is to build a program that solves that new problem in a new way. It's almost like a mini R&D project rather than a factory model where you just like turning out Widgets or whatever because of that property. You can't really break it down until okay. This is just the sum of all the functions and classes and files that you author. It really has to do with the ultimate problem, you're solving and the users that you're
solving it for. So So that's one angle. It's not lines of code. It's not like any sort of low-level metric organizations that end up, treating that as a proxy even like a first-order proxy. I think end up introducing perverse incentives into their development teams. That's how you get a lot of code written, but not actually solving user problems. The other kind of angle to look at. This is just view it. In terms of the overall picture
in some sense. Every piece of software is worth some amount of time saved or money because time is money in Certain sense. So if you look at the value that you're creating, one thing that you can look at is, how much is this piece of software being used by users who's using it? And how much are they, paying you or how much revenue are you generating from this piece of software?
Because that gives you, I think a better proxy for how much productivity you're generating because at the end of the day software is all about solving and automating problems for people, saving people time having the computer do something. So that a human being doesn't have to do this wrote in repetitive process. And so if you want to measure productivity, Ava T at a high level. You start with the end user think about time, saved or money, saved for the use of your
software. Now, that's the high level principled way of looking at it. How does that actually help you if you're an engineering manager or a program around a development team, inside a large organization, the connection between your individual work or your team's work to the overall revenue of the company or the time saved across all the users of the product is often a hard line to draw and that's why you get things like the Organization coming up with points like
Sprint points. If you use a scrum or a drawer, one of those processes, one of the responsibilities of the product organization is to roughly map user value time saved, whatever it matters to the company into the customers to this point system, that reduces all those complex factors and their interrelationships between different teams to rough Point. Allocation. That's easy for engineering teams to reason about. And so then you're talking about how many points are you doing in
a given quarter or so? I don't know if that's a satisfying answer but that's roughly how productivity is measured organizationally, but I think at an individual level it's almost like this feeling. It's almost like this binary thing because a developer you either feel productive, you don't feel productive. If you've done any amount of software engineering, you know, that feeling the difference between being in this Flow State where your brain feels like.
It's firing on all cylinders, you know, you have all the contextual information you need is coding away and you're like, holy crap. I'm getting a lot done versus the other. Which is when you're like, I don't actually know how this piece of code that I'm
contributing to works. So all the changes I make are like to local or I keep breaking things, or I keep having to ask this other person, a bunch of questions or the iteration Cycles too long for me to get feedback to get into that, kind of Flow State. So, an individual level, I think it often reduces to just, which mode are you in? Are you in kind of paralysis mode? Or are you in getting stuff, done? Shipping code mode? I like the way that you Explain
that. At the end of the day, it has to bring value either to your users or brings Revenue to your company. But obviously, this is a very tough thing to Define because how do you actually map back what you do as a developer, you know, like churning code, maybe debugging testing and map that back into how you bring value to the user? And I think it's taken for granted and many companies that they don't actually do this kind
of measurement. I know, and you say for individual, we know it. When we feel it. There will be a day when you feel like, oh, you're on fire, you getting along Things done, you could just works pretty fine. But there are days where you are judging to maybe processes. Maybe. I don't know like very difficult box to solve or even competing priorities between multiple tasks that you need to do.
But in the first place because in many companies, I'm sure that there's no such thing as a good well-defined developers productivity metrics or measurement, but do you think that we should start measuring and investing our time in coming up with this developers productivity, numbers match? Alex, or even like some kind of rough proxy. Yeah, that's a great question. I think it is important to measure things in to quantify them because I think numbers,
keep people honest. Obviously, they're never going to capture everything that you care about. But in order to have something falsifiable and checkable, it's good to define a metric. And then track your progress toward it. Because if you just Define things in terms of qualitatively, it's very easy to trick yourself and other people into thinking that. Oh, they actually hit that where maybe you didn't actually
deliver. On what you were trying to deliver on so metrics and quantifying things is important, but it's really important. What metric you choose and how
you quantify that right now. I don't know if there is like a single Universal metric of developer productivity, that it makes sense to adopt across every software organization, because, I think, whatever metrics that you use are in some sense, tied to your unique organization, and in some sense, tied to the problem that you're solving or the type of software that You're building. So the way I think about it is if you're a single developer working on a solo project.
That's completely self-contained. Let's say you're an indie developer. You're going to build a simple tool and then you going to sell that for money your metric. The one that matters is really Revenue. How much money are you generating? How much our users willing to pay for this piece of software? Because that tells you how useful this piece of software is to them in some sense or it's at least a lower bound on how useful this thing is to them.
And you scale that up to a hundred person or or thousand person or At the top of your product engineering organization. There's a person may be their title is VP. Maybe the title is sitio. Maybe it's director, who is responsible for delivery of the whole system and is in some sense responsible for ensuring that system is delivering as much value as it can to its users. Whether its measured by Revenue that product generates or time spent using the product.
Whatever it is. They probably have one or two metrics that really are the kpi key performance indicator of how. Well, They're doing their job when they report to the CEO and they report to the board. That's how they're evaluated, at least in most well-run suffer companies. It's that person's job to then, go break it down, like the people reporting them. Okay, what do I need them to do in order to achieve this goal? In some cases? They might be able to break it
down, such that. Okay. This part of the product is very independent from this other part of the product. So I'm going to have to reports of mine. You're responsible for this one and you're responsible for that one, and I'm going to look at the engagement numbers of each separately and I'm going to tie your compensation. Shinto those metrics, because that's a really good proxy of the overall usefulness of each
part of the system. Now, that's the lucky case, if you can take apart the things and it breaks down nicely without interdependencies. I think more commonly is that. There's a lot of interdependencies. You can't really say that you can't evaluate any piece of an end user software application in isolation, without reasoning about its impact on other parts. And there's also a cross-cutting concerns such as technical debt test coverage and things like that. You have to worry about like the
overall health. Our team. And at the end of the day, it really is that person's responsibility to figure out how this all breaks down. Yeah. I unfortunately don't have a satisfying answer here because I think every organization is a little bit for any. It starts from the top that person has a number that they're trying to optimize that as a proxy for the overall, end user value that they're trying to
deliver. And then part of the art and craft of building, an organization is figure out what the right breakdown of responsibility is and what metrics are going to. Used to evaluate how each constituent part of that organization is working. So, in the beginning, you mentioned things like story points, right? These days almost every company's running, some kind of a jaw methodology, scrum or
T-shirt size, whatever they use. A lot of them actually starts using that as the measurement of productivity or either like the product engineering team or even like individuals, and I know these are like two edged sword. Sometimes it works. But many cases actually, it doesn't work because, you know, you can game the system, you create a As many points as you want to, but actually doesn't mean anything or sometimes you just take it. Okay, this story points.
Let's assume it's done. Let's create a new card. So it's an illusion that you're making progress. But my question on this is that do you start seeing these as a problem coming up with the proxy that actually doesn't work. And if so, what will be from your experience, some basic indicators that may be any team could start with that. Doesn't lead them into something that mislead the actual things
that they want to measure. Yeah, so if you're using any sort, Points-based system, I think one of the things you want to be wary of is ensuring that you're being honest with yourselves about Point allocation. A lot of. It depends on the specific people. You have on the team if you have someone who's experienced are running scrum or agile, who knows how to work the system that can make the difference.
Between this being a working system for your team and it being a not working system for your team. Another thing that a lot of organizations do is, there's a separation of product from engineering. So that product is responsible for allocating the points and then engineering is responsible for executing the projects
Associated those points. And because there's that separation, you remove one of the incentives to game the system because in some sense, engineering is measuring themselves by velocity, but they don't get to choose. How many points are associated with each project. So that's if you're using a points-based system. It's not for everyone, full disclosure Source graph. We used a points-based system, the past we currently don't on
any of the engineering teams. We might use one on select engineering teams in the future, not ruling it out. It's perfectly good system, but it's not the one that we're currently using. I think there are a lot of other proxy metrics that you can. Look at that.
Give you a sense of is your organization overall in a healthy or not healthy State. I think one thing you can do is you can send out survey just surveying your developers to see how satisfied they are and ask them how predicted they feel relative to their ideal level of productivity, or the levels of productivity that they attain working at previous companies. I think a lot of developers have a And innate sense of how
productive they feel. Because again, it's this binary thing where you either feel like, okay and getting stuff done, or know there's a lot of barriers and paper cuts that are hurting me, preventing me from getting my job done. You can also look at the Ops and deployment side. There's a standard set of metrics now, or I should say an emerging standard related to how quickly you deploy and ship software. So those I think they go by the name accelerate or sometimes Dora metrics.
I think there's four of them. It have to do with like how frequently you deploy decode, how much time there is between submitting, a commit into the Upstream git repository, to when it shows up in production, things like that. Those are good things to minimize. If they're too high, then that's an indicator that your organs is likely, not as productive as it could be. In fact, probably much less productive than it could be. So those more quantitative things are tied, more to the
operational side. In my view. The operational side tends to be a little bit more quantitative because Ops is all about taking. Taking the software, the source code that you've written and just getting it into production that process tends to be these days. A more automated process. A lot of humans have been taken out of the loop there with things.
Like see ICD. You have these systems are designed to take the changes to source code and then push them through some sort of q, a pipeline, until they're automatically deployed on the more product in application engineering side. It's tougher to quantify and that's why you have these Point systems where that's an attempt at quantification through product prioritization. But you know, unfortunately, I don't have a great answer other than talking to your developers and asking them.
Hey, are you hurting right now or do you feel like you're being really productive? There isn't really a good Universal set of metrics that you can look at on the application engineering and product side that gives you a good idea of how good your team is operating.
I think that side of the picture is almost like emotional intuitive empathetic and I think that's why I put these such an important quality and Engineering managers because they kind of have their finger on the pulse of the development team. They can a know what the potential of the people is on that team and how close they are to realizing or actualizing that potential. I like the way that you simplify all this just by talking to your developers. So sometimes a lot of managers,
a lot of leaders. They forget this aspect, maybe also relates back to the empathy thing that you mentioned, because maybe they try to find the perfect system, either like a gel story points. Technical debt lines of code, being tested or whatever that is without actually thinking back on The Human Side. And the developers themselves, they might give a lot of insights into a daily
productive. Or are they being hindered by processes or politics teams or even technical debt, which has been surrounded over a period of time. So I like the way that you relates back to Developers for leaders out there. I think it's very easy. Just talked to your developers. Ask them. How do they feel when they work on some code lately? Just real quick. I just want to clarify.
You're absolutely right. You should have your finger on the pulse of the development team and you should be talking to your developers daily. I'm not saying that you can't come up with any sort of quantitative. Metrics on the engineering side, like a lot of organizations do, let's say you identify Tech debt as a big problem and then you come up with a way to evaluate code for Tech tip. Maybe there's some simple parsing or linting mechanism.
That is a proxy for that or if there's not enough testing your code base. If that's a clear need, then you can spin up a coverage tool like coveralls are code Cove and track that metric down because that gives you a line by line, but I think you can't use those as top-line indicators of overall health. They are just kpis that you track to Specific problems. And you always keep the human in the loop here. You're always in touch with your
development team. To ensure that as the number goes down, people also feel better about it. So, in the context of that technology Industries these days, right? Obviously, there's this Enterprise to traditional Enterprises companies who have been around a long time. They might have a well-defined software development process and there's also the startups which probably arguably do not have any process in place, and they just figure it out along the way.
So, do you think they are? To set of problems where from the Legacy Point of View, maybe you have traditional ways of doing software development, either the waterfall or the software development lifecycle that companies adopt and also the Other Extreme, which the startups who are trying to come up with a good way of making the app developers, more productive turning out more features for their products.
Do you think that these two extremes that you need to think about, especially for those who work on these set of companies? Yeah. So between startups and Enterprises, I guess the question is, how different are they actually do? We need? A completely different set of rules and best practices for how to do great software development inside large Enterprises versus startups. My cents on. That is it's always yes, and no but I think fundamentally, it's
the same, right? Like software developers wherever you are, whether in a big company or a small company. There's a couple things that you really care about that. Make you really productive fast feedback loop. Are you able to build tests compile the code quickly? How long does it take between offering a change and seeing it in production and getting the Users, all those Loops you care about making as quick as possible. You care about automating as much as you can.
So if there's any certainty way that you need to do in order to ensure that your suffer meets a certain quality bar. Ideally, you want that automated without human beings in the loop. So that everything just works. You can focus on the creative parts of the job. And you want a really robust development environment where it's easy to look up the context that you need to have in order
to be productive. One of the things that might be undervalued at both startups and large Enterprises is the amount of the job of being a professional software engineer or developer that is reading and understanding code before you can actually write the code that builds the specific feature that you're implementing. You have to understand the code base that you're contributing to. You have to understand how your change fits into that.
You have to understand what other shared libraries are packages exist in your code base or perhaps out there in the open source world that you can use in leverage because you don't want to reinvent the wheel. And then when you go to submit the change, most organizations these days, practice code review. So someone else Team has to go and review the code and understand your change. Understand how it impacts the rest of the code base.
Ideally, understand it, as well as the person who authored the change, which is no small feat. In some ways. It's more difficult, but we can get into that later. That's a problem. That applies. I think universally from the largest Enterprise development organizations, all the way down to the individual developer. I think where it gets different between the two scenarios is just the constraints that you have to work with.
So, if you're working inside a large existing organizations, especially one that has an established product. Duct, the ton of users you're going to be working in an environment with more constraints. Those constraints are sometimes bad, but often good, they always exist for a reason, you have quality constraints to ensure that code that breaks functionality for end-users
doesn't make it into production. You have the constraints of fitting your change into the overall code base following the stylistic conventions meeting the necessary test coverage bars passing Architectural Review from the senior engineers and architects, who are Possible for maintaining a certain level of consistency and quality across the code.
These are all constraints that are necessary to make the team functional at a larger scale, all in the service of building more complex product, that's able to do more stuff from where users and so dealing with those constraints, especially in the
context of a larger code base. I think it becomes a bigger part of the problem solving role of software engineering inside larger organizations, because you're not just solving, for the needs of the users directly, but you're solving also for these Strains imposed by the overall organization in the service of the overall user set, or customer set of the organization, definitely makes sense. I think you brought a wonderful point for developers.
Most of the time you don't work with like Greenfield project. We don't come up with code from the scratch. Even if we do right. We normally utilize code from somewhere either like open source projects, somewhere like, stack Overflow, and we just search and maybe copy paste and adopt. I think it's a good segue as well to talk about. Code search, which is what you're doing with Source graph. So maybe you can share, why do you think companies should invest in building tool?
Like so scrap or having a code search capability within a company? And how does it differ with normal IDE, which can search code as well. So, maybe you can share a little bit more about code search. Yeah. So code search is a solution to a problem and I'll start with the problem. The problem is this collection of challenges and difficulties that people who are starting to call Big code.
Big code is a buzzword. It's like big data in a sense, I guess, but it would encapsulate Siz all the things that get much harder when you're developing code at scale inside this world and universe with way more code than there used to be and that applies to both to co bases inside organizations. And now you have more and more companies that are more and more software driven, and they have larger and larger code bases
than ever before. And a lot of these companies, by the way, are not like what you would call tech companies. Traditionally. There are a couple He's in other sectors, like agriculture, Healthcare or Finance, or whatever. The some really old established companies that are now mainly software companies. So that's one side a bit code is the code inside your company's these large proprietary code bases. And the other side is, the exploding world of Open Source.
Open source has been around for a while now is decades old, but the past 10 years have seen this, just like sharp uptick in the volume and the number and the diversity of Open source libraries in packages that are available a lot of this driven by proliferation of the web and web services. All of which is Howard by these open source. Libraries. There's not a software organization today that isn't heavily reliant on open source
code, which is amazing in a way. But with all this progress also come challenges, right? So the challenges of working inside a large code base. It means that there's more kind of context that you might be unaware. That there's more to kind of read it and understand before you. And start writing your new feature with confidence. There's often this like Paradox of choice or embarrassment of riches issue that comes along with you know, you start writing something.
You don't want to reinvent the wheel and you're like, oh, yeah, there's probably an open-source package. That already does this thing that I'm building. Let me go find it. And turns out there's not just one there's five and then you have to go and figure out which one is the right one to use. If I choose incorrectly it could cause me a lot of pain down the road. So this important decision, but how do I evaluate things all
these challenges? Has are related to the fact that we're operating in this world, of unprecedented, volume of code. And all of it might be relevant to the task. You're doing and you want to take advantage of that wealth of knowledge. And at the same time you have to also satisfy the constraints that volume of code might impose on you. So this is going back to the proprietary side.
A bit code. If you're contributing to a large established codebase, you need to ensure that you follow conventions Union. Sure that the change that you commit doesn't just work in isolation but interacts. Well with other components you need to And essentially like how it's going to fit in. So anyways, all these challenges, they add up and they add up so much that at some point, reaches, a breaking point.
It's some point it reaches the point where the constraints that have to work with and the context that you have to load up and your head starts to be too much to fit into your IDE. You can't just clone all the code that you need to know about to your local machine and one by one set up their development environments and spend up in your IDE.
And at some point, it can get so bad that inside some of these larger organizations, the customers that we've worked with the Developers. Is literally say like we don't feel like we can get stuff done. Like, we're not shipping things anymore because the burden of the existing code base is so great. And so that's where code search
comes into play. Because code search, is this tool that 20 years ago, there wouldn't be a need for it inside most organizations because code bases were smaller back then. But these days because everyone's working inside these large code bases, where they want to know about the code. That's not their local machine. They want to understand what's out there. They want to see what libraries exist. And how to use them. They need a Rule to search over this giant Corpus of data.
Just like, in the early days of the internet. You didn't necessarily need a search engine because it was smaller and you could use something like Yahoo! And that got you everywhere, that you might want to go.
But then as the volume of data and knowledge increase in that Corpus of data, you wanted something that allowed you to instantly jump to what you're trying to get to and make accessible the kind of long tail of things available in that Global Knowledge Graph. And so with code, it's things like, okay, how do I discover the libraries that? Are available either internal or external. They might help me get my current job or task at hand done. Once I find the available
libraries. How do I use them? How do other people use them? The best way to learn a new libraries Often by example, so I want to see like, how this particular function is called, but other folks, my teammates or maybe other open source contributors out there once I start using it. Maybe this is a bug that tap is in production. Now, I have to go and explore this unfamiliar. Codebase this code that I didn't write someone else wrote. Maybe I've never met that
person. Before. But now I need to go and debug their code and figure out. Okay, is this error message coming from within this library? And if it is, how can I fix it? And sometimes you're doing it. While production is down, so that your company is literally burning money while you're trying to fix this problem. So all these things go back to like you're operating in this
giant codebase. You never know when you're going to have to understand a part of that code that's unfamiliar to you or that you've never worked on before and in order to get there as quickly as possible with minimal. Friction, with minimal context, switching, you need something like Code search. That's optimized to help you discover and understand those unfamiliar pieces of code. So, this is my first time I could a hearing, the term be
coat. I mean, we all know about big data, but because I think the way that you explain it, I think it makes sense. We see a lot of more and more code, even if you just count GitHub repositories with the open source world. I think it just increases exponentially, it will keep growing as more and more people are into engineering programming and things like that. I think the other term it could certainly make Coming back to the use case, here of the code search.
Maybe you can share some of the case studies from some of your customers. How does actually something like code search help to improve their productivity. Is there anything for a particular research that you have done that actually owe you? But using code search really improve something by X. Yeah. So there's a couple different case studies and are irrelevant. I'll start with kind of the most concrete one, which is when your site goes down, when there's a production issue, you want to
get this. Right back up as quickly as possible, and you want to solve the root problem as quickly as possible. But in either case, you want that time to be as short as possible. Ideally seconds. If not minutes, definitely within the same day when an issue arises, the first question engineer gets paged and often woken up in the middle of night, and you wake up, wipe the sleep from your eyes and log onto your machine. The first question is, what the
heck went wrong. You're trying to root cause that issue. So you go through the logs, you see an error message in the logs, and now you have to go and actually make that change in. Code or maybe you have to identify which commits caused that change and revert that a lot of our customers. When they start using Source graph. It becomes our go-to tool for doing this. You just take whatever are is appearing. The logs.
You pop into the source graph and nine times out of 10 that gives you exactly what you're looking for. You click on the result. You see exactly the line of code, that's throwing the air and then from there, you can quickly figure out what's going on and write the fix and commit. It get production back online. So, we've heard from customers who have honed in on that use cases.
Has we've turned some incidents. That would have been like hour-long, downtime incidents, two things that were solved in a matter of minutes, which is huge. I think, especially in this age when a lot of applications are web services. So if there's a production error means, your site is literally down and means you can't make any money off the site. In your users are also yelling at you because this thing that they probably started to depend on rely on is currently unavailable.
So that's the very like a cute thing that we've been able to solve for at the opposite end of the spectrum. There is what impact? How we Had on just like the day-to-day developer productivity that, as I said before, is a notoriously difficult thing to measure. So, one of the challenges that we face is, when we go sell the companies. They're like, okay, you know, we'd like to measure the impact that you're having on our organization and then we say, okay, great.
How do you currently measure the productivity of your developers? Some people that use Sprint points in which case we'll say after adopting Source graph. You'll see an increase in overall velocity measured in points, but if they don't use One of those Frameworks and they don't have a good way of measuring developer productivity. Then one of our roles is sort of becomes our responsibility to
help them determine a metric. That is a reasonably good proxy for the productivity of their developers. And sometimes it literally just reduces to developer satisfaction. So we had cases, especially over the past year, with covid and the pandemic. We actually had zero customers churn through the past year when we went and asked our contacts at a lot of customers and said, you know, it's interesting. I knew that your business was really hard. We expected you to be a
potential turn risk. They said well, when covid first hit we did like a survey of all the third party vendors and tools that we use to see which ones we could possibly cut. Because remember February, March last year. The sky was falling and everyone was like, okay, if we have to cut things, what can we cut when they got to Source graph? And I like, hey, development team. How would you feel if we cut Source graph? The response is just overwhelming.
This is a need to have tool. If you remove this, to, I literally won't be able to get my job done. The Us, was that intense for that reason. It's saved our necks last year. So that's like the opposite end of the spectrum. And then there's this like third angle, which touches upon one of the other metrics that I mentioned before, which are these kind of ad hoc organization specific metrics. If a particular organization has
a goal in mind. Let's say it's like increased test coverage by X percent, or maybe we want to deprecate. This old hacky API that, we've been trying to get rid of for the past couple of years, but somehow it's impossible to Stamp Out. We've recently built into our app, the ability to track usages of particular functions and things like that and also the ability to track this numerically on like a chart basis.
So if you have one of these metrics that you can Define, in terms of a small piece of code, that takes your code, bases input and spits out a number for. What is this metric? We can actually help you track that number as it goes down. Another case study is like particular customer has this, use case of deprecating this API and so we actually have a burndown chart over here are all the out. Outstanding usages of this API.
Now your goal as an organization is to make this go to 0 by the end of the year and here's how you're doing right now. Wow, I left all those use cases that you mentioned suddenly I can relate and I really like the true validation of your tool when things are in crisis. Like in the covid situation, the true validation of developers, most of them who would like to have social graph still in place.
I think that's a true validation of how your tool or code search in this place is the true measurement of how Developers Can be productive, definitely some of the companies or teams might find it fancy for these use cases, but I think if you asked developers, these are definitely very important, but there is just no tool around to solve this problem and thank you for solving this problem. Definitely, few years ago.
You wrote this blog post about current state of developers tools and I think code search with Source. Graph is one thing. What are the other tools that you think Worth to try out an experiment, whether you can bring the same impact like Source graph into a team. Yeah, totally. I think this is a really exciting time to be working on developer tools.
And I guess to be a software developer in general because what we've seen in the past five years or so, but probably even the past like two or three years. There's just been this explosion in the number of available open source developer tools and developer tool companies. It used to be very much the case. I think that really good developer tools were associated with a single proprietary. System of think windows in the 90s they had visual studio and dotnet and c-sharp, I guess to
some extent. You could argue like the Java ecosystems fostered by Sun and then Oracle and that was the case for a long time. But I think in recent years perhaps driven by the proliferation of cloud and web services, you've just seen this explosion of tools that are not tied to any particular vendor ecosystem. They're just vailable for anyone to use regardless of whether you using this particular technology stack, that's been a huge Boon
because it's freed. He's up a lot of room for Innovation, but it's also been a bit of a challenge because now, there's like a bunch of different choices to choose from, it's no longer. Just choose your vertical, choose, which stack your on, and then adopt all those standard tools and that ecosystem. Sorry. I have degress from the original question here, which is what tools would I recommend that are really useful. Yeah. So I think there are a lot of
great tools. I think that there is a lot of great work being done and the devops or op side of things right now, so couple of tools that we use at source graph on the monitor. Inside we use this tool called Centre. It's like application are monitoring think of it as like a stack Trace Explorer but on steroids for your production systems. That's great. We use this tool called honeycomb, which is for observability.
Observability. Is this new way of thinking about how you keep an eye on the state of your production systems. It allows us to instrument our application and then go and explore the data set of production events in this kind of open-ended fashion, which is very important for debugging. Being anomalous events. Hopefully you're at fixing issues as they pop up in production, but that also means that every new error in production is one that you haven't seen before.
So, in order to be able to diagnose as effectively, you kind of want this more, open-ended exploration tool, which is what honeycomb is great for. On the employment side. We use a tool called Palou me, and we also use care reform from Hashi. Corp. The infrastructure management deployment tools have gotten a lot better since 2013. The tools were using. I think it was like AWS cloud. Nation and doctor didn't even exist back then.
Nowadays. You got tools like terraform and plumy that make it much better to manage all the configuration state of what you want deployed in kind of decorative fashion. So easy to reason about and then the system takes care of spinning up. The proper infrastructure. There's obviously Doctrine kubernetes, which have made deploying software a lot easier. I think especially multi-service applications, which source graph is a multi-service application.
It's also been critical for making your software available. Oil in a self-hosted way the conventional wisdom these days is the world is moving to Cloud. But from our experience many, if not, most large Enterprises still prefer a solution that they can deploy into their own environment, especially a product that indexes their code, which is very sensitive data.
And so without Doctrine, kubernetes it be much harder to offer a self-hosted product because you wouldn't have as much control over the application deployment environment and a lot of this is going to be organization specific, right? So what works for us might not Certainly be the right thing for you. But those are the tools that we use, right? And I couldn't also relate that to another article that you wrote which is an ex-googler this guy to develop a tool
during your internship. You exposed to a lot of internal tools that maybe Google build, or maybe they used from it open source or wherever they find it. Yeah. Why do you think it's important for you to write it? And what's your mission in? Educating people to know about what tools exist in Google and how it helps back to the world. Yeah, so I have my own experience as an ex-googler, but to be fully up front. I did an internship at Google. So I was there for all of three months.
It's been a while since I've been at Google and I had that experience. But what we started to realize as usage or Source graph group was that we were Landing in those customers because an ex-googler would bring us in. Because the pattern that we saw over and over again was someone would leave Google, they would miss Google code search which is Google's, internal code search utility index does all Google's code. Accessible to every developer.
They would look around and they would find source craft. And they would say, this is awesome. I'm going to introduce this to my new company. Google as a development organization, is one of the most advanced and sophisticated in the world.
They really pioneered a lot of tools and technologies that have since, either made it into open source, or the inspired, similar open-source counterparts, but because all these kind of Technologies are things were first built at Google and there's like, Google specific tools inside, Google. Google has become something of this like evolutionary the analogy, I like to use is it's kind of like Australia, you go to Australia, you look at the animals there.
They look similar to animals elsewhere in the world, but they're all different. They got Kangaroos and stuff. Google is like that where they branched off from the rest of the world, a while back and they've been on their own evolutionary path. A lot of the tools that they use internally are similar to ones in open source, but they're not
the same ones. And so one of the challenges that we saw in speaking with all these X googlers who became source, Users and customers was like Hey, you know having some difficulty mapping from Google internal technology to what's available outside and that's a pain point because I think the developer experience inside Google is so good.
That one of the first things that developers do when they leave Google is they try to recreate pieces of that developer experience, because like I felt the absence of code search was super painful. And so wrote that blog post, based on conversations with a bunch of such X googlers and tried to put together a kind of a quick, Explainer for how do you map from Google internal tools to ones available, outside to be a front. This was not the first post that
touched on the topic. I think there's a GitHub repo actually, which I think we linked to in the post that just gives a comprehensive listing of all the Google internal tools and their external counterparts. And our goal in the post was just to present some of that information in more of a narrative fashion and talk about, not only what tools you might want to look at. But like how do you bring those tools into your organization as an Blur entering the outside
world. So for those of you who would like to learn more about all these mapping what tools exist in Google and how it maps with others in the world. I put that in the show notes also having seen inside Google itself. I can validate that actually these tools really brings a lot of productivity and they are well integrated end-to-end from the beginning of your software development life cycle to the end up to your code being deployed on even maybe the locks and monitoring observability and
all that. So sometimes the integration part probably is one of the most Crucial piece of all this developer tools because without that, you'll still scrambling insiders when you want to debug something that you go to this tool, but when you find another problem, you go to the other tools, so I think it will be great one day in the future.
Hopefully soon that we will start seeing all these interoperability between different developer tools, and it'll be great experience for developers to have that as well. So, coming back to the developers productivity, right? So, we talked a lot about from the product engineering point of view, company, team and all that, but what if I as an individual as a programmer. I would like to also improve my productivity. What should I do? Maybe you have some perspective here.
Yeah, so on an individual basis, I think you always want to start with the pain that you're trying to solve. When you're coding everyday, make note of what you find difficult, what annoys you where you're kind of like context
switching away from code. Like anything that takes you out of that flow state, where the state of flow is this kind of mental state, where you just feel feel like your thought processes is extremely fluid and you just cranking out code and he takes you out of that or anything that prevents you from getting into. That. That's something that you should address and you should address it in the way that programmers address. Every problem, which is figure out how to automate it.
Chances are if it's causing you pain. It's because it's something wrote or repetitive or manual, or potentially unnecessary something that is boring to your brain, but you have to do. And so when you go to automate it, there's obviously two ways of doing that. You can either build something that It's it for you, or you could find a tool that already exists, that solves the issue, do a quick Google search and see if anything exists.
So that's like a very needs driven way of addressing pain points. The other angle, I'd recommend going about, it is just find developers on the internet, who write blog posts or maybe they tweet about their work flow or the tools that they use and use the tools that they use.
Because I think one of the ways you get good at any sort of craft as you learn from the Masters that craft and It's not even about learning from masters of The Craft per se because I think software development is one of those things where there's so many tools that chances are if you talk to anyone who's been doing this for at least, some period of time. They'll probably have at least one tool that they are aware of, that would be really useful to you. That would be really cool to try out.
In fact, we actually started doing this thing during quarantine on our development team. We're a bunch of Engineers on the source craft team would join like a live stream and we have one person just screen, share their Dev environment. Up and walk us through. Like how do you make a code? Commit? Show us your editor? What command line utility is to use on a daily basis? And that was awesome.
On the first one that we did. I walked away with three new tools that I had to try out that weekend because it was like, wow. You can actually do that. I had no idea. I don't think it's announced yet, but we're going to turn that into web series because we think that that sort of conversation is so interesting and useful especially in the covid world or as the world becomes more remote. I think one of the things that we lose out on On is the ability to look over.
Our teammates, shoulders walk over to the desk and be like, oh, what is that on your screen that sort of like social connection. I think it's just a great way to discover new tools and Technologies. I totally agree with you on this. So from my experience, with the way I figure out other tools is when I normally did pairing with other developers.
So, during the time when we can sit together side by side and also yeah, like what you said on Twitter, people sometimes post random things that looks cool or maybe it's just new open source tools that someone We just wrote it and publish it. You think that it will help for your productivity. So I think I totally agree and I think the permutation of many things that we can apply in our developer workflow.
I think it's just tremendous, there will be a lot of things that you can try to improve in terms of your productivity. So a little bit here. I know that you mentioned. This is the time, the current exciting times for developer productivity tools, right? What are some of your predictions seeing this, maybe? I don't know. Five years forward. What will be the state of this developer productivity? Activity tools.
Yeah, so I think right now we're in this explosion of different tools in the kind of diversity of offerings in the parts of the software development lifecycle that they tackle and it's amazing. I think that will continue over the next five years. I think this is a trend that's already playing out which is more. And more of these tools are either open source or opencore including the ones that enterprises pay for.
And so there's a lot of companies that are pioneering business models built on top of Open Source Technologies, which I think is really cool. As developers care a lot about the openness of the software they build on top of, and also the software that you use. I think that kind of gets at the importance of being able to introspect into your tools. If something goes wrong, you can poke in and see what went wrong. This is also, this almost like philosophical feeling that.
Hey, if I'm building a substantial portion of my workflow on top of this tool, then I deserve a certain level of freedom in terms of ability to continue using that tool to some extent. I think people are still burned by Proprietary lock in which was a common strategy. I think for software vendors in the 1990s, but increasingly the consensus is that open is the way to go. And so I think you'll see more and more companies that are building tools where the tool that's relevant for the
individual developer. That part is going to be open source, and then there's like an Enterprise component that is more important for teams or organizations. That might be kept proprietary. In order to be able to build a sustainable business on top of the technology. So that's one Trend that I think will continue.
I do think that with the explosion in diversity of tools that we're currently looking at there, will inevitably be the pendulum swing backwards, a little bit into more consolidation, especially around particular, ecosystems and targeting specific aspects of the software development life
cycle. You'll see essentially like winners and losers to certain extent, and the winners will probably buy up some of the losers and they'll be some amount of consolidation and standardization, but I don't think it will ever go all the way back because I think we're really in this. World. Right now. I think in the old world, the model was you had these proprietary vendors that built up. These vertically integrated ecosystems that they control, they controlled the marketplace.
They controlled what Partners were able to get in front of customers and they control the technology stack. Microsoft built out the tech stack for Windows and made it a really lucrative environment thing. And I don't think that models necessarily wrong it obviously yield a lot of value over the years, but I think now we've gotten to the point where there is no single entity that can Capture all the creativity and Innovation that can happen with software and you see that with a web.
Why is the web? The platform that appears to have won out over all the existing, kind of incumbent desktop operating systems? And I think that some of it is tied to the openness of the web. The fact that it's not under the control of any single entity and that permits this kind of flourishing and diversity of ideas that can spring up and find users and use cases. That would have been Unthinkable, even a couple years before. And so I think that's going to very much continue.
You're going to see the continual flourishing of this, like third-party independent developer tools ecosystem, that's going to resist complete consolidation into that kind of world. What's going to become more important are these kind of like meta tools or aggregators that unify multiple tools into a single coherent experience and at some point I think as software continues to eat the world as software development becomes ubiquitous at some
point. The devtools market will just become like the Petit tools Market because code will become so synonymous with knowledge work that the vast majority of people in the economy could be building software in some shape or form and that's going to be like in this huge vibrant diverse ecosystem. The last thing I'll say there is that in that explosion of different offerings, anytime. You have a large data set or Market that explodes like that.
I think search is going to be an incredibly piece of functionality because it's just going to be like so many things to consider in a good search engine. I think, we'll, I started to make sense of it all. And I hope Source. Graphing, continue to play a good role. They're definitely very exciting. You're right. I mean, software is eating the world.
So more and more people get into programming writing code, more and more open source, projects are being published, and it's going to be tremendously growing over the time. And I'm sure developer tools will be a plenty as well for us to try out. And I think, don't forget as well, the code these days is not just sometimes, impacts, right?
There are a lot of audio, something like this, or even like videos on Tube. I think those probably are some of the opportunities that haven't been tapped by people who are teaching tutorials on the videos. Maybe sometimes they just want to search, which part of the video that actually teach this. Maybe it's going to be cool. So thanks so much for sharing all these stories. I think due to time, we have to cut it out.
But before I let you leave, normally, I would ask this one question for all my guests to share with my audience here, which is the three technical leadership wisdom. So being do you have any kind of wisdom that you want to share with all of us here? Yeah. The one piece of advice that I got early in my career, as a tech lead or an engineering manager that really stuck with me, is when it comes to motivating the people on your team and unlocking their potential.
You always want to start with the why aspect of their jobs. So, rather than telling people do this, do that. You always want to give them a kind of high level goal that they can creatively, use their creativity to find a solution to, for a junior engineer. That high level goal might be something that is Ultimately low-level go and make this button work really well for users, but the should always be that component of here is what the end goal is.
And I do not care how you do it. That's up to you to figure out. That's up to you, to exercise, your human brain and creative inspiration to figure out how best to solve that problem. Because I think that Creative Drive is what motivates every developer. It's like why we got into
programming, right? That Act of Creation, that active creative problem solving, I think, oftentimes especially as the organization Rose and you're trying to solve these bigger problems and you're solving them by breaking them down into smaller bits can be very easy to just become more prescriptive or more imperative when it comes to telling people what they need to do, but what I found works for both myself and the people I work with is always keeping that goal in a way that
the person actually, satisfying that goal is able to work creatively. That's good advice. I would say for both, like tech leads when you're talking to others and explaining a problem that you want them to solve. And for the person who's receiving that If your manager is explaining this problem to you, ask the questions about what is the end desired State here rather than just taking
down the orders. Because I think that will ultimately help you do your job better and lead to a happier manager because you've done a better job, he found a more creative solution to the problem that you were asked to solve. I think another piece of advice is, if you find yourself doing anything twice that you don't like, doing find a way to automate it or look for a tool that automates. It that is your kind of job as a programmer is to Make things.
And if you're not automating your own life, then you're not living your own values. And then, lastly, just remember to have fun. It's amazing that we get to work in a job like this because coding is so fun. You get to create these like applications that are used by dozens or hundreds or thousands or maybe sometimes you millions of users and you get to do all that just by sitting at your desk and typing things into a computer. I think that there's a lot of complexity to deal with
professional software. Engineering is there's a lot of people issues. Shoes, there's a lot of understanding existing context. There's a lot of slog to the job. But I think at the end of the day, the best way to make yourself, do a good job and also not get overwhelmed by the complexity of deal with. Is to remember what got you into program in the first place?
Focus on that, and try to experience that as many times per day as possible because I think, if you do that, it will drive you to focus on more of the creative aspects of the job. It will also motivate you to automate the more mundane or wrote aspects of the job. And it'll just make you a happier person, because it'll feel like you're doing something that's truly exercises. The human creative aspects of your brain. Thanks for sharing the wisdom.
I like the last one. Definitely, we all got into programming because of something something that's, you know, spark when we try it, writing some code and something magically happen. I think it's also synonymous with the productivity concept of a developer, right? When you are productive. Basically, you can generate code, that actually works, may be impacted many part of people's lives, but I think we have to be conscious over the
time. Let's say we don't enjoy what we are doing the programming things. Maybe the productivity goes down and we are not having fun. So you always have to look back and think what got you into the programming in the first place. So thanks again for your time. Being for people who wants to find you online or learn more about social graph. Do you have a place where they
can go to? Yeah, first graph just go to short script that calm you should be able to search over all the open source code from that URL or you can download and run your own source graph instance. And for me. Probably the best place to reach me is just on Twitter. I'm at biamby ey A and G. Feel free to reach out or DM me or tweet at me or whatever you like cool. So, thanks again Beyond. I hope one day I would be able to use your tools in my development workflow, and it will be great.
I think to have all these superpowers for developer. Awesome. Thank you, Henry, and thank you so much for having me. This is great. Thank you for listening to this episode and for staying right till the end. If you highly enjoyed, please share it with your friends and colleagues who you think would also benefit from listening to this episode. And if you're new to the podcast, make sure to subscribe and leave me your valuable review and feedback.
It really, really helps me a lot in order to grow these podcasts better. You can also find the full show notes of this conversation on the episode page at technology. No, the death website. Including the full transcript interesting quotes, and links to the resources and mentions from the conversation. And lastly make sure to subscribe to the show's mailing list on technology node are deaf to get notified for any future episodes. Stay tuned for the next technique Journal episode.
And until then. Goodbye.