Well, I learned that snowflake is like really practicing go direct. So, you know, if something you don't like, you'd like to fix, you have to go to a person who made the decision. Usually you learn much more about the decision. There's a good conversation. Sometimes you convince the owner of the decision to do something different. I think nothing beats going direct because any other means of trying to change certain decisions at a point of view indirectly is ineffective, causes frictions and ultimately energy is dissipated.
The Engineering Leadership Podcast brought to you by ELC, the Engineering Leadership Community. And I'm Patrick Gallagher, and we're your host. Our show shares the most critical perspectives, habits and examples of great software engineering leaders to help evolve leadership in the tech industry.
Welcome back to the show, and hopefully from a wonderful holiday weekend, or if the 4th of July is not a holiday you celebrate that you still have the opportunity to spend quality time with your friends and family.
So, in the next episode, we have a conversation with Greg Czajkowski, SVP of Engineering at snowflake, discussing everything from the secrets he's learned about great teams, including some incredible perspectives like the power of eliminating hierarchy, going direct, how to reduce energy dissipation in your teams, removing friction in your org, creating higher innovation per time unit, and many other great perspectives.
Let me introduce you to Greg. Greg's career has made him a verified distributed systems and org scaling expert. Prior to snowflake, he spent 13 years at Google as the VP of Engineering responsible for a broad portfolio of Google Cloud data analytics and machine learning products and for internal services, addressing data analytics needs of all of Google's businesses. Before Google, Greg spent six years at Sun Microsystems working on Java runtime environments and operating systems.
Greg also holds over 50 patents. We're also joined by guest cohost Clarence Chio. Clarence is CTO and co founder at Unit 21 and he's a long time ELC member. Enjoy our conversation with Greg Czajkowski. So the first thing I wanted to do is I just want to say welcome Clarence, really excited to have you join us as a co host for this episode. So thank you for for joining us today.
I'm excited to be here and excited to dive into Greg and snowflake. I've been a big fan with been customers of snowflake at Unit 21 for over a year now and we're really excited, really impressed by the technology and also by the leadership of the growth of the engineering team. Absolutely. Greg, how are you doing? What's what's going on? I'm great. It's very nice to be here with you guys. So let's go for it.
Let's do it. So to set the stage a little bit, you know, we're going to explore some of the different perspectives that you've gained around building great teams and some of the lessons that you've learned around operating in a really highly competitive SaaS environment that that snowflake exists in to begin our story and exploration of your experience.
So the next thing I want to do is to bring is there a team from your past career at like at Sun, Google or snowflake or otherwise where you think back are you reflect and you say, man, that was just a superb team to be a part of. That was an amazing experience or so a team that's like we need more we need more teams like this. Is there an experience that comes of mine there?
I'm talking maybe blessed event because in every single team I was on was like that since my early days as a micro systems and Google now it's no flag I oversee a whole bunch of teams right but I see great teams and to end and you know, I don't want to name one team because I would do a disservice to many others, but I'll just say that it's all those teams have is one unifying theme.
People on them are focused align on what they do they move fast and they are empowered to bring concerns as they arise right so essentially people can make decisions of course in a bigger context of what we want to accomplish. So I also try to create such teams. Both Google and Sun are legendary companies I think you know no one can really contest that but what makes nothing different like what made you want to join snowflake? What's that to you?
So first I was at Google I was in charge of all of the data analytics tools at the time when snowflake started to rise to prominence first I learned about snowflake from one ads on highway 101 we have very creative marketing department I love their ads.
I got intrigued right so I started to debunk snowflake and it was very early after the launch but of course it was my competition so I started to pay attention big time and I was increasingly impressed with a boldness of vision and the execution velocity.
It was just kind of amazing how such a small company could essentially run circles around bigger ones and deliver innovative ideas to market quickly a lot of innovations per time units so to speak right so so I was re attracted to that and then I met the founders and I was really impressed by how great a conversation was how well we clicked.
I talked with Frank Stuttman our CEO and you know then he shared with me some of his core tenets I'll talk about later but no go direct so how essentially a lot of problems can be solved by go direct and I've always been like operating in the go direct mode but till Frank told me was kind of this subconscious you know idea then he verbalized it so I was very impressed I was I was also concerned I was going for a large from a large company to a then start up right how people operate how culture is different and actually turned out that it was not.
It turned out that it was not much different from the one at the Google maybe some decisions were made faster you know there was I think more focus because there was only one product and we had to focus but it was not a very painful transition fact it was a very pleasant transition.
You mentioned one of the things that really stood out to you about snowflake was the moving fast doing really innovative things and this conversation with Frank and understanding sort of the core leadership tenets or like the core company tenants that that he applies like also sounds like that was something that really stuck with you so my the next question was going to be like well what are the secrets like what have you learned about.
Building great teams and how's that shape your approach now so maybe you can share some of those those lessons that have shaped you and you mentioned go direct would love to dive in deeper into that one if we wanted to start there.
If any group of people there always no different opinions different experiences different backgrounds and so which would shape different decisions and quite often you might not be happy with some decision or some take on on something right then in many places the typical
more operation than is to not go to the source of this decision but to go to somebody else complain avoid a discussion and you know not every discussion used to be difficult in fact if you kind of practice going direct you you will learn how to bring even the most
difficult topic in a respectful and proactive way well they learned that snowflake is like really practicing go direct so you know if something you don't like you'd like to fix you have to go to a person who made the decision usually you learn
much more about the decision so it's not like you go there and someone say oh you know we made this decision on a whim and now we have to overturn it there's a good conversation sometimes you convinced the owner of the decision to do something different but I think nothing beats going direct because any other means of trying to change certain decisions
certain point of view indirectly is ineffective causes frictions and ultimately energy is dissipated another thing I learned snowflake again I think in hindsight maybe I knew it all along but I was able to like consciously verbalize it is that it's very important to pay attention to energy
energy is pretty much like the only thing we have and we can manifest it in different ways and if we spend time on planning we spend time on like figuring out what to do and then do it well energy is not dissipated it's conserved and used to the highest purpose then we can have this effect of high rate of innovation per time unit
right otherwise you can have two steps forward two steps back so focus alignments go direct those are the things I learned here that's fascinating right I love to double click into the good direct point there because how do you balance between that and the
topology or structure because going direct for you know people that have maybe problems or issues or questions about an engineering team doing something that maybe bypass the typical channels for efficiency I love that it's so much so much efficiency that you gain and so much common ground that you you gain by just talking to the person but how does this balance out with structure and organization
first of all it starts with being accessible the founders sets I think a very good kind of practice that snowflake of being visible and accessible anybody can walk up to them discuss any topic you know I'm trying to the same thing and my directs and so on right so so it's about being accessible then is about being open to ideas right any
question and concern goes and is a management to answer it and sometimes the answer could be you know sorry we cannot change that but here's why another thing is to build trust if people go direct to somebody and then you know
they raise some topic and it's used against them it's obviously a bad go direct right so people need to be open to critique they need to be open to new ideas and and again we practice it in the management team and I like to think that it's propagating down obviously there could be pockets was not happening but it's really important to kind of walk the walk I am
very interested in removing hierarchy as in hierarchy of course exists right managers know have more input to their employees performance assessments those kinds of things right but most of the time hierarchy is not present that the way snowflake operates so in all kinds of design or implementation or engineering reviews if you kind of had some observer from the outside you
who is the most senior person level wise versus who is the most junior person we invite everybody's opinions I like to think that this combination adds up to the uniqueness of snowflake I would love to talk a little bit more about eliminating hierarchy because that was a practice you know when I was doing a little bit of research
in the prep like that was a practice that stood out to me as really distinct the particular practice that you had mentioned about the accessibility for senior leaders and executive leaders to anybody in the company can you
share a little bit about how you eliminate that hierarchy and maybe the value of having that accessibility like is there a story that illustrates like when you make senior and executive leaders accessible to people in the company the value that that can bring to velocity increasing innovation
and things like that yes I can tell you story about it's for my past so not from snowflakes from Google I was in a very junior employee I just started and I was at the same time taking a management class and the mid term was to interview your CEO
was a class for working professionals so usually there was a CEO for everybody and my colleagues were pretty sure that their CEO will not be a respond to their email and VPs will maybe dain to talk with them but the email Eric Schmidt in my email I said hey Eric I know that you don't know me but I'm very curious about to meet you and by the way I've got this mid term I would love to learn a few things that got a great grade
I expected silence or some email from his admin which would go like Greg got a grip but instead Eric found time for me in a week full hour at lunch time we talked about all kinds of things I got knocking a little bit more and to me the lesson was that if a CEO can find time for me for something so honestly freevales within a week that I have no excuse you know if someone wants to talk with me about anything I have to be there within one day
and it also taking further you know I think I think a good manager has to kind of have it in him or her that you know you don't want to block your team if your team asks you a question one's a decision one's some kind of approval you cannot be the bottleneck right so we are really trying to be there you know be very accessible to people not to be a bottleneck and if you practice you get really good at it such an incredible story
you mentioned this phrase higher innovation per time unit and I wanted to dive in deeper into that I think the big thing is like are there are there certain practices that you see as like input to create higher innovation per time unit like are there specific practices that you see as like the dials to help out with innovation tell us a little bit more about your perspective there
usually when I when I talk about innovation or execution I like to add per time unit right because you know you may finish the project it's going to take you 20 years right so yes you got to finish line but way later so maybe I'll first talk about execution and then I will talk about innovation because they're kind of you know they're not disconnected
first it's not like it's really important to make sure that once we focus on some implementation we know exactly what are going and chances are very low that we'll have to change tack right so I talk about energy energy dissipation one of the biggest energy drains is when you start doing something and you do it for six months let's say and then it's like oopsie now if we actually do you mean that we have to change tack do something else you wasted those six months so we try to front load all the design discussions with and of course all the progress discussions with our great
PM team and then we do what we what we plan to do a part of it is also managing responsibly technical debt and it's kind of easy to move fast when you keep a crewing debt and you never page off and then you're going to bankruptcy of some kind and then you have to reamerish from it right so managing long and short and is very important now innovation let's assume that we actually have execution down
path we know how to run quickly now how do we come with ideas which are worthy of execution first of all it starts with the boldness of the company so the founders instilled the sort of sky's the limit mindset think big actually it's one of our corporate values think big very encouraged and do not self censor right if the idea looks great for a moment don't think about resources needed or time you let's explore the idea because maybe there's
something there and maybe we can make it work right we have a few means of soliciting ideas from the org we have a process which I call from ideas to launching we have a hackathon once a year where a lot of cool ideas come up and we listen to our teams right so at the planning the planning time we ask every team to you know within some larger framework to provide
their own plans and it's also a venue for teams once a quarter to offer ideas what they would like to do correct I want to dig down a little bit more than that because the innovation per per unit time concept is really really fascinating something I've always struggled with is how to know that a team is performing at the peak of their output so how do you know that the
team is maximizing the innovation per unit time is there a metric or measure I've never gotten anything very convincing about this but everyone has a different perspective you know it's a great question because honestly asking you know is there some kind of version of speed of light right which you cannot break but when you're added you know that there's you cannot get any better and honestly I don't know the answer but I'll have a few insights here one is that if you have done
everything possible to eliminate distractions if you have given the team all the right tools then at least you know you you have a shot of being optimal at many companies you've got this dilemma do we pursue quality or velocity and with velocity we go very quickly but we really break things and we
pursue quality but you never launch right and it's almost like no X or or either or right but it's not like what we concluded is that actually it should be quality and velocity because you know velocity is an illusion to the next rollback right if you go too fast so I talked about responsible technical management but essentially you know it's very important for planning to say okay here are new cool ideas but at the same time imagine that we only focus on
innovation but never focus on quality you could imagine headlines right no it wouldn't do anybody in a good right so we have to combine thinking about quality and velocity as as one concept almost and of course at times you kind of tilt it slightly to like
go faster or more quality but ideally on both metrics you get better over time and at this point it's not like it's not a dilemma it's an accepted fact that both dimensions need to be pursued in concert two things that you shared that really resonated team
execution and performance is like the speed of light and then velocity is an illusion for the next rollback I thought really really great quotes to capture some of the sentiment there I'm sure Stonthlick releases a lot of a lot of products at a very high
velocity and I'm sure there's tension between pushing business progress pushing for market capture revenue growth and also balancing this quality metric that frankly I think engineering maybe has the biggest view inside per view over how do you balance that how do you set the
expectations for the business side of the house the revenue center house products at the house to create this balance of a non dilemma between quality and velocity I should also add that you know I I sense probably every six months reminder to the product org that you know quality is much more important than deadlines right so ideal of course we don't everything on time in fact tomorrow would be great but in practice you know we we do pay attention to quality the point where you know
if we discover some issues we'd rather not launch than launch and regret it kind of has many interesting aspects to it but one of them is we spend a lot of time on alignment across the org and our sales colleagues know very well that quality sells in fact we are very proud of our NPS 72 recently there was a blog post 72 in our business is tremendous the high score and it sells right so same with self-managed service with the no ops no nops approach push the limits we are very proud of
stuff they just works right don't need to configure you don't need to do a whole bunch of things that other systems require now obviously there's always a temptation like we can just give this one no for this one customer do this other
thing for this other customer and kind of maybe move faster but the aggregate result of such coronary cutting would be going from a great product to something mediocre so if full understanding across the company same with automation now you may say that okay maybe this quarter we could go faster and
deliver some more features or we could automate some processes so in future quarters we can move faster right those debates actually are not very difficult because our program management team our sales team
has a very understanding what helps is great track records the same approach has been serving us very well a few quarters ago we decided with with our business colleagues that we will not do some things on the business side in a given quarter to make room for automation so that in future quarters
we can go faster and it was also a fairly easy decision and discussion and so anyway alignment does help track record does help also what helps is that at Snowflake we by and large have pretty good understanding
of the product so all the execution telemetry all the statistics sit in something called snowhouse so the snowflake account used internally but pretty much all of engineers have access to it I use it even in some statistics and the knowledge of the products from using it every day also helps
to make it better because otherwise we suffer great I wanted to ask you a follow-up question about energy dissipation because you'd reference it a couple times and you pointed out a few really precise sources or trouble areas of energy dissipation and so I was wondering if you could share a
little bit more about your perspective there and maybe how you coach teams to identify or eliminate that because as you know you become more senior you become more abstracted from the team work yes how do you do that it's really important to me and my colleagues at Snowflake truly empower
everybody you know to bring ideas to a table to be a partner in many things we do and a part of it is to tell us if you know maybe one process went too far or maybe if some tool is not so great or or you know essentially friction we actually have every roughly between six to nine months something
which we call friction survey it goes to everybody in the product org and there's essentially one question what slows you down and can be anything I even once got a response like you know on the it was before the pandemic on the third floor after 3 p.m. there's no coffee that was pretty
easy to fix right but there of course deeper problems like you know let's say security reviews would take a long time or we actually were kind of struggling two different source code systems and and all that right so it can be all over the place and with our tpm org we are trying after every
survey to go after those those items certainly we have a lot of work to do now the focus is our dev tooling to make it faster again you know how to increase the unit productivity of engineers so one source is essentially learning from everybody you know what hers and trying to fix it and
there is a kind of large observation I talked about it a large source of energy dissipation is changing plans so we really spend time with with a pm team and architects and everybody has some input on defining what to do next and I've been here for you know three years now actually last Friday
I turned three it's not like and I haven't yet seen like a major change of plans and it's not because we are so stubborn and kind of work against reason but it's really because planning is done very well and then you can just execute smoothly the coffee one I think is really fun because
that becomes like a really tangible easy thing to do and like a no-brainer that I love hearing that that particular story I mean it was I don't drink coffee so I wouldn't know but I accepted it as a fact of nature that coffee is important yeah I think it just it just goes to show like how the
level of detail that you have to go into to understand what causes an increase or decrease in productivity what kind of grinds people's gears because everything builds on one another to create a perfect working environment that maximizes all the metrics and productivity but it also shows
a bit I think of kind of trust you know I think that people bring in up even such seemingly small problems but of course for them important it shows that it's at a level of trust like I trust that at the minimum you know nobody will use it against me that's information will make we'll make fun of
it at the maximum some of the will fix it and I always tell the team that there are no kind of problems too small because if something bothers you certainly it's most likely worth fixing no I'm actually curious about doing this at scale because Snowflake has a big engineering company
that company has grown a lot especially in the last few years and especially since you took over engineering and support and I'm curious how you do is at scale I'm guessing there's a lot of empowerment that you rely on the managers you rely on the structure of support that you've built
in the org some curious about the management structure what do you look for the manager with the role of a manager in engineering it's Snowflake so I would say that it's Snowflake the role of managers is not that different than it's other you know Silicon Valley companies right so so people are very
engaged in setting manager strategy for the teams recruiting for the teams what being of the team's execution right so kind of standard management tasks what I think is unique at Snowflake is that managers at least the ones I know I don't know everybody really well but the ones I know well
they're very open they really embody the values of you know go direct help others know try to figure out how to improve essentially challenge status quo it's very important we also spend quite a lot of time on helping managers grow right so one way to think about is that you know you may
commission a class from some outside vendor and then this vendor will teach you something and this is the useful approach we took some more different approach more home grown so we have a group of managers which form the working group now it's five of them which runs monthly meetings for
managers where we bring all kinds of interesting topics to discuss and they are kind of crowdsource from the management team as long as as well as they come from the working group recently we've even started you know retrospectives on execution as usually you the retrospectives or post mortems
when there is a production failure right so something went wrong and then you debug the problem discuss the root cause the timeline and formulate action items we took that concept into management right so there was some plan actually we got to finish line but maybe we could have gotten
there sooner so let's x-ray this execution let's have a post mortem just like in production they are blameless so obviously some of us involved but we don't care about who did it the idea is okay here's the shape of things here is how it went well how it didn't and what we can learn from it
and as far as I can tell you know what I'm hearing from managers who listen and participate in those meetings people really like our focus on management development because it's coming from us it can be instantly customized it talks about specific events that we saw as opposed to some more abstract
events from elsewhere I think it works quite well it's really interesting since we're talking about managers one of the elements of how snowflake is leadership structure is set up that was really interesting to me was sort of the composition of small teams and the focus on having small teams
in small unit sizes and so I was wondering if you could talk to us a little bit more about that team structure my understanding is that it's four person teams and like the idea is like make it small and then use those as like the units to sort of build up is that accurate can you tell us a little
bit more about the small team structure and how you build those up at different levels it's sort of a goal I wouldn't say that we always organize like this could be three or five and maybe maybe some teams do differently but in my own experience when I was a developer and also many of my senior
colleagues share this opinion that they were the most productive and the most happy as developers and a fairly small team so it's not like you work in a four person company and of course there's a larger structure but even very large teams can be decomposed into smaller ones and then smaller
and then at the bottom a small group of people right usually they do their own code reviews designs they own something together and then they interact with other small teams and then they build something larger it's not always possible you know in some roles you've got to be extremely
horizontal and talk to everybody and understand a lot of things but I think it's you know most developers I know 80% could easily fit into this kind of small team model and it's not like I see a lot of it happening you know it makes people happier because they know their small number of
colleagues very well they learn each other's habits they learn their strengths they also learn faster because there are fewer people to learn so to speak you can you can just learn a few and then move fast forward and I think it works to a large extent but again it's not like I am like the last
person to tell teams how to operate so for instance we don't prescribe whether teams should you should use agi-methology or something else as long as you know code is clean reviewed tested and all that teams are totally at liberty to decide how they create their artifacts in fact we have
some people who do per programming it's totally fine too so how they operate I will not tell them I'll just suggest that you know here's a really good way to do it based on my or others experience I love that appartializing teams to choose how they operate as long as it's most effective for them
and creating the same quality of output as everybody and I love the idea of owning having people own something together is there like a particular way you think about shaping how those units all interact with each other I think a successful way to run a large org has two elements one is
telemetry so how do you know what's going on well I like to ask teams for you know to send every week or every two weeks a very short update email update to a certain mailing list and people can subscribe to it also once a week we send all those updates and all the you know review decks the
whole org but to me you know I read those very carefully and I learned a lot about okay on the team things to green no issue on that team something started to go south need to pay attention the trick there is to explain to people who send those updates and again we have a lot of new employees
they come from different places different habits some as well different fears it's important to explain that it's totally okay to share bad news quickly but it's absolutely the wrong behavior to try to you know something should be read but it's yellow we're so z yellow and then you know
yellow is the new green and all that right so so be really honest in your status reports this way you can have a kind of quick bird's-eye view of the whole org this one thing second you know you know I don't you know I am engineering and then one of my direct runs a large program and
then his directs run another large program it's more like this kind of recursive structure and you really pay attention to to how the org is structured you can get a lot of leverage from that team is composed of people who own that specific topic and a different team own something
different and they interact in a loosely coupled way right they don't need to be like totally totally closely coupled and especially comes to the fore when you create new sites especially in far away geographies right because if you if you don't pay attention to what I would call
a spaghetti dilemma where everybody is talk to everybody else you wind up with a situation for example when people in Europe have to spend many pretty much every evening every single day to be on zoom with their colleagues on the west coast of the US we have now five offices you know
Bellevue in Washington and San Mateo California and Toronto Warsaw and Berlin so how do you protect people from essentially having no life because they have to be always on zoom and talk to your colleagues right if you kind of have some attitude that instead of state you know it should be at
most one evening a week it kind of forces some decisions right it forces grouping of work okay that team owns that part the other team owns the other part and they communicate through you know documents and files they don't need to emitting all the time great we're we've got time maybe for
for one or two more questions then we have a couple rapid fire ones to close on I wanted to bring up one dilemma that's come up in a couple engineering leader peer group conversations that that I've had and and that's this dilemma around how team size can often be time oftentimes be used as a
proxy for power authority or a way to demonstrate I'm progressing in my career in reference to the the team structure elements that you were laying out earlier in like really trying to help preserve this feeling of intimacy for somebody who is like who is stuck on this idea that my team size is
a proxy for my own power and authority how do you deal with that with with somebody who maybe is trying to use that as a way to demonstrate power and preserve then the intimacy of a team yes so first of all we are trying to decouple the team size from impact as in of course the bigger the
team size ideally did the bigger impact but if there are even small teams with high impacts the small team size is not the deterrent to promotions for example or visibility conversely you know if somebody's team is too big it can be a detriment you know so you I'm asked a lot of people but you're
not doing good things with them therefore you know it's a thing as opposed to some kind of advantage right but we we essentially tried to kind of in real time look at the manager so I think it's it's not like it's impossible to like quietly emass an army of people and nobody will notice right
we actually even know we grow quickly and there's plenty of headcount and plenty of hiring I'd like to operate kind of in in the you know every headcount is precious and how do we give it right so usually I don't give somebody like 20 headcount I give them three and then three and then three and then let's let's see how it goes I also if you kind of go with the you know the larger the team the more important you are it can go or it can cost lowering the hiring bar all right because then
if you want to quickly hire an army of people you just stop paying attention to quality as I said you know we we really try hard to decouple the notion of of impact from from team size it feels like such a such an important notion to to communicate to your team I'm going to
I really appreciate that you have removing size from promotion consideration Greg are you ready to wrap up with some rapid fire questions yes I am perfect okay what are you reading or listening to right now so on the technical front in recent there was a great database conference sigmots
I've got you know a bunch of papers to read uh not technically I'm reading a biography of Robert Openheimer I think there's a lot of interesting lessons especially in his case he was a great engine no engineer great scientist engineer at some level two but also he was a great organizer
so it's pretty fascinating his biography a great organizer that's a great way to capture that what tool or methodology has had a big impact on you here hands down it's it's test driven development you know what to do styles from test cases build the rest a powerful uh is a powerful punchy way
to capture the whole concept I love that and then the the last one what is a trend that you're seeing or following that you find really interesting or something that maybe hasn't hit the mainstream yet now so it's always fascinating to look at ml what kind of news cases come up I'm fascinated by
this kind of AI-powered code development assistance so you know github copilot a few things of of the way how far can you push ml to write code starting from intent I think this is the next great thing and you know let's see how it unfolds how far can you push code starting from intent
that's great Greg thank you so much for for joining us clearance thank you so much for for co-hosting and diving us into some really great conversation points um Greg we really appreciate your time this was a ton of fun thank you Greg was great and a time thank you if you
enjoyed the episode make sure that you click subscribe if you're listening on apple podcasts or follow if you're listening on spotify and if you love the show we also have a ton of other ways to stay involved with the engineering leadership community to stay up to date and learn more about all of our upcoming events our peer groups and other programs that are going on head to sfelc.com that's sfelc.com see you next time on the engineering leadership podcast