One of the questions in the back of my mind, throughout my career was why some companies are so much more successful at producing, high value, high impact, products than others, and the principles are pretty well, understood. You need to be customer Centric. You need to be evidence driven. While evidence guided, you need to be able to adapt your plan or just stick to roadmap, but some companies are better at
implementing them than others. Google was an example and I found it. The difference comes to four areas of change or four areas that we need to tackle.
Hey everyone. My name is Henry Surya, we Robin. And you're listening to the technology, you know, podcast the show where I'll be bringing you the greatest technical leaders practitioners and thought leaders in the industry to discuss about their Journey ideas and practices that we all can learn and apply to build a highly performing technical team and to make an impact in your personal work. So let's dive into our Journal.
Hello again, my friends and my listeners, welcome to the package, you know, podcast the podcast where you can learn about technical leadership and Excellence from my conversations with great thought, leaders in the tech industry. If this is your first time listening to Tecla Journal, subscribe and follow the show on your podcast app and social media on LinkedIn, Twitter and Instagram.
And for those of you longtime listeners who want to appreciate and support my work, subscribe as a patron at technology. Not death / Patron or by Me a coffee at technology.
Not deaf /. Dip my guest for today's episode is itamar, Gillard, itamar is a coach and author with deep experience in product management, strategy and growth and was previously, the head of Gmail's growth team in this episode, we discussed all things about product management, and how to build high value products itamar.
First shared his journey at Google, growing Gmail to 1, billion, monthly active users, and some of his lessons Earn on managing large scale, product changes getting users feedback and dogfooding itamar than explain in depth. His just framework as an alternative to the product roadmap, a collection of methods, and best practices for producing high value. And impactful products he shared some challenges working with product roadmap and how teams can create better alignment instead.
He also shared how we can do product privatization better by using the icy technique and his confidence m. Towards the end itamar, share the different ways of how companies can conduct product experimentation, and how to use the Jesus board to improve the way we execute product development. I really enjoyed my conversation with itamar hearing some of his lessons. Learned growing Gmail and techniques to improve product
management and development. If you also find this episode useful, I would appreciate it. If you can help share it with your colleagues and your community, so that more people can learn from this episode. Also leave a Our rating and review on a podcast and Spotify. Let's go to the conversation with itamar after a few words from our sponsors. Are you looking for a new cool swag? Taglit Journal. Now offers you some swags that
you can purchase online? These wax are printed on demand based on your preference and will be delivered safely to you all over the world where shipping is available. Check out all the cool tracks available by visiting technology. No, dot f / shop, and don't forget to break yourself once you See if any of those wrecks. Hello everyone. Welcome back to another new episode of the attack. Did you know podcast? Today we have a topic about product management.
I'm really happy to see itamar. Gillett today, itamar is a very experienced product manager. He has worked in multiple big companies like Microsoft Google. And in fact, it has helped grow Gmail product into 1 billion monthly active users. I think this is a lot of expertise that we are going to learn here from itamar. So thank you for this opportunity and looking forward to our conversation. Thank you for inviting me. And yeah, I'm looking forward to
it as well. So it a, I always like in the beginning to us about your career Journey, maybe if you can mention any highlights or turning points that you think will be good for us to learn from, you know, for sure. I started out as an engineer and I worked in engineering for a while. As a software developer, I really liked it. I started rising up the chain of command and I found myself an engineering manager of sort and I felt that was not the right career path for me.
So I kind of switched to the dark side a bit and became a product manager that was in the year 2001 so you can see how old I am actually and that was probably a better career choice for me. I spent about 15 years in product management mostly in Israel where I'm from, but I worked also for some International companies and then I spend a few years in Switzerland as a product manager working for Google during these 15.
The years, I had a chance to work for startups for Scallops for Microsoft and then for Google initially for you too. But then I spent a few years in Gmail as you mentioned and that was an overwhelming experience. We had, as you said, billion active users, give or take and a lot of impact, but at the end of this period, I felt it was time for yet another change.
So, I stopped being an active product manager, and I became a coach for product management teams for leaders, and I started writing a lot about product management, some Ideas or some of aggregation of things that tell you right from other people, might smarter than me. And that's where I am today. I'm coaching I'm presenting. I'm writing and I'm having a chance to speak to you. Thanks for sharing your story.
So, if all of you do not know yet, Adama also, wrote newsletters and he also published several self-published books. Those are really short Snappy, but very concise and dense. So make sure to check out his website. But later on, I'll put it in the show notes as well. So it down and let's start from your Gmail. They'll experience because I think this might be a good opportunity for many of us to
learn from. So, maybe if you can identify when you started joining Gmail, how big was the product back then and what was the evolution, like growing from that stage until 1 billion monthly active users? So, I joined remember 2011, it was pretty massive by then it had. I'm using official number 420 million users already so I cannot really take any of the credit for the big success of Gmail. It was wonderful product that Con transformed email, people
really love the amount of value. It gave. I think it stayed a very honest product and really try to help the users and help the customers. And while it showed ads, we had a clear separation of the ad team being there to monetize the product and the rest of us were there to create value for the customers. And this balance of give and take is very important in every
product. But we weren't there just to sell more ads, I can tell you and that's one of the The things I really liked about Gmail. The customer focus. The fact that we were constantly striving to add more value and to make the product still relevant. And that's one of the key takeaways that I always teach start with the customer, put them at the focus of everything. Nice, we always need to start from the customer solving the
real problem, right? Not like imaginary problems, but maybe specifically, for some product management techniques. Is there anything that you do differently at that slot scale number of users? Because I think any small change that you do sometimes I also experienced that whenever Gmail change the UI ux, some people will like it, some people will really hate it. So how do you do this balance for product management, in order to cater for building number of
users? Yeah, when you work on this curl, there's a number of complications that are not typical for any other kind of p.m. role that I experienced. At least the one thing is that you mention any minor change, you change the color of the send button, someone will hate It it someone will love it. It might affect the productivity of some people. So you need to be more cautious and especially with email because it's kind of a mission-critical product for so many people.
You need to have a gentle. And so we were bit kind of risk, aversive would say, in a sense that we tried to while creating value not to ruffle the product too much. The was just, as I joined a major launch of redesign of Gmail, that was very controversial. Some people really loved it. It modernized Gmail, Before it was kind of ugly but very functional and people liked it. And then we launched this redesign that went across.
All of Google, by the way, the same time there was this new design language launched and a lot of people really hated it because it makes the information less dense. They so fewer lines or fewer threads inside of the email. So that taught me a lot about how sensitive we should be about this changes years later. I had an opportunity to launch a major Innovation if you like or major.
When we introduced the tabs. So we realize that a lot of users are not really organizing their inbox in any sort of way, everything that comes in stays in the Inbox and if it goes to the other page, it's as good as all cards for them. It's really hard for them to sift, through the piles of email, the advanced uses, the power users, head techniques, the knew how to use labels and filters and all the other tools of Gmail. But the vast majority of uses
didn't. So we had to develop a system that enables them to A bit more organized without having to do a lot of work. And that led to after research to this development of the social tab, the promotions Tab. And there are few other optional tabs you can activate. This was a major change in the user experience and in order to reduce the risk because it was very risky, we took a long time to develop this thing. It took about 12 to 14 months, we've done numerous user studies.
Some of them including hundreds of customers or users, we've done a lot of Of data analysis. We've done a lot of usability test, tens of thousands of googlers volunteers to actually use this on their personal email. I really was appreciative of that and when you do such things, you are reducing the risk. So when you launch eventually, you're not surprised. You're hardly ever surprised because you actually already learned all the bad sides of that might happen.
All the negative aspects and you fix them along the way. And that's really informed my thinking as well as about how can the right way to develop product. You don't need to be as risk-averse as Jim and obviously a new product, you should maybe take more risk but still these techniques. I think of are useful. So there are two things that I have interest as well.
So when you roll out this change, for example, introducing this new tab, it's like the new workflow for many people and sometimes it could be destructive. How do you actually engage critical feedback from the users knowing that your product is being used in multiple countries, multiple geographies multiple cultures language might be different as well. So how do you do this research? And how do you actually get the feedback that actually allow you to take the risk and make the change?
So, first of feedback for me, the word feedback is usually after you launch and there's some feedback tool to let you know. I assume that's not what you meant, but don't get to this point and then just learn for the first time whether or not they liked it, that's a very common mistake and a very costly one because there's statistics to show that most leaders actually are not good. If you look at A/B experiments, Etc, that Netflix Microsoft go
will run. The best case scenario is one in three ideas, actually create any sort of measurable Improvement. So don't wait to the point that they need to give you feedback after you launched it to tell you that was actually a terrible idea. Usually the feedback you get is that they will just not use it because it's just doesn't do anything good in their lives. Much better is to start as early as possible with validating this idea and sometimes you can
validate it just on paper. You just just do a little back of the envelope calculation. Say how many people actually have this problem? Let's put a guess, how many of those will actually see that change that we created? Because not all of them will, how many will convert and start using it? And how many of those will actually see the benefit to expect and retain keep using it?
And sometimes a lot of ideas just on paper will realize are not as strong as you think either because they don't solve a big enough problem or because the likelihood that they will solve the problem to these people is small. So just this This, which is very cheap to do. Will help you a lot. Then you have a lot of techniques to fake the product, before you build it, you do a Wizard of Oz test. We've done this for the tabbed
inbox, the very first test. We've done, is we invited people to his ability test, they saw Gmail already with the tabs and inside the tabs by Magic, they saw their messages which were actually came from their inbox. How did this magic happen first off? They signed an agreement that we Are allowed to process their inbox a little bit. Not actually go into the message is just the subject and Center.
While the interviewer was interviewing them, we kind of scraped the top 15 messages from their inbox, with Chrome extension. And then at the back, we move the manually. We guessed. This looks like an update. Let's put it here. This looks like a promo of this looks like social. So after a few minutes we give the researcher, a green light and said Okay show them the new inbox.
But of course it was a facade, this wasn't really G. But this was just a facade that showed him how it might be and we got incredible feedback just from this simple experiment and we saw a determine this value in each for most of these customers. The first ones were 1210 out of them. Absolutely loved it. And wanted this right away and then there were two. It said, no, because I already have a system to organize this, and this will conflict with my system.
Those were the power users. And this ratio of about Twelve to fifteen percent of people that don't want the inbox in hate it, when we showed it to them repeated throughout our research. So it was really interesting to see, incidentally, a lot of my colleagues were in this group. So it was a hard sell to convince them that people actually need this because they were like, we have solutions for this, why do we need another one?
And also the technical press before we launched, we sent it for preview to a lot of article, writers and newspapers. Technical reviewers ahead. Time and day of the week to review and the reviews are very tapi. They were like, I don't understand why this is needed, no one was excited but we knew by then we tested so much with the regular users that I actually loved it and it does exactly what they need. So we didn't care.
So that just comes to show you that even the technical press and even the expert, can't really predict always what's work. What does work? Well, this is a very valuable sharing especially for people who like to do experiment of their products introduced new. Diaz, right, I think always validating with your customers, your users, not following some reviews from some experts of prayers or even maybe the internal employees. Sometimes, you can't trust them as well.
So always validate with your users relating to the internal employees. I think there's one thing that I also love from what Google is doing, which is called the dogfooding, you mentioned about it as well, right? Before you launch, you do very thorough, thousands of googlers using it. Maybe tell us a bit more about dogfooding why it is important and how we should do it.
Maybe more properly. Solutely, by the way, it's not just Google. It's a very common practice in Silicon Valley or in Tech in general, historically, Apple relies on it extensively, they don't call it dogfooding but Apple doesn't like to do external user research but they use their employees extensively to test their products for
years. Sometimes, Microsoft, the same when I joined Microsoft in 2003. The first thing I noticed was that Outlook mail client, was completely buggy, ask people what's going on and said oh that's because we're all. Dogfooding the next version of Outlook.
So that was the norm. So it's a really powerful way to get your colleagues to participate and to give you feedback, early feedback, there are much more tolerant of bugs than the customers would be the much more supportive and also many of them are technical enough to tell you exactly what's going on what's wrong. But you mentioned a very important point. They don't always represent your ideal customer the one that you want. I mean, most of the people you find in your company.
Very technical or tech-savvy. I have a certain level of Education, income, Etc, certain age groups, so be aware that this is not necessarily representative what we did with the inbox. For example, I invited dog, fooders first off, we did fish food, which was the team itself. Started testing a very rough version of the product on our own in boxes just to convince us of the. There is value there, that's yet another test.
I recommend trying, if you are in the target audience, there's also something called a back dish. Even if you are developing some, I don't know B2B system for, I don't know. Doctors for information retrieval. You're not in the target
audience. You're not going to be able to dog food this in your real life, but have the T in come for a day or half a day, and just try to complete some tasks with the software developed, and you'll be surprised how much more enlightened they become by the end of this experiment.
So, bug crashes. Another interesting, way to get into the shoes of the customer with the Little tab in books, we launched it to anyone who wanted but I also send them a survey to try to understand what kind of inbox management during
employing already. So we had the big in boxers which were the people were not cleaning their inbox and those were the ones we listen to most closely we had the small in boxers in the zero in boxes which are like the I don't know, the ninja Cult of inbox management, you know inbox zero almost no one does it it requires extreme. Appling to do it. But in Google, a lot of people thought that that's how people actually should use their inbox completely irrelevant to the regular user.
So when a is 0 in boxer, came to me and said, you know, here's a problem. How would qualify this with this kind of? I know that person has a certain mindset that is not necessarily representative of the larger population. So very powerful tool but learn to qualify who's saying what your managers are not. To representative of the audience either external experts, there are experts is great. To listen, to them is great to talk to them. They'd kill not necessarily predict.
How people will see the product, people, react, to products in a very complex way. Partly emotional part of the intellectual part irrational, probably not the social and limit. It's a very hard thing to predict, really. So don't trust logic alone or the views of experts. Thanks for sharing that I think it's really valuable or these lessons learned from you. Or Gmail experience. So let's move on to what you're
doing. Now, you are now coaching people more on the product management, teaching them how to probably do product management much better. And you have one technique, which I think called just G is, T, which stands for goals ideas, steps and tasks. Maybe can you share a little bit? What is this G is about, and what kind of problem are you trying to solve by having this
framework? Sure. So, one of the questions in the back of my mind, throughout my career, was why some companies There's also much more successful at producing high value, high impact products than others. And part of the reason, is Corporate culture and some companies are a bit more behind than some of the more advanced, but I don't think that covers all of it. Sometimes companies with bad cultures, managed to do a good product and vice versa.
So I was looking for the kind of the best practices or the principles that really helped, and the principles are pretty well, understood. You need to be customer Centric. You need to Be evidence driven or evidence guided, which I think is very, very important. You need to be able to adapt your plan or just stick to roadmap. This is true on the road map. Level is true on the Sprint level. You need to be truly agile, giving you information and you need to empower people because
it doesn't work. If you centralize all the decisions in the hands of a few very small people, you need to distribute the decision to create an organization that is intelligent enough to coalesce around the problems. Try Thinks it's a. So those are the principles. Those are well understood for decades but some companies are better at implementing them than others.
Google was an example. I saw in Google how some of these were very powerful and you could even see some of this coming out in the examples I gave you. And I could compare this to my experience in Microsoft that was a completely different place. I would say on this aspects, maybe it's changed since then and I found it. The difference comes to four areas of change. Or four areas that we need to tackle. The first is how we set up goals and what's in this goes so
everyone uses. Okay ours this data is great but okay, I was can contain terrible goals. Really bad goals in the contain, the right type of gold 16 contain dozens of objectives and cures on it contained just three but very focused one. So what is the right practice? Stir and we know some of the roles outcomes over output less is more tied to actual behavioral metrics of Customers not just Revenue but the actual guidelines of how to do this escape, a lot of companies I work with.
So that's one area that I think a lot of companies can improve and that's the goals layers. Each one of these goals are tied, but each one is semi-independent. You can Implement just the gold changes. For example, to start with, you can set the nose down metric, you can set your metrics for you, can start using our kiosk correctly and those are, some of the things I teach the second
change is, how do you choose? Which ideas to invest in that the ideas layer and idea, just for clarification is a hypothetical way to achieve the goal, could be launching. A new feature could be partnering, it could be starting to use a new API. It could be purchasing a company that does exactly what you want. These are all ideas, hypothetical ways. As I mentioned earlier, the statistics suggested that the best case one is Raiders will
work. And in reality in most companies and most cases it's 1 in 10 unfortunately, which means Maybe 90% of what's currently in your product backlog in your roadmap stuff. You don't really want to do. It's a waste, it's going to do nothing and it's not hard to see the result of this. Just look at the existing product. How blow to their with features? That get almost no usage. This is true for every product. I worked on these features are a liability. You have to keep maintaining them.
You need to keep sustaining them, they create bugs for you in the future, the complicated, your cue a matrix. So it's best not to launch these things. It's best to invest in. The thing that really move the needle. So participation is a key challenge. It always comes up as one of the top challenges. When I speak to product teams, we don't know how to choose the best ideas. How do you choose based on opinions based on consensus based on the opinion of the highest person?
You know the system and it is a very unreliable the risks so they send us again and again into the wrong direction. So what I teach usually is how to use ice which again is very common is very well known. But to go very deep on the eye on the ear, especially on the see the confidence, how to choose ideas and how to attribute confidence them based
on evidence. And for that, I created a tool called The Confidence meter, which is now gaining popularity quite a few companies are using it if you want, we can talk about it later. So that's the idea layer, how to choose their gears that are most likely to succeed, but there's a big quotation around most likely because this is, again, just as Set of assumptions and a set of guesswork and some work with
evidence. It doesn't guarantee that if you do good participation, you necessarily land on the road and take dere's no-one can actually predict, definitely not ice. So you need to test these ideas, and that's the step layer. So, the steps are is about how to experiment, how to get ideas from concept. To a launch feature by combining learning and building at the same time and I give some examples. So, a Wizard of Oz test is an
early stage test. Before that you can do back of the envelope calculation, before that you can do ice analysis before. You can do assumption mapping. There's a ton of things and later on, if you see the deer still worthwhile investing. If you also, you pivoted, you improve it along the way, then it becomes worthwhile starting doing the more expensive experiments, you know, the dog foods of the fish Foods, the early adopter programs, Alphas, betas, Etc. ABX permit really depends on the
type of idea. If it's a big idea or smaller, Idea, if it's risky, or not risky in your level of risk, I mentioned in Gmail, we were very risk-averse, but some companies are should be less risk averse than us. Based on that. You need to decide how many of these tests you need to build along the way. So sometimes you can do just one test and launch the idea, sometimes you can rely on expert opinion, because it's a very tiny change, and it doesn't create much risk.
It's an art form. You need to learn to combine. So that's what I teach in the steps layer. Before me, a step is like an experiment or an iteration. Teach us something. Then the big question is how to run this whole thing? How to do project management with an agile team, especially we all have this scrum teams or kanban teams or whatever, how to bring this, to their lives and how to prevent the managers and the stakeholders form. Randomizing us constantly and coming in with new ideas and
pushing us around. And so that's the task layer. This is how to get the teams to work on the right things with tremendous amount of context. Because what I see is a lot A lot of engineering teams or development teams caught up in protective edger layers protected from the customers protected from the business. They don't need to be bothered with this thing's. Just give us a prioritized product backlog. That just break each item in the
backlog into user stories. I don't know if you ever seen this process and after that, we will build it for you. That's our job. That's a terrible mindset, I'm sorry. This is not how good companies build products. You should not work this way at all. What you should do is A lot of context, the goals, they they are the steps. All this creates a lot of contacts, in the minds of the people do the work and kind of Empower them to a suggest ideas.
Be invent some of the experiment and see become explorers in a sense. The become discovers not just people who deliver, but people also discover and that should be part of their job description. And this should be part of what they're compensated on that. We should move away from this output Focus. That about of teams living. So, that's what I teach in the task layer. It's really jammed packed and I think there are a lot of
insights. Maybe we will cover one by one, but before we start with all these goals task steps and all that you touch on something that is pretty common in many startups or product company. Right? It's about product roadmap and I know that specifically you wrote something about problems with product roadmap. Some companies do like the yearly product roadmap or even quarterly. So tell us why this is a challenge. And what does this do differently with product roadmap?
All right, so the criticism of product world map is widespread, it's not just me. Let's start with the Positive. Why do we need Road? That's what function road maps, provide so, road maps. Provide some sort of sense of security, or some sort of sense that we are, well, planned.
And we know what, to expect this feature will end in this quarter Etc. And then we can plan the rest of the companies, where the marketing team can start preparing their marketing materials and training, the sales team, and the sales team can prepare and maybe inform the customer. Customers and depending on which company you work for the CTO is happy? Because he or she can level the results so that kind of know how many they need to recruit each quarter planning.
It's wonderful. Everyone loves a good plan and this is a mindset that we inherited from the industry that preceded us. You know, the 20th century kind of manufacturing and probably that worked when you're producing calls or packaged consumer goods or all these other things, the challenges we Face a lot more. Uncertainty. Then these industries of the past were classic business management was dominant.
There's a lot of uncertainty in the markets because the markets are very Dynamic with software and the internet two guys or two girls in a garage can disrupt our Market within a few years. Thus far fewer barriers to entry. The customers have a lot of choice. The customers are very fickle, they can change their mind, so we can't afford to just lock yourself into a plan that It doesn't react to new information that will be very very bad for business and they're bad for a product.
The second thing is these plans never come together is expected to just need to look at last year's roadmap, how well did we deliver when it some things were delivered very late? Some things were canceled new ideas that we didn't even disagreed on in the Middle Road map. All of a sudden hop to the top and that's my baby, okay, because that's actually makes us a bit more agile.
Just shows that This heavyweight process, this tremendous effort in actually prioritize The ideas and choosing and putting them on the roadmap. And scaling, the results has. It's not worthwhile doing. We need to actually try to do something. That is more agile, and I can go on about all the different negative side effects, cultural and other that kind of being attached to roadmaps create a lot of organizations understand this and they moved from yearly
roadmaps to quarterly roadmaps. And then after that, there's a bit of a non-committal world, my power, I call it the next and later sort of concept but even committing Into a quarter. Each time means the during this quarter, you don't really have the freedom to launch. Whatever you think you need to launch. It's imperative to have this Freedom. Equality is a lot of time in most of our businesses. So I think a programmatic approach to this is to realize that this certain types of ideas
where we gained high confidence. We're already tested and we went through our dog food and the early, whatever. We're pretty sure this is a good idea. And by that point, we're already built the product to Early in order to test it that we know already. Also, we have pretty good confidence about how much effort it will take to finish with these ideas. I'm pretty comfortable to put it on a timeline and saving we're going to launch this q1 even specific date and q1 because the
risk is pretty loaded. By committing to, this were actually doing something bad. It's a good idea. We want to launch it. We know how. So for those feel free to commit the other ones, the ones that are in process of being evaluated, I would I'd say don't commit to them just color them and say you know these are the
errors are medium confidence. We only tested them to this level, they might happen, they might not happen, we don't commit, we don't know for sure and then there's ideas that are low confidence etcetera, so that's add ideas level at the goals level. I would say that most company strategy, stretch multiple gears. Then there's a yearly. Okay? Are for the company that says this is where we want to be by the end of the year. But you could actually own the
timelines. Say we want this particular key result of this particular objective 2 complete by and of H1, right? So by this data so you can kind of plot your goals, your company level goes until Level goes on a timeline, that's sometimes called an outcome-based roadmap. So what do we want to achieve by one? That's also how to predict it's also a bit brittle but it's less likely to break than committing to a set of Of concrete ideas. I assure you. So it's worthwhile doing this
exercise. Then there's the question. Okay, but what do we tell the customers. How do we prepare? What about marketing and sales? What about resource leveling? These are really important questions. I'm not discounting them. I do think that if you want to be truly agile, these people need to learn agility is with the CTO needs to learn to plan in a more agile way. And to learn that, sometimes he or she will need to recycle people for project that failed
to another project. And that's just the way it is the marketing and sales people. I talk to Not necessarily in love with Rodman's either. They know that there are late. They know that most world maps actually don't come together as expected. They know that a lot of these features that are promised are not the best ones, they really want to create value for the customers to because that makes their life so much easier, right?
High value is what the customers want that aligns all goals with their goals. So the business seems really understand this. I saw a lot of business tips being willing to be much more agile and adaptive, and Lynn than the executives, Give them credit. They don't want to tie the engineering team necessarily to set of deliverables if they know they can make their business goals. And that's really where we need
to all aligned. Not only set of launches, but of set of business goals that the team's commit to be a question of trust there. You need to build the trust in the engineering of the development team that they can actually deliver this. And this is a big hurdle to Traverse. But if you're there, the companies that are in this position, the much better lined, the spend much less time on Learning.
They have much better outcomes at the end of the day, because everything they do is towards business and user goals, and they don't need to invest as much resources. Actually use the resources in a much better way. So this is I think is the ideal. You need to Aspire to thanks for explaining that. I think it's really important to understand why product roadmaps failed and what are some of the things that you propose here to make it better?
I think apart from trusting you also touched on in the beginning that the team also needs to know a lot of context, not just like okay, here are the features that we want to build, but also tie it back to like, why we want to do it. Which customer, what kind of problems? What business goals that we want to do? And, I think another challenge that I normally see is about alignment, we have multiple teams in the company. We have one big goal, but you have to align between multiple
teams. How does maybe just do differently to align these teams or different people. Different managers so that they can go towards the same goal. Do you have any tips here? Well, alignment is a major Challenge and there's more than one Allowing problem or cause for misalignment, we just I'm trying to help deal with some of the causes I cannot guarantee that's going to help with everything one is that we try to align teams around projects, around ideas. Let's start from the bottom.
Let's say I have an idea in my team. I want to do something, then I'm going to your team because I'm depending on you. There are dependencies and I want you to commit to my earlier. I want you to work on my project from experience as a very experienced. Product manager, I tell you most of the time this feels they have their own ideas. They don't want to commit to your project. You'll get a lot of pushback or sometimes they will say, no, I
would disagree. This is the best way, I suggest aligning on goals, so come to that other team. It's a listen. We want to achieve this outcome. We want to reduce the number of, I don't know, hacker break-ins into account by 50% next quarter. Do you think this is a good goal? We depend on. You are you willing to commit to this code? This is Much easier discussion because then they can say yes, or they can say yes, but we have more important goals. So maybe talk to me next
quarter. Both ways are good, because if they say no, you know, you shouldn't commit to this goal because you're not going to achieve it. If they say yes good, then you ask them to copy this goal. This. Okay, are into their okay, ours, which is a form of commitment, doesn't mean you'll get it
necessarily. But it does mean that there are serious about it. This is called a show Tokyo's disallowed in Google. So that creates alignment across teams, for example, what really helps teams the line is. If from the top there are clear goals and you'd be surprised in done. How many organizations that's not the case at all? There's no clear no style metric. Does nose clear? Set of metrics trees, there's a million goals.
Everyone's idea becomes a goal. Essentially though key ours or not about outcomes are about the set of ideas that we've decided to implement and that sends people in various directions or they get a lot of Teams to work on this one of these massive projects as well. It kind of creates alignment but it's really just creates a massive project for us to collaborate on. So I suggest process of starting with very few company, level goals.
If you are medium size company, don't do a middle level goals, don't do departmental goals or disciplinary girls, don't do ux goals versus engineering goals. We don't need any of this do team goals, and each team needs to ask yourself how can we contribute to the company goals? And the answer often is, I need to collaborate with these three other teams in order to achieve
that particular goal. Middle managers are very good at helping the team's connect the dots and helping them actually forcing them to collaborate, but you don't need the middle managers to create goals in order to do this in a larger company, where you start having business units, Etc, then? Yeah, it makes sense to have more middle level goals. So, fewer goals, if you are mid-level goals alignment on
goals instead, Set of ideas. These are just some of the ideas can provide, thanks for sharing that. So, one thing that I pick interest is about fewer goals. Some companies, of course, if not all, they have many ideas and they have multiple stakeholders. Right? So these stakeholder has maybe more Revenue. The other is more about active users or some other about cost-saving you touched on
earlier about prioritization. So, we have so many ideas within company, maybe they are all aligned to the same goal. Maybe they are all different. How do you do this privatization? Maybe can you Tell us a little bit more about this ice technique and coming to the confidence meter that you also touched on earlier as well. All right, cool. And by the way, you raise a good
point. I talked about alignment inside the product organization in between tears, but there's even a bigger question for alignment against business stakeholders Etc, but participation kind of comes into that cuz that's really where we feel some of the pressure, right? Everyone comes with their favorite idea. Everyone is convinced that this idea is to be at the top of the list, sometimes it turns into Political.
Pressure escalations, and often as a product manager, I had the least power, least influence, and the list political skills. I didn't know how to escalate. I didn't know how to do all this stuff that these guys were experts it. So how do you defend against it? First and foremost, you need good goals. I'm sorry to go back to this. If there's no clear agreement of what your company in the world, to your team is trying to achieve. Everything is a good idea. It's really open market.
Just whatever pushes harder will get their idea. So you need to be very clear about what's your goal. If you're the on boarding team, your goal is to improve the percentage, completion rate of onboarding.
For example, that's your local no style of your team and this should be reflected in your goals and when someone comes to and with an idea and says, you know, what will be cool, we really want to do this cross company initiatives, that puts emojis in everything, you say that wonderful, but unfortunately, it doesn't stack up very high in my list of
priorities. Because look, This is what I'm committed to and this was reviewed by my managers, and my, managers managers in the stakeholders, and everyone said, yes, the on boarding team needs to focus on these things. So now you're coming with this other thing, I'm sorry, this quarter, I cannot help you and derailing to Tech director Min. Write it down, put it in your idea back but don't necessarily without the goals. It's really hard for you.
You have nothing to back on to rely on to push back, sometimes own ideas. So that's number one. Then when you have a bunch of ideas, You need to somehow stack rank them and create what I call a hint, which ideas you want to test first and that's ice, essentially, some people consider eyes like magic. It's like it will tell me exactly what I should build. And after that, I just build it. I don't need to test it anymore because I said that's a completely bogus interpretation for dices.
It doesn't work this way. It will never work this way. There's no magic, no one can tell. You is just gives you an a hint. How does it do it? By breaking the question 2. Three parts. One is what is the impact impact on? What on the goals which particular goals?
The usually it's either the North or metric of your company or your business unit or if that's really how to measure than you know, style metric is a team if that's not the case of maybe on a key result of specifically result, right? We wanted to shorten the average onboarding time from 30 days to two days. Which ideas, what's the impact? On this thing, of course it's a gas but this gas will improve.
The more you test. The idea you start with a guess and then it's becomes a much more educated guess with some of these techniques. I mentioned. The second question is, is that the e at the end? That's basically the opposite of person wakes, in most development teams in a marketing team, it might be marketing dollars. It, whatever is the most scarce resource and easy ideas wanted, we can launch relatively quickly and with low-cost and expensive, Dear a low easy.
There is the opposite and then there's confidence and confidence is basically how sure are we that the impact and the ease of what we think they're. So when they're our guests and all their best owners, my gut feeling, the confidence should be near zero because if there's an unreliable sort of evidence that Self Conviction, every terrible idea that was ever out
there in the world. Someone thought it's a good idea, and we can see now in some very notable companies, how very Enterpreneurs. Push them to do really stupid things because there are sure there are good ideas and there are very convincing their ability to predict the future. So don't fall into this trap, but then if you start doing back-of-the-envelope calculations and reviews with your peers and with experts and the market research, competitive, research, customer interviews, Etc.
You go up a scale of confidence that is logarithmic, by the way, it goes up the test become harder and harder. Most, I Diaz will fall along the way. This is the confidence M. I mentioned it kind of Encompass that and then you can give yourself higher and higher confidence scores along the way. And what happens usually, it's, you're also able to adjust the eye in the E, the impact in the is because you know, much better. Now how impactful this is and how is it going to be unusually?
The impact goes down, and it goes up because you realize it's not going to be as impactful as does. The something called the planning fallacy. And now in, Logical mechanisms. We have we tend to underestimate the time of tasks and overestimate, the impact or the benefits, they will give us. So this is a good way to adjust for this. So that's how I recommend using
ice. Do it once just to rank your ideas and then keep doing it as you'll get more evidence, don't stop there and definitely don't rely on ice as a one-time magic algorithm to tell you what to build. Right, so I think I read one of your blog as well when you illustrate how to use this ice for building. I don't know, like a internal portal with this chat bot or something like that. That is probably one. Good illustration, how we should do this?
Ice technique, right? Do it once but also doing iteratively and as you can see the I see Andy will change based on more information more data that you have and I think that's a very good exercise for people to look at. So now we have done all this privatization, we have good amount of ideas. How do we break it down? How do we deliver? What you call experiments or steps, which is the next step. How do you actually scope them? And make sure how we deliver them?
Because ideas are just ideas. The most important thing is how we do. We roll it out to the customers, to the users, that will use that idea, right? So experimentation, right, everyone wants to do its magic words. I hear a lot of companies actually are interested to introduce it.
What I notice is that a lot of people because the word experiment sounds very rigorous day, Think either of a b experiment or betas or what they call MVP, which is often and near complete version of the product with basically, all the features, all the bells, and whistles, maybe it's not super polished. When I was young, this was called a better or maybe an alpha, but it's more like a bit of really, that's way too late to wait to learn whether or not. It's a good idea or not to invest.
It is basically waterfall. It's just basically building the whole thing, but I notice a lot of companies. Don't realize, what, two large gamut of experimentation techniques, validation techniques. We have at our hands at our fingertips to validate ideas and some of them don't require actually experimenting at all.
So I like to break it into five buckets this assessment, which is just doing it on paper or just looking whether or not their idea or aligns with the gold assumption mapping to see how much risk is in there. Dear.
These are all things you do without collecting external data even and even those a lot of Allow you to eliminate a large swath of your ideas Park them say for now don't look the most promising, then there is fact finding which is looking at data that you already have usage data, behavioral data other things, conducting user interviews, all relying on user interviews, you've done in the past because it's best not to do them just on demand but to to them on an ongoing basis.
Look at competitive result like look deeply at your competitors. Doing field research, there's all sorts of ways to get data. Data. And then asked whether or not this data, actually, confirms our assumptions over the ideas and that's a key Point. By the way, what we test is not their leader entirely but some of the assumptions within their dear. So it's sometimes good for large idea to break out and ask what are the assumptions?
This is a very good tool called assumption mapping by Davey J blend very highly recommended and then the next phase is testing the idea starting to build but of course in the first stage, you don't need to build a whole thing. You can fake it, you can do. Tortoise landing pages. Call-outs buttons in your you either do nothing except when you click on them, it shows the intended.
People actually are interested in this thing and then you show them a little pop-up that says, you know, we're not quite ready, but do you want us to notify you when it's ready, which is another test, of course, companies of validated the whole business model in this way, it's a really powerful technique to human operator test. We talked about Wizard of Oz, there's something called concierge test.
There's a lot of techniques and of course, the The fish for Dimension. So there's a lot of ways to test ideas in a very early stage before they're even close to being finished. Then there's another layer which is kind of mid-level test Alphas, early adopter, programs. There's all sorts of ways to test ideas that are not polished. Not fully implemented. Just the core scenarios, not scalable, not build.
Like we would build them to launch, and then later on there is late stage test like, betas previews Labs. All the most rigorous This which are A/B experiments, which have a controller limit, even the launch itself is another test.
So we have tests and then we have experiments, I call experiments, only the things that have a control element, then this release, the release itself is an experiment to like when we release the tabbed inbox we did this very gradually, we launched 25 percent and we monitor these five percent. If there's anything - starting to happen with there, any of their metric then we launched to another 10% at the 15% we
surveyed The people. And sometimes you do a whole back, you launch it to 99.5% and then you hold the 0.5% for another few weeks out of the thing just to see. Still, if there are differences, you have all these techniques to use. So basically, it's the matter of scheduling them, and putting them choosing, which ideas you want to test, and then asking yourself which are the key assumption. Then, what's the first step we should work on this tab, what's next etcetera?
And that kind of brings us to the next layer, which is how to project manage. This how to test multiple ideas in parallel, how to allocate resources, how to Stage IT Etc. Is there any specifics about project management or tasks that you mentioned by a Java project management? I guess in most companies these days they practice Asia whether it's fake Asia or maybe some kind of a gel.
Is there any tips that you think based on your Consulting anything different that we should be aware of about this project management for the task? I would say that angel in general is a very positive thing the especially compared to what Waterfall, as I practice it when I was a young engineer, so it's brought a lot of advances, over the years. Especially scrum became more and more strict mode, more structured, more and more full of process.
And I think some of the things team's practice today. Honestly, a lot of the founders of it or not. Super Keen about, I will not name names, but it became a very process everything. And then there's a lot of process people involved, waves dedicated scrum Masters and dedicated agile. Coaches, just To drive the process forward. So it's kind of becoming Gorman isn't of the days of project management, in the days of
water. Focus waterfall, was such a heavy weight thing that we needed dedicated project managers as to push through it. I'm not trying to break up all this. And if it works for you, keep using it, what I try to do with gist is to interoperate with it in a way that doesn't require the scheme that you're using today to change very much, it might require some changes but not a lot. So what I suggest Tim's Do is use what we call the gist mode, which is basically a new process
where you put on a board. It could be physical Board of the digital board. Three columns, one is your goals, just a key results, you committed to, and generally, for a team, productive of 10 Engineers or less. I recommend no more than four key results for quarter. Even for is a lot, because teams just cannot move that much that many key results. Those could be business, or product or user-facing cures on. Could be also technical or
design key results. Mean, if we need to cut down technical death or close a bug backlog, those are very important goals as well. Don't kick them out to side because the teams will do them. Without you. Knowing you need to a very conscious decision with your other leads, the designer and the engineering lid, what's the ratio? And then you create a border around that you pick the top ideas.
You want to start with based on ice or whatever method you like to use and you put them next to it in the ideas, column just the once. I'm starting to work on right away. Then next to those, you put the steps not all steps, just the ones you planning to do in the next couple of weeks, maybe because that part will change a
lot. The whole board has to be very Dynamic. With some ideas might turn out to be better than you, need to remove the entire a deer with all its Steps From the board and put another one. Instead some steps complete some steps need to be redone. So it's a very Dynamic thing. It's the job of the leads to keep managing the ball to keep it up to date, but I highly recommend meeting. With the team at least every week or every other week to review the board to talk about
the next steps. And that's a perfect kind of segue into the planification or the Sprint planning or whatever cycle planning you're using because that brings them a lot of context. It reminds them. What are the goals? What are you doing? Are we pursuing? What are the steps were? Not just coding. We're not just trying to push stuff into production. Market is done and move on to
the next ticket. We're actually trying to complete an experiment or what delivery and That's tied to trying to validate an idea and that idea is tied to a gold. So a lot of Engineers, tell me this really was missing. Before that, I didn't know I was working in this vacuum. I didn't know. No one actually told me, I didn't understand once they understand a very nice side benefit that I experienced a lot in Gmail, is that they don't
have to tell them that much. They understand what needs to be done. They come up with much better ideas than you of everything. I launched in Google.
Maybe 60% wasn't my idea at all just people were creative and I'm up with much better ideas in mind, and that's exactly what you want to do. Because then you have more free time to focus on researching the market, understanding the needs Etc. Instead of trying to spoon feed people with exact requirements, which is enormously tiring, and it doesn't get you what you wanted, the end of the day, because no matter how explicit your user stories are going to
be, they're still getting wrong if they don't understand the context so much more to give them the context. So that's how it works. And at the end of the day, once you have your Jace board and you have this column of steps that turns into the backlog that feeds into the scrum or kanban process, you just need to prioritize it. And that's exactly what they need.
The big change is that maybe meets printer will be changes, maybe meet Sprint, we will realize this steps, we thought were useful or not useful because we learned new information, we need to scrap this plan. So for teams that are very, very strict about Planning the entire scope and estimating it and not willing to change. This could be a problem. You can either do shorter Sprints, all you need to challenge this assumption that this is actually a good thing is really helping I would argue
that being willing to change. The scope of the Sprint is really true to the nature of AJ but I'm not the developer here so it's easy for me to make this statement. Another you had a product manager because sometimes product manager wants to change agenda as well. So I think that many people refuse To do this in the middle of spring or whatever that is right.
But I think the true, a gel will cater for that, either dropping the previous ones or looking at the capacity, whether they still can do it and they will accommodate that. If let's say, there's a good impact is good, business outcome, right? So thanks for reminding that. And I think hearing what you explained about, G-spot, it seems like a very powerful technique, very interesting. Because yes, I can see it all so many development teams, they don't understand the big choir, right?
The goals why they're doing certain does for example, if even if you add a A button, right? Sometimes we don't know how this will trickle down to the company goals. So thanks for explaining that as well. Yeah it'll mark thank you for this elaborate explanation about this. I really learned a lot and I'm sure many listeners here would also learn from your sharing about just unfortunately, we have to wrap up the conversation because of the time.
But I have one last question that I always ask from all my guests, which is to share, what I call three technical leadership is them. I guess for your case you can also do a tree product leadership is them, I'll leave it up to you which one that you Want to do, but can you share, maybe some wisdom for us to learn from you? All right, so tip number one is to recognize what type of
company you work for. And adjust your expectations and your model for operations accordingly, multi Kagan gave a really good kind of dichotomy of company types of the ones that you treat product organization which they often call, it is a delivery organizations, you work in a delivery tear, you basically there to do whatever the business tells you to do. They don't trust you at all to have your own. Onions or to know what needs to be done. Maybe that works for some types of people.
Incidentally, these organizations tend to be the strictest Embraces of This heavyweight scrum, and safe. It said rather know why, maybe because the liver is the name of the game and those like scrum today's. So targeted at delivery. If you are working in such an organization, a lot of what I just described doesn't apply to you. It will not fly. Don't expect your organization to actually be able to mutate. Into this much more modern thinking type of organization.
I don't want to be critical. Maybe this model is working for certain types of Industries or certain types of products as so you will be endlessly frustrated. If you expect them to, you should understand the rules of the game and within this very limited confines, try to be more creative or more evidence guided the second type of team that the market equation describes the feature team, which is basically a feature Factory. There's a lot of more autonomy
to decide how to implement. The feature, what to do inside a trust you on this and maybe you invent the features too. It's basically still very output focus is like launch feature. After feature of the feature, you don't challenge the feature so much in the middle. You don't do a lot of Discovery basically, just decide through some sort of protestations scheme, what to build and then you build it. And there there's a strict focus on road maps.
And a lot of time is spent on planning, that's one of the Key signs. I think there is a huge potential for improvement. Moving from feature to product in a sense, this is really the potential. So you really need to understand whether or not your company is ripe and ready to actually make this transition. If you can find allies and then you can start bringing some of these techniques. The third type is a product organization where they actually understand the importance of
outcomes overall output. They understand the importance of Discovery and they do it to some level and there. No. It is actually 100% pure and doing it correctly and not opinionated and none of this. It's impossible and maybe we shouldn't strive to be in this situation. But there you can find a lot of places where you feel, there's still a lot of pain is still a lot of ways being created and try to fix those things.
Move the organization to a slightly better place and think just applies to these last two groups. So Jason and everything. Of course, it's part of a much larger ecosystem of Failure. So recognize where you are, what kind of organization, how much willingness they have to adopt this new things and adjust your expectations in your room work plan accordingly, if you are want to be this change agent, of course. Recommendation, number two is on your Voyage, to try to change things.
Use evidence a lot. Here's the thing I discovered, if you go to a battle of opinions with a person who is more influential than you or more senior, you will almost always lose on principle, no matter how good your rationale or opinions are. If you come to that stem discussion with evidence, like that person will say, you know, what's the best thing in the world.
We need to build this nft or this scenario, machine learning thing, and it would say no, you No, we just did this longitudinal study and that shows that people really want and analytics dashboard. Look at the data, that's our Mall, interesting discussion, you might get more positive results, out of that, they might say. Okay? Yeah, but I want you to test my idea, fine. You have all these techniques to
test ideas, that's fine. But at least they're giving you the wrong to talk at their level. It's not just instructing you to follow their idea. So evidence is such a powerful thing. I've seen very authoritative CEO that used to come to Development teams to tell them build this build the other.
I just told them a few techniques and then the junior p.m. came and changed the CEO and said, You know, here is a business model canvas that shows the two latest idea will not work and the CEO was very pleased. It's saved the company. A lot of money, they didn't need to pursue another wild idea, just to discover months or years later, it doesn't work. So everything is such a powerful thing, don't over estimate, don't expect it to be magical. Some people still are very opinionated.
Dated but learn to use it and the confidence meter. That's why it's so popular because a lot of people were looking for such a tool to have in the back of their pocket. When people try to push their ideas to show them, that that's based on really weak evidence and we shouldn't have that much confidence as they are expressing right now. Tip number three is to learn to let go. I had a challenge as a product manager for very long time.
I felt tremendous responsibility over the success of the product. And I wanted to in Trucks, everyone exactly what to do and how to do it and that include designers and Engineers. I wanted it to be at a very high level based on my kind of criteria and that doesn't work very well. They don't like you first off for doing that and second, it's not the right way to do because they're the experts in their
area of expertise. So part of the reasons I could opt to Jesus because that enabled us to have a much more kind of balanced discussion. And part of the problem is that a lot of the responsibility does lay on the shoulder. As of the PM's. I mean, if you're an engineer and most companies, If the product idea, felt that's not your fault, you deliver the code. Look at the product manager made a mistake, right?
So change the Dynamics and said, no, it's our Collective responsibility to meet the goals. If one of us fails all of us first and then it's much easier for you to also delegate to them decisions because they're optimizing for the same thing. As you do, not optimizing just for code quality or just for implementation scrum proper, they're optimizing it to achieve the The goals of the team goals, it's much easier to let go with this model.
Also, you need to realize what your personality type is. I am a bit of a control freak, but still, it really helped me when things went this way. So, these are my three tips, recognize the type of company you work for and adjust to it. Use evidence where you can and learn to let go learn to let your team members guide, some of the decisions.
Right? I guess they're letting Go part also applies, not just Our product manager for every roles engineers in testers, whatever that is, I think we always can use some kind of a letting go, right? So that we don't use ego and pursue our ideas, mostly but also listen and contribute with others. Thanks for sharing that wisdom. So it dominate people love your juice framework. They want to learn more. They want to connect with you. Is there a place where they can
find you online? Yeah, there's a couple things. I suggest one is go to eat tomorrow. Gilad to Chrome / resources and then you'll Find all of these things. Some of them as downloadable template, some of them as stocks, some of them are just links to articles, all of my stuff is online, it's very well to share. Then you can start using it. The juice board template, for example, is there, the confidence meter is there. Also the spreadsheet.
The other thing is I regularly share, new tools, and new of things and your insights, with the people who follow me on my newsletter. So I suggest just subscribing, etymology live.com, / newsletter, I assume you We'll be putting these two links in the description suddenly. I will put that in the show notes as well. So, thank you so much itamar for this crash course about dies and product management. Thank you for sharing my pleasure and thank you for inviting me.
And I hope this will be useful for the listeners. Thank you for listening to this episode and for staying, right until the end if you highly enjoyed it. I would appreciate if you share it with your friends and colleagues who you think would also benefit from listening to this episode. And if you are new to the podcast, make sure to subscribe and leave me your valuable review and feedback. It helps me a lot in order to
grow this podcast better. You can also find the full show notes of this conversation on the episode page, at Tech, Legion of death website, including the full transcript interesting. It's and links to the resources mentioned, from the conversation. And lastly, make sure to subscribe to the show's mailing list on pack leader dot f to get notified for any future episodes. Stay tuned for the next technology. No episode. And until then goodbye.