Lessons from the Dot-Com Bubble: How Product Leaders Should Think About AI (with Greg Petroff)

And as our organizations change, how do we need to change with them? My guest today is the wonderful Greg Petroff, a professor, design thought leader, and seasoned executive whose resume includes companies like SAP, GE Digital, Google, ServiceNow, Compass, and Cisco.

And as Greg and I were discussing these changing times before recording the show, he made the point that we've been here before, with the Dot-Com era, but the difference is the pace of change and in turn, the required pace of adaptation. You're about to hear Greg talk through the tools, organizational changes, and strategies that executives need to be thinking about right now—because, Bob Dylan says, and warns us very well, ‘you better start swimmin’ or you'll sink like a stone. Let's jump in.

Welcome back to The Product Manager podcast. I'm here today with Greg Petroff. Thank you so much for joining us today, Greg. How are you doing?

And so curiosity is very important behavior to cultivate right now. And I think in the Dot-Com era, there were lots of companies that sat on the sidelines and didn't really understand what it was. And then there were a bunch of people that didn't really understand what it was, but they were curious and just jumped into it.

And started making things and I think we're in a very similar moment where it's moving so fast that the way that you have to find your way into understanding how you might use AI is to be using it as much as possible.

Also, I think some organizations are very orthodox about the role definition and they say, this is what you do and you don't do things outside of that and. This is what your cross functional peer does. And I think that worked for a while. I've never been really orthodox. I've been one of those people in blended boundaries in my career. And sometimes I've been accused of being a designer who does product management or a product manager who does design.

And my perspective is it's about product culture. And I think one of the things that's happening right now is that the tooling that we're using is so different than it used to be. It's so much faster and it's more accessible for a wider audience. And so those of us who might have been in a silo can start to do some of the work that might have been a peer's contribution. And the question is, where do those boundaries map? And, so the Venn diagram is collapsing a little bit on itself.

And I think, the future is going to be teams where, there's this term T shaped people, right? So you have, you're very deep in one area and you can navigate horizontally and I'm almost feeling like it's going to be a double T, we're going to get to a point where you're going to find these sort of hybrids, someone who can design and write some code or someone who's a product leader who came from design or an engineer who loves product management.

And so I think the more important thing is that whatever organization you find yourself in, that there's an open conversation around the tools and what they enable and who's on first, basically, who's going to tackle certain aspects of the product development life cycle.

And it gives us an opportunity to shift from like an engineering led culture or product led culture, or a design led culture to a product outcome culture and where everyone is contributing, based on the skills that they can bring to the table.

And so I think we'll see hybrid organizations that may look very different along the way, in the project management space, I think, the definition of what to build is going to become much more important because it's becoming easier and easier to build things.

And the incrementalist mindset of just adding a feature or adding a piece of something you can do that, but for the same amount of effort, you could do something that's much more valuable for end users and customers, and you can imagine things that might have been not possible before and so.

That's the thing I'm interested in is like, how do we create the space for that cross functional partnership to be bolder in the things that they're trying to do, because it's actually more straightforward to actually accomplish that stuff.

Let's talk a little bit about AI tools and product development, because you hinted at the tools are changing, tooling is changing, and this kind of goes down to discernment, because we have a lot of tools available to us, and we're really at an interesting juncture where we really have to exercise good judgment about which tools do we decide to use and how. So how do you help teams develop that good judgment or that, that kind of gut feeling about when and how to use AI tools?

There's a whole bunch of tools now that you can do, you can build custom rag models on your research insights and, for organizations that have multiple years of research, you can take all the transcripts from your research and build models that you can ask questions of. It's almost like having a virtual persona. I know that's controversial. And I wouldn't call it a virtual persona.

I would just call it making the knowledge base that you already have more accessible and easier for people to traverse. And specifically when you onboard new members of the team, helping them upscale their domain knowledge because they can have this corpus of information that's really useful to them. I mean, it's not easy to do, but it's one that once done, and I've been experimenting with that with some friends.

And, the fact that it comes up with Insights and elements of knowledge that may be commonly understood, but maybe not as clearly described is really interesting, certainly in the design side of the tooling side, you're looking at, there's a bunch of things coming in the Figma space, Figma itself is a new tool, it wasn't here really 5 years ago, and it's changed the way that we work.

I think teams are working straight high fidelity, and I'm not 100 percent sure that's the best idea because sometimes the low fidelity allows you to see things that you might not see once you're looking at something that feels like it's a completed product, but it's so much faster. And so teams can iterate and make very quickly so that, when the design team can make faster than you can think, then it's often useful for product teams to make to think versus thinking to make.

And what I mean by that is the conversation you have with mistakes that you make in the artifact that you're creating, inform you about the outcome and the customer problem in ways that just talking about it sometimes are insufficient. And, pictures like, a thousand words.

What we're finding is that design teams that work with their product teams very early and just iterate, and they're not really making the final product, they're just iterating and making stuff, can basically replace the PRD with a prototype and vet the prototype and actually have something that they know that would be of value in a way that was expensive to do in the past and took too much time. And so, so that's an example.

And then certainly on the engineering side, I mean, People are experimenting with things like Copilot and tools that allow them to move faster. I think we're still early days there because I think engineering folks are on the fence about its utility. But the fact that large parts of code bases could be partner written with you. Also, it frees up the engineering teams to work on maybe harder problems. So all these things are there.

And I think one of the things I think we have to have a mindset of collectively is how do we disrupt our work, because most of us are trying to place these tools into our products and we would understand better the outcomes that they could provide by also using them and, our day to day product development processes.

So that kind of plays well into our conversation about the value of institutional knowledge, particularly in research teams, I think that research is an area where the use of AI is a little controversial. So, how do you see that relationship between human expertise and AI powered knowledge management evolving?

However, in large organizations, there's usually a lot of content available and not many people actually access it. And the research team would like more people to access it because they feel like it will drive better insights and those insights will drive better outcomes. So I think one part that's interesting is building custom knowledge graphs of your research corpus. And then enabling anyone in the organization to ask questions of it. And that's not replacing research.

All that's doing is allowing you to see the patterns of insights that collectively you have. And it shares that knowledge over a wider audience in the organization. So it's not just owned by the researcher anymore. The researcher has done the work and it democratizes access to that knowledge. In a way, it helps everyone understand who are we serving and why? And then it allows the research team to keep feeding that. Their job is then, at some level, to continue doing the work that they do.

And I'm a big believer in qualitative research and even quant methods where, you're instrumenting et cetera, and not giving that up to an AI to do for you. The ability to synthesize that knowledge and share it more broadly, I think, is going to be something that's going to be really interesting. And I've always looked at democratizing research. I know people feel controversy around that idea.

Whenever I've seen access to something that's a value in an organization become more open and more available, the demand for expertise actually goes up. And what I mean by that is that if more people see insights, the more they want them. And the more they recognize that they need professional insight gatherers on their team to do the work that, they know how to do well and do it in a methodologically solid way.

And so I think that's the area that I'm interested in is how do we make sure that everybody is deeply seeped in the knowledge of who they're serving and taking that information to make better products, I think is things that actually make better products.

It's more about making that data interactive like what we have developed and what we have learned. And similarly, we did a couple of episodes on the product analytics tools that are Gen AI enabled that are really fascinating. We did one with Mo Hallaba, who's the CEO of Datawisp and also one with Mario Ciabarra at Quantum Metric.

The way they put it is the democratizing of data and making it more available and more interactive to people who don't have that data scientist competency or that skill set. And I agree, I think that once, once data is more accessible and we can really see, no matter what kind of stakeholder we are in the organization. We can really understand how those insights benefit us and interact with them in a way that makes sense for our role.

Then it really does open up the door for, okay what does a data informed culture, what can it really look like if we're really using those tools effectively? So I think it's a very exciting moment in product.

So in some organizations, the, those who have the most domain expertise have the most influence, I'm not going to say power, but most influence in the organization, what happens in an organization where everyone has access to a deeper understanding of who you're serving and why I think that part will be interesting to see how that unfolds and if it helps or if it breaks some conventions around, the structures of how we see ourself, that part's interesting.

The last thing is, I get in trouble sometimes for saying this, but I think the role of research is to de-risk product decision making. And what I mean by that is, yes, we are trying to understand and empathize with our end users and understand deeply the problems we're trying to solve and what outcomes are important to them. Product leadership makes assumptions about what features and functionality will benefit end users and also make a great product that people are willing to pay money for.

And oftentimes some of those assumptions are loosely held, but are expensive to implement and they're not vetted in a way where you have some degree of certainty that if you do that, people will actually value it. And so I think this is 1 of the things that researchers can do is if they want to move up the stack in terms of importance.

It's to be able to obviously give the general understanding of who you're serving, but also find the parts of the product development process that there are lots of assumptions about that haven't really deeply been validated and use their strength to provide more signal so that when leadership makes decisions on what they want to execute on, they have more certainty that if they do that work, the expected outcome will be the expected outcome.

We've got, in SaaS businesses, we have renewal rates and then we can do cohort analysis over which customers are renewing with the least amount of effort on the sales organization, et cetera, and you can take all those signals and historically, they've been information that has been owned by customer success or owned by the team and owned by this corporate strategy team and owned by the research team. And I think there's an opportunity for research to see all of that in a big 360.

And so that you can get a health check of the experience that you're creating, the business that you have, the behavior of your customers, et cetera. That was just almost impossible to do before. And it's not easy to do. I mean, there's only a few people I know in the research community have done that.

I think that's where AI can also help is to start melding different data sets on customer success and user success together to get a better picture of the health of your product and then also what needs to happen next.

And then there's also this core expertise that kind of when we have many roles as an individual contributor or a leader, I think that there's some questions about what is the core expertise that you're bringing into the table and how do you defend that when you're in a situation where other people have overlap with what you're doing, what you're contributing? There's also AI tools that contribute some of that.

So how do we help teams navigate that uncertainty of role evolution and maintaining their core expertise and value and kind of protecting like the area of expertise that they're bringing to the team?

You and I think 1 of the things interesting about this moment is we've actually had stability in the roles for the last 10 years. And so a lot of people have grown up in that era. And have a solid definition of their value, and a solid definition of who they are, and a solid definition of what they do. Haven't experienced having the rag pulled out from underneath them because there's some new way of doing things or a new piece of technology.

Some of us have been around long enough, it's happened to us so many times that we just see it and say, oh. Okay, the rules just changed, okay, now we got to be more adaptable. And I say, expertise is really important, right? You know what I mean? So I think you should know what you're great at. And I think the way I sort of coach people is lean into your strengths and value them.

But don't get caught up in your role definition, we're moving into a moment where it's more about the outcome and the problem definition. And then the execution, there are a lot of different ways to get there. And if we hold on to the orthodoxy of our role or position, we might just be fighting the currents that are around us.

And so, stay curious, have a mindset of evolving and constantly, learning, and at the same time, don't shift away from what you love and what you're good at, because the best teams are made up of people who are exceptionally good at something. I'll give you an example. When I was at SAP, this was almost like 15 years ago. I was part of a team. We weren't a design team. We weren't a product team. We weren't an engineering team. We were an innovation team.

And it was made up of, we had anthropologists on the team, we had engineers, we had product managers, we had designers, and we would be tasked with a project, and we would assemble a team, and the team was always an eclectic group of people, and we would just look at the problem and figure out how we were going to work on it together, and I anticipate there's going to be more of that kind of behavior in the future.

I think in large organizations, you have to organize in a certain way, because It helps your product development process, but in smaller organizations, I think that we're going to be looking at an environment where, we're just gonna be more nimble about the tools and the things that we have. And if we stay orthodox about what we do, it'll actually get in the way of actually doing the work.

Now that we're on this anecdote train, would you mind sharing a specific example of how a shift like this has impacted product development outcomes that you've observed?

And historically, there's been some incredibly successful people in the product management space who behave that way. So many earlier career product leaders will look at those people and say. I want to be like them and I'm going to do the same thing the challenge I've seen, especially in large organizations, is that there's a behavior that can happen where if resources are constrained and the incentives are set up for you to deliver product leaders will compete with each other for resources.

Because if they over deliver, they get the next job promotion. If there's no one at the top sort of saying, these are the things that we're not going to do incentivizes people to try to get more done in the time that they have. And what they don't recognize in that moment is they place strain on the system because there's only a finite set of resources. And so they're actually using those resources less efficiently because they've put too much into the system.

And therefore, you end up get a bit of crazy town, and then people move into kind of command and control to try to whack a bullet into place. I've also seen the other side of it, where you're seeing people who are like, hey, I'm entering this project, and I don't know what we're going to do. I don't have to be the person with all the answers.

But I'm going to assemble a team, and collectively, we're going to learn together as quickly as possible, and we're going to build a set of experiments along the way that allow us to learn as quickly as possible, and we're going to collaborate and share that information.

And the project leaders who do that, I find, get more out of their teams, they have more clarity, there's more visibility on the work, people feel like they have autonomy in that, they can find their role in the work and the effort. And it may be an indirect observation, but it feels to me that generationally, it's more earlier career or younger PMs who behave this way. And I think it's partially because they're digital natives to begin with.

Some of the more senior people, aren't really if you look at demographically and have grown up with great software and want to create a space for value for the design team and for the research team and for the engineering team and the product team to collectively solve a problem together.

And, I was at the Canadian friends conference this fall, and it was just really fascinating for me to hear the conversations around how that community of product leaders thought about how they worked with a cross functional peers. And it was really heartwarming for me because it was a very open conversation. It was that it was a 1 where people were willing to be vulnerable about that. They didn't have the answers, but they were organizing their team to go find the answers and.

Yeah, that's a little bit different from feeling like you have to have the answers and your team's counting on you to have them. And so you fake it till you make it and pivot a lot until you get there. Right. And so I think there's sort of two strands that are happening that product development community that I would be curious to see if, better outcomes happened, one law or the other. I don't have great examples for you, but I just, these are observations of behavior.

And are really starting to have this nominal effect on how people lead teams and work with people cross functionally is it's really heartening to see how that's really enabling a lot of people to do some of their best work. But I'm gonna put you on the spot a little bit since we're talking about this, you're a leader, you're a seasoned executive, and you've managed a great deal of teams.

So what strategies have you found to be very effective for fostering, as you mentioned before, curiosity, experimentation, while maintaining this high standards of quality?

So it cascades down so people can see their role and how it adds up to it. It's hard for organizations to say no to things, but I think the organizations that declare what they're not going to do are, very, it's useful because it stops eager beaver behavior around Hey, we could do this, or I have this pet project that I want to do. And if you say, no, we're not doing that doesn't mean it's not, we're not ever going to do it just means we're not going to do it now.

So that people, again, that's a part of that sort of kind of clarity aspect. I think giving feedback in a courteous way, but also holding people accountable. I think the notion of creating environments where people feel safe makes sense. And it's incredibly important. But you also have, you can't shirk away from giving constructive feedback.

And when I was at Cisco that one of the tenants that we had that I really loved, and it came from Duo, which was a startup that Cisco had purchased about five years ago, and they had this tenant that they call be kinder than necessary. And what that meant was if you had spinach on your front teeth and you're walking into a meeting, you'd want someone to tell you that you have spinach. It's not about you. It's just that this is something that you should know about.

And so I think it's about constantly giving feedback to each other and holding each other accountable to a high standard and making it about the work, not about you. And that creates the conditions where people learn to listen to that feedback and don't take it personally. They take it like, Oh, okay, that's an opportunity for me to improve my game, or that's an opportunity for the project work to, to be in a different direction. So I think those are the two things that for me.

I'm not saying I'm great at it, but I materially try to practice is try to be as clear as possible and try to give as much feedback as possible.