Beyond Dashboards: How Data Teams Earn a Seat at the Table - podcast episode cover

Beyond Dashboards: How Data Teams Earn a Seat at the Table

Jan 05, 202649 minEp. 495
--:--
--:--
Download Metacast podcast app
Listen to this episode in Metacast mobile app
Don't just listen to podcasts. Learn from them with transcripts, summaries, and chapters for every episode. Skim, search, and bookmark insights. Learn more

Episode description

Summary 
In this episode Goutham Budati about his Data–Perspective–Action framework and how it empowers data teams to become true business partners. Gautham traces his path from automating Excel reports to leading high‑impact data organizations, then breaks down why technical excellence alone isn’t enough: teams must pair reliable data systems with deliberate storytelling, clear problem framing, and concrete action plans. He digs into tactics for moving from reactive ticket-taking to proactive influence — weekly one‑page narratives, design-first discovery, sampling stakeholders for real pain points, and treating dashboards as living roadmaps. He also explores how to right-size technical scope, preserve trust in core metrics, organize teams as “build” and “storytelling” duos, and translate business macros and micros into resilient system designs. 

Announcements 
  • Hello and welcome to the Data Engineering Podcast, the show about modern data management
  • Composable data infrastructure is great, until you spend all of your time gluing it together. Bruin is an open source framework, driven from the command line, that makes integration a breeze. Write Python and SQL to handle the business logic, and let Bruin handle the heavy lifting of data movement, lineage tracking, data quality monitoring, and governance enforcement. Bruin allows you to build end-to-end data workflows using AI, has connectors for hundreds of platforms, and helps data teams deliver faster. Teams that use Bruin need less engineering effort to process data and benefit from a fully integrated data platform. Go to dataengineeringpodcast.com/bruin today to get started. And for dbt Cloud customers, they'll give you $1,000 credit to migrate to Bruin Cloud.
  • You’re a developer who wants to innovate—instead, you’re stuck fixing bottlenecks and fighting legacy code. MongoDB can help. It’s a flexible, unified platform that’s built for developers, by developers. MongoDB is ACID compliant, Enterprise-ready, with the capabilities you need to ship AI apps—fast. That’s why so many of the Fortune 500 trust MongoDB with their most critical workloads. Ready to think outside rows and columns? Start building at MongoDB.com/Build
  • Your host is Tobias Macey and today I'm interviewing Goutham Budati about his data-perspective-action framework for empowering data teams to be more influential in the business

Interview
 
  • Introduction
  • How did you get involved in the area of data management?
  • Can you describe what the Data-Perspective-Action framework is and the story behind it?
  • What does it look like when someone operates at each of those three levels?
    • How does that change the day-to-day work of an individual contributor?
  • Why does technically excellent data work sometimes fail to drive decisions?
    • How do you identify whether a data system or pipeline is actually creating value versus just existing?
  • What's the moment when you realized that building reliable systems wasn't the same as enabling better decisions?
    • Better decisions still need to be powered by reliable systems. How do you manage the tension of focusing on up-time against focusing on impact?
  • What does it mean to add "Perspective" to data? How is that different from analysis or insights?
  • How do you know when you're overwhelming stakeholders versus giving them what they need?
  • What changes when you start designing systems to surface signal rather than just providing comprehensive data?
  • How do you learn what business context matters for turning data into something actionable?
  • What does it mean to design for Action from day one? How does that change what you build?
    • How do you get stakeholders to actually act on data instead of just consuming it?
  • Walk us through how you structure collaboration with business partners when you're trying to drive decisions, not just inform them.
  • What's the relationship between iteration and trust when you're building data products?
  • What does the transition from order-taker to strategic partner actually look like? What has to change?
    • How do you position data work as driving the business rather than supporting it?
  • Why does storytelling matter for data professionals? What role does it play that technical communication doesn't cover?
  • What organizational structures or team setups help data people gain influence?
  • Tell us about a time when you built something technically sound that failed to create impact. What did you learn?
  • What are the common patterns in dysfunctional data organizations? What causes the breakdown?
  • How do you rebuild credibility when you inherit a data function that's lost trust with the business?
    • What's the relationship between technical excellence and stakeholder trust? Can you have one without the other?
  • When is this framework the wrong lens? What situations call for a different approach?
  • How do you balance the demand for technical depth with the need to develop business and communication skills?
  • How should data professionals position themselves as AI and ML tools become more accessible?
  • What shifts do you see coming in how businesses think about data work?
    • How is your thinking about data impact evolving?
  • For someone who recognizes they're focused purely on the technical work and wants to expand their impact—where should they start?

Contact Info
 

Parting Question
 
  • From your perspective, what is the biggest gap in the tooling or technology for data management today?

Closing Announcements
 
  • Thank you for listening! Don't forget to check out our other shows. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used. The AI Engineering Podcast is your guide to the fast-moving world of building AI systems.
  • Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes.
  • If you've learned something or tried out a project from the show then tell us about it! Email hosts@dataengineeringpodcast.com with your story.

The intro and outro music is from The Hug by The Freak Fandango Orchestra / CC BY-SA

Transcript

Tobias MaceyTobias Macey

Hello, and welcome to the Data Engineering Podcast, the show about modern data management. Composable data infrastructure is great until you spend all of your time gluing it back together. Bruin is an open source framework driven from the command line that makes integration a breeze. Write Python and SQL to handle the business logic, and let Bruin handle the heavy lifting of data movement, lineage tracking, data quality monitoring, and governance enforcement.

Bruin allows you to build end to end data workflows using AI, has connectors for hundreds of platforms, and helps data teams deliver faster. Teams that use Bruin need less engineering effort to process data and benefit from a fully integrated data platform. Go to dataengineeringpodcast.com/bruin today to get started. And for dbt cloud customers, they'll give you a thousand dollar credit to migrate to Bruin Cloud.

You're a developer who wants to innovate. Instead, you're stuck fixing bottlenecks and fighting legacy code. MongoDB can help. It's a flexible, unified platform that's built for developers by developers. MongoDB is ACID compliant, enterprise ready with the capabilities you need to ship AI apps fast. That's why so many of the Fortune 500 trust MongoDB with their most critical workloads. Ready to think outside rows and columns? Start building at mongodb.com/build

today. Your host is Tobias Maci, and today I'm interviewing Goutham Budati about his data perspective action framework for empowering data teams to be more influential in the business. So, Gautam, can you start by introducing yourself?

Goutham Budati

Thanks, Tobias, for inviting me to the show. My name is Goutham Badati. I have led data teams in ad agencies and fast growth startups, and I'm super excited to talk about my data perspective and action framework that I've been operating with for the last ten years.

Tobias MaceyTobias Macey

And do you remember how you first got started working in data?

Goutham Budati

It's hindsight, it's turned out to be a very funny story in terms of how I started in data. I was a software developer for the first few years of my career. And then I always felt like I was distant to how decisions were made and how my actions turned into customer products. So I wanted to switch gears, come closer to working at a customer

function. I figured advertising would be the right place. And given my technical skills, I could be a PM in an ad tech company. So I started interviewing in ad tech in the Bay Area, and one thing that I heard consistently was, you don't have an ad agency experience. So I was like, Okay, let me go back to ad agency, and I'll go back as in try out working at an ad agency.

And there was an analyst role, and I just applied and got it. And it was an interesting role. It was basically building Excel reports. Was taking about a month for the team to do it. So, you know, they hired me. It was not my best first job, but I had to get in somewhere. And then my boss left on day one. So I had no one to oversee me. I was reporting directly to the CMO of my ad agency.

And I automated my work in about a day. I mean, anybody in this audience would know it's not difficult to automate a few Excel sheets. The workflow is very simplistic. We are building media mix models, so collecting data for those. And then I had this time to think about what to do with my role so that I can create value. So I was pitching forecasting

and predictive analytics of how to identify levers, and by changing these levers, how can we create value for the business? So within a year, we grew the size of the business in terms of media spend for the agency by about 20% through the forecasting pitching I was doing across all the clients.

And that was a turning point in my career. The role was called data analyst, and then it turned out to be analytics role. And then I started building data teams. And then I picked opportunities in similar sized companies where there were no data teams or the first data teams were struggling. And I came in and turned around the data teams at different junctures. So that's how I started in data, and it turned out to be an exciting journey for me.

Tobias MaceyTobias Macey

And over the course of your career, I presume that this is learnings aggregated from multiple experiences, but you are presenting what you're calling the data perspective action framework. And I'm wondering if you can give a bit of an overview about what that is and some of the story behind how you came to this conclusion or formulation of the problem.

Goutham Budati

Yeah. Thank you. It's an articulation of my experience in terms of having seen some of the success and failure stories of how data teams were generating value at all the companies I've been in. Oftentimes, I characterize the work into two areas. One is your craft, which is the data, the work we do, in terms of the building, the reporting, and, you know, the analysis that we provide.

And then the storytelling part of it. It's not just the analysis, but taking it forward to a level that it influences the roadmaps of any you know, of the product team or the marketing team or the supply chain team. So looking back at all of these, data teams oftentimes struggle by perspective, not being in the room where decisions are being made. It was always serving the stakeholder, not serving the business problem.

And, you know, that really you know, that framing really helped me identify what areas to influence. And I came up with this formulation of, you know, it's not just the data that matters. It's the perspective and the actions. Ultimately, we're trying to create impact with decisions.

And a little bit bold statement is there's not much data in the data roles. There's a lot of, like, decisions and decision making is involved in the data roles that really matters, ultimately, at the value stage. So that's a bit of background on this framework.

Tobias MaceyTobias Macey

And so in terms of the day to day practitioner for somebody who they understand their job to be, I have to make sure that all of the company's data is collected, aggregated, and delivered in a reliable fashion. What are some of the ways that using this framework changes the ways that they think about their responsibilities and the day to day work that they're engaged in?

Goutham Budati

So if you are a data engineer or an analytics engineer largely, you need to always continuously think how your work provides value to the organization. And I'm talking also

more specifically relevant to companies in the series B to series C, fast growing startups where the teams are leaner. You're you're in the business of not building infrastructure. You're in the business of creating value to the organization. So in the data perspective and action framework, a typical role focuses largely on the data portion, which is building the infrastructure, the reliability of the data, the consistency in terms of how you are providing data to your stakeholders.

But I would extend it to providing your perspective and calling into action what the business is not doing. And be the inner voice of the company. That will bring you into the session rooms that you're not in right now. And if you look back, it'll actually help you develop better roadmap for the data infrastructure. Because oftentimes,

within the companies, there is light shining on one portion or one part of the business, but there is light not shining on various parts of the business. And these would come to you as reactionary requests.

Rather than making these as reactionary requests, you can be proactively seeking and developing a roadmap and have a very proactive posture in the company. Ultimately, data teams can have a seat at the table, not be a reactionary data team, but rather a very proactive, you know, visionary data team in the company by following this, you know, approach. TLDR,

you will have a seat at the table, people will listen to you, and, you know, you can help frame decisions and be part of the decision making.

Tobias MaceyTobias Macey

There are a number of aspects of the day to day work of data engineers. A lot of engineers who do work and particularly at the deeper, more complicated levels will often take pride in their technical prowess,

and they don't necessarily have either the exposure to or potentially interest in the higher order business requirements. They're very focused on those low level details of making sure that all of the file layouts are appropriate to get the best IO throughput or making sure that their delivery pipelines

have a high reliability, that they're doing lots of data quality tests, etcetera. And I'm wondering how that can be a potential limiting factor in their effectiveness and just some of the ways that people who are very technically focused should be getting that exposure or educating themselves on how to think about those higher order business problems that are oftentimes three or four levels removed from the specific problem that they're actually involved in in their day to day work.

Goutham Budati

Very good point. And by design in companies, people are siloed in part of their functions because you start specializing into different parts, and you start focusing on, you know, what's your day to day responsibility. But if you want to grow in your career, that is one aspect. And two, if the business design needs to continuously grow, there is so much wealth of perspective that a data engineer or an analytics engineer can bring on. I would say that we need to force

some time during the week to get away from the day to day of the work, step back, zoom out, and look at the business context. And it's not difficult to get this because you have peers on the business side, or you have your manager, or you have analysts who can really help you gain the context. By time boxing and forcing yourself to have a perspective, it'll really help you develop a macro perspective over a period of time. So I'm not saying you have to trade off between these two. It's about carving that small 5% time within your week or 10% of the time and really

focusing on building perspective that really helps you draw a line from what's in the data, what's buried in the data, to the business. Because oftentimes, decisions are at a high level, sometimes made based on emotions, and are rationalized later through data. But objectively, the data team could be looking at things deep in the data that are not passed on to the business. So building this perspective

really builds a handshake with the business. Now, how do you trade this off? It's not easy in the starting, but I think one mechanism that I have built that really worked for me and my teams over the past few years is doing a weekly narrative or a one pager on the data and then the perspective and action. The perspective space

really is flexible. You can come up, you can dream on what needs to be done on the business and get instant feedback from one person, that could be your manager or your peer or someone on the business side. And this cycle of weekly one pager and feedback really helps you build a strong intuition on the business. And once you have the strong intuition, that will help you think about your systems differently.

Maybe you have these noisy alerts that no one is looking at and doesn't even matter to the business. Or maybe there is this, you know, 100 pipelines that you built and you are very, you know, you consider them sacred, but maybe they are not. And maybe there is a change in the business strategy that is coming two years from now, and you're not aware of it, and you are maintaining this, and in the end, you have to dismantle all of this and change the direction. And these would be valuable things for an individual contributor to seek out and not rather wait for the roadmap to be shared from top down.

Tobias MaceyTobias Macey

There's definitely a lot of value in being that personality that does identify and seek out solutions to problems before other people recognize them, but it can also be difficult to understand where those problems exist if you don't have a feedback mechanism from the broader business where maybe as far as you're aware, everything's working great. You have your data pipelines, your data quality checks are all aligned, you have your data catalog for people to be able to find the different dashboards, etcetera, but you never hear about how the consumers of those dashboards or the consumers of those data assets actually have to export them to a CSV file, load them into Excel,

add all of their own formulas, and then do a dump of that every time they want to do anything. And I'm wondering how you can gain some of that insight or encourage some of that feedback if you don't already have that built into some of the organizational

practices and just the the communication patterns within the organization where oftentimes it can be very one way where maybe a request comes in and you deliver the thing that was requested, but you don't understand why it was requested. You you don't either have the capacity or the time to do that deeper digging of what is the problem that they're actually trying to solve versus the technical solution that they're asking for.

Goutham Budati

Yeah. A very interesting problem and something very close to my heart. There are two ways here. One is a request comes to you, and what do you do when a request comes to you? And the second one is how do you go and hunt or scavenger be a scavenger of hunting for problems? Let's talk about the request side of this. Whenever a request comes to you, what are you going to do with that? Many teams might have briefs or know, some templates in terms of a ticketing system, or maybe teams brainstorm and sit down and talk about problems. But you have to have a product mindset here. You need to always ask these three p's. The first p is, what is the problem? Really clarifying what problem we are solving. Next is who are we solving this problem for the person? Is this a set of people on the stakeholder side? Are we solving it for a few functions? Really articulating that really helps. And then why now? Oftentimes, you'd be surprised that a lot of these requests are curiosity requests that probably will not sustain the, you know, prioritization

scheme, if you really stress test it. So asking these three questions really helps you form an understanding of whether you are seeing the same request coming from different stakeholders at different points in time. Should you build an infra that is rather self serve? Again, self serve is a loaded word, and this group would know a lot more. But that would really help. And then following through by launching not the perfect version, but rather a version 0.8

and co creating either the dashboard or the end result, which could be sending data back into a marketing channel or

sending it to a vendor. Working with your stakeholder will help you develop this perspective. So in a reactionary manner, take it as an opportunity to dig into the problem, who are you solving it for, and why is it important in the context of the business. That helps build perspective, these micro moments. But when you look at the proactive state where no one is coming to you, but you rather want to ask, you know, find out where the problems are. In a very classic fashion, you could sample a few stakeholders

and interview them, what their pains are, what are they seeing as it relates to data, or not even related to data, the decisions that they are making. And you would see a ton of things that people are doing that are inefficient, that as an engineer, as an IC, you'll be able to solve it maybe in a day or two, right, with simple tactics.

So that would start building stakeholder trust. Ten years ago, and this is true even now, I once met, you know, a data engineer who was trying to understand, like, what he should focus on. And my advice to him was talk to five people. And then he came back the next week with a set of problems and things he can solve and how it can be better for the organization. I think data engineers and analytics engineers have so much power to solve problems.

If they are not waiting, I mean, not most of them are waiting, but if they are not, seeking out, you know, they wouldn't be able to enrich the organization with these benefits.

Tobias MaceyTobias Macey

And so we've talked a little bit about some of the perspective seeking and how to gain that broader understanding in the context in which you're doing the work. And how does that then factor into the types of system designs, the types of data artifacts that you produce, and how you think about the presentation and discovery of the data that you have where

maybe somebody has requested, oh, I need this dashboard with these 15 columns, but really they only use two of them and then do an Excel export to add a third one. And I'm just wondering how you can think about once you have that broader perspective once you do start to build up those feedback loops of interacting with the consumers of your data assets how do you then design the systems to reinforce those feedback loops as well as to ensure that you're delivering the actual requirements versus the stated requirements.

Goutham Budati

Yeah. You brought up a good point of stated requirements versus actual requirements. Oftentimes, the stakeholder who is coming to you probably doesn't even know what the full requirements might look like. So as an engineer, it's your responsibility to ensure that you clarify what the underlying requirements might look like. This could be done in the form of design sessions, building some prototypes, because ultimately,

you know, there are surfaces that the stakeholder needs to, you know, see and visualize. That could be in the form of a dashboard or a report or maybe, you know, a simulator of sorts, maybe if you're building like forecasting, you know, tooling. I would emphasize the design layer. As part of the perspective, the design layer is very important, the surfaces that people see and consume. After iterating on that a little bit in terms of the

question and answer style, I once had a stakeholder write down 100 questions that really matter to them. And in the last three or four years, once I started to ask them, none of the questions came out of the boundary of those 100 questions. So it gives me a gamut of what people are trying to ask and what people are not trying to ask. And then roadmap it in such a way that we deliver on the core requirements of, let's say, looking at three or four columns. And then in the future, enrich the data. Oftentimes, this goes back to your first party data. Can enrich it with, you know, the external third party data to see a complete view of the customer. And then maybe, you know, add in additional modeled segments of data so I can slice and dice the data by certain, you know, model segments. So this is a broader framework of how typical analytics would work, and I would frame it in that design session. So if you're going to get the core data, then later you're going to get this, and in the end, you're going to get this. My favorite way of operating is actually having your dashboard as a living roadmap. So in the example that you brought up, the third column would just say TBD or it's coming or, you know, have some, you know, messaging there so that people know instantly that when they're consuming, this is coming on this particular data or this part of the roadmap. Driving clarity through your process is very important. And I think seeking out perspective

in this manner and doing design sessions and using surfaces as a way to communicate largely, you know, clarifies to your stakeholders on what you're working.

Tobias MaceyTobias Macey

And then the other aspect too of having that broader impact and, you know, digging back into some of the technical requirements is that while it is great to have buy in and communication with the business users and the stakeholders of the data, you still need to make sure that it is reliable and has high uptime, which is a nontrivial effort.

And what are some of the ways that focusing on the core needs can maybe narrow the scope of the technical systems and the technical requirements around it to allow for a more concerted focus on those systems and ensuring their uptime while maybe avoiding some ancillary work that can ultimately become a time suck.

Goutham Budati

Let let's take a more binary example of real time data versus, you know, my data could be, you know, lagged and delayed, or can be refreshed, you know, once a day. I think, even if you have a requirement for, like, real time data of sorts, like focusing on the most simplistic outcome with more batched systems, is one way of coming up with a solution.

Because oftentimes, requirements also evolve over time. So it's very important to focus on the bare minimum or an MVP, or I call it the point eight version of building the systems. Now, it's not just the technical aspects of the system in terms of the uptime and, you know, the latency in terms of how the dashboards

are producing reports. It's also the usability aspects of it. In a funny way, the users have different ways of selecting filters or slicing and dicing data. So it's important to consider all these through the early builds.

And once you're certain of the requirements and this is going in the right direction, then carving out a sprint and going deeper onto the robustness of technical aspects of the data is important. I wouldn't focus on the technical aspects of making this the perfect system without getting

buy in in terms of this is the direction we are going. So navigating that in the consumer world, this is typically you would have a product manager and an engineering group where you would manage these, and the engineering group has some, you know, flexibility on focusing heads down on this. In the data world, data product manager is relatively new. And in the orgs that I have designed, I've had analysts play that role, so that because they're close to the stakeholders, while the analytics engineers and data engineers have some headspace in terms of focusing on the technical aspects of things. But in this set of fast growing startups, it's still a luxury to have, you know, both these roles. So playing this well in terms of

ensuring the requirements are really finalized before you build the robustness of the system is how I would focus on. One example I want to give from the past is, at some point, based on the requirement, our analysts and analytics engineers built about maybe, you know, 50 dashboards. And they went too far in terms of the build aspects of things, in terms of making sure that there's alerting on every possible data stream that we're bringing in. And all the data is presented,

and all the queries are templatized, and people can actually self serve in that manner. But once you started working with the stakeholders and figuring out how many they use, people use no more than two dashboards there. And all the, you know, rest of the dashboards were organizational, you know, I wouldn't call it a waste, but rusty, a thing that you could have avoided, and rather vertically integrated on top of those two dashboards

itself. So this kind of trade offs need to be taken by the analytics and data engineers because there's no clear data PM role stable in most of the companies I'm talking about.

Tobias MaceyTobias Macey

And then the final phase of the framework being action, that's definitely one of the long standing challenges of data work where you can provide all of this insight, you can generate a lot of context,

and tell a good story around what the information is telling you, but then it's left as an exercise to the reader to decide what to do about it. And oftentimes, that also requires shifting to a completely different system or platform to even be able to take action, and that's where systems such as reverse ETL or what is sometimes called data operationalization

is intended to help address where you can actually maybe feed some of that data back into the system that is where all that activity happens. I'm just wondering how your thoughts around the formulation of the framework and the the change in focus of the technical role can help to facilitate understanding what action to take and support the execution of that action potentially even without having to switch contexts.

Goutham Budati

Yeah. This is one of the hardest, in in the data side of things. And one of the models I've been recently excited about is the forward deployed model in terms of the AI world, where you're not leaving your data to the stakeholders.

There is a big white space between your recommendations and how decisions are made. And if you're not extending yourself into that white space, if you're not being strong from your perspective and mapping out what actions to take, the stakeholder is going to do what they would do, both from like a, you know, domain incentive that they might have in terms of the immediate things that they need to act on, and also maybe losing to some misinterpretation of the data. So I think where perspective

comes in play is it's not just an insight, it's about why this insight matters, what do we do with that insight now, and then clearly mapping out the actions to take. If it is marketing, it's about sending data back into the marketing platforms. And here's how we are going to send the data, and how would you collaborate with the marketing partners,

their data teams or technical teams who are receiving the data, and ensure that everything is tight and buttoned up. So extending yourself and deploying yourself a little bit more into the action layer would help in that your CTL case. In the case of like roadmap prioritization and planning, extending these actions into opportunity sizing, because a lot of things, what will happen is you'll generate hypotheses, and these hypotheses would have opportunity sizing. Do not leave this to your stakeholder. Have a version. Have an opinionated view of what are the opportunity sizes for this and why we should be doing,

you know, certain things, and be an advocate for it. Ultimately, the PM or the decision maker decides, but you're giving them sufficient enough information

to challenge you, and maybe, you know, you're objectively right, and it maps into their action. The third thing is repetition. Just because you mentioned it once, just because you presented it only, you know, a couple of times, it won't make it to the roadmap. It might be a patient journey over months for you to, you know, get into the decision making action layer framework. Ultimately,

any insight is no good if it is not action long because you're not getting feedback back. Maybe it's a good idea, maybe it's a bad idea. But you need to maintain a list of learning agenda,

and you are the advocate for influencing the learning in the organization. That's how I'd see it. I've had lukewarm success in this action area, even with this framework, but I'm feeling very optimistic over a period of time, having looked back on the compounding actions that I've taken by being a strong advocate for recommendations. As a corollary to being able to provide recommendations,

Tobias MaceyTobias Macey

there is the ever present challenge of trust where that is often used in the context of is the data trustworthy? Is the presentation trustworthy? Is do you trust the accuracy and

freshness of the data, which is where a lot of the focus of a lot of the technical exercises comes in of, well, I need to have data quality checks. I need to have metadata so that people can see when was this last updated. I need lineage to understand where did this data come from. I need documentation to say these are the business rules around the transformations and why they were done that way. And so in order for you to be in a position of providing recommendations of, okay, well, this is what the data says. This is the action that you need to take. There's also that prerequisite of building up trust in the work that you're doing and the ways that you're presenting it. And that is something that is hard to build and easy to lose. And I'm wondering how that factors into as well some of the strategic elements of how data teams should be thinking of whether and when to provide those recommendations of this is what it says. This is why you should take this action, in particular, if they're coming from a perspective of, well, I'm still struggling just to make sure that the data is even right. So I don't know if this is the right action to take and maybe how that can help to

how taking that perspective of I need to provide a recommendation can help to improve their ability to maybe short circuit some of that feedback loop of waiting until a data consumer tells you that something's wrong to take action against it.

Goutham Budati

Yeah. I think one of the reasons this framework is going to help and be very self serving is

if you yourself cannot be confident in the decisions to take make based on data, you will empathize with the stakeholder pain a lot more than seeing the pains of the stakeholder as, you know, maybe complaints or, you know, stakeholder is always unhappy. They are unhappy probably for a reason, and because you cannot replicate the decisions based on the data consistency that systems are providing. So if take a step back, there is obviously a phase of trust building where

you will discover a lot of the reasons why stakeholders don't trust data. And you, as an engineer, will go back, or the idea is would identify gaps in the technical infrastructure and try to strengthen the technical infrastructure. If you don't have a cataloging

platform, and that's the reason that people are not able to trace things back, There should be a bare minimum, you know, cataloging solution, either through open source or, you know, some of the vendors that are already available in the market. Or if it is because of the nature of your metrics changing over a period of time because you're not factoring in refunds and credits, and as a result, your, you know, subscriber numbers keep changing,

you would there should be an item to actually fix these things. This would actually help build robust data systems. And as you just said, it's easy to lose trust. And it's not just a function of data team. It's a function of every team. If you're not consistent in terms of your actions, and, you know, if there is data quality and data loss happening, trust will be lost easily.

So it's important to make sure that you are continuously providing reliable data, and at least a core metric. So I have one thing that I have formed for my own self is know your macros and micros. And there are very few metrics that you always need to know. And there are few input metrics that you always keep track when the macros move. Those should not change. And if you preserve the sanity of those metrics at a baseline, trust will be continuously captured, and you

would get permission to yourself to actually advocate for the perspectives. And if you're not advocating for perspective and actions because data is not right, it is a roadmap item for you to go and fix these things before you form these habits. That's how I would put it.

Tobias MaceyTobias Macey

And as far as those macros and micros, those are elements that engineers will be very familiar with in the technical domain of, well, I know what are the microscopic challenges of being able to process a thousand gigabytes a day, and I know that I need to have accuracy of the time stamps down to microsecond precision because of the rate of events that I'm processing.

But how does that translate to some of the business domain of the macroscopic challenges and some of the microscopic focus that people are going to put on the details that matter to them?

Goutham Budati

So on the engineering side, there are the technical, you know, details and the system robustness metrics and certain team goals that matter. On the business side, there are going to be the business metrics. Ultimately, every business needs to make money, and that is basically increase in terms of the revenue, keeping your costs lower, and your profit margins. And now you segment every metric by

the different types of customers you might be having. And every metric would have a certain baseline. Like every day, I would get certain number of conversions of sorts. This forms like a nice triangle of things to measure, which is your metrics.

Every metric has a baseline, and every metric can be cut by a segment. Now, these business macros, it's important to draw a lineage to how the data is being shaped. Every revenue is made up of different order level metrics and customers across different geos and how you collect this data. And working backwards from the business macros and preserving the, you know, robustness across every step in the, you know, transformation process

is probably the way to go. The way the business sees metrics is a little bit more, you know, working backwards, like looking at the business metrics and looking at the inputs. The way and and this is not blanket for everybody, but in general, technical teams look forward in terms of what we are collecting to how the metrics are being shaped. So having both these dimensions and both these views for an engineer would be helpful in terms of speaking a shared language. Because in the end, data,

if I were to oversimplify it, we are building a shared reality and shared intuition so that we can make the right decisions. That's what is the power of an analytical system and people are. That's how I would think about it in terms of having that two dimensional view, both from a forward out into the business metrics and then working backwards from business into the data side of things.

Tobias MaceyTobias Macey

And digging now more into the organizational structure that helps to encourage this collaboration and perspective building for technical folks as well as for the nonengineers to be able to build empathy with the challenges that the technical team is facing. What are some of the team structures and organizational dynamics that you have found to be most effective in supporting the technical team to be able to enact this framework and build more of that rapport with the business?

Goutham Budati

I played around with different org structures, and also depends on your budgets and how much you can hire. But one of the ideal structures I have found to work for me is data teams are divided into two parts. One is the build team. That is the engineering team, typically, the data analytics engineering team, bringing data into the warehouse, building the surfaces,

and making sure the surfaces produce reliable data in a consistent manner. And then the second part is the storytelling group, which is the analysts, the research folks bringing the qualitative data, as well as building this perspective. If I look at my team as a whole now, you have the team that builds data. You have a team that shares perspective and drives action with the stakeholders.

And there's always a duo here. One cannot operate without the other. You can't just have an engineer. You can't just have an analyst. Both needs to operate together. And in an ideal environment, you would have a central

build group with data engineers being the central horizontal part, with analytics engineers tightly coupled with an analyst on the business domain. So for example, marketing, functions could be larger, but marketing would have one analytics engineer and one analyst. And then product teams or supply chain would have one analyst and one analytics engineer, or a group of them, acting as deals, where you're not just providing data where data where the analytics engineering is focusing on the robustness of the system, and the analyst is focusing largely on the storytelling, the perspective, and the action side. That's my best structure, if

there is sufficient leverage in terms of creating the structure. When you don't have that, in more leaner teams, I would still follow this, but rather have these embeddedness at the storytelling level keep the build group a little bit more horizontal

so that they can focus on building the system. This typically happens at early maturity levels of the company, where data is just forming, there's no infrastructure, and you need to do a lot of large infrastructure build. And you also need to keep the lights on and, you know, go towards the ambition in terms of growing. And this typically happens at the fast growing Series B level. But when it comes to C, C and F, evolve into, you know, such structure of horizontal at the data platform level, but playing a duo of analytics engineers and analysts at the, embedded level.

Tobias MaceyTobias Macey

And now bringing this around to some of your experiences working in this fashion, what are some of the failures that you encountered that helped to inform this positive formulation of the framework where you said, okay. Well, that didn't work, so I'm gonna make sure that I don't do that next time by ensuring that I am engaging with the stakeholders to bring in their perspective and ensure that I'm building something that people actually care about.

Goutham Budati

I think I'm constantly surprised by the curveballs and

many failures I have along the way. And this is all fuel in terms of developing a better perspective and strengthening the depth of the framework that I have for myself, which is the data perspective and action framework. People change in companies oftentimes. Like every two or three years, people rotate, new people come in, and they wouldn't have the context that the old people had. So context building becomes such an important aspect. And how quickly you can onboard people, especially stakeholders,

as well as your team, really becomes important. If you don't have that, now you are very lean on the perspective layer, because perspective is built on context. And perspective can be leveraged only if you have the right set of, you know, actions you can take. So bringing people up to speed on that is very important. Two is, oftentimes,

people see data teams as service teams, and this is more true in more of the early stage companies. And shifting that is very important that you're the partner in organization. Even perspectives sometimes don't help. One of the learnings I had was data folks should become operators to a level that you're running these weekly meetings of metrics, or you're questioning

teams objectively in terms of the initiatives that people are taking so that there is objective dialogue that is happening. And plugging yourself in and being seen as an operator sort of helps, is a forming thought that I have right now with some of the many failures I have seen. Number three is clear writing.

I think writing narratives really helps. And I've seen in my experience that data folks oftentimes share links to a SQL query or links to spreadsheet or dashboards. But now you're leaving everything to a stakeholder to make sense out of it. But writing this perspective really ups your game. It immediately goes to 10x, and you have a way to make it 100x if you can actually build the right kind of relationships. I would also close this by saying seeking feedback is important. Just because you put things out there doesn't mean that people are reading. You need to force feedback coming in, and it can happen in bite sized meetings or maybe, you know, Slacks you send about, you know, what do you think about this? And then you're forming a shared language. So I'd say, like, I'm focusing on augmenting this based on the experiences. I'm actively putting it out there and, getting feedback by colliding this with reality.

Tobias MaceyTobias Macey

That aspect of writing things down and explaining them also brings up the interesting impact of generative AI and agentic coding tools and agentic data systems

that forces us to change the layer at which we're operating if we want to be able to maintain velocity and stay relevant as the ecosystem shifts. And that does force us to do a lot more natural language writing and explanation of the problem that you're trying to solve for in order to get the agent to do the thing that you want it to do. And I'm curious how you think about the impact of this shift in tooling, this shift in the degree of effectiveness and velocity

and how that forces us into this more into this mode of thinking that you're encouraging through this framework?

Goutham Budati

Yeah. One caution in the generative AI space is, I would say, maybe outsource your writing, but don't outsource your thinking. Writing for me is thinking, so, you know, I cannot outsource my writing as well. And how would you put this in perspective is very important. Oftentimes

and the second one is, you know, make sure your writing is actually read, and you get feedback. Someone should not be taking your writing and then putting in a summary and then consuming the summary of it. I think you need to write in such a way that people need to consume it raw. And this is important in the age of agentic AI, I would say. At some point, it would become pretty normal that people would stop sharing just the data, but add a commentary, add a summary, and that becomes a new normal. But the perspective layer is continuously

asking you to go beyond. Go beyond the normal. Punch a little higher. Make a little bit more emotional. As long as humans make decisions, you know, you need to make sure it appeals to the human and not the system. But if systems are making decisions, in the case of reverse CTL and all, you don't need to have an narrative of, you know, things that, you know, the system consumes. Obviously, you need to have metadata,

but not in a way that you're writing in to drive action. So that is how I see it. As long as you preserve the thinking space and operate in the thinking space by using your, you know, narratives with perspectives as a way to operate in the thinking space, I would see efficiencies gained in the Gen AI space. You would have better thinking partner. You don't need to ask a peer, get go. You could probably, you know, ask, you JGPD or Claude to, you know, fine tune it, find the blind spots, and go with a much more prepared narrative,

I would say.

Tobias MaceyTobias Macey

And as you have been developing this framework and speaking to other people about it, what are some of the most interesting or innovative or unexpected either implementations of the framework or outcomes that you have seen people achieve?

Goutham Budati

I think one of the things I've done early on was having different people write the same same, you know, problem or tackle the same problem. And I've seen the variance in terms of the, you know, data perspective action narratives that different people came up with. And initially, people saw this as redundant. Like, why are we doing this? This is a waste in the organization in terms of different people doing the same thing. But I think after spending about two to three months of doing this repeatedly,

we now see different dimensions of attacking the same problem. And I, quote unquote, being the chief editor of sorts, taking in pieces from different parts of the narratives and building a much more cohesive or a different kind of narrative and influence roadmap, it has given me a lot more ammo. So the

unlock here for me was, while people saw it as redundancy in the early stages, now people see the value of it because now we are expressing data very differently. We are talking about things in a way that it really resonates.

Insight resonates with the stakeholder, with the leadership, with everybody who are able to make decisions. And once it resonates, actions happen quickly. And it goes back to my validation that decisions are made a little bit with emotion and rationalized later. And this this kind of, like, fits in that narrative, but we are actively trying to be objective so we are making better and bigger decisions, I would say.

Tobias MaceyTobias Macey

What are the signals for an organization or a specific focus or role. Maybe it's something to do with the scale of the business where spending your time focused on the business perspective is a detriment to the core work that you're doing perhaps because you are working on a platform team and that core technical execution is really the thing that is going to provide the best impact. And just what are some of the signals of when to focus more on the technical versus the organizational?

Goutham Budati

I think someone in the platform team, I would this goes back to the structures again. I would strongly suggest they have a deal or a partner who they rely on for some of the business perspective sharing or give them some space when they are working on the platform related things to advocate for why this platform thing is important.

So finding partners is important. It could be a formal partnership, like your org structure allows for it when you have a PM or an analyst who's actually advocating for you. Or in leaner teams, you need to have these spaces where, let's say, a weekly business review or a monthly business review, you are connecting the dots. You're abstracting

what you're doing into the value of the business. Oftentimes, you could be saving, you know, dollars on, you know, your data warehouse and the systems you're building, and that's about 30%, you know, savings for this year. Or maybe you're making your system ready for scale for the next three new initiatives that the business is going to launch, and the time to develop at that point in time is going to be

cut down from three months to ten days. These are the kinds of value things that you absolutely need to be thinking no matter what, Because it helps in both ways. One is, are you doing the right thing? And also, next is, are you doing this thing in the right way, which is basically, you judge it from an engineering standpoint. But are you doing the right thing is always called into question. And if you practically hit this on, this is going to only help preserve

the, you know, data infrastructure well. I've seen projects get killed. I've seen good projects get killed because, you know, there's no narrative around it, or people don't understand why they're doing it. And if you just draw a line to the future on what the impact is gonna look like and if the company wants to do work on the future related things versus not, it's a good decision to be made.

Tobias MaceyTobias Macey

And so for somebody who is very focused on technical aspects and wants to change their level of impact, what are some of the immediate tactical actions that they should be taking to start to move further up the value chain or move further up in terms of the level of abstraction or visibility that they can have into the actual business problems?

Goutham Budati

There are structural things that the company can place to actually allow for this. And then there are no structural things in place for a growing organization. I would say, as an individual, find spaces

and find the people where you can share out who are not close to your work. It just flexes your storytelling muscle of what you're doing and what value you're generating to the organization. So this could be in the form of demos or sharing an update to a team that doesn't know what you're doing and seeing if they get it. I think this will really help you flex the storytelling muscle. Two is product market your work, I would say, which is essentially

showcase the value in the venues you already have. Every company generally has business reviews and roadmap planning sessions and reflections.

It's very important to not let someone else tell your story, but rather have a story coming from the work that you or your team is doing. And that could be if it's a team lead. Do they need to be representing it in the cross functional meetings? Or if it's your individual work? Performance reviews happen. Like, how are you going to tell your story of the value that you're going to provide to the organization?

So focusing telling the story by finding the spaces and finding the people to tell to would immediately up the game in terms of how you can turn on the perspective that you have within you. And then it's a matter of repetition and not losing patience, because it takes time. Like, again, repetition is very important. Doing it over a period of months will really help you see a different perspective within yourself.

Tobias MaceyTobias Macey

Are there any other aspects of this strategic formulation of how to be most effective and impactful as a data professional that we didn't discuss yet that you would like to cover before we close out the show?

Goutham Budati

I think, this formulation is similar to some of the things that other people might see in, you know, the predictive descriptive,

predictive, prescriptive approach, or, you know, what happened, why it happened, or what should we do about it. But this formulation really helps in terms of helping with the data, which is the core, which focuses on the systems to build and the infrastructure that you need to preserve, and then the perspective, which layers in not just everything from data, but also things from

your opinionated view, because those opinions matter. And bringing in the opinions, the hypotheses, and putting it out there is very important,

and then creating actions. But then the important thing to do is this is habit forming. If you don't form habits on a weekly consistent basis, you cannot break through in terms of getting to the perspective of the next layer. And this habit forming will help the organization build an operational rhythm towards generating insights consistently and tackling the bigger bets.

That's the hope. And formulating in this way and connecting it to the bigger bets and making sure that insights don't happen by chance, insights are not exception. They should be happening in a continuous rhythm, and you should be finding bigger, better bets is is what my hope is.

Tobias MaceyTobias Macey

Alright. Well, for anybody who wants to get in touch with you and follow along with the work that you're doing, I'll have you add your preferred contact information to the show notes. And as the final question, I'd like to get your perspective on what you see as being the biggest gap in the tool or technology that's available for data management today.

Goutham Budati

I think the biggest gap, from my opinion, that's available is goes back to my framework of enabling spaces to have good narratives beamed through the organization. It's not about just the hard data. It's also about what it means. And there is little tooling or little design solutions that are available out there that are very prescriptive.

So I would love to see something solved from a system design or org design or tooling design standpoint that can just help us. I think with AgentDK AI and generative AI, we are almost nearly there. So I'm very hopeful that this pattern would be solved for as well.

Tobias MaceyTobias Macey

Alright. Well, thank you very much for taking the time today to join me and share

the framework that you've developed and your thoughts around how to implement it and some of the ways that it shifts the focus and impact of the work that we're doing as data professionals. Definitely a very important thing to be considering as we progress in our careers and as the nature and shape of data work changes. So I appreciate the time and effort you're putting into that, and I hope you enjoy the rest of your day. Thank you so much.

Thank you for listening, and don't forget to check out our other shows. Podcast.net covers the Python language, its community, and the innovative ways it is being used. And the AI engineering podcast is your guide to the fast moving world of building AI systems. Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes. And if you've learned something or tried out a project from the show, then tell us about Email hosts@dataengineeringpodcast.com

with your story. Just to help other people find the show, please leave a review on Apple Podcasts and tell your friends and coworkers.

Transcript source: Provided by creator in RSS feed: download file
For the best experience, listen in Metacast app for iOS or Android