Product Management Essentials for Data Professionals

Links:

• Greg’s LinkedIn

Did you like this episode? Check other episodes of the podcast, and register for new events.

Transcript

• Greg’s background
• Responsibilities of Data Product Manager
• Understanding customer journey
• Interviewing business partners and decision-makers
• Products sense, product mindset, and product roadmap
• Working backwards
• Driving the roadmap
• Building a roadmap in Excel
• Measuring success
• Advice for teams that don’t have a product manager
• Conclusion

Alexey: This week, we'll talk about learning product management. We have a special guest today, Greg. Greg works as a technology manager at Amazon, where he builds AI products from the ground up. You've probably seen him many times on LinkedIn where he shares a lot of useful resources. Welcome to our event, Greg. (1:06)

Greg: Thank you so much, Alexey. Thank you for having me, letting me talk to you and lending the ears of the audience today as well. Hopefully, you're doing great on your side and that your year is off to a good start. (1:28)

Greg’s background

Alexey: Thanks for joining us. So before we go into our main topic of product management, let's start with your background. Can you tell us about your career journey so far? (1:43)

Greg: Absolutely. I started in manufacturing – I'm an industrial engineer. I also have a Master's in engineering management with a focus around supply chain and business. In manufacturing, I had the opportunity to rotate around different positions, whether it's supply chain ops management, quality/safety management. In manufacturing I also had a chance to be a product manager who was responsible for value chain optimization, and especially pricing optimization, which was very fun. That was before I joined Amazon in building some systems that are either AI-based or non-AI-based, which make sure to keep customers safe when they go online and purchase the products. (1:54)

Alexey: So you were managing actual products, I mean, physical products before that, right? (2:45)

Greg: Yeah. Before that, it was physical products and understanding the value chain from the time you put a strategy towards purchasing raw materials to putting a strategy behind where you want to manufacture these products – how much it costs you to produce them, which are the distribution centers, the price of the products, and the marketing strategy – all that stuff. All of these strategies in data that you're pulling – the role is to optimize the margin for all of these products given that you're creating value for the customers that you serve as a B2B company in the chemical industry. (2:51)

Greg: It was quite interesting to be able to collaborate with different groups from the commercial team, sales folks, and also finance, marketing, and transportation teams. It was good to be kind of in the middle, and to coordinate all of this – to make sure that the execution of each department was done on time and delivered exactly what the customer expected to receive or beyond. (2:51)

Alexey: So a product manager is not an IT-specific role, right? (4:19)

Greg: No. I would say, it even varies by department and not only by company. Instead of a company, there could be different roles a product manager can take. My latest role in the manufacturing setting as a product manager was developing what I call “data products”. What I call data products is kind of like a hub aggregator of data that provides certain services to its clients. My clients were internal – when you think about a group of sales folks who often struggle to understand where to get data, how to get data, and perform some quick analysis to make quick decisions or informed decisions. (4:24)

Greg: I would build key dashboards that were run on top of curated pipelines. That helped them build different things such as putting together a contract in less than five days, where they are able to explore the distribution routings, the transportation capacity, assess the clients’ demand over a period of time, forecast this demand, assess the pricing, assess the pricing of their competitors, work with marketing data to understand what kind of discount this customer needs to have, or even understanding what the historical data of that customer was – to understand what kind of new value proposition they can put on the table for that customer. (4:24)

Greg: The goal was to generate these kinds of contracts very fast so they don't lose the opportunity to book long-term contracts with these customers. For that, they needed access to curated data and to minimize the time it takes to transform that data in order to generate insights. (4:24)

Responsibilities of Data Product Manager

Alexey: That's quite a lot of things. You mentioned marketing, pricing, transportation, and then building this internal data product. Is this what a typical product manager does? Let's take a data team at a usual IT company, not a manufacturing company. Let's say the company has a website – could be an ecommerce website. What could the role of a product manager be in the data team of such a company? What are they usually responsible for? (6:41)

Greg: It's good to make the distinction between data product management and traditional product management. In traditional product management, there are a lot of data products that are being sold to external customers – that are being managed, developed, and implemented via the traditional product management principles and best practices. What I’m talking about, specifically, is data product management for products that are being served internally. In this case, your customers are sales folks, marketing, finance, supply chain – any program teams that can leverage these services, or these products, to consume and make decisions that generate value for the company. (7:13)

Greg: When we think about the traditional product management best practices versus data product management best practices, they have a common principle that they both share, which is to always start with your customers needs – always start with identifying the customers’ pain points, and then derive a strategy for addressing that pain point. This is where they both come together under that same principle, from which they can work backwards to figure out how to devise a solution for these problems – pain points – and also how they can build a roadmap based on agreed upon strategy that they can deliver for the customer in order to generate value. (7:13)

Greg: Most of the time, generating value means that the service or the agreement that this product has made is served with regards to expected service-level agreement. Whatever the problem these customers are facing, the mission of that product is to solve that problem. If you think about a sandwich, the mission of that sandwich is to fill up the belly to satisfy somebody's hunger, for example. After the customer consumes that sandwich, the mission is that hunger goes away. Just a simple example of the term. And if you look at the simple definition of a product by itself, it simply means “an artifact or a substance that is manufactured into a service”. So it’s manufactured into a final output that generates value for a consumer. (7:13)

Alexey: Basically, the role of a product manager is to identify the needs of a customer, think how these needs can be addressed, and then derive a vision/strategy. Then you said that they work backwards and eventually build the service that solves this particular pain point – this need. This whole thing – this service – is what a product is, right? It’s something that solves a particular problem for a particular group of people. (10:34)

Greg: That’s correct. (11:07)

Alexey: Then you mentioned that it is common for both traditional product managers and data product managers – both have clients and both solve problems, except that in one case, traditional product managers solve problems of external clients, while data product managers usually address the needs of internal clients. Is that right? (11:08)

Greg: Correct. For example, in data product management there could be a recommendation system, which is something that I would call a data product. A recommendation system is often used by external customers. Therefore, a traditional product manager is capable of managing a recommendation system, given that this product manager gathers specific skills to manage these recommendation systems. Let's explore some of the skills in general. You're talking about designing skills, strategy, analytical skills, and technical skills – in other words, skills that let you become able to manage a data product for external customers and to be able to have conversations with engineers from a technical design/technical implementation or architecture perspective. (11:32)

Greg: A recommendation system can be managed through traditional product management, but a leg up that traditional product management has is that there is this UX design aspect of it, where, for example, these products can be served on the website, they have a good design on the website. Whereas internally, when you think about pure data products, the UX design is probably lacking – meaning you're collecting all of the data, aggregating them under one roof, and then serving them to downstream clients who can consume that. Think about how you would design a user interface for something like this. There are limitations. There's an opportunity for internal data product management to invite UX designers to understand how to design that user interface for them. I'm hoping I'm making the distinction between the two. (11:32)

Understanding customer journey

Alexey: With all these responsibilities – regardless whether it’s internal or external – it's quite a lot of things to do for a product manager. In your opinion, why is it useful for data professionals who are not product managers, such as data analysts, data engineers, data scientists – why could it be a good idea to also learn a bit of product management? (14:03)

Greg: I think it helps because it prepares your mental model for always starting at the end – with the end in mind. “End in mind” meaning that you’re envisioning a “North Star” service or product that addresses key pain points for your customers. With that mental model, you're capable of figuring out “Who are the stakeholders that allow you to fulfill this mission?” With that, they are also capable of understanding the customer journey, which is a key part of designing a solution for your customers. (14:30)

Greg: What that gives you, as a data analyst or data scientist, or any other data professional, is the ability to acquire domain knowledge. If you're able to express and understand the customer journey for a data product, served on web, for example, being able to design express details of that customer journey from the time they enter the website, navigate through, click the different products until conversion – meaning until they accept to purchase the product or service and consume it and feel satisfied – being able to map that out is something that will make you a more powerful data analyst, a more powerful data scientist, or a more powerful data professional. Being able to acquire these skills can only help to bridge the gap between technical and non-technical teams. (14:30)

Alexey: What is a customer journey? I think I understood from your description that this is a journey that a customer makes. So let's say they want to purchase a pair of headphones – they go to an ecommerce website. First, they need to find what they need, – wireless or wired headphones, big ones, small ones. So first, they look for what they need, then maybe look at reviews and then finally they make a decision, click the “Buy” button, and then enter their credit card details. A few days later they unpack their thing and start listening to music. So that's the journey, right? (16:40)

Greg: That's the customer journey. Exactly. If you're the product manager for the website that serves all these products – headphones and things like that – whether you're a product manager or a data professional, you need to understand that customer journey. You can design key solutions, once you identify those pain points, or the roadblocks that would prevent a customer from purchasing your product. (17:22)

Alexey: How can I understand the customer journey better? Let's say as a data scientist – what do I do for that? (17:54)

Greg: Obviously, you will not be able to be on the grassroots of things where you're deep diving all the time. So that's why you need different teams. The best practice is probably to do interviews. Most of the time, data scientists don't have access to the external customers, or the final consumers. Therefore, it's best to interview your business partners, the product managers, or the department heads, business analysts, etc. to help understand the current standard operating procedures or operational steps when transactions are made for that product. (18:01)

Greg: That's probably the best way – read documentation about these processes – at this point, you will be able to gain knowledge. But one of the most powerful tools that people often forget to use, which is available to us all, is the will to ask questions, or rather people forget to ask the right questions at first. Asking the right questions comes with experience. When you're new in the field of data science or anything like that, I always find that those who become very powerful, very effective, and very efficient, are the ones who are willing to ask a lot of questions so they can learn and adjust and discover. This is a soft skill that's definitely needed when it comes to discovering domain knowledge. Good data scientists, or good data professionals, are good detectives. That's probably the best way I can think about that. (18:01)

Alexey: So you need to be a good detective and find business partners, business analysts, product managers, domain experts – and ask them questions. It doesn't have to be great questions. You just need to ask something to understand the topic better. With time, you will learn how to ask the right questions. (20:04)

Greg: Absolutely. (20:26)

Interviewing business partners and decision-makers

Alexey: You mentioned that people need to interview. How would you go about interviewing these business partners – how would you structure this? Let's say we want to set up a meeting just to learn more about the customer journey, more about operations. First, you just find a domain expert and then schedule a meeting with them. So how do you structure this meeting? How do you structure this interview? What do you ask them? (20:28)

Greg: What I would advise in that case is, again – start with the end in mind. It's not good to just schedule a meeting without a structure. That's why I like this questioning strategy, because you want to set a meeting with purpose. What is the purpose of setting up the meeting? Are you setting up a meeting to simply learn the general operations of the business? Or are you trying to surface the pain points of the business? Or are you trying to learn what the key goals are and then setting up a framework for identifying what the obstacles are that could prevent you from reaching these key business goals? Or are you trying to surface opportunities that will create an uptick or an increase in those metrics that help you reach those business goals? (20:58)

Greg: Start with the purpose of the meeting. Most of the time, it's to understand what the business is trying to do, which is understanding what the business goals are. Then you need to create a list of questions to identify the different obstacles that you're experiencing – that are posing a threat to reaching these goals, or even identifying the current limitations you have that could represent a threat to reach these goals. Then these data scientists can go back and figure out how to generate, or deploy, new features that allow them to reduce these limitations, add more capabilities, or remove obstacles so that these goals can be met. (20:58)

Alexey: How do you identify these obstacles? What kind of questions to ask? Is it like, “Okay, tell me about your day? What are you struggling with?” Or what kind of questions should be asked? (23:07)

Greg: Yeah. That can be done through some specific use cases, right? One of the most famous ones could be, “Well, last year, we made a million dollars (for easy numbers) and last year, around the same time, we're already at a million dollars in the year. This year we're down 30% compared to last year. We're not making money and we're trying to figure out what's going on.” There are a lot of frameworks, and me being from manufacturing, I typically go with the “five whys” when I interview people. “Well, why do you think that's happening?” “Well, customers are leaving us. They're not going to the website anymore to purchase.” Asking those “five why's” allows you to gather preliminary answers from the business, which most of the time I think data professionals can take with a grain of salt. (23:20)

Greg: Business folks tend to leverage their intuition and their experience about the market to kind of deduct certain things. That is if they didn't have time to deep dive into data. This is where data scientists, or data professionals, can go back into the data and kind of reconcile these hypotheses, or make correlations between what they've first heard about the problems. With that, that could be a reason why customer churns are happening. With proper design skills, they can understand how to address this or find the root cause. (23:20)

Greg: For example, for customer churn, they can find out whether it was due to a feature that was released on the website that caused the customers to be confused, and miss clicking the “Buy” button since that feature launch, which has caused a reduction in sales. So all of these “working backwards” tasks help you identify what happened. Maybe it was an external event that happened. For example, let's say that COVID was the event that caused a downturn in your sales. Being able to work backwards to figure out that root cause is what data scientists can do when they're analyzing data. (23:20)

Products sense, product mindset, and product roadmap

Alexey: Yeah, thank you. We have a question from Peter. Actually, I have a similar one. The question from Peter is, “What is product sense?” What I would add to this is – is it similar to the product mindset? To me, they seem like synonyms “product sense”, “product mindset”. What are these things? Do you know? (26:25)

Greg: Product sense? I don't hear that term often, but I have an idea. There's no firm definition for this. Product sense to me is a mindset, I would say – the mindset of being able to understand the characteristics and limitations of your product with regards to your customers’ needs and being able to derive a roadmap and strategy that address these expected and unexpected needs of your customers. (26:49)

Greg: It’s about understanding the trends of these products, the trends of the market, the trends of your customers’ behaviors – because needs change over time. A product sense to me is more of a mindset that entails different things from understanding the external and internal forces that drive the performance of your product. (26:49)

Alexey: This is something that everyone can have? It's not only a thing that product managers can have, but everyone on the team, every data professional, non-data professional, frontend engineer, backend engineer – everyone can have that. Right? It's about understanding the characteristics of your product and understanding the limitations. You also mentioned defining roadmaps and strategies. This is also part of that. So how, as the data professional, can I take part in defining a roadmap? I assume this is usually what a product manager does. So how can I help my product manager work on the roadmap? And what do I need for that? (28:05)

Greg: If you're a data professional, you're on the business team as a data analyst, a business analyst, or you're on the tech side, even an ML engineer or a data engineer, etc. Since product roadmaps are led by product managers, the best practice is involving all your stakeholders into this. You need to understand whether you will be able to deliver what you're promised to deliver. That, especially the technical side, will be responsible for “T-shirt sizing” the efforts it takes to deploy the key features of your product. (28:53)

Greg: As a data professional, the contribution that you can make is to provide that “T said sizing” in terms of effort. For example, if a feature requires the build of key data pipelines that require real time inference from machine learning, a data professional on the technical side is responsible for providing advice and aligning on timeline to ensure that the underlying infrastructure is either present or there's a strategy for establishing that infrastructure which gives you those key pipelines, which in turn enable these features that you want to release in an XYZ timeframe. (28:53)

Greg: When you think about it, it's definitely something that data professionals are already doing. It's a matter of whether your organization is applying these best practices where product managers are inviting everybody at the table to provide their opinions and advice on how to move the needle. (28:53)

Alexey: What is “T shirt sizing”? Is it taking something big and complex and chunking it into smaller things? (31:14)

Greg: “T shirt sizing” is simply providing “How long does it take? What is the cost of me spending time to build this? How long will it take me to build this?” etc. Basically, it's determining the effort that it takes to build something. (31:21)

Working backwards

Alexey: I guess another thing – I'm not sure if we talked about this – but you mentioned it a few times that you should start with an end in mind. I guess this is something that we can also do. Let's say, when we discuss any feature, we can say “Let's think about what kind of problem we're solving for the customer.” Right? Then we work backwards on that problem, instead of trying to come up with a solution that is technically interesting, or good, but doesn't really solve the problem. (31:45)

Greg: Exactly. If you think about a product manager, a data scientist’s customer is the product manager in this case, right? When the product manager comes up with this roadmap which has a list of features that are prioritized based on the ones that speak closer to the business goals, then, the data scientist needs to work backwards from this – to figure out how that solution will be delivered. It’s not up to the product manager to dictate how that solution will be delivered. (32:15)

Greg: For example, as a product manager, I cannot go to the engineers and say, “Use this tech stack and use it to deliver this feature.” It should be the engineer’s responsibility to tell me what the best solution to achieve this is. Their role is to gather the requirements from me, meaning “This feature requires high availability, high concurrency, low latency, etc, etc.” And then they figure out how to design the final product for me for the final release. (32:15)

Alexey: Okay. So you would say that a PM is more problem-oriented and a data scientist is more solution-oriented, right? (33:37)

Greg: They're both solution-oriented, if you think about it. A PM is solution-oriented for their customer and a data scientist is solution oriented for their customer, who's the PM. A PM is solution-oriented for their customer, who’s the external consumer. So it depends on which angle you're looking at this. They're both working in tandem to provide a solution for the ones they serve. You can abstract that to any position. Whether you're an accountant, or a supply chain manager, or you’re a buyer, or a planner of production – you're serving a downstream customer. And you can create a roadmap. (33:44)

Greg: This is why when you think about a roadmap, to me, that's one of the most powerful skills. Knowing how to build a roadmap is a very powerful tool to use for career advancement. You don't have to be a product manager to be able to build a roadmap for yourself as a supply chain manager, for example. That shows leadership – that you're thinking beyond your daily activities. You're thinking about the future transformation of your business. And that’s what gets you promoted, because you're thinking about future transformation. You’re thinking about improvement. You’re thinking about the growth of your department. It's a great skill to have. It's a great tool to use. It shouldn't be limited to product managers only. (33:44)

Alexey: So how can I do that? How can I build this roadmap? Should I maybe ask my product manager to help me? Or how do I go about that? (35:34)

Greg: Now, when I say “roadmap,” a data scientist building a roadmap is different from a product manager building a roadmap. A roadmap built by a data scientist and their group members – that’s going to be about building the underlying infrastructure that helps them better serve their clients. For example, a group of product managers may come up with 10,000 business use cases that require some sort of ML algorithm to address these use cases. In a sense, you can have a group of data scientists who strategize on how to implement certain frameworks, in this case, to keep up with all of these demands. (35:43)

Greg: What I call “demands” are – all of these business use cases are coming out, these product managers are coming to knock on their door and saying, “Hey, I need help.” That's a good use case for building a roadmap for ML Ops. So figuring out all of the different components of ML Ops, assessing where you are today, and where you want to go tomorrow, to fully deploy an ML Ops framework that allows you to keep up with all that demand. That ML Ops framework provides you with the right supply that meets the demand of the business teams. (35:43)

Greg: Again, a roadmap for the technical side may be different from a roadmap for a business non-tech side. But it's the same process with regards to having to focus on the customer in mind. You don't want me to come up with a bunch of requests and you can't answer to any of these requests because you don't have the underlying infrastructure to support my demands, right? So you gotta have a plan to be able to respond to this increased capacity here. Again, I'm thinking about manufacturing. (35:43)

Alexey: What I also heard from you – I don't know if it's the right summary or not – but let's say people come to me as a data scientist with problems, and these are usually product managers, or maybe product analysts, or other people. So they come to me with problems and for me, the ability to generalize all these problems and then see the bigger picture, and then put all these problems into a roadmap and say, “Okay, with this thing we can solve this bunch of problems. With this thing, we can solve this bunch of problems.” And then you see the bigger picture. This is what I mean by coming up with a roadmap, right? (38:26)

Driving the roadmap

Greg: Coming up with a roadmap is kind of like understanding what strategy you will use to serve your customers. Meaning “Are you going to deploy a project or assign a group of tech folks to each product project that your customers come up with? Or are you going to build a framework that can scale to address these needs? Do you have a process in place to understand which of your models can be reused or repurposed for different business use cases?” Versus having to go through the long cycle of project management. All of that is about, “What is the vision for your data or tech department?” To serve its client. How you build that vision is by identifying the issues, or the obstacles, that you've seen along the road when you're working through an unscalable way. (39:01)

Alexey: And what is that? What is this non-scalable way? (40:22)

Greg: When I say “unscalable” I mean the “manual” way. If you're having to allocate time, effort, people, resources, for each business use case that comes to you – which is a long cycle of Project Lifecycle Management – versus having an ML Ops infrastructure that allows you to quickly build, test, deploy models to shorten these project lifecycles. So you'll have to draw that roadmap, which helps you get to that maturity level. (40:24)

Greg: Because today, you may make an assessment that helps you identify the gaps – whether you have the right professionals, whether you have the right tech stack, whether you have the right skills to manage these tech stacks. You need to decide, inside of that roadmap, which of these are most important. You need to go after this by prioritizing until you get there. So it's all about drawing the roadmap to reach a maturity level. (40:24)

Alexey: I think you said “driving the roadmap” multiple times. Let's say I work in a team that has a product manager, and a bunch of other people. So then “driving the roadmap” of this team means taking active part in discussions when we talk about what we want to solve next, like, “Okay, I think this is the problem we should be focusing on right now” and being able to defend this position – to convince everyone on the team that this is indeed the problem. So, “We as a team should focus on solving this problem in the next quarter (for example).” This is what it means to drive the roadmap, right? (41:44)

Greg: To me, a roadmap should be a three-year plan, never less or never more. It should be a three year plan because something that is a roadmap for five years – that’s quite loose. Nobody really knows what's going to happen in five years in terms of trends. So you have a better sense of the future trends or within the next three years. And then inside of those three years, you kind of prioritize what you want to go after. For example, if your biggest needs instead of that roadmap are “We need more people. We need more specialized folks. We need more data engineers, because we cannot keep up with these demands to build key data pipelines that serve different purposes. We need to hire more and better qualified data engineers.” Or “We need to create a strategy for putting together better training materials for these data engineers so they can keep up with the industry, etc, etc.” (42:27)

Greg: You identify the limitations that you have and then prioritize the solutions to these limitations over the next three years. You prioritize by what you need the most to what you need the least. You also prioritize by how much effort it will take, whether you want to buy third party software or you want to build it from the ground up, the cost, the effort, and the impact. The impact is a big factor, because maybe hiring more data engineers gives you a better return on your investment in terms of being able to create pipelines that serve downstream customers. Again, a roadmap is inside of three years and accounts for impact, effort, cost, timeline – so you can strategize on what you need to focus on. (42:27)

Alexey: Would you say this is more like a project management skill or product management skill? (44:40)

Greg: It's a product manager skill. (44:45)

Alexey: So when I think about things like cost, impact – all that – to me, this is like a project that you need to finish. This is different, right? (44:47)

Greg: This is different. Think about a roadmap as the future transformation of a product, a system, or a department and the strategy that helps you get there – from point A to point B. (44:55)

Alexey: A three-year-long plan seems like quite a while, right? So how would you go about that? Like if I don't even know what will happen in one quarter or half a year – how do I plan for three years? To me, that's quite a long-term plan. (45:20)

Greg: Yeah. To me, that's more of a “think big” kind of thing. Then don't forget – you have a three year roadmap. If it's more than five years, then you're kind of shooting into the air with no aim. Three years is a good estimate of what the future needs are – it's being able to anticipate where the trends are going and being able to anticipate what the future needs of your customers will be. That gives you a good leg up. And remember, a roadmap is not a static strategy – it's an adaptive strategy. (45:37)

Greg: You give yourself three years, and once you have alignment on this roadmap, you start working at year one on what you decide is more important. But things change in the market and your roadmap needs to be up to date. A roadmap that you write for today – for the next few years – will change next year, most likely, because the needs of your customers will change with it. So don't think about the roadmap as a static document, but more of a dynamic one that provides flexibility and even proactiveness – being able to anticipate what your customers will do and what their needs will be. (45:37)

Building a roadmap in Excel

Alexey: How can I best learn these skills? There is actually a question from Patricia “How can we best develop our roadmap-building skills?” Is there a good resource, maybe there is a product management course that we as data scientists can take in order to learn these things? Or do we just need to watch how product managers do their job and learn from them? (47:18)

Greg: I don't know of any specific product management courses. You can probably YouTube or Google, “How to build a product roadmap” and kind of pull the skills from this and apply them to your department. But I'm not sure that there is a set way of doing so. You can come up with your own framework for this. For me, it's about starting with the end in mind – which is “What is the problem that you're experiencing right now?” (47:46)

Greg: List all of your problems in an Excel sheet, where each row has the list of problems and the column heads are “the problem”, “the driver of this problem” or “potential root cause of this problem”, then you can have another column called “solution” or “potential solution”. Then after the solution, you can have “justification”. After justification, you can have “stakeholders that will be affected”. Then you have “impact” – what is the impact of that solution? Is it customer experience improvement? Is it an uplift in sales that the solution gives you? Then you have a column for “effort” where you can estimate it – if you're already on the tech side, you can better understand what the effort will be for that solution. Then you have SMART goals. The SMART metric for that solution is time bound and measurable. That’s another column that I would add in a roadmap. Then the last column, I would say, would be “prioritization.” Is it priority number one? Number two, number three? (47:46)

Greg: Then once you fill out all of that Excel file, you can filter by priority – where you prioritize P1s first. Then in terms of that SMART goal, you will know when you need to deliver based on the effort. There will be trade-offs – there could be something with high impact that may take six months to build. Are you going to prioritize that versus prioritizing three smaller effort solutions that when combined can give you the same impact as working on one that can give you a big impact. So you have to understand how to make trade-offs between all of the things in this list. So there you go – I just created a weak framework for building a roadmap for your department. (47:46)

Measuring success

Alexey: Yeah, thanks. That's useful. Also, there is a question from Elena about setting up metrics for an internal data platform. I think that's also interesting and this is something that I think product managers also do, right? Let's say we have a problem that we want to solve. Now, we need to think, “How can we actually measure that we are solving this problem?” Is this something that product managers do? (51:11)

Greg: It's a product management 101, especially in designing solutions for their customers. You want to identify the affected customers and the pain points. You want to brainstorm what that “Northstar” is for these pain points – meaning the vision of a product that would solve these pain points. Also you want to describe exactly what that question is asking – describe what the success criteria is. Success criteria are where you embed success metrics. (51:39)

Greg: For you, a solution may be that you want to reduce latency of the data pipelines that you serve to the downstream team, for example. Or you want to reduce the SLA (service level agreement) for your teams. You want to increase engagement on your website by 10%. You want to reduce customer churn. Product management design 101 – success criteria. Success metrics are key to monitor and validate that the solution you've implemented was effective. (51:39)

Alexey: This comes to the SMART framework that you mentioned. The M there is “measurable”. So we always need to think about a way to measure that we're actually solving this problem, right? (53:15)

Greg: Correct. (53:26)

Alexey: I think there the question was about internal data platforms. And I think, I can imagine myself, “What kind of problems might I have with a data platform?” Maybe there are data quality issues, right? And then, “How can we measure that?” Well, fewer people complain, for example, or some other ways of measuring that. Then it can become your success criteria – fewer people complain about quality. So we can track complaints and then after half a year – if there are no complaints, we can call it successful. (53:27)

Greg: Also pipeline failures, for example. “Are you experiencing less pipeline failures? Are you addressing your pipeline failures within the service level agreement that you're established? For example, you may say, “High priority business-critical pipelines – when they fail, you have a service level agreement of 48 hours.” So your success criteria is anything 48 hours or under – you can use that as a metric. You can say “98% of your business-critical pipeline failures will be addressed inside of that agreed upon service level agreement.” So with that, 98% is your success metric. So you need to do that or exceed it. That's another metric for internal data platforms. There are many others that you can think of, but that's one easy one that I can think about. (54:02)

Alexey: I guess when you do this exercise that you mentioned for coming up with a roadmap, you do the Excel spreadsheet – if you fill in all the columns, it will give you a lot of great ideas for what the right metrics can be. “How can we ensure that we're actually solving these pain points?” (55:09)

Greg: Correct. (55:31)

Advice for teams that don’t have a product manager

Alexey: Another question I have for you is – I know that there are teams who do not have product managers, (for some reason or another). I have worked in such teams at some point. I know that what can happen in such a team is that people work on solving problems that don't exist. I have tried solving problems that don't exist and then I was surprised why my solution is not used. Right? (55:32)

Alexey: Do you have any recommendations for what people in such teams can do when they don't have product managers who can tell them, “Hey, you're doing something that is not useful”? How can we make sure that our projects in such teams are more successful? (55:32)

Greg: Yeah. Whether you have a product manager or not, you still have a customer. If you're a data science team or a data team, you're doing some work for somebody downstream to consume. So, the first thing I would say is identify those folks. Identify the business decision-makers or the decision-makers of the departments that you serve. Who are the decision makers? Identify them and interview them. Surface these pain points, and then brainstorm ideas to tackle these pain points. Use data to brainstorm ideas, and use data to solve these ideas. That's probably the most powerful thing that data scientists have, right? They use data to explore and they use data to solve. (56:14)

Greg: At the end of the day, depending on the company culture, you may find that you don't need the middleman – you don't need the product manager to guide you in this journey. You just need to understand who your customers are – who you serve – what they are trying to solve and how to solve them. So best practice is – get in touch with the decision-makers. (56:14)

Alexey: Yeah, I remember a PM who I worked with in a team who said that he would call himself successful if a team no longer needed him – that the team can autonomously function without him, come up with all these roadmaps and all these things. I thought, “That's an interesting thought.” If a team no longer needs him, what is he going to do? But that's an interesting thing. I guess he will move on to a different team that maybe still needs him to help? (57:44)

Greg: Yeah. I think what he or she said is more of a mental model piece. When the technical team no longer needs a PM, that’s because the mental models are aligned and that makes it an even more powerful team that tackles true problems and works on the right things. So the mental models are aligned on the right things. (58:15)

Conclusion

Alexey: Yeah. Okay. I think we should be wrapping up. Do you want to say anything before we finish? (58:42)

Greg: Yeah. I guess my message is – wherever you are, be open to learning different things. I think you may hear more conversation about data professionals, or data scientists, ML engineers, software engineers, who want to transition to product-based positions. You don't have to wait until you're ready to make your next career move to learn about the skills. You can learn about these skills on the job – at your current job. Being open to learning a skill outside of your immediate responsibilities, or outside of your skill set, makes you that much more powerful at your current job. And it gets you promoted faster. Be open to learning something outside of your daily responsibilities. Hopefully everyone who listened here takes that as the inspiration to have a great career. You're in charge. Don't forget to ask when you want to get from point A to point B, whether it's in your career, or whether you want to understand the pain points of a customer – ask. Again, best of luck. If anybody wants to follow me on LinkedIn, I'm happy to answer additional questions. With that – thank you, Alexey, for your time and for hosting me. Hopefully, the audience and you enjoyed the conversation as much as I did. (58:49)

Alexey: Yes. I see in the comments that people are thanking you for sharing all these amazing tips. I’m joining them in thanking you. Thanks a lot. The other question I was going to ask is how people can find you. I think you answered that – LinkedIn is the best way. Great. So, thanks a lot. Thanks for joining us today. Thanks, everyone, for joining us today also – for asking questions. Have a great rest of the week. (1:00:47)