What is product management – from a data-driven perspective?

I was trying to find a definition that focuses on a data-driven perspective.

Of course, you can look up a definition for product management in the different textbooks concerning product management.

I would also assume that, if you go to a product management conference and you ask 30 people, you will most likely end up with 20 different definitions of product management.

And those definitions can differ very much from your own (internal) definition. And I think that’s a natural thing. Product management, in comparison to other business areas, is usually a field, where people come from different areas and bring different experiences into the job.

You have the possibility to learn about product management at university, and you also have the possibility to start a (product) management career right off, and you immediately get into your first job. But this is a rather new situation. Product management is still a very young discipline and most people who have been in product management for 10 or 15 years already, have had different backgrounds (before they got into product management).

My background, for example, is a more technical background. So, I studied computer science and economics, and I have always been focusing more on technical projects (before I went into product management). Well, my style of working in product management is usually rather a technical one and not, for example, that of a different product manager who was in an UX job before.

Usually you will find people in product coming from a UX background, a PM background or a tech background but rarely from a data background.

If you take me, I had a technical and economics background (controlling) when I got into product management. That explains a bit why data and analytics played an important role in my product management setups. But it was not a defining one.

After my product jobs I have been working now for 8 years as a data consultant and now coming back to work on products again.

So I am taking this thought experiment now on how my views on product development have changed after spending 8 years in data.

So does a product management approach change, when you are exposed for 8 years with data-driven approaches.

Spoiler: it did change significantly.

Don’t miss my next Building Data Products post

Get my new posts automatically delivered in your inbox. And get exclusive access to my data products library I am currently building (huge Miro board).

We respect your privacy.

So what has changed? My initial definition of product management was highly influenced by Marty Cagan and his famous book ‘How to create tech products that customers love’. That was my initial motivation why I went into product management. I wanted to end up with products that make the life of their users easier in some way.

So that it enables them:

  • to do things that they couldn’t do before,
  • or that they could save a significant amount of time
  • or that they could get rid of tedious tasks
  • or that they just had some fun

And to be sure, that is still my core motivation when building products.

The changes after working for eight years in data, are that I started to question the impact of product decisions.

So what has changed?

During my eight years as an analyst and as a data consultant, I have worked with a lot of companies and helped them to set up a data framework (a data stack, technically speaking).

And the reason why we did this and why we invested so much time into this, was that we wanted to make better decisions for the product and marketing.

Especially product analytics is an interesting area and it is the field I love most in analytics because it’s challenging in comparison to marketing analytics. Marketing analytics is a lot about how different marketing activities convert. The most complex thing you can do here is how to apply a proper attribution model on your marketing data (so that you can make the right decisions about your marketing channel performance).

Product analytics is more about figuring out which parts of the product make the user stick to it and love your product.

So what are the features that are important for the user after signing up for your product?

If you, for example, are responsible for a SaaS product you want to find out which features are so essential for your users, that they will not stop using your product or switch to another one.

In fact, getting this information simply from tracking data, is a very hard task. So you often combine product analytics with quantitative and qualitative data analysis tasks. And this is the fun part to figure out how to do this in a very efficient way. One of the learnings I took from it was: The number of features (that a product team releases during a year) has no significant impact on the data.

And this was a revelation to me because, if you work as a product manager, you’re usually so focused in the process of of product/features out of the door.

The process usually looks like this:

Initially you research in the problem space. Usually using interviews to identify problems and pain points. Sometimes you can also get these from the analytics data.

When you know more about the problem you come up with possible solutions to the problem. You then have to decide which of the solution has the best match of effort and chance of solving the problem. This can also been done with data (smoke test) but often is done with UX research (paper prototypes) or even more often based on experience of the product team, which quite a good way in this early stage.

Then you develop a first initial version of the solution and deploy it (in best way with an A/B scenario).

Now comes the essential part (at least the one that is different when you work in a data-driven way).

In data-driven product development you would define the KPI which this feature should impact. And you should give a forecast about the impact (best impact prediction training you can get for PMs).

Once the feature is deployed it is closely monitored by data (and yes, this means the tracking is in place from day 1). And not only for 2 weeks but for at least 6 months.

What you want to learn is, if it really has an impact on the KPI you have defined and how your forecast works out.

And now comes the important part:

If you don’t see an impact on the KPI after 2-3 weeks, you need to investigate. Potentially set up interviews, make data deep dives. You need to figure out as soon as possible if this feature is worth in your product.

If it turns out that this feature does not solve the product, awesome! You can cross this out and make for the other solutions. But then get rid of it.

Products slowly die by features left in them even they don’t make the difference.

But what about small incremental improvement? Isn’t it worth to invest more to improve the feature.

In some cases, yes. In most cases, no. You usually should see a small amount of impact in the initial data or you will hear positive signs when conducting user interviews. But I admit, it needs some experience.

But doing this step by step, step by step incremental improvement product development costs you so much energy, time and resources that might be better applied somewhere else.

In a lot of products, where I analysed the data we found in the end, that there were only very few features in a product that basically were defined from the user’s standpoint: as the essential features (as the reasons why they use the product).

I would even go so far and say that some products might perform in a better way, if they are reduced to these core features (that the users have seen as the essential features).

What I saw very often (especially with growing product teams), you start to extend here, extend there a little bit. And in the end, you increase the UX complexity of your product, you increase the technical debt of your product and you make it harder for users to understand and to define why the product is really solving the core problem.

Don’t miss my next Building Data Products post

Get my new posts automatically delivered in your inbox. And get exclusive access to my data products library I am currently building (huge Miro board).

We respect your privacy.

This is my essential take-away looking at product management after spending years in data: A lot of stuff we do in product management does not improve our products. These are hypothesis that turn out not to fulfil the promises we put in them. We have to become a lot better to identify these.

This is where data-driven product development helps so much.

#1 Define ‘the three to four core functions’ of a product which will solve the users’ problems. 

#2 Come up with events/actions that you want to define for these core processes. Track these properly. And define KPIs, Funnels, Segments and Cohorts based on them.

This gives you a very clear set up of KPIs, that are constantly monitoring the quality of your core product benefits.

With this in place you can start to research extensions of the product. If you follow the data-driven way described above you make sure to not decrease your product quality:

Define the problem, come up with possible solutions, define the improving KPIs, forecast them, test them, monitor them, see if they show signs that they improve the problem, if not sunset them.

This whole process can lead to something that I call ‘a slow product’. And I know it is quite a ridiculous term for it, but basically, it comes a little bit from the context of slow food.

Don’t miss my next Building Data Products post

Get my new posts automatically delivered in your inbox. And get exclusive access to my data products library I am currently building (huge Miro board).

We respect your privacy.