Metrics & Analytics: Crucial Product Management Skill 10/20
12 December 2024 ‧ 3 min read
This post is part of a series of short articles about the 20 most crucial product management skills.
What it means
Metrics & analytics skills refer to understanding and being able to define key metrics for the product and individual features, as well as hypothesizing metrics impact of product changes. Product managers skilled in metrics & analytics are able to independently access and analyze data through the available tools (e.g., product analytics, BI, spreadsheets, SQL).
Why it is an important skill
Analytics allows objectively measuring the success of a product. While qualitative feedback from customers is invaluable, it is also easy to get biased by the loudest voices or by your own cognitive biases. Numbers, on the other hand, don't lie. If customers talk to you about the product with enthusiasm but they are not engaging, numbers will tell you that. If you are trying to optimize onboarding success, conversion rates, or retention, understanding how to measure the baseline and if you are improving is the only way to make sure that you are improving things (and not making them worse), especially given that most product improvement efforts fail.
Numbers can also be a great communication tool, especially with parts of the organization that are more focused on numbers, eg. Finance or Performance Marketing. Being able to prove that certain things are working (or aren't) is very powerful.
What great looks like
Product managers with great metrics & analytics skills understand the key metrics for their product and their product area across the customer lifecycle, from acquisition and activation through engagement and retention to monetization. They are keeping track of those metrics on a regular basis and are sharing relevant insights with their team.
When they are making changes to the product, they collect relevant data as decision input, hypothesize about the input of changes on key product metrics, and are measuring the impact (eg. through A/B testing). They also know how to design A/B tests in a way that they have a strong hypothesis about how the change will impact user behavior, and that they can even learn from if the hypothesis is invalidated.
They are also self-sufficient at using the analytics tools available to them and running their own analysis – be that through product analytics tools like Amplitude or Mixpanel, BI tools like Looker or Tableau, writing SQL queries or creating spreadsheets, they don't get blocked waiting for someone else to run the numbers for them.
They also understand the limitations of data and don't become too data-driven.
How to improve your Metrics & Analytics Skills
If you have a data or analytics team, partner closely with them and get them to "show you the ropes", both in terms of the tooling and data sources that are available, and getting them to teach you how to get to insights yourself. Most data teams will be very happy if you can do part of the work for them.
There is also a lot of good content about metrics and analytics. In terms of books, Lean Analytics by Alistair Croll and Benjamin Yoskovitz is a pretty good introduction. Also, the classic post about Startup Metrics for Pirates is still very good and helps think through the most important metrics to measure.
In addition, make data and analytics a part of your routine. Set up dashboards with relevant metrics in your tools and review them regularly – you can even schedule them to sent to you via email or chat on a regular basis. Review those numbers with the team, too – that helps build a data-informed culture within the entire team. You can also create reminders for yourself or as part of your product processes and documents to consider analytics, eg. how by defining up front how you will measure the success of a given product change.
I hope you found this article useful. If you did, feel free to follow me on Twitter where I share thoughts and articles on product management and leadership.