Product metrics. We all know them. We all know their importance  (We do, right?!). Our product teams also take those metrics at heart. If talking to our users is the Alpha, checking product metrics is the Omega of how we look at product management. Product metrics allow us to feel the pulse of our users, spot new opportunities and detect lurking problems.

In this blog post, we'll tap into our past work examples. We'll show you how we've used product metrics with our clients to make the products we build for them even better.

Spotting lurking problems‍

If you don’t have product metrics, you don’t have problems. In a very wrong way, that’s true. If we’d followed this (obviously bad) advice for MyColruyt, we wouldn’t have noticed that people were not using the functionality to display their Xtra card at checkout. Having the possibility to whip up the Xtra QR code within the app, without switching between apps (or even worse: looking for your card in your pockets), is a priority to MyColruyt users. So, obviously, something wasn’t right.

This got the app team thinking about the optimal user journey to show the Xtra QR code. Ultimately, the team added a couple of additional ‘entry points’ via which a user could open their digital Xtra card. They even added an automatic pop-up when a user checked off all items on his/her shopping list. Additionally, they figured out the optimal icon for displaying the QR code through a series of A/B tests. Changing the icon itself already resulted in an increase of more than 50% click-through rate.

Uncovering operational insights

When talking about spotting new opportunities through the analysis of your product metrics, we’re not only talking about new features or improvements. It also gives you trustworthy operational insights. Our PayconiqByBancontact team, for example, analyses the peak-usage times of the famous pink app to optimise the planning of maintenance and downtime moments, minimising the user friction of this widely-used app. 

However, there are more vivid examples of how they leveraged their product metrics. Accessibility tracking, for one, showed that a relatively large number of users had Dark Mode enabled on their devices, making the decision to spend time and resources on developing a dark mode version of the app an obvious one.

‍

Furthermore, detailed tracking of user and device info made it far easier to tackle multiple issues during a big (and risky) technical account migration. Given the substantial user base, there was a wide variety of devices, operating systems, and user conditions. The ability to dissect these issues based on specific platforms, app versions, and other parameters enabled us to quickly identify problems, assign them to the relevant configurations, and come up with a solid solution.

Look at your metrics in the context of your product goals

Analysing your product metrics may lead to new insights, which is great. But it’s crucial to critically assess each idea against your product’s intended outcome. Without this measure, you risk compiling a wildfire of improvement ideas that may not even contribute to your actual product goals. Prioritise strategic alignment to ensure the right focus!

Case in point: Following ENGIE’s mission statement

Let’s look at ENGIE to make this more concrete. The ENGIE Smart app is all about giving users the power to stay in control of their energy costs and avoiding those nasty ‘bill shocks’. Hello, product goal. Honing in on user click/tap data, we shaped the invoice feature - a tool that gives users a heads-up to tweak their monthly prepayments.

But it doesn’t end there. Once this new feature went live, we dove right back into the updated data to smooth out any bumps and tweak the invoice feature to perfection. All these efforts were directly lined up with ENGIE’s overarching product goal, proving that a clear focus is key to creating valuable digital products.

Case in point: Strategic featuring for Colruyt

Back to the MyColruyt project. Colruyt set out an objective of elevating usage numbers of the shopping list. Next, the product team strategically highlighted this goal to brainstorm and evaluate potential features.

• Revamping the home feed to make it more engaging and product-driven
• Enhancing user notifications on product promotions and enabling a one-tap functionality to add them to the shopping cart

Subsequently, the team monitored the metrics of these initiatives, evaluating their impact on boosting the size of shopping lists. Hence making sure they maintained the right focus.

Use metrics to better understand your user groups‍

So far, we’ve learned that product metrics help you spot problems and opportunities, uncover operational insights and should always be looked at in the context of your product’s goals. What’s next? Using your metrics to gain a better understanding of your user base. 

Product metrics can teach us a thing or two about some (often overlooked) demographics of your user base. In our products, we always try to add metrics on accessibility features (voice-over, reduced motion, adapted text sizes…) to help us decide where to put extra effort in development and testing. 

For a mobile app that’s available in over 70 countries and almost 30 languages, we found through comprehensive data analysis that a lot of users were using a larger ‘default font size’ setting. Coupled with the natural variation in word and sentence lengths across languages, it became clear that certain areas of our app were not able to display all crucial information. To combat this, we integrated a new approach into our design and release workflow by:

• Adapting our components to support multiline text where feasible
• If abbreviation was unavoidable, we ensured the least critical information was affected
• Prior to release, extensive tests were performed in several languages and font sizes to validate our adjustments

‍

One unique challenge in the same project is the support for Arabic. Unlike other languages, you need to read Arabic from right to left. Supporting this is quite resource-intensive. Our metrics, however, showed that only a handful of users were using the app in Arabic. As a result, we prioritised making the app usable in Arabic but limited the extent of optimisation efforts. 

Don’t drown in analyses, talk to your users!‍

We’ve been going on long enough about product metrics. But we have a last piece of advice to share. Monitoring your product analytics tools may be indispensable, but it’s not sufficient in itself. Or, as someone put it excellently in the comments of one of John Cutler’s recent pieces: 

“The chase for metric purity is often rooted in risk aversion and mitigation, more than anything else. It’s also why ‘measure what matters’ might be true, but not everything that matters can or should be measured.”

However ingenious your analytics dashboard is set up, it won’t reflect what is going on in a user’s mind. Metrics can and will never substitute for qualitative user research with your users. Frequently talking to your users still remains essential to uncover true user needs. 

Besides talking to your users, think of other channels to capture important insights. For some of the products we manage at In The Pocket, simply listening to Customer Support services has been the first step towards significant product improvements. All these different channels supplement each other. The more channels you consult, the more valuable pieces of the puzzle you’ll reveal.

The only question you need to ask yourself is: which pieces are you still missing?