Bring clarity to
product decisions

Download our free and easy-to-follow qualitative analysis toolkit, which is a structured way to gather insights from interviews with stakeholders, colleagues, and customers to solve complex organizational challenges.

The problem with metrics

Comic by @daveyeats

Graphs are really useful for tracking progress towards a goal and almost entirely useless at helping you move towards it.

“Graphs tell you WHAT is happening, not WHY it’s happening.”

We’ve gotten pretty good at tracking KPIs like retention and optimizing them by trial and error. Oh, that thing we did made this chart go up. Sweet. Lets do more of that.

However, once we work through our best guesses and solve for all the low-hanging fruit, it becomes harder to move the needle. We have some intuition around why certain metrics go up, and that intuition dries up without a feedback loop.

You’ll need more than just correlation now, you’ll need causality. Qualitative research can shed light on why retention is dropping, like Netflix’s stock price, or why the bounce rate is sky high.

The feedback loop

Feedback is quick in skateboarding. Without proper form and technique, the very likely outcome is your board slipping from underneath you and you landing face down on concrete. This painful experience is unforgettable and drives home what doesn’t work very quickly.

When you do skate down that ramp successfully, the incredible feeling of accomplishment deeply re-wires your ape brain to help you remember what you did right this time, helping you get better at skateboarding. And that, kids, is a feedback loop (you already knew that, but for the kids in the back, who never pay attention).

Business and software, unfortunately, are less feedback-y. You can build features for months without ever really knowing the impact they have had. You can track charts that go up or down without ever knowing why. And sometimes, when feedback is present, it is severely laggy. When clients sign up for yearly contracts, you might find out at the end of the year if they renew or not. Imagine falling off your skateboard and feeling the pain a year later. How perplexing would that be?

Building a feedback loop that tells you why something went wrong is a super power. Qualitative research is a way to close the feedback loop and tighten the time it takes for the actions you take in your business to bounce back as positive or negative feedback.

An unreal example

Being data-driven is as much about measuring important quantities and ratios over time that help quantify the impact we’re creating. But it’s also about things that cannot be measured in numbers, like how customers felt when using your product for the first time or what features your top customer support representative thinks customers would most benefit from. This is a different kind of “qualitative data”, measured in words, ideas, and thought bubbles.

How do you measure ideas, words, and insight? We think the best way to perform a qualitative analysis in your organization is to have structured conversations with the aim of solving a specific challenge.

Take this example SAAS business, the information and intuition that are present in the minds of the people that take part in the everyday interactions are about as valuable as anything you see on a chart, perhaps a lot more.


free PDF Download
qualitative analysis toolkit

  • Interview stakeholders, colleagues, and customers to gain incredible insight.
  • Stay structured in your problem solving with this toolkit.
  • Use qualitative research to solve complex problems that quantitative data cannot.
  • Unlock growth potential by identifying and solving for constraints in your product and growth funnels.

Download the PDF