Small Feature, Major Impact: How Insights in JPD Elevate Product Discovery Game (Part 1 of 3)

[Rob Mkrtchian]

In this discussion, I'm opening up about what I've learned myself and from my fellow product managers about the crucial role insights play in our product discovery efforts. We've talked about their importance, which led me to work on a new insights analytics solution wrapped into Atlassian app. I intend to build it in full view of the public—a comprehensive system for gathering and analyzing insights, built right into Jira Product Discovery, for the Atlassian community. As we go along, I'll share the progress and some key resources that could help us all. If you're curious about making your product discovery and roadmap planning more evidence-based and structured, follow along. I'd love to hear your thoughts and how you harness insights, so please don’t hesitate to share your experiences in comments.

Analyzing customer feedback is not the same as analyzing insights

Many of you are probably familiar with the concept of insights, and those new to it, I hope you'll find value in this article series. A perfect insight must be actionable, highly-relevant to idea, and evidence-based. It must be specific enough to drive decisions and actions, it must contribute to understanding idea's problem, define its interim goals, and set the direction of the solution, lastly it must be derived from facts and not from vague hypothesis and personal opinions.

We shouldn't forget that gathering insights is a team sport. It's not just product managers who are on the lookout, but also those in marketing, customer success, and sales—to name just a few.

Insights are particularly useful for tackling two big challenges—communicating with stakeholders and prioritization activities.

Insights enable us to refine our communication with stakeholders by grounding our roadmap decisions in relevant data, aligning more closely with the goals of different departments and stakeholders. For instance, incorporating market trend data can underscore the significance of an idea to your Chief Marketing Officer (CMO), while showcasing technological trends may sway your Chief Technology Officer (CTO). Even a pertinent competitor case study could compel your CEO to take a strategic leap.

It's also advantageous to recognize three distinct types of insights: behavioral, qualitative, and quantitative. When collecting them and instructing others to contribute, it's helpful to educate about the diverse nature of insights. Contributors may occasionally mistake opinions for insights; hence, it's essential to steer them towards sharing unbiased, evidence-based, and actionable insights relevant to a specific idea.

Let's examine three main types:

Behavioral Insights

These may include user engagement metrics after the release of features or navigation paths showing user flow. Understanding feature utilization can reveal whether an improvement will be embraced or overlooked. Conversion funnels and drop-off points can signal whether a new idea could enhance conversion rates. Meanwhile, session recordings might expose user frustrations not apparent from mere data analysis.

Qualitative Insights

Excerpts from customer interviews can validate an idea, while survey responses and feedback provide context beyond binary choices. Social media conversations can reflect a product's growth potential, and competitor testimonials may uncover underserved user segments or validate through their satisfaction.

Quantitative Insights

Market size and sales data can indicate the potential of new user segments. Market share insights can justify further investment for exponential growth, while consumer trends can support the pursuit of new product directions. Competitor benchmarks help in identifying areas where your product can improve based on evidenced demand.

Avoiding Risks Associated with Insight Collection

Now, while insights are a gold mine for product management, they do come with potential pitfalls. It's like navigating a maze; one wrong turn, and you're facing a wall of irrelevant data or outdated statistics. Here's how we sidestep these traps:

Data Overload

It's tempting to want every piece of data, but that's like trying to drink from a fire hose—you end up with more mess than clarity. So, we laser-focus on what matters for our current objectives, and we're not shy about filtering out the noise. JPD might not let us export insights just yet, but that's where a bit of manual effort or a clever web scraping tool comes into play. Try to import your insights into semantic analytics tool that can help you categorize, identify and analyze insights in more informed and unbiased way.

Compromised Data Quality

Internet is full of unverified information, articles and researches that can claim numbers and stats that are far from reality. Often prognosis are “sold” as truth and reality, so be careful while adding pieces of content as insights. Ensure data integrity, so every piece of insight comes with a virtual stamp of credibility. And when in doubt, we double-check the sources, because a single unreliable data point can throw off our entire game plan.

Confirmation Bias

I’m sure we all at least once caught ourselves falling victims of confirmation bias when trying to validate our idea or convince others in its huge potential. In our profession there is no room for more “false positives”. Try to bring in fresh eyes—different teams, different backgrounds—to look at the same data. Sometimes, even try tools like ChatGPT take a stab at it to compare notes. You'd be surprised at how varied the interpretations can be.

Outdated Data

Here's the thing—data ages like fish, not wine. So, we're always fishing for the latest catch. And for ideas that linger in the backlog, automated AI scraping tools can come handy when you need to check on your competitor releases or look out for new reports and when relevant piece appears it can be added to your idea as insight with a label “for review”.

Actionability of Insights

An insight that doesn't lead to action is like a car without wheels—it's not going anywhere. Before we add an insight to our strategic deck, we ask, "Can this move us forward?" If it doesn't drive decisions or improve our understanding, it's better without it.

Ineffective Communication of Insights

No size fits all and the same goes for communicating insights. We must tailor our communication using different insights fitting each stakeholder's unique perspective. Have you ever heard that you cannot pitch the same startup metric to different investors, each of them has its beloved metric that you need to use in your pitch to favor for your startup, and like startup metrics carefully selected insights could play well for you when presenting your roadmap to different stakeholders. And surely, It's not about altering the message but framing it in a way that resonates and drives the desired perception for each stakeholder.

So what's next?

In the upcoming articles, I intend to share practical examples of how to find, assess, and collect insights efficiently. We'll explore techniques for systematic prioritization and look at how product managers from various companies leverage insights for maximum value. This introductory piece sets the theoretical foundation, but future installments will be packed with practical examples. I invite you to share your methods for using insights within JPD, discussing your insight-gathering approaches, and eager to hear your stories. Your input could enrich our next piece and shine a light on diverse practices in our field.