Data-Informed Decision Making

Friday, November 4, 2022

Image Credit: https://unsplash.com/photos/graphs-of-performance-analytics-on-a-laptop-screen-JKUTrJ4vK00

After more than a decade in product roles across multiple Fortune 500 companies, I’ve seen a pattern play out repeatedly: the best product teams are those that anchor their decisions in data while staying adaptable enough to embrace creativity and intuition. Data alone doesn’t guarantee success, but when used effectively, it reduces uncertainty, aligns teams, and guides impactful action.

Access to data is rarely the issue. Most teams today are awash in dashboards, spreadsheets, and reports. The challenge lies in how teams interpret and act on that data. I’ve observed three recurring pitfalls:

Teams spend too much time debating metrics instead of moving forward.
Metrics like page views or downloads can be misleading because they don’t always reflect real value.
Relying solely on intuition or personal opinions often leads to costly missteps.

These pitfalls aren’t just academic. I’ve seen product launches falter because teams were too focused on collecting data or chasing the wrong metrics. Conversely, I’ve seen breakthroughs when teams combined thoughtful analysis with decisive action.

How might we use data to make better product decisions, faster?

This blog post is a recommendation on how to avoid pitfalls from analysis paralysis, overreliance on vanity metrics, and ignoring data for gut instinct. I’ll take you through some principles and practices that have consistently worked for me and my teams.

Focus on the Right Metrics

One of the first lessons I learned was the importance of distinguishing between metrics that matter and those that don’t. It’s tempting to chase big, flashy numbers—think monthly downloads or website traffic—but these don’t always translate into success. Instead, it’s crucial to focus on metrics tied to specific outcomes, like user retention, revenue growth, or customer lifetime value.

For example, when I was a part of Visible, my team initially celebrated a spike in “free-trial” signups through SIM distributions at high-traffic retail stores (both brick-and-mortar and e-commerce). But a closer look revealed that only a small percentage converted to paid subscribers. When we shifted our focus to the trial-to-paid conversion rate, the problem-to-solve looked very differently—and for the better. The result? A boost in paid subscriptions over the next six-month period.

Takeaway: Define success metrics that align with your objectives, and focus on outcomes over outputs. Ask yourself: What does success look like for the customer and the business?

Hypothesis-Driven Decisions

In product development, every decision starts with a hypothesis. Whether it’s introducing a new feature, entering a market, or redesigning an interface, the goal is to test assumptions systematically and learn quickly.

For instance, I once worked with a team that hypothesized adding a gamification element to our app would increase daily active users. Instead of committing to a large-scale build, we created a lightweight prototype to test the idea. By running a short experiment, we discovered that gamification did increase engagement but only for certain user segments. This insight saved us from wasting resources on a one-size-fits-all solution.

Takeaway: Treat decisions as experiments. Frame them as, If we do X, we expect Y. Then test, measure, and iterate.

Embed Data in Team Workflows

The best product teams I’ve worked with make data analysis a natural part of their workflows. It’s not something they do at the end of a project—it’s baked into every sprint.

One of the most effective ways to achieve this is by giving teams real-time access to data. On one project, we implemented dashboards that tracked user behavior metrics like drop-off rates and engagement times. Having this visibility allowed engineers, designers, and product managers to course-correct mid-sprint rather than waiting until after a release.

Takeaway: Centralize data access and integrate it into daily team rituals. Make it easy for everyone—not just analysts—to access and act on insights.

Move Beyond Vanity Metrics

Vanity metrics—big numbers that look impressive but offer little actionable insight—are one of the biggest traps in product management. They’re easy to measure and report on but rarely reflect what truly matters.

I recall working on a mobile app where we initially focused on download numbers. While downloads increased, retention rates were stagnant. By shifting our focus to engagement metrics like daily active users and session frequency, we uncovered critical usability issues. Fixing those drove real improvements in customer satisfaction.

Takeaway: Challenge yourself to look beyond surface-level metrics. Instead, prioritize metrics that directly impact user value and business outcomes.

Rapid Prototyping and Iterative Testing

One of my favorite approaches to data-informed decision-making is rapid prototyping. Instead of trying to build perfect solutions from the start, I’ve found it far more effective to test lightweight versions of ideas and gather feedback.

For example, while working on a B2B SaaS product, my team considered adding an analytics dashboard. Instead of building it outright, we mocked up a few visuals and shared them with a small group of users. The feedback helped us refine the feature before writing a single line of code, saving weeks of development time.

Takeaway: Don’t let perfection delay progress. Test early, test often, and iterate based on what you learn.

Pair Quantitative Data with Qualitative Feedback

Numbers tell you what’s happening, but they rarely explain why. To get the full picture, I’ve always paired data analysis with customer feedback.

On one project, analytics revealed a high drop-off rate during account setup. After conducting user interviews, we learned that customers found the process too complex. Armed with this insight, we simplified the flow and reduced the drop-off rate by 30%.

Takeaway: Use customer feedback to complement your data. Surveys, interviews, and usability tests can uncover insights that numbers alone can’t.

Empower Teams with Ownership of Data

In my experience, teams perform best when they own their metrics. This means giving them the tools, autonomy, and accountability to make data-informed decisions without relying on top-down mandates.

On a past project, I worked with a team that was tasked with improving user retention. By empowering them to define their own success metrics and run experiments, they developed a personalized onboarding flow that increased retention by 20%. The key was giving them the freedom to act and the accountability to deliver.

Takeaway: Foster a culture where teams feel responsible for their metrics and outcomes. This builds engagement and drives results.

Case Study: Scaling with Data-Informed Decisions

Let me illustrate these principles with an example from a major initiative I led to scale a Mayo Clinic subscription-based healthtech product. The team faced challenges with customer churn, and we needed to identify why users were leaving.

Diagnosing the Problem: We started with data analysis—identifying patterns in churn. In this case, a significant percentage of the subscriber-based churned after 3 months of initial signup.
Testing Hypotheses: Based on the data, we hypothesized that a cumbersome UX was a major factor. We looked at journey analytics—and specifically, task completion time and task completion rates—to validate this.
Iterative Improvement: Once we confirmed our hypothesis, we launched small improvements to the flow of logging food intake—measuring impact at each step.

By the end of the initiative, churn dropped, and customer satisfaction scores improved significantly. The success stemmed from continuously iterating based on data rather than relying on a one-time fix.

Conclusion: Driving Value with Data-Informed Decisions

Over the years, I’ve learned that data-informed decision-making is as much about mindset as it is about tools or processes. It requires teams to ask the right questions, prioritize meaningful metrics, and balance quantitative insights with qualitative feedback.

When done well, this approach doesn’t just improve decision-making—it transforms how teams work. It fosters collaboration, accelerates learning, and, most importantly, delivers real value to customers. By embedding these practices into your team’s DNA, you’ll unlock better outcomes for your users and your business.

Ready to take your ways-of-working to the next level? Schedule a free 30-minute exploratory call to discuss how data-informed decision-making can drive real results for your team.