Nearly every product manager I speak to desires to be a leader in his or her organization. Everyone knows that as a member of the Product Team, you must exercise soft skills like influence to nudge your product towards success. However, to few know how to gain the confidence to become the leader that they want to be. The first step toward becoming a great product leader is to have the best quality and quantity of information, so that you can attack any problem with data, not opinions.
Introducing Win/Loss Analysis
One of the most chronically under-utilized ways of getting to product data is by using win/loss analysis (WLA). WLA involves developing a deep understanding of how and why your buyers make their purchase decisions, both for and against your product. WLA data is interesting to Sales, because it could highlight breakage in the selling process, and it is highly interesting to the Product Team, because it can help you sharpen your understanding of competitive threats and positioning, what customers value the most in your product, and where to focus your product marketing tactics to match the buyers’ expectations.
How to do Win/Loss Analysis
Win/Loss analysis is typically done in phases. The first phase is almost always qualitative, and involves direct one-on-one interviews with buyers. In these interviews, the Product Team should focus on asking open-ended questions that generate new learning, such as:
- What was the main driver of your purchase decision?
- Describe the process you went through when making your decision.
- What motivated you to start looking for a solution?
These questions cannot be answered with a simple yes/no, and will create good discussion that can be explored further. If you do enough of these interviews, you might start to spot a trend. Some companies feel this is enough data and stop here. Other companies go to a second, more quantitative step, and test the trends they discovered in the first step through surveys to get more statically significant results. The upside to this is higher quality data, the downsides include: cost, time, and the requirement of a large enough pool of wins and losses to sample from. If you run a B2C product with take-it-or-leave-it pricing, the quantitative step may be easier than in a B2B model with negotiated deals.
Product Teams also need to consider who they target for WLA. Wins are obvious – they went through the entire buying process, and chose our product. Losses are less obvious. Don’t think of losses a deals that went all the way to the end, such as the RFP or bake-off stage, and the evaluator didn’t pick your product. Deals such as these are losses, but they constrain your view much too narrowly. Instead, think of the funnel that you must guide evaluators through in order to close. Every company has their own definition of the steps in the funnel, but a typical funnel might have these steps:
- Qualified Lead (from Marketing)
- Initial contact, sent marketing materials
- Meeting, assessment of needs
- Demo of our solution
- Competitive displacement
- Evaluator decision
An evaluator doesn’t have to get to step six to become a loss. Many times, the evaluator might get to step two or three, only to become non-responsive. That should still be considered a loss. What if our marketing materials don’t speak to their pain? What if we demoed the wrong solution for their needs? The result of these missteps is the same – we lost the deal. WLA searches for and finds the breakage in your process so the Product Team can address it.
Unfortunately, win/loss analysis can also be a political animal. Over the years, many Sales teams have become jaded by WLA, and view it as a way to assign blame to them for lost deals. That is not why we (should) to WLA. WLA should be about finding issues in our overall process of making the sale, of which Sales is a part – but not the only part. If the breakage is in our positioning or pricing or competitive response, these things have nothing to do with Sales, but could have a huge impact our Sales ability to sell. In order to overcome political objections to WLA, we encourage teams to put their initial focus on analyzing wins. Usually no one objects to win analysis and it is a good way to get the organization comfortable with the process.
Another important caution for WLA is who does the actual interview. WLA involves getting honest and straightforward answers from a buyer, who make be skeptical of your motivations. For that reason, Sales (and anyone involved in selling that particular deal) should not be involved in that WLA. Because buyers lie to sales people – it’s in their nature. If your salesperson calls on a buyer they lost to try to find out how and why the buyer made their decision, the buyer will immediately assume that the salesperson is trying to restart the deal – a now closed issue in their mind. So the buyer will generally say whatever they have to say to get the salesperson off the phone as quickly as possible – usually defaulting to the “easy” answers of: “we didn’t go with you because of the price,” or, “your product was missing a key feature we needed.” Is there any surprise that salespeople believe that the primary reasons they are losing are price and features?
WLA Generates Results
Win/Loss analysis can sometimes surprise you and lead you down a different path to product success. One Product Team we worked with found out through doing win analysis that the reason they were winning was not that they were the low price leader (as their Sales team thought), but rather that their buyers put a major premium on their customer service and support. In fact, the Product Team was able to illustrate that in 80%+ of the deals won over the past year, price was not the determining factor. The next time Sales wanted to discount, the Product Team had some data to show that perhaps discounting wasn’t the right strategy.
Another Product Team we worked with found breakage in their renewal process by doing loss analysis, which over the course of a year saved over $1MM in recurring revenue!
Clearly, WLA is a powerful tool, and should be exercised more by Product Teams.
The Impact of Win/Loss Analysis on Product Leaders
A few quarters ago, I was teaching a Pragmatic Marketing seminar and was approached by a product manager. He was discouraged because he didn’t feel like he had management’s buy in for being market driven and, as a result, was not getting any budget to do things like visit with the market. He asked me for advice.
One of the things I told him to do was win/loss analysis. First, it is relatively inexpensive, especially for wins. Second, nothing speaks with more credibility than the buyer. Third, he needed a quick win, and WLA can be fast. I asked him to try doing a handful of WLA’s, share those with his executive team, and see what happens.
A few months later, he reached back out to me, and his demeanor had completely shifted. He told me that his CEO was so impressed with the results of his WLA efforts, that they recreated their positioning, marketing materials, and selling process around the buyers’ expectations. All of the sudden, the organization was looking to him, the product manager, for answers about the market, and his concerns about budget for occasional travel were starting to go away. In short, a few well-timed win/loss analyses changed the company’s perception of him – and his perception of himself.
If you are interested in making the same transformation into a trusted product leader at your company, remember that first step to leading is following. Specifically, following the market. Once you have the market data on the tip of your tongue at all times, the confidence to speak with credibility about the needs of your buyers and users will follow shortly.
If you are intersted in going in-depth about win/loss analysis and lots of other topics of interest to Product Teams, consider attending a Pragmatic Marketing training. I will be leading seminars on these topics on Mar 11-13 in Toronto, Mar 19-21 in Austin, Apr 8-10 in Tyson’s Corner, Virginia, Apr 29-May 1 in Denver, May 6-8 in San Diego, and May 13-15 in Princeton, NJ. I hope to see you there! In the meantime, you can always email me at firstname.lastname@example.org with questions, or comment below.