Taking Event Support from Sales Tactic to Product Strategy

Events. It’s natural to dread them. As a product manager, you may view events as largely tactical and time-consuming endeavors. Beyond that, you may loathe being stuck in a trade show booth taking on a sales role that’s not entirely comfortable for you.

But the truth is that events aren’t just a sales and marketing tactic. They can and should be a strategic part of your market research—a way to glean feedback from your market as much as they are a place to push your message.

Rethinking the Role of Events

While some product managers have a sales background, most don’t. And when you’re put into a trade show environment, your focus suddenly becomes delivering leads. Maybe your sales people lure prospects in with a good pitch and then expect you to continue the conversation—maybe even close the sale. Frankly, a lot of product folks aren’t good at this.

Ultimately, you may be wasting your time manning a booth when you could be making events more beneficial to your team, your company and even your prospective customers. So, what can we do to rethink event support and create events that actually benefit the product team? Here’s a start.

1. Talk about your customers’ problems. First of all, let’s change the way we approach events. So often, we see our role at a trade show as one of doing demos and talking up our product’s features. Stop that. Focus instead on the problems that our product aims to solve.

When you begin a demo, talk about the problem first. Then, confirm your prospect actually has that particular problem. (If they don’t, then demo the solution to a different problem.)

You might say: “Our product does a million different things, and obviously I can’t show all those to you right now. But one problem I’ve heard a lot of your peers have is this … Is that something that you’re running into as well?”

If your market research is on point, they’ll probably say, “Yes it is. I would love to see how you’re solving it.” But they could say, “No, we actually don’t have that problem” or “We solve that with something else.”

By prefacing your demo with a conversation about problems (rather than leading with your product), you increase the likelihood of people being more open about their pain points. And what you learn in those conversations is tremendously beneficial.

You might learn that your audience has different pain points than you thought. Or maybe they think about the problem in different terms and it’s time you started using different language.

For product managers events can be a perfect opportunity to learn more about your customers and whether you are positioning your product well in the marketplace.

2. Break free of the booth. Trade shows and other events are about networking and talking to people—not just selling. So, it’s important to leave the booth. Sit down and have a cup of coffee with people.

Once you put together a name and a face, it’s easier to follow up afterward. You can call and say, “Hi, we met at XYZ event last week. I’m doing some follow-up research, and you came to mind. I’d love to get 30 minutes to chat.”

That request is easier to make if you’ve met and have started building a relationship. This can be vital to your research because you’re gaining information from potential customers.

If you manage a product team, you could even gamify this process by offering a reward for the product manager who makes the most contacts.

3. Create your own event. We typically think about event support as being driven out of marketing with the product team being there to support the marketing function.

That doesn’t have to be the case. Let’s take event support to the next level and as product teams, create events of our own.

You could create a rolling road show or a “listening tour.” Go to your customers and host a breakfast, for example, or speak on a topic that’s useful and interesting to them. In doing so, you’re creating the opportunity to converse and learn about their pain points and needs.

You could offer incentives to attend and bring a peer, such as raffling off a big prize. Or you might consider offering an advance look at your beta offering as an incentive.

From Tactic to Strategy

By changing the way you approach events—or even by creating your own product-centric events—you can engage in useful conversations and gain valuable data that you can use as part of your product development and marketing. As you create your strategic plan, bring events into the fold as part of your product marketing strategy rather than a sales tactic and reap the benefits.

On Being a Pragmatic Marketing Instructor

This week, I was fortunate enough to receive a new opportunity within Pragmatic Marketing, and accepted a new role as Vice President of Products.  It’s a rare occasion that the stars align and you’re able to do work you love for a company you love with a team you love – but that’s exactly how I describe working at Pragmatic.  If you haven’t experienced that feeling, I sincerely hope that you will, and I’d like to share some thoughts on the best job in the world with you in the hopes that it piques your interest, and points you down a path of achieving success, however you define the term.

In 2010, I worked for Dell, and managed a small team who was responsible for portfolio management of several acquisitions that Dell had made over the years.  Prior to Dell, I worked for Cisco, a company Cisco had acquired called NetSolve, and a startup called NetStreams, amongst others.  All of these were good experiences in their own ways, but at a certain point I realized that I had the itch for something smaller, where I could see an immediate impact.  I had an idea for a startup of my own that I was trying to pursue but had come to a point that I needed to either quit and do it full time, or abandon the idea.

Key Learning: Why kind of person are you?  Do you prefer the stability and structure of big companies, or do you prefer the chaos of smaller?  Know your preference and use it to narrow your choices.

Around the Spring of 2010, Pragmatic Marketing opened a role for a new Instructor position.  While I had never considered it an option before, I found myself stepping back and asking “Why are you thinking about making a change?” “What do you really like?”  My self-realization was that I did love managing products, driving strategy, and beating the competition.  But what I liked even more was seeing the successes of my team.  I got energy from seeing people from my teams grow, seeing them nail a presentation to the executive team, and even seeing them quit and move on to bigger opportunities (although that’s always painful).  Their success was my success, and I loved it.

In a “normal” career, you might manage a few dozen products, you might touch a few hundred people directly.  But as an Instructor, I’d get to affect thousands of people.  It’s intimidating to move from a “doing” track to a “teaching” track, especially if you’ve climbed the ladder your whole career.  It wasn’t until I got into the role that I realized how good the fit could be.

Key Learning: What is your passion?  Do you love what you’ve done so far, or do you need to go in a different direction?

After going through an intensive hiring process, eventually I joined Pragmatic as an Instructor in August of 2010.  Immediately, I was paired with a mentor Instructor to learn the what and how of what we teach.

Most people come to three or four training days with Pragmatic, but it’s difficult to convey how much depth in our teaching is developed behind the scenes.  In any given class, we just scratch the surface – and the Instructors on the team all come from executive backgrounds and have loads of experience across industries and business models to fit any situation.  It feels like a Marketing MBA every time you step into the room with the Instructor team, because they are some of the smartest people in the world with regard to the application of market-driven techniques and strategies.

Key Learning: Where can you work with the smartest people?  When you work with A-players, and people smarter than yourself, it forces you to raise your game.

After completing the mentor/protege process and becoming certified to teach, I started to travel the world to deliver our training.  It’s no lie to say that teaching a marketing class can be intimidating:

  • You have to master topics ranging from pricing to competitive analysis to organizational alignment to roadmaps to business planning
  • You have to command the room and have the answers to everyone’s questions
  • You have to supply energy and keep some traditionally dry topics interesting and fun
  • You have to do all of this in front a room of smart, Type A people who are or will become executives

It’s a challenge, in a good way.  In the past six years I’ve delivered our trainings in 20+ states in the U.S., the UK, Paris, Sydney, Hong Kong, Singapore, and many others.  Training is a marathon; a few years ago I was on the road 47 weeks in a year teaching.  You can’t do that for something you don’t love.

Key Learning: Love and passion aren’t enough.  Do you have or can you develop the skills you need to thrive in your new role?

Today, when I step in front a room, my first step is always to observe the class.  Who are these people, and why are they here?  Some people are eager, excited, and ready to learn.  Others are here because their boss told them to be.  Others are evaluating switching roles from Engineering or Sales into Product Management or Marketing.  None of them quite know what to expect because their past experiences with other training classes have been so poor.  We have a chance to blow them all away with the best professional experience of their lifetime, and make an impact that lasts far beyond the time I’ll spend with them.

Within the first 15 minutes of class, everyone realizes that this class will be unlike any other class they’ve ever been to.  It’s not theoretical, it’s practical and actionable.  It’s enlightening and fun.  It embodies a philosophy of asking “What problem are we trying to solve for our market?” that will stick with them throughout their careers.

It’s not uncommon for students to approach me and other Instructors at the first break to say “Wow, I feel like I’ve already got my money’s worth and we’re not even through the morning session!  This is the best training I’ve ever attended!”  I often get emails from alumni of our classes years later, who tell me that I taught their class long ago, and they still use what they learned, and that they carry their tattered books around as reference manuals and clutch their Pragmatic Framework tight to their chest as a Rosetta Stone for their jobs.  That’s the feeling I got when I first attended a Pragmatic Marketing class as a student.  That’s the feeling I want to give to all of my students.

What we teach impacts students throughout their careers, affecting them, their peers, the products they work on, and the businesses they work in.

I’m an Instructor.  I don’t “train people.”  I change lives, and businesses for the better.  When people ask me: “Why did you stop doing Product Management to teach people like me?”  I tell them: “I never stopped doing Product Management.  I do it every single day – only now, my product is you.”

If you made it this far, and you can envision yourself as an Instructor, I’m looking for the next great member of our team.  Head over to our careers page, read the requirements, and let’s get to work changing the lives of students around the world.

Your survey data is flawed, and what to do about it

Science magazine just come out with a bombshell: senior researchers at SurveyMonkey have detected that as many as one in five surveys contain fraudulent data. And we’re not just talking about a handful of surveys, they analyzed over 1000 public data sets from multiple different surveys to reach this conclusion.

“…Among 1008 surveys, their test flagged 17% as likely to contain a significant portion of fabricated data. For surveys conducted in wealthy westernized nations, that figure drops to 5%, whereas for those done in the developing world it shoots up to 26%.”

This is a major problem.  The usage of quantitative data in product teams has exploded in the past 10 years, and if you can’t trust your data, then you can’t trust the decisions you are making that rely on that data.

Fortunately, the researchers, Michael Robbins and Noble Kuriakose, point to the likely culprit:

“The basis of the test is the likelihood, by chance alone, that two respondents will give highly similar answers to questions on a survey. How similar is too similar? After running a simulation of data fabrication scenarios, they settled on 85% as the cutoff. In a 100-question survey of 100 people, for example, fewer than five people would be expected to have identical answers on 85 of the questions.

…one of the inevitable problems, Robbins says, is “curbstoning” where an interviewer sits on the curb and invents survey responses—often duplicating answers—in order to avoid risk or save time.”

Essentially, some researchers tend to create data, especially when the survey population puts the researcher at personal risk.  It makes sense, and is also why you don’t see a ton of polling data or opinion surveys on ISIS, because very few researchers are going to go there (and very few people would be willing to pay for that).  There’s a fairly low likelihood that in a long-form research survey, that multiple survey responses will be identical or nearly identical.  If the dataset includes lots of identical data, it could be a flag for data falsification.  To be fair, Pew Research takes issue with Robbins and Kurikose’s research, and is pushing back publicly on their website.  This fight will continue for awhile, because the answer is not black and white, it’s a gradient.

So what is the impact to product teams?  Probably minimal, for most teams.  Most product teams are getting validation data from a variety of areas that are less susceptible to this type of manipulation, such as A/B testing and experimentation.  Even if you are performing surveys as  part of your data collection (and you should be), most product teams aren’t building products that target the developing world or high risk populations.  If you are, ensure that you work with a reputable research team or train your team on how to collect that data responsibly, and check the dataset against Robbins and Kuriakose’s test.

If you aren’t building products that target market segments with these problems, take this as a reminder that data is just data, and before we draw conclusions, there is a set of sanity checks we need to run against the data first.  One of those checks may include “is everyone answering the questions in the same way?”

March Market Madness!

Pragmatic Marketing is running a neat Market Madness promotion – a “playoff” between the boxes on the Pragmatic Marketing Framework.  You can vote on each match up and see who the winner will be at productmarketing.com.  But half of the fun is just seeing the college logos the marketing team created for each box.




My money is on Distinctive Competencies.  I think they are underrated as a #3 seed.  Go grab the bracket (PDF) and see the seeding, and tell us what you think!

It’s time to get your Ph.D. in Sales Comp

A few years ago, I was working for a large company here in Austin.  This company had recently acquired a series of software startups and was attempting to integrate them into their larger hardware portfolio.  The product team I managed was responsible for the integration, product management, and transition.  What I learned during this time about Sales behavior was a shock to my system – and may help you as well.

Imagine a Fortune 100 company: large, established, incumbent.  Also stale, bogged down in red tape, and struggling to innovate.  A typical move for a large company is to inject innovation through the acquisition of a smaller company.  Often, this makes great sense on paper – the bigger company gets access to technology and talent they don’t have, and the smaller company now has an opportunity to both cash in their hard work and take their products to a much larger audience via an established sales channel.  If you’ve worked in the industry for any length of time, you’ve been through this cycle from one side or the other.  Unfortunately, what looks good on paper can quickly go astray if you don’t understand Sales and the tools you are giving them.

In this case, my team had responsibility for four to five smaller acquisition company products and portfolios.  Luckily, the larger company had done a good job acquiring companies in the same area who had complimentary products, both to each other and to the larger company’s hardware devices.  Essentially, our new software portfolio provided ongoing systems management for the hardware that made up the mainstay of the larger company’s business.  If we could simply start to attach our software to the millions of dollars of hardware deals that were already happening, there was a great chance of success!

To make this picture even more attractive, the larger company’s hardware business was undertaking a long, slow decline both in revenue and profitability.  They were averaging single digit gross margins on hardware, whereas our software was achieving typical software margins in the 40-60% range and higher.  By all rights, it was going to be an easy win…until we starting training the larger company’s Sales teams.

The long, slow decline of established business

Our decline wasn’t this pronounced…but it was close!

The first time we trained one of our hundreds of new Sales teams, we knew we had an issue.  We did a “lunch-and-learn” where we fed the team pizza and spaghetti, while one of our product managers did a demo of the product and talked about how it was complimentary to the products they were already selling into their accounts.  The Sales team was polite and listened while they ate for about thirty minutes.  Then, everything went wrong.

Do you ever want to scream when working with Sales? It doesn’t have to be that way if you know how they’re motivated.

At the end of the presentation, the Director of the Sales team stood up.  He looked straight at my product manager and said:

“John1, I want to thank you for that great presentation.  I think we all now have a clear understanding of your product, its pricing and competitive positioning, and how it can help the company.  I understand that the CEO has identified this as a strategic product.  And you know what?  We are never going to sell it.”

Needless to say, that was not the reaction for which we were hoping.

John tried to recover by asking the Director why he felt that way, and the Sales Director responded by saying:

“To your credit, this does seem like a really good product.  But, it doesn’t match up with how my team is rewarded.  Right now, software sales represent 10% of our quota, and hardware is 90%.  If one of my guys blows up his software number and misses his hardware number, he is fired.  If he blows up his hardware number and sells zero software, nobody cares.”

Very quickly you can start to see the problem – even though the company had spent millions of dollars on this acquisition, they had not put the tools and behavior modification in place with their existing teams to make it a success.

At this point, we needed to get educated (quickly) on how Sales at this company was motivated.  Our next stop was Finance.  We scheduled a meeting with someone that Finance identified as the “Genie of Sales Compensation.”  When we sat down with the Genie we asked her to show us how Sales was motivated, quota’d, and bonused.  That was when she opened her manila folder, and took out a taped-together 3×3 matrix of legal sized paper, upon which was printed in 6-point font an Excel spreadsheet containing approximately three dozen rows representing the different sales teams, and about fifty columns representing various ways the Sales teams were measured.  Some Sales teams had thirty or more variables to determine their quota attainment.2  In short, you needed a Ph.D. in Sales compensation to understand this system.

Yes, it was almost this big.

Many people refer to Sales as “coin operated,” by which they mean that Sales operates in whatever way will help them get the most coins.  This is exactly how you want Sales to act, but it reinforces that if you aren’t very clearly part of their compensation, they won’t spend time worrying about you, regardless of how strategic or important to the business you think your product should be.

In the end, we found that adjusting the Sales teams compensation models to account for our products was so fraught with politics and peril that it was doomed to failure.   Prioritizing our products in terms of comp meant deprioritizing someone else’s products, and every Sales comp line had an advocate in the form of a powerful executive for some other product.  That is when we realized that we needed to sidestep the issue by creating our own overlay Sales team that was only measured on our products.  This worked, but only after a lot of pain and suffering.

This month, Pragmatic Marketing’s bloggers are going to be writing a lot about various Sales tools and ways we can enable Sales teams.  Remember however that the number one most powerful Sales tool in the Product Team’s bucket is the compensation model.  If you get this wrong, or your product is not represented in it, it will not matter how slick your competitive training is, or how good your positioning is, the strength of your competitive analysis, or how well you priced your product.  As the CEO of your product, you must understand how your team is motivated – and take corrective action where required.

What other challenges have you run into with Sales, and how did you overcome them?  Weigh in below in the comments section.

1 Name changed to protect the innocent
2 This is when I realized that I prefer smaller companies to larger ones

Join me at PIPELINE 2014!

On this Friday, June 6, 2014, I will be presenting at a virtual conference called PIPELINE 2014.  PIPELINE is put on by Planview, and will include presentations about a wide variety of topics of interest to Product Management and Marketing professionals, including innovation, ideation, prioritization, and how to change your culture.  I will be giving a talk entitled “The Cornerstone of a Market Driven Organization,” and if you’ve always wanted to go to a Pragmatic Marketing training, but never had the chance, this is a good opportunity to get a taste of our philosophy.  Best of all, PIPELINE is free, and the presentations are all archived online for you to watch at your leisure – all at once or over the next year!

You can read the entire agenda of presenters here, and see what catches your eye.  Register for free at this link.  See you on the 6th!

The Problem with Building Products in an Agile World

Over the past decade, there has been a ton of ink spilled over Agile development.  Much of the Agile narrative has focused on the pros of implementing Agile methodologies, such as better planning, faster feedback, and the ability to quickly pivot and make changes.  These are all good things.  However, from a Product Management standpoint, Agile has some major problem areas that can create inefficiency at best, or completely wreck a project at worst.

Before we continue, I want to be clear: I love Agile.  The teams I have managed and worked as a part of have delivered lots of user stories and requirements to Agile teams and seen success.  Agile can be wonderful.  In my travels over the past four years as a Pragmatic Marketing Instructor, I’ve also witnessed several “Agile traps” that jump up and grab teams.

1) Agile makes it harder to be disciplined with regard to strategy

Because of the nature of Agile, multiple sprints happen every two to four weeks.  Organizationally, we now have the ability to change direction more quickly than ever before.  This can be helpful to react to changing market conditions (or the needs of big deals).

Unfortunately, this ability to quickly change is a double edged sword.  Because we can change direction quickly doesn’t always mean that we should.   I often see Agile teams building half-solutions to dozens of problems instead of actually fully solving anything, because they are flipping from solving one problem to the next one too quickly.  Agile requires extra discipline to stick to the strategy when it’s tempting to continuously change direction.

2) Agile values feedback from customers (sort of)

Agilistas would say that they strongly value getting constant market feedback at the end of every sprint.  That’s good.  In my experience, what actually happens is that most teams recruit four to five customers who are willing to provide that level of constant feedback and then roll with them – for the rest of the release or even multiple releases.

Going back to the same customers for feedback creates a two-fold problem.  First, do those four to five customers really represent your overall market segment?  Or, are they the “noisy 20%” of your market segment?  In my experience, the types of customers who are willing to sign up for regular meetings to provide feedback every other week don’t typically represent average users; they represent power users and experts.

Second, customers aren’t dumb.  Soon, those four to five customers figure out that they have a disproportionate amount of influence on your direction, and realize that their feedback can turn you into essentially a custom development shop for their particular needs – again, that’s not market driven, that’s customer driven.

3) Agile’s roles can actually prevent you from being market driven

When teams implement Agile, they often look to create new roles.  Specifically, the “Product Owner” role is one that is often tasked to Product Management.

Unfortunately, the Product Owner role is extremely noisy and tactical and requires a lot of day-to-day interaction with Development in daily standups, sprint planning sessions, and retrospectives.  While there is nothing wrong with making the Agile team run efficiently, I have seen teams get so obsessed with “making Agile work” that they start sacrificing what Product Management really should be doing – which is spending time with the market.

I speak to Product Team members every week who are doing the Product Owner role, and they all tell me that 95-99% of their time is spent on internally facing Development activities, and they rarely, if ever, speak to anyone in their market.  That’s not a recipe for being market driven.

4) Agile helps us run faster, but are we running in the right direction?

Are you running in the right direction?

Agile’s big promise is that it will help us run faster in Development (or whatever), but it doesn’t say anything about if we are building the right thing to begin with.  That is the role of Product Management to find out.  I see Agile teams often getting so obsessed with the machinations of Agile that they forget that you can be the most efficient company in the world, but if you’re building something that no one wants, it’s pointless.

How is your team using Agile today?  Are you running into any of these issues, or others?  Please share your experiences in the comments section.

Who Owns Design?

One of the most frequent questions I receive is “Who owns Design?”  Should the designers report to the Development or Engineering team, should they report to Product Management, or perhaps Marketing?  Or, should Design be its own group?  The answer might surprise you.

Before we can define where Design belongs in an organization, we must first define “Design.”  There have been dozens of books written on this topic and the debate will continue to rage online long after this blog turns to dust.  For now, let’s turn to “Paul’s Pragmatic Definition of Design”

“Design is the role in a team that understands a market problem, and comes up with a solution to that problem.”

It’s doesn’t have to be harder than that.  To determine where Design belongs, lets examine whom the Design primarily works with on a day-to-day basis.  Design works sits as the middle layer in this diagram:

Problem-Design-BuildTypically Product Management is responsible for the top layer of finding and quantifying the markets’ problems.  Development is typically responsible for the Build layer of bringing the designed solution to life.

However, to make this picture more complex, Design doesn’t mean just one thing.  At a minimum, there are two flavors of design:

  • Front-end Design, and
  • Back-end Design

Front-end design is what most people mean when they say “Design.”  The titles found in front-end design include User Interface, User Experience, Human Computer Interface (HCI), and Human Factors Designers – all titles found predominantly in software projects.  But front-end design is not limited to software.

Front-end Designers in hardware applications are concerned with ergonomic design, user fatigue, eye level, placement of power cables and interfacing to larger systems – such as how a server might mount into an equipment rack.  In services, a front-end designer might think about how many key presses a user must make on their phone when they call into an interactive-voice-response system before they can talk to a human.

Back-end Designers are often called “Architects.”  In software, they are typically concerned with items like choice of codebase, code re-use, selection of libraries, how to build a system that is both scalable and secure, and whether to use Amazon EC2 or another cloud provider.  In hardware, they might choose the clock speed of the processor, or how much memory overhead needs to be planned for in the system for future upgrades.

Design is measured differently than Product Management and Development.  We teach that Product Management should be measured on its ability to be market-sensing.  That is, members of the product team should be measured on their effectiveness at bringing accurate market data into the business to feed better decision making.  Development is often measured on a combination of scope, time, and quality.  Design is not measured using these metrics.

How to effectively measure a Design team is a large question that incurs significant debate.  But, suffice it to say that designers have their own set of metrics to test the effectiveness of their designs.  For Design, measurement needs to answer the core questions: “Does the design effectively solve the user’s problem?” “Does the design fit into their daily life or workflow?”  “Does the design act in a way the user expects it to act?”

The bottom line is: Design is a huge role, encompassing a wide variety of areas.  It’s very rare that one person highly skilled at both front-end and back-end design.

TL;DR, Just Tell Me Who Owns Design!

In my experience, back-end design almost always reports to Development.  Because of the intertwining technical nature of the decisions these Architects are making every day, it makes sense that they would sit within the reporting structure of the team creating the solution.

Front-end design is a very different answer.  In the past decade, front-end design has migrated, from reporting to Development, to reporting to Marketing, to Product Management, to their own group.  Sometimes, they are contracted resources that report to whoever hires them.

Like Product Management, there is a natural tension that exists between front-end design and the Development (and Marketing) teams.  This tension arises from differing (sometimes competing) goals and different measurements of success.  Typically, this tension is a good thing.  However, when we subsume that tension and shove front-end design under Development, or Marketing, the tension resolves in favor of the executive who owns that team.  For instance, a Development team who owns Design and is measured on “hitting the date,” will naturally pressure its designers to create designs that help them hit their date.  This is probably not conducive to good designs for the user.

So who owns Design?  The designers, of course.  And who owns the designers?  I believe that the goals of Product Management are highly aligned with Design, so I can easily see these reporting up the same structure.  I can also see success for Design as an independent part of the Product Team.

Wherever you place Design in your organization, ensure that you have an honest conversation about how designers are measured.  If you measure Design like Development or Marketing – the results will be sub-optimal.

Becoming a Product Leader with Win/Loss Analysis

Nearly every product manager I speak to desires to be a leader in his or her organization.  Everyone knows that as a member of the Product Team, you must exercise soft skills like influence to nudge your product towards success.  However, to few know how to gain the confidence to become the leader that they want to be.  The first step toward becoming a great product leader is to have the best quality and quantity of information, so that you can attack any problem with data, not opinions.

Introducing Win/Loss Analysis

One of the most chronically under-utilized ways of getting to product data is by using win/loss analysis (WLA).  WLA involves developing a deep understanding of how and why your buyers make their purchase decisions, both for and against your product.  WLA data is interesting to Sales, because it could highlight breakage in the selling process, and it is highly interesting to the Product Team, because it can help you sharpen your understanding of competitive threats and positioning, what customers value the most in your product, and where to focus your product marketing tactics to match the buyers’ expectations.

How to do Win/Loss Analysis

Win/Loss analysis is typically done in phases.  The first phase is almost always qualitative, and involves direct one-on-one interviews with buyers.  In these interviews, the Product Team should focus on asking open-ended questions that generate new learning, such as:

  • What was the main driver of your purchase decision?
  • Describe the process you went through when making your decision.
  • What motivated you to start looking for a solution?

These questions cannot be answered with a simple yes/no, and will create good discussion that can be explored further.  If you do enough of these interviews, you might start to spot a trend.  Some companies feel this is enough data and stop here.  Other companies go to a second, more quantitative step, and test the trends they discovered in the first step through surveys to get more statically significant results.  The upside to this is higher quality data, the downsides include: cost, time, and the requirement of a large enough pool of wins and losses to sample from.  If you run a B2C product with take-it-or-leave-it pricing, the quantitative step may be easier than in a B2B model with negotiated deals.

Product Teams also need to consider who they target for WLA.  Wins are obvious – they went through the entire buying process, and chose our product.  Losses are less obvious.  Don’t think of losses a deals that went all the way to the end, such as the RFP or bake-off stage, and the evaluator didn’t pick your product.  Deals such as these are losses, but they constrain your view much too narrowly.  Instead, think of the funnel that you must guide evaluators through in order to close.  Every company has their own definition of the steps in the funnel, but a typical funnel might have these steps:

Typical funnel diagram

  1. Qualified Lead (from Marketing)
  2. Initial contact, sent marketing materials
  3. Meeting, assessment of needs
  4. Demo of our solution
  5. Competitive displacement
  6. Evaluator decision

An evaluator doesn’t have to get to step six to become a loss.  Many times, the evaluator might get to step two or three, only to become non-responsive.  That should still be considered a loss.  What if our marketing materials don’t speak to their pain?  What if we demoed the wrong solution for their needs?  The result of these missteps is the same – we lost the deal.  WLA searches for and finds the breakage in your process so the Product Team can address it.

WLA Cautions

Unfortunately, win/loss analysis can also be a political animal.  Over the years, many Sales teams have become jaded by WLA, and view it as a way to assign blame to them for lost deals.  That is not why we (should) to WLA.  WLA should be about finding issues in our overall process of making the sale, of which Sales is a part – but not the only part.  If the breakage is in our positioning or pricing or competitive response, these things have nothing to do with Sales, but could have a huge impact our Sales ability to sell.  In order to overcome political objections to WLA, we encourage teams to put their initial focus on analyzing wins.  Usually no one objects to win analysis and it is a good way to get the organization comfortable with the process.

Another important caution for WLA is who does the actual interview.  WLA involves getting honest and straightforward answers from a buyer, who make be skeptical of your motivations.  For that reason, Sales (and anyone involved in selling that particular deal) should not be involved in that WLA.  Because buyers lie to sales people – it’s in their nature.  If your salesperson calls on a buyer they lost to try to find out how and why the buyer made their decision, the buyer will immediately assume that the salesperson is trying to restart the deal – a now closed issue in their mind.  So the buyer will generally say whatever they have to say to get the salesperson off the phone as quickly as possible – usually defaulting to the “easy” answers of: “we didn’t go with you because of the price,” or, “your product was missing a key feature we needed.”  Is there any surprise that salespeople believe that the primary reasons they are losing are price and features?

WLA Generates Results

Win/Loss analysis can sometimes surprise you and lead you down a different path to product success.  One Product Team we worked with found out through doing win analysis that the reason they were winning was not that they were the low price leader (as their Sales team thought), but rather that their buyers put a major premium on their customer service and support.  In fact, the Product Team was able to illustrate that in 80%+ of the deals won over the past year, price was not the determining factor.  The next time Sales wanted to discount, the Product Team had some data to show that perhaps discounting wasn’t the right strategy.

Another Product Team we worked with found breakage in their renewal process by doing loss analysis, which over the course of a year saved over $1MM in recurring revenue!

Clearly, WLA is a powerful tool, and should be exercised more by Product Teams.

The Impact of Win/Loss Analysis on Product Leaders

A few quarters ago, I was teaching a Pragmatic Marketing seminar and was approached by a product manager.  He was discouraged because he didn’t feel like he had management’s buy in for being market driven and, as a result, was not getting any budget to do things like visit with the market.  He asked me for advice.

One of the things I told him to do was win/loss analysis.  First, it is relatively inexpensive, especially for wins.  Second, nothing speaks with more credibility than the buyer.  Third, he needed a quick win, and WLA can be fast.  I asked him to try doing a handful of WLA’s, share those with his executive team, and see what happens.

A few months later, he reached back out to me, and his demeanor had completely shifted.  He told me that his CEO was so impressed with the results of his WLA efforts, that they recreated their positioning, marketing materials, and selling process around the buyers’ expectations.  All of the sudden, the organization was looking to him, the product manager, for answers about the market, and his concerns about budget for occasional travel were starting to go away.  In short, a few well-timed win/loss analyses changed the company’s perception of him – and his perception of himself.

If you are interested in making the same transformation into a trusted product leader at your company, remember that first step to leading is following.  Specifically, following the market.  Once you have the market data on the tip of your tongue at all times, the confidence to speak with credibility about the needs of your buyers and users will follow shortly.

If you are intersted in going in-depth about win/loss analysis and lots of other topics of interest to Product Teams, consider attending a Pragmatic Marketing training.  I will be leading seminars on these topics on Mar 11-13 in Toronto, Mar 19-21 in Austin, Apr 8-10 in Tyson’s Corner, Virginia, Apr 29-May 1 in Denver, May 6-8 in San Diego, and May 13-15 in Princeton, NJ.  I hope to see you there!  In the meantime, you can always email me at pyoung@pragmaticmarketing.com with questions, or comment below.

ProductCamp Global Coordination

If you are a ProductCamp leader, or are thinking about starting a ProductCamp in your area, this post is for you.

There is a problem with how ProductCamps are organized and run today: they are very tribal.  Knowledge is passed on a one-to-one basis, or by physically traveling to another city to watch how that camp performs.  This is inefficient and error-prone, and leads to a “multiplicity” problem where we are making a copy-of-a-copy-of-a-copy and missing out on the chance to share what is working and what isn’t to create a kick-ass experience for everyone.

To solve this problem, I have joined with two other leaders in the ProductCamp community to kick off an new initiative: a Global forum for ProductCamp coordination.  Joining me is Colleen Heubaum, past President of ProductCamp Austin and small business owner.  She brings loads of experience in managing teams, wrangling personalities, and guiding ProductCamp through transition and growth periods.  Jon White is a board member at ProductCamp Seattle and the Product Management Consortium and has a broad range of experience in managing these same issues with ProductCamps.

Our goal is simple: provide a place where leaders of exisiting and prospective ProductCamps around the world can share best-practices, learn from each other, talk about challenges and provide solutions, and collectively raise the experience that we provide to the ProductCamp community.  To kick off this program, we are hosting a conference call/web meeting this coming Monday February 11th at 11:00AM Central time.  Leaders from over 30 camps have been contacted and we look forward to a good discussion about where we need to focus in the future.  We envision a monthly cadence of calls, which will be recorded and posted for anyone interested to listen to and gain knowledge from in the future.

If you would like to join the call, please feel free – we want this service to benefit everyone.  Details to join are below:

Step 1: Dial-In
U.S. & Canada:    866.740.1260
Australia: 1800642582 (Toll Free)
France: 0800942408 (Toll Free)
Germany: 08006645316 (Toll Free)
United Kingdom: 08004960576 (Toll Free)
Access Code: 7761906
Step 2: Web Login

If you can’t make the call for whatever reason (there is never a perfect time for everyone), please drop your info into this form, and tell us what topics you would like to hear the community of leaders talk about in the future.

If you have questions in the meantime, please feel free to email me.  Talk to you Monday!