How to Have the Worst Beta Ever
If you run Product Management, especially in a smaller company, you may find yourself running the beta program. This is a tactical activity, and you will be knee deep in beta user qualification, feedback, administration, possibly even support, but you can take positives from a good beta. Or you can fail miserably if you don’t understand that there are different flavors of beta, and different motivations and goals behind each.
One of the biggest issues I have run into with beta is that there are several types of beta. It took me (too) long to figure this out, so hopefully I can shortcut the learning process for you. Some flavors of beta:
- Post-Alpha Beta - This is what most people think about when they say “beta.” A final test to find bugs and make last second tweaks before you roll up for release. Depending on the size of company, sales model, and other factors, you may have “release candidates” that come out of beta, or several iterations of beta code.
- The Sales Beta - Sales is drooling over a feature in the next release and has to “get their customer into the beta” to make the sale.
- The Google Beta - A beta that never ends. Gmail is still beta after how many years…really?
- BINO (Beta in Name Only) - Your development cycle went long, and you don’t really have time for a good beta, and you can’t move the release date. Hope your QA is good!
- Not Ready for Beta - The opposite of BINO, your code isn’t ready but your beta testers are, so you put something in their hands that is better described as Alpha and they freak out.
You never want a BINO or Not Ready for Beta, and the good news is that Product Management controls those dates, or should. The Sales Beta is the worst, because they always promise up and down that their user will offer good feedback and will be a great beta participant - don’t believe them, because they lie! What really ends up happening is that Sales forgot to mention that this product was beta, the user gets the product and then they are both confused or angry that it is not ready for primetime. That is a sure fire recipe for failure.
The Google Beta is interesting because having a product in perpetual beta is convenient. Don’t like that bug? Sorry, it’s beta - see the image at the top left that says “beta” in 8pt light grey font? That might fly for free or ad supported products, but I doubt a paying customer will accept a product in forever beta.
What other kinds of beta have you seen?
10 Responses to “How to Have the Worst Beta Ever”
1 Tzvika 29 May 2008 @ 2:07 am
Great article!
I’ve also seen another kind of Beta: The so called “vanilla” release. Some organizations don’t do “Beta” - in stead they release a x.0 release, and assume that only early adopters will install it, and that it will be buggy in the extreme. It is taken as a given that the product is only reasonably expected to actually work as of the first service pack, the x.1 version.
2 Saeed Khan 31 May 2008 @ 6:45 am
I really push back hard on the “sales beta”. You can’t recognize revenue on beta software. The endless (i.e. Google) beta is an interesting concept. Not sure what that actually means anymore. Wonder if there is some legal liability issue or something that’s keeping it in perpetual beta.
BTW, I wrote a pretty detailed article about running successful beta. A link to it can be found here.
http://onproductmanagement.wordpress.com/2007/08/20/some-pm-reading/
Saeed
3 Jukka Eklund 1 June 2008 @ 8:35 am
Maybe there is also “a startup beta”, a product that’s built and released out to find more funding and possibly test the business model. Many of the “web 2.0″ sites have that feeling, don’t you think?
4 adam bullied 5 June 2008 @ 8:30 am
The alpha/beta label is used far too improperly. Especially in start-ups with consumer products, people will put the label on the product just to make it OK excuse that it’s buggy, but their VCs or management force it to get pushed out there.
If your product has a clear, distinct reason to be released as a beta (i.e., it’s time-limited, you want specific test results, etc…) like in the good ole days of software development, then cool. But slapping “alpha” or “beta” on your logo and calling it day when there are several SEV-1s is a major concern - that should not be part of your product lifecycle.
Really, if you can’t come up with a good reason to do run a beta program (with goals, and distinct purpose) than the product is probably not ready for public consumption.
The best article I’ve read on properly structuring a beta program (with distinct reasons and goals) is here: http://tinyurl.com/6b4m9g.
5 Weekend Reader: 7June08 » The Productologist 7 June 2008 @ 2:56 pm
[...] How to Have the Worst Beta Ever [Product Beautiful] [...]
6 Gopal Shenoy 12 June 2008 @ 12:51 pm
Hi Paul,
Great article. In my previous company, to avoid confusion between alpha and beta internally, we had the following definition:
Alpha is to find from customers whether you build the right thing to solve their problem - focus on functionality and not bugs (finding bugs is just a bonus)
Beta is to find if you build the right thing right - the focus here is nothing but finding bugs and fixing them.
Gopal
http://productmanagementtips.com
7 Michael Ray Hopkin 12 June 2008 @ 11:01 pm
Here’s a twist on the ‘Google’ beta: the SaaS beta. Develop or convert your products to a hosted/SaaS model, ratchet up your release cycles to monthly, then you can call it a ‘release’ or a ‘beta.’ Either way customers get their hands on the new functionality.
The SaaS model is great in many ways, but the traditional concept of beta changes dramatically. -Michael
8 What does “beta” mean for Software as a Service? (Jarrett House North) 16 June 2008 @ 8:06 am
[...] Johnson at Pragmatic Marketing points to an interesting article on five different types of betas. One of Steve’s commenters suggests there is a sixth kind, the SaaS beta: …ratchet up [...]
9 David Locke 17 June 2008 @ 11:06 am
Why do you let sales sell anything that hasn’t been released yet? I’ve seen that one kill a company. It was like the current release had never been released. Everyone wanted the next release. In the meantime, we had to get more investor money.
10 Anthony 19 June 2008 @ 6:47 am
“Why do you let sales sell anything that hasn’t been released yet?”
I’m sure several readers will empathise with me when I say that’s more easily said than done…
Leave a Reply