How to Have the Worst Beta Ever

BetaIf you run Product Management, especially in a smaller company, you may find yourself running the beta program.† This is a tactical activity, and you will be knee deep in beta user qualification, feedback, administration, possibly even support, but you can take positives from a good beta.† Or you can fail miserably if you don’t understand that there are different flavors of beta, and different motivations and goals behind each.

One of the biggest issues I have run into with beta is that there are several types of beta.† It took me (too) long to figure this out, so hopefully I can shortcut the learning process for you.† Some flavors of beta:

  1. Post-Alpha Beta – This is what most people think about when they say “beta.”† A final test to find bugs and make last second tweaks before you roll up for release.† Depending on the size of company, sales model, and other factors, you may have “release candidates” that come out of beta, or several iterations of beta code.
  2. The Sales Beta – Sales is drooling over a feature in the next release and has to “get their customer into the beta” to make the sale.
  3. The Google Beta – A beta that never ends.† Gmail is still beta after how many years…really?
  4. BINO (Beta in Name Only) – Your development cycle went long, and you don’t really have time for a good beta, and you can’t move the release date.† Hope your QA is good!
  5. Not Ready for Beta – The opposite of BINO, your code isn’t ready but your beta testers are, so you put something in their hands that is better described as Alpha and they freak out.

You never want a BINO or Not Ready for Beta, and the good news is that Product Management controls those dates, or should.† The Sales Beta is the worst, because they always promise up and down that their user will offer good feedback and will be a great beta participant – don’t believe them, because they lie!† What really ends up happening is that Sales forgot to mention that this product was beta, the user gets the product and then they are both confused or angry that it is not ready for primetime.† That is a sure fire recipe for failure.

The Google Beta is interesting because having a product in perpetual beta is convenient.† Don’t like that bug?† Sorry, it’s beta – see the image at the top left that says “beta” in 8pt light grey font?† That might fly for free or ad supported products, but I doubt a paying customer will accept a product in forever beta.

What other kinds of beta have you seen?

Previous Post Next Post

14 Comments

  • Reply Tzvika May 29, 2008 at 2:07 am

    Great article!

    I’ve also seen another kind of Beta: The so called “vanilla” release. Some organizations don’t do “Beta” – in stead they release a x.0 release, and assume that only early adopters will install it, and that it will be buggy in the extreme. It is taken as a given that the product is only reasonably expected to actually work as of the first service pack, the x.1 version.

  • Reply Saeed Khan May 31, 2008 at 6:45 am

    I really push back hard on the “sales beta”. You can’t recognize revenue on beta software. The endless (i.e. Google) beta is an interesting concept. Not sure what that actually means anymore. Wonder if there is some legal liability issue or something that’s keeping it in perpetual beta.

    BTW, I wrote a pretty detailed article about running successful beta. A link to it can be found here.

    http://onproductmanagement.wordpress.com/2007/08/20/some-pm-reading/

    Saeed

  • Reply Jukka Eklund June 1, 2008 at 8:35 am

    Maybe there is also “a startup beta”, a product that’s built and released out to find more funding and possibly test the business model. Many of the “web 2.0” sites have that feeling, don’t you think?

  • Reply adam bullied June 5, 2008 at 8:30 am

    The alpha/beta label is used far too improperly. Especially in start-ups with consumer products, people will put the label on the product just to make it OK excuse that it’s buggy, but their VCs or management force it to get pushed out there.

    If your product has a clear, distinct reason to be released as a beta (i.e., it’s time-limited, you want specific test results, etc…) like in the good ole days of software development, then cool. But slapping “alpha” or “beta” on your logo and calling it day when there are several SEV-1s is a major concern – that should not be part of your product lifecycle.

    Really, if you can’t come up with a good reason to do run a beta program (with goals, and distinct purpose) than the product is probably not ready for public consumption.

    The best article I’ve read on properly structuring a beta program (with distinct reasons and goals) is here: http://tinyurl.com/6b4m9g.

  • Reply Weekend Reader: 7June08 Ľ The Productologist June 7, 2008 at 2:56 pm

    […] How to Have the Worst Beta Ever [Product Beautiful] […]

  • Reply Gopal Shenoy June 12, 2008 at 12:51 pm

    Hi Paul,

    Great article. In my previous company, to avoid confusion between alpha and beta internally, we had the following definition:

    Alpha is to find from customers whether you build the right thing to solve their problem – focus on functionality and not bugs (finding bugs is just a bonus)

    Beta is to find if you build the right thing right – the focus here is nothing but finding bugs and fixing them.

    Gopal
    http://productmanagementtips.com

  • Reply Michael Ray Hopkin June 12, 2008 at 11:01 pm

    Here’s a twist on the ‘Google’ beta: the SaaS beta. Develop or convert your products to a hosted/SaaS model, ratchet up your release cycles to monthly, then you can call it a ‘release’ or a ‘beta.’ Either way customers get their hands on the new functionality.

    The SaaS model is great in many ways, but the traditional concept of beta changes dramatically. -Michael

  • Reply What does “beta” mean for Software as a Service? (Jarrett House North) June 16, 2008 at 8:06 am

    […] Johnson at Pragmatic Marketing points to an interesting article on five different types of betas. One of Steve’s commenters suggests there is a sixth kind, the SaaS beta: …ratchet up […]

  • Reply David Locke June 17, 2008 at 11:06 am

    Why do you let sales sell anything that hasn’t been released yet? I’ve seen that one kill a company. It was like the current release had never been released. Everyone wanted the next release. In the meantime, we had to get more investor money.

  • Reply Anthony June 19, 2008 at 6:47 am

    “Why do you let sales sell anything that hasnít been released yet?”

    I’m sure several readers will empathise with me when I say that’s more easily said than done…

  • Reply Michael Fine September 25, 2008 at 10:29 am

    Hilarious! As the author of “Beta Testing for Better Software” and one of the founders of the beta testing firm Centercode, I have to say these are just the tip of the iceberg.

    There is the “Process Demands It Beta” in which the company is committed to beta testing; includes it in the schedule but doesn’t care what comes out of it.

    The “Good News Beta” is when the testing is done to just to grab marketing data and testimonials. Sadly, the bugs and issues get neglected to make a release date.

    The “Bed News Beta” is the opposite where they use the beta test to help EOL a product. The product manager thinks a new version is needed and ignores key data that could help future development.

    I could go on forever with examples of common beta mistakes, misuse and issues. I have conducted more than 500 of these tests and I have to say, it is one of the most undervalued processes in product development.

    Good product managers embrace beta testing and make the most of it. The results of a good beta test are invaluable in so many ways. Thanks for the fun read!

  • Reply The Cranky Product Manager December 19, 2008 at 5:25 pm

    Apologies. The Cranky Product Manager did not realize you already had a post outlining and describing 5 types of Beta tests, including one called the Google Beta.

    Alas, the Cranky Product Manager posted her one too-similar list yesterday. She honestly was not aware of your article ahead of time and apologies for any overlap in content or concept.

  • Reply Eric Honda June 6, 2009 at 6:01 pm

    You should look at the company called VOConline

  • Reply Beta Testing in a SaaS World | Cloud Computing & Bad Behaviour April 26, 2010 at 5:19 pm

    […] read with interest a comment on one of the posts from the Product Beautiful blog.¬† The topic of the post was about the 5 types of beta programs that organisations commonly end up […]

  • Leave a Reply