Problems of Estimation

If you are PM at a startup you will often find yourself being the “go to” person for all G2M activities up to and including coordinating from requirements through First Customer Ship (FCS). Even though the Product Manager isn’t/shouldn’t become a Project Manager, you will often find yourself “owning” the date. Lots of us still miss our ship dates, but that doesn’t make it right. It sucks to miss a date.

Missing dates screws everything up: advertisements placed months in advance must be pulled or adjusted, customer expectations need to be reset, and most importantly, revenue realization from the new products and features is delayed. I’ve been at this for almost a decade (gulp!) now, and I’ve started to see some patterns that hopefully we can all take advantage from. Let’s use this space to facilitate a discussion of why schedule slips happen, and how to prevent and mitigate schedule changes…

When I first started my career as a programmer, I thought that most schedule misses would come from “the problem.” Meaning that the problem that Development was being asked to solve was so difficult and unprecedented that it would be difficult to solve. I was wrong – almost all of the issues I have seen a development team experience have been problems of estimation, not delivery. Estimation and delivery are explicitly linked – give a team forever and they can solve any problem, but the rest of the business can’t wait that long. Product Management writes requirements to scope the size of the problem down for Development to assist in developing estimates, so assuming you’ve helped your Development team by providing the scope of the problem size, the next major area of breakage I see are estimation accuracy and what I call “launch completeness.”

When a Development Lead or Architect looks at a new problem, they ask the question “is this problem like any work we’ve done before?,” and “can our prior expertise assist us in solving this issue?” If the answer is yes, than they can estimate with more accuracy. If the answer is no, there is a large chasm of uncertainty where there is estimation risk. The Dev Lead may attempt to account for this risk with a variety of tactics such as padding or asking his team to provide more detailed estimates (Joel on Software’s approach is my favorite because it forces the developer to actually think through the implications of a design before giving estimates). Development may even want to transition to more radical methodologies such as Agile, where schedules are fixed and scope is flexed (may be appropriate for software, more difficult for hardware). There are also tactics that the Product Manager can take to protect the business from mis-set expectations, which I’ll describe further down.

Launch Completeness is a concept that one of my managers illuminated for me in a team meeting long ago. I’ll never forget the scenario: the Executives were in the room asking why a product/service couldn’t be launched in 90 days vs. the longer time that Product Management wanted. He put up a slide entitled “Work Items Required for Launch.” The first item, in tiny, 12-point font at the top left hand corner read “Development Complete” … then he proceeded to build the rest of the slide which added 3 additional columns of work items required for launch such as “Complete Documentation” “Train Support” “Train Sales” “Publish Demos” and so on. The point is you need to account for the other items you need at launch so you don’t suffer what I refer to as “stumbling across the goal line” and have an ineffective launch just because development finished writing their code.

Tactics to Win

Some of these tactics I’ve used with success, others I am still fine tuning:

  • Separate your Roadmap from your Release Plan – this allows you to focus on those products where you have resources committed and launch is imminent, while looking at futures through a different lens.
  • Use a “Model Schedule” to estimate – work with your Development counterpart to create a model or baseline schedule for a theoretical product. Overlay this schedule on top of new projects as they are starting and you will be able to quickly tell if you going wildly out-of-bounds.
  • Make use of a safety zone – For products on the Release Plan, don’t announce them publicly until you hit Pilot (hardware and software complete). Count on the other activities for Launch Completeness taking X days (we use 90) and you can always launch early if you finish early. If you do this correctly, you will know at the beginning of every quarter which products you can launch: if they’ve hit pilot, you will. If not, you won’t.

The last one is the hardest – especially in a startup. The business wants to move so fast that the idea of slowing down the process is a tough sell. Look at your process today, are you “stumbling across the goal line?” Do you get to your launch date and Engineering is still finding bugs? Do you slip your ship date by 2 weeks every 2 weeks? If any of those are true, you’re already moving slower than you’d like, it’s just your process that hasn’t caught up.

Previous Post Next Post

3 Comments

  • Reply Tom Grant January 24, 2008 at 1:34 pm

    Good suggestions, all. I’d even expand the list of difficult-to-estimate projects to user experience changes. They’re not as technologically dense as, say, re-building the core libraries of a complex middleware application from scratch. However, it’s just as hard to estimate a UI overhaul, since you can’t really tell from the beginning how many different tasks or use cases are going to need a thorough drubbing.

    How valuable would it be to get hard data, across the industry, on the real sources of product slips? I think all of us in the PM profession can tell good stories, but at an aggregate level, are there any patterns? I’m asking, in part, because I now have the luxury of putting my time into studying these sorts of questions.

  • Reply Paul January 24, 2008 at 4:08 pm

    You’re right, GUI changes are basically impossible to estimate because you can go through the entire design process, build it, and then find out that it failed – only by putting it in front of a user.

    I for one would love to know where product slips happen at an aggregate level. Even better would be to segment it by hardware companies vs. software vs. services, and/or by vertical e.g. consumer electronics vs. networking vs. Web 2.0.

  • Reply Switching Teams | The Productologist: Exploring the Depths of Product Management December 1, 2008 at 7:15 am

    […] and its role and effects on Product Management on many famous Product Management blogs: CrankyPM, Product Beautiful, All About Product Management (there are many more, just click on any of the blogs on my Blogroll […]

  • Leave a Reply