May
02

Why Product Management is Open Source’s Fatal Flaw

By on May 2, 2008

GNUFree Open Source Software (FOSS) is great – I have released code under the GPL, LGPL, and similar licenses. There are mountains of FOSS available right now for download from sites like SourceForge and others that save businesses millions of dollars. More importantly, open source software offers feature sets and mixes that often aren’t available in commercial products because the market is too small, commercial companies don’t understand it, or the problems aren’t profitable enough to solve.

The great promise of open source is that you can have equal or more functionality than commercial software for free, and you have access to the source code if you have the desire, time, and skills to hack it into something new. This model was perfect when developers were writing tools for each other, like text editors such as VI and Linux. Most FOSS projects aren’t under the stewardship of a commercial entity (although some of the most successful ones are, such as RedHat, Firefox, and OpenOffice), they are built by and for a handful of developers “scratching an itch,” and they are not working with a Product Manager. Unfortunately, FOSS has become a victim of its own success, and today, open source developers are facing a problem that threatens to turn legions of users against the software they rely on.

Most FOSS projects are a meritocracy, meaning that the developers care about developing for themselves and their own problems. If non-contributing users problems’ happen to be solved, great. If not, well – “you have access to the code, feel free to build that feature yourself!”

Users understand that free software comes with limitations: there is typically only ad hoc support, updates are only as frequent as the developers care to make them, and bugs may go un-addressed forever. But users don’t care because they are getting something for nothing, and were willing to put up with the lack of polish found in most FOSS. However, now free/open source software has become so widely distributed and used that the boundaries in user’s minds between FOSS and commercial software has become blurred.

Open source developers have created products so good, that they are nearly indistinguishable to an end user from commercial software. This has changed the mindset and expectations of users to think that they are the persona that the developer is writing code for, but are they? Some applications, such as Firefox, have made the leap and are clearly developing for an end user. For an example of a FOSS project that hasn’t, look no further than Pidgin.

PidginPidgin is a free and open source instant messaging (IM) client akin to MSN Messenger, Google Chat, or Trillian. Recently, in their 2.4 release, they changed the GUI action of the text field where the user types their IM from a manually re-sizable window, to a fixed size window that auto-re-sizes based on the amount of text typed. On the surface, this sounds like a minor change, but it triggered a massive user revolt! Why?

First, the Pidgin developers violated the Principle of Least Astonishment. You never take away functionality from the user when they upgrade. Second, the Pidgin developers let “Perfect become the enemy of Good.” If you take the time to read through the entire discussion (I don’t recommend this unless you have plenty of time), you see statements from the developers such as (paraphrased): “We want to find one solution that fits the needs of all users; we don’t understand/don’t agree with the use case that calls for a resizable input window.” Third, the developers became more and more entrenched as the discussion progressed, rationalizing the feedback as a “vocal minority,” and recommending that the users try to use other applications for their needs. Finally the thread devolves into the developers reminding everyone that they do this work on their own time, for their own enjoyment, and by the way they are closing the bug report and tagging it as “will not fix.” Harrumph!

The most interesting reply on the bug report is from Dan Livingston:

I teach “Collaboration in an Open Source World” at a local college. I have been searching for, and in this ticket have found, a perfect example where communication between open source developers and users fails at multiple, fundamental levels.

Obviously, the motivations of open source developers are varied; some do it for technical enjoyment, others enjoy knowing they are contributing intellectual capital to a better world. The problem is when the motivations of open source developers conflict with the expectations of users.

Consider every wildly successful open source project: the users are enthralled with their ability to perform new activities in ways previously unimagined. Rabid dedication grows, and an evangelical fan base results. Pretty soon, it’s obvious why users would not want to go with non-open source software alternatives.

What happens when those same newfound powers are taken away? What happens when the developers impose their personal dogmas upon the project? Even for as small an issue as chat window resizing, a minority (or majority) of users will emphatically express dissent.

It’s easy to see why open source developers could develop dogmas. Some like to fantasize about the theoretical limits to which a design may become “pure”, developing a vile repulsion to anything which steers away from purity. Others become obsessed with metrics such as maintenance effort per line of code, even though they often worry about features and lines of code which only contribute to 1% of the complexity of the application. Yet others develop fixation on “ultimate user simplicity”, feeling that two options are better than five options which deliver more power. The most dangerous dogma is the one exhibited here: the God feature. “One technological solution can meet every possible user-desired variation of a feature.”

The initial lure of open source software is that quality software should resoundingly meet the needs of users. As demonstrated up until Pidgin 2.4, the fan base has emphatically been extolling the virtues of Pidgin. But when developers take a feature away, presumably to implement a “better version”, and that better version in fact is a step backwards from the functionality previously available, they had better have a damn good reason. Such a reason is lacking here.

“This is how IM should be used.” “Our design is better.” “We will only consider a ‘pure’ design in which we can accomplish the old functionality in a paradigm that also supports the new functionality.” “An additional checkbox is too detrimental to the user interface.” “Maintaining two branches of logic within the dialog sizing component will be untenable.” “We have no interest in not pushing our shiny new object.”

These are all statements, which if executed within a corporate arena, would get developers fired. Developers, make note: you are doing a disservice to the community you claim to represent, and are doing so with false illusions that you are “right” because you have convictions in your justifications.

It does not matter that you are open source developers with the autonomy to ignore your user base. It does not matter that a plugin “could” be developed to solve the problem. It does not matter that you feel your default solution is superior. It does not matter that you only want to consider solutions which can be implemented through the new solution framework. It does not matter that your users should abandon your product if they don’t like it. It does not matter that someone could fork the code base. It does not matter if 11 thousand people download your source code per day, and only 270 complain about it. For each of these, there are very valid rebuttals

<snip>

The fact is that typing letters into an IM window is THE most critical task of an IM program. Users have varying needs, needs which can not be addressed by your limited attempts to come up with “one solution for everything” that incorporates “shiny new logic” that demonstrates how smart you are. You are ignoring the fan base with a dedication to your convictions that is alarmingly evident to even the most unobservant of followers, and as such, you are demonstrating that you no longer deserve to be in the position of servicing the needs of your user base.

For the sake of everyone involved, I hope you find your path back to the light.

Predictably, the very next reply is:

Do you have a lecture about insulting hobbyist developers you’ve never met on their bug tracker?

Later, Professor Livingston dresses down the development team with some well-placed satire, tongue firmly in cheek:

From now on, the user base of Pidgin should just assume that the following fictitious letter was ghost written for the Pidgin development team:

“Dear Users:

You are most undoubtedly reading this page because you want to know how to manually resize the text input area in Pidgin. It is a feature that you have perhaps grown accustomed to and comfortable with, but please make note – this feature is no longer supported.

We have received many complaints from a very small minority of the user base who nonetheless persists in being very vocal about their displeasure. Please take comfort in the fact that they are only a very small percentage base of Pidgin users, and if ignored, will go away and bother some other open source project.

Nonetheless, we feel it very important to make the following proclamation regarding on our stance on this project: we will not “fix” it. In fact, please notice that the status of this “bug” is “wontfix”. So would you please just get this idea through you head and go away now?

We are developers of this software, and we develop Pidgin so that it may fulfill our explicit needs and desires. If you want to join us for the ride, then fine. Just shut up, though. Please, if we’ve made a feature a certain way, it’s because WE WANT IT THAT WAY. Is that so hard to understand? All day long our bosses tell us what to do. Our wives tell us what to do. Our government tell us what to do. YOU will NOT tell us what to do.

Some say that as the developers of the premier open source IM client, we have a “responsibility” to serve as wardens of our precious charge, nurturing it into a fine, outstanding, model citizen of the open source community. That’s rubbish. The last time we checked, we didn’t sign up for day care. We signed up to write software that WE want to use.

So please take your ideas and go elsewhere. If you want a development team that responds to the desires of their user base, hoping to release world-class, quality software to millions of people, then start your own open source project. It’s not that hard. It’s free. All you have to do is commit your time, just like we commit ours.

The development team would very much like to come up with a solution that meets the needs of ourselves and the general user base. However, we cannot understand your needs. You speak in a foreign gibberish, gobbledy gook language that none can understand. “I just like it that way!” That is not an answer! You must enumerate the metrics and aspects of your preferences and desires in ways that we can evaluate and then assimilate into our collective. We cannot currently assimilate any of your idiotic reasons for wanting a resizable text box. And by idiotic, we mean “any solution which does not fit into the scheme of our cleverly intelligent auto-resizing text field.”

So, just to make it clear: we will not listen to your suggestions unless your suggestions make sense to us, and we like them. If you do not like it, there are plenty of other ways on the Internet in which you may occupy your time.

Regards, Pidgin development team”

Obviously, there is a huge gap between the expectations of the users and the developers. Who normally bridges that gap? Product Management.

If Pidgin had a Product Manager, they would have raised a flag on the change in functionality and would help the engineering team prioritize feature sets based on the needs of the target users. Unfortunately, most FOSS projects today don’t have Product Managers, and don’t have written personas or target users; they have developers working for themselves.

I propose that Product Management should take a more active role in FOSS, not telling the developers what users need, but teamingwith developers to identify the target audience and prioritize the users needs. If developers can donate their time working on these projects there is no reason Product Managers can’t do the same thing.

Another issue we can solve is one of expectations. Open source projects should adopt a clear tag, license, or some other marking indicating if the goal of that project is to “scratch an itch,” or if the project has gained the maturity to really care about servicing a base of non-contributing users. Free software programmers work on donated time, and as such are never obligated to serve anyone but themselves, but indicating the goal of a project up front at download-time (for the user), and at code writing time (for the developer) would set expectations appropriately and sideline these kinds of conversations.

Even on a “serviced” project, users need clear guidelines about what is and isn’t acceptable feedback. The discussion above started out as civil, but quickly turned into a shouting match, with each side digging in. At that point, no one could compromise without losing face. A FOSS Product Manager could have diffused the issue early, by involving users in the design (pre-code), and being a neutral party to explain the decision making process and trade offs.

Is anyone currently contributing Product Management to a Open Source project? Would you (as a PM) be willing to contribute Product Management expertise to a FOSS project? Are there any free software projects out there that would be willing to experiment with using a Product Manager (I am referring to FOSS projects that are not backed by for-profit corporations)? FOSS developers – would you be willing to work with a Product Manager on your project? Why/why not?

This story was also covered on Slashdot.

Others have discussed Open Source Product Management in the past.

Comments

  1. [...] Why Product Management is Open Source’s Fatal Flaw | Product Beautiful (tags: opensource productmanagement) [...]

  2. Don says:

    And proprietary software has such a great track record on product management? Like Vista? Sure.

  3. trev says:

    Aren’t you describing the role that the companies from Red Hat to Kickfire are playing? In some cases, like OpenBSD, the developers attracted users who want the same evolution of the system; no commercial entity need exist for product management. In the case of Red Hat, the niche opened where enterprise users needed a company to be product manager for the OS everyone wanted their commercial Unix apps (eg: Oracle) to run on.
    Some companies will live and die being niche product managers for FLOSS projects they add value to; others will prosper.
    To make money off FLOSS, you need to be both part of its development community (or at least have their respect) and able to find clients who already want FLOSS and are willing to buy your services. You need to be a FLOSS product manager.

  4. Bob says:

    A Project Manager is not needed. That is a commercial corporate mindset. In the FOSS world, there is a different method. FORK.

    That has already happened.

    If the users like the old way better, then they will go with the old system. The developers who like the old form better will pick up users. The developers of the new redesigned interface will change when they see the user numbers plummet. It happens.

    There is also a method in FOSS for the two groups to come back together. Just combine the code from both branches. It happens in the Linux kernal all the time. I hope it happens here after a while too. Time will tell.

  5. jpipkin says:

    And this is why the GNOME philosophy of “the user is stupid and will not understand these features; they must be removed” is an epic fail, and why I laughed like hell when Funpidgin forked to start putting features back into what was once a fairly usable IM client.

  6. Paul says:

    Forking is an answer, but IMO a very lazy and horribly inefficient way to solve the problem. This goes back to expectations setting. If the goal of a FOSS project is to scratch the itch of a few core developers – it should be labeled that way (or users should assume that *all* FOSS projects are unless labeled otherwise), and then users won’t expect reaction to their feedback.

    However if the stated goal of a FOSS project is to “create the best $whatever,” than “just FORK it” is a poor answer since “best” is defined by the userbase (as you indicated by stating that users would flow to the fork they liked best).

    Developers make hundreds of decisions on a project every time they sit down to write code; do we need to fork every time we disagree? NO…

  7. Sum1 says:

    The exact same thing happens in commercial software as well. Why does mac os not have a “cut” feature in its file manager, why has windows vista, arguably the most managed product in software history been the target of large numbers of irate users?

    Product management may help, but it may not help at all, and just distance your users from your developers, which always reduces time to live. Forking is far more efficient than waiting for a rival company to provide the answer – the code base is already there.

    Social inertia, rather than fiscal inertia becomes the only limiting factor to the rate of forking. Arguably the whole process is somewhat darwinian, and will produces software in the end that satisfies the majority of the users.

  8. Sarah says:

    It seems to me, the developers of pidgin have a major bug up their ass about this text box. That’s fine. They are, by their own definition, hobbyiest dev’s so they don’t program for a living, hopefully the fork will acquire some professional devs who’ll STFU and listen to their userbase before going “Ya know? Perhaps both would be a good idea”.

  9. oiaohm says:

    Nop its part of natural development of open source projects. Forking is a test of strength of a idea.

    If a Forks reasons are valid over time it gets enough strength to either kill of its parent project or cause it parent to change direction.

    Most cases forking is avoided and a all round agreement is found.

    There have even been cases where the problem is both groups are right.

    Forking does have negitaves. Project management is not just a Open Source problem. Windows Vista is a great case of that. Except open source does provide you with a way to get passed a bad selection. Fork the project. Vista we are kinda stuck with its evil bits if we want to use Vista.

  10. Bob says:

    Paul, Nice of you to respond, thanks.

    The stated goal is probably wrong for the true (amateur/hacker) free software developer. They aren’t trying to build the ‘best’ software, they are trying to build software that mostly works for them. Best comes later. It’s a result of continuous improvement (often by others) and time. It is frequently more of an evolutionary approach. That’s why there is so much road kill in the repositories. Only the best of breed survive. Often as combinations of previous projects, now both dead.

    What you are describing is more the approach that Corporate entities use. They need to conserve their resources. There are Corporate entities in the FOSS community, so you are not wrong either. Life sure is complicated.

    I think it’s a question of resources. The type of efficiency you are advocating is necessary when there are a limited number of developers, and will produce a better product under those circumstances.

    The way I described earlier is better when there are a relatively large number of developers. Then, the limiting resource is user attention. Eric Raymond sort of described that process in his Cathedral and Bazaar essay. If you haven’t read it, go ahead. In those terms, you are on the Cathedral side, I’m seeing this case (Gaim/Pidgin) more in Bazaar terms. It’s history is certainly one of going from a barely usable system to best of breed in fits and starts. I believe they have taken wrong turns before, too.

    Other projects are different. I believe that Open Office comes down more on the Cathedral side, for instance. Gnome is trying to be on both sides of the road. Really, each project is different.

    I do agree that the Pidgin project (and probably most others) could benefit from the discussion you want. Understanding is always good.

    Is that what you are really trying to say?

  11. [...] Why Product Management is Open Source’s Fatal Flaw | Product Beautiful (tags: opensource productmanagement) [...]

  12. Dave says:

    You mean to tell me, Pidgin devs couldn’t put a check-box in their settings area that could activate/deactivate resizable IM windows???

  13. Paul says:

    Dave, they could have…but chose not to.

  14. [...] I’ll define as users who are not also developers. Paul Young of Product Beautiful presents an thought-provoking case. Open source developers have created products so good, that they are nearly indistinguishable to an [...]

  15. tomek-k says:

    Answering to the questions put at the end (as a FOSS project developer) I think this is a great idea to try out the Product Management. I think it could be useful.
    Some commenters above claimed that in case of FOSS this is not necessary, but maybe it would be good to look at it from a bit different point of view than “Product Management as a way to set goals for a FOSS project”.
    When you start a FOSS project, that is intended for users (particularly for a large user base), then one of the primary goals, that is common to all user-oriented projects, is that the piece of software you want create has to be useful to the users (either practically usable or entertaining). You have designed the project in a way so it theoretically will achieve the goal, but you can never be sure what the users want. You don’t sit in their heads, you look at your project from your own, subjective point of view. Besides, if the project is big enough so it takes really lot of time to acomplish it, then the way the project should ‘be’ can also change with time.
    Product Management is the thing that could be truely helpful in such situations (I write “could”, because I’ve never tried it before, so I don’t know for sure), adjusting the little parts of your ‘great plan’ so the program becomes realy good (and useful). BTW read the famous “Cathedral and Bazaar” once again – there is a very good example of what I mean (but in that case the developer himself communicated with his users, so I don’t know if it could be considered a PM).

  16. Well, we’ve all certainly seen products that were bad, for whatever reason. Sometimes it’s because the product was designed by committee like Vista and OS/2 and the Edsel. Sometimes because it was designed by a bad designer or not designed at all, as Alan Cooper expressed in Inmates.

    I’ve worked with bad developers, bad designers, and bad product managers. Haven’t you? But I’ve also worked with brilliant developers, designers, and product managers. I hope you have.

    The reason that we love the iPod is because Jonathan Ive has brilliant taste in design and Steve Jobs values it. The Blackberry is terribly ugly but it absolutely meets the needs of the sales people for whom it was designed.

    If we want to program to ourselves and people who value what we value, then there’s no need for product management. Developers can say “I want this” but they typically cannot say “I know the market wants this.” Every agile method advocates a customer representative. Who in your organization is meeting people in the market to understand product requirements?

    Product management can help technology companies decide what matters.

    Developers say “this can be done.” Product management can say, “This should be done” and explain with supporting research. But if product managers are saying “this is my opinion,” then I’d rather just go with what development wants to build.

    It’s not big company versus small that is the question. It’s business or hobby. With limited resource, we have to ensure that those resources are working on the right things. If you’re in a hobby you guess and assume. If it’s business, you need to know–based on data, not opinions.

  17. Anonymous Coward says:

    You’ve just made a case for
    Professional Open Source Software (AKA the Beekeeper Model).
    http://wiki.pentaho.com/display/BEEKEEPER/The+Beekeeper

  18. [...] kicked over a small ant hill last week with my post about the (lack of) Product Management being a fatal flaw to open source [...]

  19. [...] Why Product Management is Open Source’s Fatal Flaw | Product Beautiful: Building Product Manag… I like how the initial defense from The Faithful Commenters is “we’re just as bad as commercial software.” Way to run to the knives. (tags: via:bwhichard opnsource productmanagement usability) [...]

  20. [...] Why Product Management is Open Source’s Fatal Flaw | Product Beautiful: Building Product Manag… While many good points, it would be nice to find more data points besides the pidgin idiocracy around the resize window, which I think is open source at its absolute worst. (tags: opensource productmanagement pidgin) [...]

  21. [...] Why Product Management is Open Source’s Fatal Flaw | Product Beautiful: Building Product Manag… “I propose that Product Management should take a more active role in FOSS, not telling the developers what users need, but teamingwith developers to identify the target audience and prioritize the users needs.” (tags: opensource productmanagement) [...]

  22. [...] exactly what happened to the Pidgin project recently. In their 2.4 release they changed the GUI action of the text field where the user types [...]

  23. [...] exactly what happened to the Pidgin project recently. In their 2.4 release they changed the GUI action of the text field where the user types [...]

  24. ADev says:

    Well, I participated in an open source project which had a member who acted like a [project manager] / [Director] for some time. His direction increased the efficiency of the team and produced several happier users…but it didn’t work too well.

    Firstly, developers started feeling pushed. Part of this was the Director aspect: setting deadlines, following up with devs on % completion, etc. Being hobbyists, the developers generally tried to avoid all questions of ETA, or responded with an ETA that could please this director only to attempt to postpone it later (having to deal with extra pressure because they had already promised a feature). Well, this is off-topic though. This is not on the Project Management side of this.

    On Project Management it was actually worse!!! In the beginning, his role was welcomed. It seemed that he really checked what users wanted in general, weighted and prioritized tickets, streamlined communication and development, etc. However, soon enough it all came down to “the project manager is a hobbyist, so he’ll allocate his time as he pleases” which also translates to “partially” checking the users needs, based on the importance that the components had for HIM! In other words, instead of prioritizing globally, he prioritized among tickets from select components of the project, leaving other components behind.

    Taking it a step further…how does a person become involved with development of an FOSS? Well, firstly one needs to find the FOSS. There are many ways and reasons one may find a project, but it is probably fair to say that two are the most common:
    a) A Person has a need for a tool (as a user), searches around, and lands on an Open Source project. First becoming associated with the project as a mere user, he/she eventually decides that he/she can donate some free time and utilize his/her skills to make the project “better”.
    b) A person knows he/she wants to develop a certain type of project and searches for existing solutions or initiated efforts. Then joins the original team being happy to have found it.

    Among those two, #a is by far more common (in my experience at least). So…you have a bunch of hobbyists who started as users (i.e. have or at least had a personal need for the product) and are now building it. This is the devs developing for themselves aspect…Fine…but what happens when the #a case is a Project Manager?

    Even if one attempts to be objective, even if one is a highly experienced Project Manager who KNOWS how to deal with users and devs, the whole process changes when he/she is a user him/herself! As a hobbyist, there is a limitation in the time allocated and often in the attitude with which time is allocated. As a user, there is an implicit subjectivity…one will always want to push forward features that are more important for one’s own person. Then there is the sub-component bias, as I mentioned earlier. Given limited hobbyist time, one may only deal with 1 part of the project, doing a great and objective job there, perhaps, but having made the subcomponent choice more subjectively.

    And finally, even if the Project Manager is 100% excellent in the job…there is no guarantee than devs will WANT to listen to him/her. Because of all of the above (and more) reasons and expected behaviors, devs are (or may grow to be) biased in treating the Project Manager as a mere user, even if he/she had been accepted as a non-coding team member. In that case, it doesn’t matter how objective the Project Manager is. The recommendations are perceived as subjective and are as ignorable as the loud voices of the “270 out of 11000 users”.

  25. Ibod Catooga says:

    The problem with the product manager idea is that the Pidgin devs are huge douches who don’t know their asses from a hole in the ground.

    Also, they want to Richard Simmons videos. I caught them doing it. I was disgusted. I put in a Jessica Biel video, but they started to cry.

  26. Grammar Gnome says:

    “This model was perfect when developers were writing tools for each other, like text editors such as VI and Linux.”

    Linux is NOT a text editor!

  27. [...] … product management in open source? That can’t be, and in some cases its true (See the summary of Pidgin’s resizable textbox).  In other cases there is too much “traditional” [...]

  28. Conroy says:

    I stumbled upon your website, i think your web site is awsome, keep writing.

  29. [...] … product management in open source? That can’t be, and in some cases its true (See the summary of Pidgin’s resizable textbox).  In other cases there is too much “traditional” product [...]

  30. […] of all, let me introduce the “causus belli”: Pidgin Bug Report, an article on Product Beautiful and a further Article on Product Beautiful on the […]

Leave a Reply