Skip to Content.
Sympa Menu

internetworkers - Re: [internetworkers] Re: Software Development Nightmares

internetworkers AT lists.ibiblio.org

Subject: Internetworkers: http://www.ibiblio.org/internetworkers/

List archive

Chronological Thread  
  • From: BigLee Haslup <biglee AT haslups.com>
  • To: "Internetworkers: http://www.ibiblio.org/internetworkers/" <internetworkers AT lists.ibiblio.org>
  • Subject: Re: [internetworkers] Re: Software Development Nightmares
  • Date: Wed, 05 Mar 2003 13:02:35 -0500


Compared to implementation and testing, the cost of change is lowest in the
design phase. The cost of change is counted in minutes, not weeks and months.
In spite of this, the importance of design is often neglected.

This is one of those statements which, while altogether true, turns out to be somewhat less useful than it first appears. In point of fact, the way it is stated almost guarantees that it will be misapplied. I would argue that the development methods used in the open source community tend to minimize the costs attributable to design errors, but for reasons nearly opposite to those that have been put forth.

Consider the cost of picking the wrong number when playing the lottery. A single error -- picking a seven instead of a six, say -- can make tens of millions of dollars difference in the outcome. It is terribly important to pick the right number. And yet, so many people pick the wrong numbers that the lotteries are a consistent source of revenue for their sponsors. Why? that's easy: because there is no way to know the right numbers before you buy your ticket, and once you buy your ticket you are committed.

Any software development project beyond a certain degree of complexity is in some ways a bit like playing the lottery. It is too big to hold in your head and perceive in its totality. Because of this failure of imagination, the reality of the project will always have some aspects that are unexpected -- surprises that will call some of your design assumptions into question. Additional work on the design can reduce the number and severity of these design errors but the process is asymptotic -- no amount of effort will reduce the error count to zero and there is a point of diminishing returns.

In concrete terms this means that all major development efforts start with a process in which the users agree to accept functionality which will turn out not to be what they want or what they need. The developers, in turn, promise to deliver a system which they will not be able to implement. Careful and painstaking design work can reduce the size of the reality gap but no amount of work will eliminate it. As the design phase draws to a close, traditional project managers schooled in the importance of the design phase will hold some sort of ritual, designed to stress the supreme importance of doing careful work. The design is reviewed, approved, and all participants put a bit more of their professional credibility on the table as a guarantee of its correctness. Then, if they have missed something important, the death march commences.

What is different in the open-source world is that "design" is not a phase it is a process. By the time most open source projects release the first official design document several participants will have partial implementations that they have been playing with. After the design is released some participants will get on with working on the implementation and others will immediately begin second-guessing the design. Individuals will switch back and forth between coding and carping about the design, depending which they feel is the greater need. Anyone who feels strongly that a major mistake has been made can take the code and start his own branch. If the dissenting developer is a crank then he cranks away harmlessly and is ignored. If he is on to something then others will start defecting from the main body of the project. Sometimes this results in a new project being started. Sometimes the central controlling organization will notice the interest in the alternative design and merge the changes back into the mainstream.

Because open-source designs are always, to some extent, theories about the best way to solve a problem, they are always subject to experimentation and revision. There is very little stigma associated with questioning the design and so the process of re-evaluating and correcting the design is ongoing during implementation. From a point of view that prefers certainty early in the process this may seem inefficient, but it allows problems to be dealt with as they come up, rather than insisting that problems be dealt with before they come up and characterizing any later corrections as failures. It may be expensive to make changes to the design of a project six weeks into implementation but it is vastly more expensive to do so six months in.

The result is that successful open-source projects tend to follow most of the tenets of Agile Software Development; Individual interactions, the primacy of working code, collaboration with users and flexibility in planning are all there. Or, stated another way: "If you put smart people in a room, they'll know what to do" with the saving grace that people who don't buy into the program can go start their own rooms.




Archive powered by MHonArc 2.6.24.

Top of Page