Our Thinking

Software History: Waterfall – The Process That Wasn’t Meant To Be

Posted by Chris Kessel on Jan 14, 2013 5:33:37 PM

“In my experience, the simpler method has never worked on large software development efforts…”

– Royce, Winston (1970), "Managing the Development of Large Software Systems", Proceedings of IEEE WESCON 26 (August): 1–9

The quote is from Winston Royce’s original article on the waterfall process where the “simpler method” he mentions is the waterfall process as most of us know it, with rigid handoffs from Requirements to Analysis to Design and so forth. Royce creates this idealized process as the starting point for a discussion on the complexities of managing large software projects. However, Royce immediately acknowledges such a simple model “is risky and invites failure” and proceeds to offer suggestions on how to tailor the process to be more useful.

Unfortunately for all of us, the simple version escaped from its roots as a discussion point and took hold in the industry. Somewhat ironically, Royce’s own suggestions move towards a more iterative model. Though the beginnings of true iterative development were in practice in the early 1970s at places like IBM’s Federal Systems Division (with some of the earliest writings by Harlan Mills), it would take well over two decades for iterative models to slowly displace waterfall as the dominant software development process.

Royce’s Five Steps

After acknowledging the simple waterfall model’s limitations, Royce offers five suggestions to help improve the model and give it some flexibility. It’s interesting to contrast some of these points with what’s become commonplace in more modern iterative models.

1 – Program Design Comes First

Royce notes that a key problem with the simple model is that there is little technical constraint feeding into the analysis phase, which can lead to analysis models that simply can’t be implemented, such as expectations of unrealistic performance or storage requirements. Royce advocates a “Preliminary Program Design” step before the analysis phase. In many iterative models, this is termed something like the proof of concept or architectural baseline. Agile process advocates (sometimes called agilists) may recognize this as akin to a spike or “iteration zero.”

Royce advocates this preliminary design because, as a result of the knowledge gained, “all the analysts and all the program designers will contribute to a meaningful design process.” In a nutshell, Royce wants to uncover technical risks early.

2 – Document the Design

Early on, there is no program and so the only means to convey program specifics to a large group of people is through good documentation. While Royce does advocate what many today would deem a heavy level of documentation, he does make a one very specific recommendation that may sound reminiscent of iterative and agile approaches:

  • “Until coding begins these three nouns (documentation, specification, design) denote a single thing.” Royce recognizes that early in a project’s lifecycle, an overdose of rigid delineation in documentation is counterproductive. He advocates a gradual model where such documentation is fleshed out as the project evolves. This may not quite match the iterative models, but it does speak to the iterative heart of specifying more as we learn and avoiding over specification up front when our knowledge still has significant gaps.

3 – Do it Twice

Because of the complexities in creating a fully functional system in one sweep, Royce suggests that the Preliminary Program Design “is actually the entire process done in miniature, to a time scale that is relatively small with respect to the overall effort.” For anyone whose read Mythical Man-Month, this may sound similar to Brooks’ suggestion to Plan to Throw One Away. And again, it echoes the common iterative step of doing a proof of concept. For agilists, the motto “the simplest thing that could work,” typically a vertical slice through the application, may come to mind. Whichever way you look at it, Royce is advocating an early exploratory iteration.

4 – Plan, Control and Monitor Testing

Royce advocates early and incremental testing. Technology has moved in giant strides since Royce’s article which makes some of his suggestions a bit dated, but two of his suggestions capture best practices now heavily ingrained in iterative and agile approaches.

  • “Every bit of analysis and every bit of code should be subjected to a simple visual scan by a second party.” Royce advocates for document and code reviews, but notice his use of the phrase “simple visual scan.” He’s not looking for heavyweight processes, but rather simple reviews such as those encompassed by informal document reviews and agile practices like pair programming. That’s not to say that formal reviews don’t have their place, but the goal is a constant level of small reviews rather than infrequent large reviews.
  • “Test every logic path in the computer program at least once with some kind of numerical check.” In today’s world, such checks are embedded in the iterative and agile practices of continuous builds (or at least daily), unit testing and code coverage analysis.

5 – Involve the Customer

The value of involving the customer is well known today, but Royce specifically recognized it as critical long before agile processes elevated customer interaction as a best practice, and even before techniques such as use cases were well established. There’s little more that need be said and Royce hits the heart of the matter: “the insight, judgement, and commitment of the customer can bolster the development effort.”

Conclusion

As a framework for discussion, Royce’s waterfall model works well to elicit questions such as “What belongs in the Analysis stage?” and “What’s affected if the Design has a flaw?” As a process for actual development though, waterfall as we know it was never meant to be. Royce’s suggestions even hint at practices eventually given fuller consideration in iterative models. Although we may lobby for a particular model or set of practices in our own projects, we are typically constrained to the process favored by our employer or client. Should you find yourself on a project requiring the waterfall process, think back to Royce’s five suggestions and see what additions you might be able to implement to make the process better serve the project’s needs.

* An interesting historical note: the term “waterfall” is never actually used in Royce’s original paper.

Topics: Design, Methodologies

Our Thinking - The Online Blog is a source for insights, resources, best practices, and other useful content from our multi-disciplinary team of Onliners.

Subscribe to Blog Updates