Our Thinking

Software History: Learning from our Legends

Posted by Chris Kessel on Oct 22, 2012 3:24:59 PM

Learning from those who have come before us is a practice that transcends profession or industry. Professionals of most any industry learn what the masters of their craft have added to both theory and practice. Can you imagine a classical musician who knew little of Mozart? Or a physicist or mathematician who was unaware of Isaac Newton’s contributions?

Sadly, the software industry places little emphasis on its own history. Very few software developers or managers have read Frederick Brooks, Martin Fowler, Robert Martin, or any of the many luminaries in our field despite the fact that lessons from 40-50 years ago still speak to issues we see in every project. How have these lessons aged so well in an industry with such fast moving technology? The answer lies in the fact that the difficulties in software development tend to be due to human factors rather than technical issues.

Today, I’d like to do my part to spotlight one of software’s iconic figures, Frederick Brooks, and discuss a couple highlights from his seminal work: The Mythical Man-Month.

Background

The Mythical Man-Month is a compilation of articles published by Frederick Brooks in 1975, probably the most famous of which is No Silver Bullet. Each article discusses an issue Brooks ran across in his experiences with large software development projects. In the 20th anniversary edition, Brooks reflects on how the articles have aged and comments on other developments since the original publication date.

The Mythical Man-Month

This article is the namesake of the book and vies with No Silver Bullet as the most well-known of Brooks’ work. The short summary is that software projects are often late because, as an industry, we don’t do a good job of either estimation or measuring progress. Additionally, when our projects inevitably run late due to these difficulties, the natural response of adding people often aggravates the problem rather than helping.

First, a look at estimation. (For an excellent book on estimation, see Software Estimation: Demystifying the Black Art by Steve McConnell). Brooks notes several issues around estimation, but a key issue is that our estimates tend to operate on the total sum of effort which is then divided by the workers available, hence the term “man-month.” If a project would take 12 months to finish, then the assumption is that four people could finish it in three months. Brooks notes the core problem lies in that assumption:

“Men and months are interchangeable commodities only when a task can be partitioned among many workers with no communication among them.” 

It takes effort to partition tasks and coordinate people and that effort raises exponentially the larger the team gets. Further, some tasks are not well suited to partitioning. Our estimates are rarely, if ever, refined to add overhead for these human factors.

Second, what happens when a project starts to run late? The natural response is to allocate more people, but this response is also built on the assumption that tasks and effort are easily divisible. The reality is that adding a new person to a project requires training, ramp-up time, and reallocation of tasks, all of which are significant efforts that aren’t reflected in the original estimates and typically require time from members already on the project. If a project is late enough in the game, adding people can make things worse rather than better, which is known as Brooks’ Law:

“Adding manpower to a late software project makes it later.”

We often hear about the software triangle of scope, schedule, and resources.  The Mythical Man-Month drives home the lesson that manipulating the resources leg of that triangle also impacts the costs in the other legs.

No Silver Bullet – Essence and Accident in Software Engineering

This article is probably the most well-known of Brook’s essays, at least by name, if not in content. The title is often shortened to just No Silver Bullet and taken to mean that no single thing can be the silver bullet, which defeats a project’s problems. While that straightforward meaning has some truth, it unfortunately misses the key points about the essence and accident of software development and it’s in those points that the real value of the lesson is hidden. Brooks starts the article distinguishing between essence and accident as follows:

“All software construction involves essential tasks, the fashioning of the complex conceptual structures that compose the abstract entity, and accidental tasks, the representation of these abstract entities in programming languages and the mapping of these onto machine languages within space and speed constraints.”

The essence is what’s unique in every project, the problem to be solved, the design to create, and the strategies for testing and validation. The accident is crafting that solution with the tools and systems at hand. Brooks goes on to say the essence is the hard part of software development:

“I believe the hard part of building software to be the specification, design, and testing of this conceptual construct, not the labor of representing it and testing the fidelity of the representation.”

This isn’t to diminish the efforts of coders and testers as Brooks has other chapters extolling the importance of skilled workers, but to emphasize that their skill is in their ability to understand the problem and craft the solutions, not in the speed with which they type or draw diagrams.

Most improvements in software development productivity have reduced the accident: faster computers, better IDEs, automated testing tools (e.g., code coverage analyzers), etc. All of these things reduce the time it takes to implement or validate the solution, but none really address the essence of creating the solution itself. Accident is also an ever diminishing factor as our accidental roadblocks, such as time to compile or deploy, can only shrink so far. Moving from a compile time of hours to seconds is a huge cost savings, but going from seconds to milliseconds just doesn’t have that same payback.

When Brooks says there is no silver bullet, he means there is no fundamental way to reduce the work required to attack the essence. No tool, language, or software process can be a silver bullet since such things only attack the accidental costs in development. Such things have desirable incremental value, but should someone come to you and say some new tool or technique will grant vast increases in productivity, be wary and consider whether it’s truly essence that is being addressed. Robert Glass, in his book Facts and Fallacies of Software Engineering, notes that because software development is inherently difficult, the industry has been particularly vulnerable to its equivalent of what Glass calls the “snake oil salesmen.”

It’s worth noting that in his 20th anniversary retrospective, Brooks calls Object Oriented techniques “a brass bullet.” Brooks made this observation in 1995 and I think it’s safe to say that OO techniques have since become well established as best practice.

Summary

There is much valuable material beyond these two articles in Mythical Man-Month and while a portion of the articles in the full book have become obsolete (such as issues around managing scarce mainframe resources), the book as a whole is still well worth reading and some of the lessons learned are truly timeless. If we do not learn from our past and build upon the knowledge of those who have come before, then what chance do we have to fundamentally improve?

In my next post, I’ll delve into one of the great books on software culture: Peopleware by Tom DeMarco and Timothy Lister.

 

Topics: Methodologies

Our Thinking - The Online Blog is a source for insights, resources, best practices, and other useful content from our multi-disciplinary team of Onliners.

Subscribe to Blog Updates