Waterfall, Agile, and Death Marches

I’ve grown weary of internet debates about software development methodology, argued by people who don’t really understand the vocabulary they use. Allow me to provide a brief lesson on three of the most commonly misused terms.

The Origins of the Epithet “Waterfall”

A little history: Winston Royce is attributed with defining the methodology now called “waterfall” in his 1970 paper, Managing the Development of Large Software Systems although that term never appears in the paper. Royce was offering his opinion on what worked and what didn’t work after nine years of software project experience. While software for internal use could get by with just analysis and coding, paying customers needed a more rigorous approach. However, Royce argued against using a purely sequential process for developing software. In his paper, he critiqued several diagrams, beginning with the one below.

Royce noted that the lack of feedback between the sequential activities was a fatal flaw, but that the feedback flows had to be managed in order to avoid runaway scope. He explored several other diagrams in his paper, which became progressively more complicated until they resembled a spaghetti-and-meatballs dinner for sixty or so. Plainly, feedback was necessary but it was possible to become paralyzed.

The earliest use of waterfall in referring to a software development model may have been in a 1976 paper on software requirements by Bell and Thayer, which referred to Royce’s paper. During the 1980’s, there were several “structured” software project management methodologies marketed by various consulting firms that looked much like the more complex diagrams in Royce’s paper, but no one has actually used any of those methodologies for decades. None of them called their methodologies waterfall, either.

Time Marches On

Technology has changed dramatically in the last half-century. Where the primary cost constraint in 1970 was computing time, it is now knowledge-worker time. Where the primary technical challenges were data storage and code efficiency (does anyone remember dates with two-digit years?), storage and retrieval technology now provides terabytes for the price of a good dinner, free relational databases, structured and ad hoc queries that return results in milliseconds, and a global network with connections to tens of billions of processors. As a result, the nature of software products has changed from calculations of orbital mechanics and industrial controls, written by and for large corporations and government agencies to (overly) personal communications and videos of cats, paid for with marketing messages.

In Royce’s era, programmers and users submitted Hollerith cards and (maybe) a day later scanned the results on 132-column green-bar paper. Today, users have the expectation of point, click, fill-it-in-for-me, and tell-me-what-to-correct. Where once coders spent most of their time writing code to detect and handle logic breaks and invalid inputs, they now focus on the presentation layer. No one optimizes for memory usage, and execution speed only becomes a concern when responses are returned in more than two seconds. There are standards for virtually everything, reducing the effects of obsolescence, and a global communication network that facilitates collaboration across and between continents and cultures.

Software development today bears as little resemblance to 1970 as modern agriculture does to Mesolithic-era farming in the Fertile Crescent twelve millennia ago. But there are still farmers and there are still programmers, and the micro-economics of production still apply. And the same seven activities identified in Royce’s diagram are still needed, even if we organize and execute them differently.

Agile is Not a Performance Characteristic

Most modern software development organizations will tell you that they use agile methods. The term refers to the Agile Manifesto, composed by seventeen thought leaders at a conference on lightweight software development methods in 2001. While there are several software development methodologies marketed as “agile,” in most organizations “agile” is a synonym for Scrum. Other common “agile” methods include Kanban, XP, and DSDM. Each of them addresses the seven basic activities, although the level of detail varies widely.

Note that “agile” in this context is a label, rather than a performance characteristic. While many organizations use these methods (or combinations of them) effectively to produce reliable, maintainable software that creates real value, many others flounder. An entire industry of consultants and trainers offer their services to those who want to succeed with agile methods. Many of these experts (and their students) disparage what they assert are “non-agile” approaches with the generic epithet “waterfall.” That doesn’t make them so.

Complexity Requires an Even More Rigorous Approach

As noted, the Agile Manifesto arose from a conference on lightweight methods. While that certainly describes much of modern software development, many software projects require compliance with a variety of contractual, legal, and regulatory requirements, in addition to functional and technical requirements. These complex projects tend to include a lot of requirements elicitation, analysis, and design tasks that must be finalized by people who are not coders before decision makers will give their approval to proceed with actual coding. Because of these controls and process requirements, numerous parallel workflows by people who are not coders, critical decisions that often have nothing to do with code or coders, and the higher cost and longer duration of complex projects, they are typically managed using predictive methods—in other words, estimating task duration in advance, in order to schedule activities so that knowledge workers who are not coders can manage their workflows.

Complex projects include most government-funded projects, most projects for heavily-regulated businesses, and nearly all products in the medical devices or aerospace industries. Agile methods are used for some project activities, in parallel or in series with other activities using other approaches, when it makes sense to do so. People who don’t understand (and don’t care to understand) anything about these external controls and processes often dismiss them as command and control or waterfall or some other epithet. The rest of us understand that you have to skin a rabbit before putting it over the fire—see “Payne, Podrick” for an illustrative misuse case.

Death Marches Aren’t About Methodology

A project should be approved based on a specific business case, with parameters such as a scope, delivery schedule, budget, a staffing model, and performance and quality goals that are realistic. Ed Yourdon defined a death march project as, “One whose ‘project parameters’ exceed the norm by at least 50 percent.”

Acme Tornado KitThe business case should be updated whenever the business environment changes or assumptions prove to be invalid. An oversight committee should meet periodically to assess progress and risks, along with the probability of achieving the goals in the business case. This committee should be empowered to adjust any of the aforementioned parameters, and even to cancel the project outright if it seems necessary. Thus, death marches are not a failure of methodology but a failure of project governance.

Although there are still many organizations that will continue death march projects even in the face of evidence that the project will fail to achieve the intended business goals, few of them are profitable and essentially all of them have other bad management habits. In the words of Eric Bogle, “… year after year, their numbers get fewer. Someday no one will march there at all.”

Managing Globally Distributed Project Teams

I started managing projects that included team members or customers outside the US in the mid-90’s. In the beginning, it was one other country. Then two, and so on. As I progressed in my career, working with globally distributed project teams became my norm. A typical project would include people spread across five to thirty countries, three to five continents, and from three to seven time zones. As you would expect, it’s very different from managing a few folks clustered together in a cube farm. Preparations must begin before the first team meeting.

Working Calendars

It is important to be cognizant of the non-working days for the people in your team. Set up the holidays for each country in your project planning system—here is a list of commonly observed national and religious holidays in several countries for 2020, and here are instructions for updating the working calendar in Microsoft Project. In addition, ask your team members to record their planned vacation dates in a shared location—I usually just use an Excel spreadsheet, to keep the technical overhead down. Also, find a culturally sensitive way to inquire about maternity leave!

Time Zones

One of the biggest problems with working across oceans is the impact of time zones and the international date line on available windows for team meetings. Even if the organization adjusts working hours to get some alignment, it can be a burden for those who are always either getting up early or staying up late. Try to schedule meetings in a way that shares the burden.

Also recognize that not everyone observes daylight savings time, and those that do, don’t all change their clocks on the same day—Europe and North America are a week apart. And the Northern and Southern hemispheres are on completely different schedules. Here is an excellent resource for finding the current time and time zones of most of the large cities in the world, and here is their daylight savings time page.

Visibility into Workload Conflicts

Most globally distributed, cross-functional project teams include some number of people who have additional work responsibilities. The project will always be in competition with that other work, and you won’t necessarily know when priorities change. To avoid delays, maintain contact with your team member’s manager, or a proxy—someone who can act as a remote source of information and as a person of influence, should that be necessary. Not all cultures will openly discuss doubts and conflicts, especially with a distant colleague. It is vital to have a way to identify and resolve conflicts, and getting a periodic pulse check from someone on site can make a huge difference.

A Common, Bland English

For most global organizations, English is the common language. That doesn’t mean everyone speaks or understands it fluently, and it certainly doesn’t mean that everyone is familiar with all the local idioms, slang, and cultural references. When on the phone, speak deliberately (but not too slowly), as it can be difficult to parse out similar-sounding words. Work to avoid misunderstanding by keeping your spoken communications as jargon-free and non-colloquial as possible. And try to take the edge off your regional accent – I work at sounding as much as possible like a “generic American,” without the drawl. I tried to raise this with a colleague from Houston a few years ago, who replied, “What accent?” Note that British, Australian, Canadian, Scottish, Irish, and New Zealand accents and idioms are just as real and just as hard on the ears as Indian or Texas English. Speak to be understood by your audience.

There are many resources available online that can help you build your Cultural Intelligence, and even if you aren’t managing global teams right now, you almost certainly will be before long. I learn more about how cultural differences impact work and communication with every project, but I generally find that if I assume people are doing their best until I have reason to doubt it, my life is a lot happier.

Final Roundup: PM Articles for the Week of March 23 – 29

This is the 500th and final roundup of new project management articles published on the web during the week of March 23 – 29. Thanks for sticking with me all these years! Over the next few weeks, I’ll publish a few articles that I wrote for other venues but never got around to posting here, and in November I’ll publish my annual list of public holidays for 2021.

And this week’s video: Camaroonian musician and composer Emmanuel N’Djoké  “Manu” Dibango passed away on March 24th due to complications from COVID-19. He was 86. His 1972 hit, “Soul Makossa (I will dance),” was widely sampled and influenced generations of composers and performers around the world.  6 minutes, safe for work. “A true musician never dies; he just stops performing live.”

Ethics, Business Acumen and Strategy

  • My wife made 39 disposable face masks for a local hospital

    Fred Schmalz interviews Sunil Chopra on how companies can prepare their supply chain for the next disruption by balancing efficiency with resilience. 5 minutes to read.

  • Thomas Choi and colleagues say that the COVID-19 global pandemic proves the value of supply chain mapping as a disruption response tool. 5 minutes to read.
  • Greg Satell argues that we need to prepare for future crises—from artificial intelligence to climate change to the ballooning debt—like we would prepare for a war. 5 minutes to read.
  • Randall McAdory analyzes Tesla’s diverse collection of core technical competencies, and asserts that their market advantage is essentially insurmountable. 9 minutes to read.

Managing Projects

  • Cornelius Fichtner talks with Karthik Ramamurthy on seven techniques for effective project scope management. Podcast, 30 minutes, safe for work. Plus an article, 3 minutes to read.
  • Elizabeth Harrin announces the creation of a new professional community for project managers: #ThePMTribe. 5 minutes to read.
  • Glen Alleman argues for capabilities-based planning—providing an outcome or effect without an a priori specification. 2 minutes to read.
  • Mike Clayton tells us what to put in a project risk register, and how to maintain and use it. Video, 11 minutes, safe for work.
  • Praveen Malik explains how to save a baseline in MS Project, and how to use it once you have. 5 minutes to read.
  • The nice folks at ActiTIME suggest six ways to think of (and track) project costs. 6 minutes to read.

Managing Software Development

  • Stefan Wolpers curates his weekly list of Agile content, from the new future of remote work to the latest Cynefin iteration to enterprise agility. 7 outbound links, 3 minutes to read.
  • Cesar Abeid interviews Johanna Rothman and Mark Kilby on their new book, remote project management, and distributed agile teams. Podcast, 48 minutes, safe for work.
  • Nicole Segerer suggests that evolving software business models require new approaches to establishing and maintaining customer relationships. 4 minutes to read.
  • Kristin Jackvony reviews Enterprise Continuous Testing, by Wolfgang Platz and Cynthia Dunlop. 4 minutes to read.
  • Gojko Adzic did a comprehensive survey to see what’s changed since his classic book, Specification by Example, was published ten years ago. 22 minutes to read, but worth the time.
  • Henny Portman recaps the 3rd edition of Agile NXT, a downloadable magazine from Xebia. 3 minutes to read.

Applied Leadership

  • Karolina Toth interviews Steven McCord, SVP of Technology at WhyHotel, on how to succeed in your first 100 days as a tech leader at a new company. Podcast, 30 minutes, safe for work, or read the notes in 11 minutes.
  • Tom Cagely begins a series on toxic meeting cultures with diagnostics: how to recognize them. 3 minutes to read.
  • Lisette Sutherland interviews Danny Page on socially responsible outsourcing and the finer points of working as a distributed team. Podcast or video, each about 40 minutes, safe for work.

Cybersecurity and Data Protection

  • Das Rush interviews Joel de la Garza on the transformations of information security practices and capacity as more of the workforce goes remote. Podcast, 22 minutes, safe for work.
  • Erich Kron and James McQuiggan advise us what to do if we discover that someone is impersonating our company in a phishing campaign. 2 minutes to read.
  • Ben Dickson shares some additional security precautions to take when connecting remotely to the company network. 4 minutes to read.

Pot Pourri

  • Pascal Papathemelis recaps his webinar on active listening. Great content, too bad I missed it, but this is almost as good. 7 minutes to read, plus a video, 52 minutes, safe for work.
  • Francesco Marcatto makes the case for dot voting as a way for a team to converge on a limited set of alternatives. 4 minutes to read.
  • Esther Derby gives us a cheat sheet for helping those who are new to remote work to conduct better meetings. 2 minutes to read.

Peace be with you!