Mar 08 2019
Project management has existed in various forms throughout human history. Everything from organizing more efficient hunting and gathering techniques to building a “smart city” requires implementing methodologies through a specific process (or set of processes).
Although the tools are more sophisticated now, and we’re continuing to improve on our ancestral successes, the failure rate for IT projects still hovers around the 50% mark, and the “lack of clearly defined objectives and milestones to measure progress” is the number one reason cited for the failure.
Given the pervasiveness of technology, all companies are essentially tech companies in some way, shape or form. For this reason, enterprises are inevitably tasked with undertaking IT projects. These can be as simple as building an app or as massive as redesigning their database infrastructure (e.g., moving away from a legacy system and to cloud-based services).
Within the software development world, Agile is the dominant methodology. Meanwhile, it’s predecessor, Waterfall, is still present, but lurking in Agile’s shadow. The truth is, both have their place within project management. Indeed, depending on the size and scope of the project, they may be used in tandem.
Are there projects where Agile makes better sense to use? Yes. The same can be said for Waterfall. But, it may not always be easy to determine when to use which. The guide below will give you a brief set of heuristics for deciding when to use one versus the other (or both).
The Waterfall method was conceived by Herbert D. Benington over 60 years ago and was the primary method of software engineering prior to the advent of Agile. In short, there are 6 phases, each requiring completion before the next phase can begin:
Different authors may term each of the phases something slightly differently, e.g., using “operations” instead of “maintenance.” Also, there are Waterfall variations; but, the concept is still the same: Waterfall moves through a specific set of linear processes where each phase is completed before moving onto the next activity.
Throughout the project, the requirements are kept stable and unchanging. The Waterfall method is beneficial when there aren’t any rapidly shifting functional requirements present. This method is handy and helpful when the system’s requirements need to be designed and analyzed only once, correctly, and will stay the same for an extended period of time.
This aspect, in many cases, will help development teams avoid the core restructuring of applications as the project progresses. In particular, this is of benefit for smaller projects, where requirements can be easily understood by all stakeholders in advance of development.
As stated in the opening paragraph, measuring progress (or the lack of performing this function) is a prime reason as to why projects fail. Given the frontloading of detailed requirements as agreed upon between the developers and end users, the Waterfall method creates a comprehensive scope of work at the beginning of the project.
Milestones are clearly defined. Since it’s expected that these requirements are unchanging, progress in relation to cost, scope, and time is more easily tracked and measured throughout the project life cycle.
Waterfall works well when the clients or customers have a clear set of requirements. Certainly, these can be hammered out through a stakeholder meeting during the initiation phase of project management. But, consumer demand changes quickly and Waterfall has a “no turning back from here” lock-in for its processes. So, client requirements can neither be vague nor lacking in detail. There’s minimal opportunity for revisions once you arrive at the 3rd step (Coding).
Due to the sequential nature of Waterfall, once the software is developed and in the testing stage, iterations based on user feedback are difficult (if not impossible) to incorporate into the end-product.
Aspects of the product that weren’t thoroughly thought out, or for which requirements have shifted during production, will be more difficult to fix if user feedback is negative. This increases product risk as end-users may reject all or part of the finished software.
Due to Waterfall’s emphasis on detailed planning before development, the planning stage can take an exceptionally long time. This aspect of Waterfall makes it a poor fit for long-term projects where there is a risk that requirements will change during development.
Extensive planning for a software that doesn’t meet end-user needs can be an expensive mistake if assumptions about the product or market are misplaced.
The Waterfall method is optimal for projects that
Agile’s core tenants, as expressed in The Manifesto for Agile Software Development, came into being on February 13th, 2001. Thought-leaders from various process-focused firms convened at The Lodge at Snowbird. The participants included representatives from SCRUM, DSDM, Feature-Driven Development, Crystal, and Pragmatic Programming (among others). The Manifesto for Agile Software Development includes several fundamental principles:
The above is frequently translated into the following:
Agile development has made application development far simpler in the rapidly evolving world of functional requirements. However, it’s not a magic bullet. Notably, rapid-cycle iterative development is not always the best fit for every project.
Agile’s regular release cycle allows for continual feedback from the actual end-users. By providing working demos, the Agile method can reveal unexpected issues and generate ideas for future iterations. The development team can then use these insights to improve the likelihood of the product’s success. This feedback loop improves product alignment for both business value and end-user value.
This benefit, in part, is driven by the Minimum Viable Product (MVP) release. The first iteration of the application must focus on delivering business value that substantiates on-going application development.
The Agile method has embedded characteristics that ensure functional requirements are visible to end users throughout the iterations of application development. There are no “black holes” of requirement specifications.
Agile encourages the application development cycles to be “time-boxed” into Sprints that last from one to four weeks in duration. The length of the Sprint cycle is declared in advance, and end users then have the assurance that application iterations will be delivered within predictable time frames.
Agile-based project delivery is divided into units of activity that include cycles of collaboration, development, testing, and deployment. Since these cycles occur on a repeated, iterated basis, applications undergo constant review for improvements and quality assurance.
Certainly, enterprises can (and do) apply their own systemization to the Agile approach, but that’s not what it was designed for. The goal is more flexibility in producing and maintaining the end product. Thus, scope creep, overestimating and underestimating costs, and losing control of the project schedule are problems that can occur.
While speeding towards an MVP through iterative cycles is a better way to ensure that the end-user is receiving an optimal product, strong documentation and end-user specifications are required to decrease the probability of project failure.
With less emphasis on planning and documentation, and having different cohorts produce each unit in a non-sequential manner, using a pure Agile methodology tends to cause project fragmentation. As a result, the project goals and objectives can easily go off the intended path.
Without the usual concerted effort focused on detailed planning and documentation, it’s much more difficult to reallocate human resources should developers and engineers leave in the middle of the project. Less experienced developers will be hard pressed to pick up where the others have left off since documentation can be scant.
Additionally, progress metrics are an essential component of determining the who, what, where, when, how, and why of all resources. The “fix and implement things as we go” subverts the resource allocation process which can easily lead to scrambling to meet the “just in time” reach for resources and, in turn, sets the stage for costs spiraling out of control.
Agile is most beneficial to use when projects:
So, what’s the best option for your project? The “Best of Both” is a common solution for balancing out the disadvantages of solely relying on either. The list below provides examples of application development and deployment scenarios where a mix of Agile and Waterfall techniques will produce the best delivery results.
If the product is not well defined, and further feedback is needed from the market to validate requirements, Agile is the best choice. An Agile approach that incorporates constant input from end users will allow those who will actually use the product to guide requirements, thereby increasing the chance of project success.
Waterfall is to be avoided where requirements are less clearly defined, or likely to change. However, the project can be planned at a high level using Waterfall, and then executed and controlled (as much as is possible for Agile) via the Agile methodology.
The first two phases of Waterfall (Preliminary Design and Detailed Design) provide the necessary steps to estimate overall project scope. For projects that are competing with others for a limited company budget, IT teams often want to get their budget established early and upfront. Agile then becomes an excellent method to iterate versions of the larger system once the overall budgetary requirements are estimated to a reasonably accurate extent.
Waterfall can be a good fit for shorter projects, where the project timeframe limits the risk that requirements will change. For longer projects, the length of planning required and the possibility of underestimating or overestimating feature importance can add unnecessary cost to a development project. Again, Waterfall can be implemented during the initiation and planning stages with the caveat that product development will deploy Agile.
SAP, Oracle, and other enterprise-class platforms are not commonly deployed through iterative development. Waterfall is the front-runner in this scenario. However, Agile is an excellent method for the development of adjunct applications such as business intelligence and dashboard systems.
As is the case with enterprise-wide applications, ETL-driven projects tend toward deployment in “one-big-leap.” Loading data into a repository still in development will often produce no useful system capabilities. Waterfall’s 6 step progression provides a solid path toward getting the data in one place and with the right structure. Yet, Agile techniques can be the key to developing responsive end-user systems that meet the immediate needs of data visibility.
The deployment of large or intricate E-commerce stores can be compared to that of their large brick-and-mortar counterparts. The store’s interior has to be completely built-out, and the POS systems must be operating correctly. Yet, the actual “storefront” requirements can shift quickly.
As an example, there may be new products to feature or cross-selling system features to adapt. Agile is a perfect fit for the development of customized user interfaces that must undergo a swift change-and-adaptation phase, while the larger and less flexible e-commerce ecosystem is developed using Waterfall.
Research from the Project Management Institute (PMI) suggests that organizations embracing Agile methodology are more likely to succeed with IT projects. But this isn’t to say that Waterfall is dead.
As a matter of fact, it’s still in use for the systems previously discussed. For example, the Waterfall method is robust when applied to enterprise-wide application deployments and with multiple organizations involved.
The application of this method can result in better team cohesion and customer relationship building.
One approach to client service is to bring rigid perspectives on Agile vs. Waterfall then pitch one or the other as the only methodology to consider; this is a sign of a lack of understanding with regard to the nuances of project management. Indeed, an experienced project manager will be able to recommend ways to either merge Agile and Waterfall (the Hybrid approach) or offer alternatives, e.g., Scrum, Critical Path Method (CPM), Extreme Programming (EX), etc.
Depending on the project, there are many different reasons and use cases for utilizing either methodology, a mix of both or a combination of other approaches. Summarily, a plethora of alternative choices exist for project management, and the options should be weighed carefully to ensure the project’s objectives are successful.
Want more? Head back to the Tivix blog