This page describes both Waterfall and Agile software development philosophies. In reality developments are not 100% waterfall or agile, but a blend of the two.
This is a ‘back to basics’ look at the two philosophies. Separate pages will consider the implications, and necessary modifications, when each is applied to different bespoke software or product software projects.
- Why yet another page?
- waterfall: modelled on construction projects
- We have an agreed design and the process to build this is well known.
- agile: modelled on engineering design projects
- no one has ever done this before
- The Team
Why yet another page?
In short, to try to separate the ‘religion’ from this topic, and to step back from the ‘bespoke only’ or ‘product software only’ perspective in almost every other page on this topic I have found.
An example of the ‘religion’ attitude is an article here which suggests Apple computer has poor software quality because they use ‘waterfall’ rather than ‘agile’. It is almost certain that Apple would use a methodology encapsulating some of both ‘agile’ and ‘waterfall’, but the implication is that label ‘waterfall’ is itself sacrilege. While a company (and even perhaps Apple) may not embrace sound software methodology, evaluation is more complex than simply a label.
The other issue is to take each concept, and envisage only a chosen adaptation for the authors field of software. Not only are bespoke and product software almost polar opposites, but even within each category there are factors that affect ideal methodology. So most pages compare only for a specific environment, which is great if all your projects match that description, but less helpful for those with different or varied projects.
Waterfall: Modelled on constructions projects.
The process of building something (i.e. a building) has been practiced and refined over centuries. The waterfall approach is simply taking this construction model, and applying the model to the building of software. Using a gantt chart, the steps look more like a cascade of waterfalls. But if you view a cascade from the front (see pic) then you have the appearance of water falling from step to step. The waterfall is from step to step, not all at once.
The principle of waterfall is that each step is completed before the next begins. A complete understanding of requirements should be in place before design starts. ‘Building” or development, should not commence without a completed design. Once built, tests verify the building meets the design. These are the same steps that have been proven to work for construction projects over centuries.
Agile: Modelled on engineering design.
While waterfall considers a software project as similar to a construction project, agile effectively considers the entire software project, as being an engineering design phase.
With engineering design, you have a problem, and the challenge is to find a solution.
If it is know that some existing design can be modified to provide a solution, then there is no engineering design required.
An example from the tangible world is the design phase for mass produced products. You have a team of designers who engineer a design, but although you may have a fixed budget and time frame, it is only at the end of the design phase that you know what the final product will look like.
The principles of agile can be considered be:
- what is being created is new and innovative
- new ideas will need prototyping and revision
- the best solution is not always clear at the outset
- actual product requirements may not even be clear until the product is trialled
The process of agile software development can be considered as:
- define initial set of requirements
- build an as simple as possible prototype as quickly as possible
- now repeat the following steps until satisfied:
- analyse what how the prototype could be improved, and add those improvements to requirements
- analyse steps that can be made within 1-2 weeks to meet more requirements and perform those steps
Note: this is representing the agile philosophy itself. Any actual implementation of agile will also depend on some other choices as to how agile will be implemented.
Consider a car manufacturer who produces a new model of a certain car every year. The design team, with an allocated number of people and resources, is given a fixed time to come up with next years design. Agile development is similar to this process.
Hybrids? Solutions using a combination.
Waterfall: We have a specification and the process to build this is well known.
With construction projects, you could be building an exact replica in an effectively identical site, but surely there will be something new? In fact, the more daring the building, the more often that some part of the project will actually encompass an engineering design project, even with prototypes! The fact that almost every waterfall project encounters some steps which better fit the agile pattern, is the reason so many waterfall projects suffer time and cost blowouts.
Agile: No one has ever done this before.
But Surely someone has done part of it before? Won’t there are least be construction projects to build the prototypes?
The reality is that both agile and waterfall can start to approach a series of short waterfalls placed back to back. Each just approaches from the opposite side.
A very short waterfall stage loses many of the characteristics of waterfall, and a long build of a single prototype becomes more waterfall. In the end no project is ‘pure’ and all have elements of both. Software is always doing something new, but just how new?
The more new, the more no one has genuinely done anything like this before, the greater the need for an agile mindset. The more ‘just like a previous one but with these straight forward but time consuming changes‘, the more a waterfall mindset may work.
No one can predict the exact cost of an exactly specified system containing elements that no one has ever done before.
As waterfall is based on the construction model, the budgeting process it quite like getting quotes having a house built to match the plans you have had drawn up. You can have faith you will get something that matches the plans, even if it may be a little late and the real problems only occur if you realise during construction that result will not match what you actually need.
Agile is more like working with the architect. You may have a fixed quote but you have to commit blindly without knowing in advance if you will like the result. With agile you can control the budget, but at the expense of accepting what is produced within that budget.
The result is that many projects are forced into a waterfall mindset from the budgeting phase, and can only live with an agile mindset for the limited spend of the design phase.
Just as waterfall breaks a project into steps, it separates the roles needed for those steps. Design people are expensive, but during ‘construction’ you may only need labourers who cost less. Similarly a waterfall methodology separates project managers, annalists, and programmers. Programmers who just program become lowest cost resource, and can be more easily outsourced and managed by project managers.
Agile requires the whole team act like engineers. As the work is always different even the construction has less guidance and more decisions on the fly by the ‘engineer’. The result is more costly resources and a much flatter team structure. The team project manages themselves using scrum or other methodology, but this requires additional skills.