The world is full of uncertainty, and when it comes to software engineering projects or new product development, uncertainty is a familiar conundrum to be negotiated in the process of development. In our projects, how do we best deal with uncertainty?
The degree of uncertainty varies of course according to the level of complexity and unknowns in a product development or project. On the one end of the spectrum are serious innovations which require the inventor to heroically dive into unchartered waters and move beyond conventional wisdom. James Dyson explained that it took 5,127 prototypes over 15 years to develop the cyclone technology vacuum. Starting with cardboard and duct tape, then moving on to ABS polycarbonate, his phenomenal vision, and feat of blistering persistence lead to a blockbuster consumer product.
Dyson’s iconic cyclone technology vacuum.
“By the time I made my 15th prototype, my third child was born. By 2,627, my wife and I were really counting our pennies. By 3,727, my wife was giving art lessons for some extra cash. These were tough times, but each failure brought me closer to solving the problem. It wasn’t the final prototype that made the struggle worth it. The process bore the fruit. I just kept at it.” (James Dyson, 2011)
An exploratory method, trial and error are necessary for true innovation like the design process of the cyclone vacuum. It is interesting to note that Dyson, prolific inventor that he is, clearly a very wealthy man, attributes his persistence to excelling at long distance running as a young man.
There is an important lesson here: persistent systematic testing of a product and prototyping inherent in a design process encompasses learning and overcoming uncertainty. Dyson did not know upfront all the requirements and specifications of his final product. However he understood that his inventing the best vacuum cleaner in the world involved learning through a design process and in the process overcoming uncertainty in relation to optimizing the invention under various constraints.
Uncertainty in Software Development Projects
Similarly in software development, there is a difference between plan-driven and agile projects in their approach to uncertainty. A plan-driven project attempts to minimize uncertainty about the product by making all the decisions about the requirements upfront. However that is not realistic in many situations. By making decisions too far in advance on complex projects, there is the risk of making erroneous assumptions based on incomplete knowledge.
Requirements Uncertainty Principle
Someone said that the only certainty in life is death and taxes. In software engineering, I would add another certainty is requirements uncertainty! It was best articulated by W. S. Humphrey, whose well-known Requirements Uncertainty Principle, which states that:
“For a new software system, the requirements will not be completely known until after the users have used it. The true role of design is thus to create a workable solution to an ill-defined problem. While there is no procedural way to address this task, it is important to establish a rigorous and explicit design process that can identify and help to resolve the requirements uncertainties as early in the developmental process as possible.”
Since the publication of Humphrey’s book: “A Discipline for Software Engineering”, the agile movement has emerged and created a different approach to managing uncertainty to traditional project management. Agile planning gradually reduces uncertainty as the project proceed. This is illustrated in the following diagram.
source: Charles Cobb, 2014-18
According to Kenneth Rubin in his popular text, Essential Scrum: A Practical guide to the Most Popular Agile Process:
“Plan-driven, sequential processes focus on using (or exploiting) what is currently known and predicting what isn’t known. Scrum favors a more adaptive, trial-and-error based upon the appropriate use of exploration.”
An agile project is built around recognizing, managing, and reducing uncertainty as the project progresses.
I have a strong appreciation that using testing and an experimental method deals directly with the problem of uncertainty by systematic exploration of functionality and risk under various conditions in moving towards a system to becoming fit for purpose.
Disciplined Learning
Alistair Cockburn, an early advocate of the agile methodology, refers to the systematic exploration as disciplined learning in very small doses (for illustration see Cockburn’s knowledge-acquisition curve diagram below), where each step provides information that can be used to adjust the four categories of learning:
- What a project team should be building (rather than what they thought they should be building at the outset);
- Whether they have the right people on the team;
- Where the technical ideas are flawed;
- How much it will cost to develop. (Cockburn, 2014, 15)
I have personally experienced, as Cockburn describes, the phenomenon that in many software development projects, not necessary just waterfall projects, the major integration occurs towards the end of development. This leads to much learning about the interoperability and related risks associated with the integration after most of the development costs have been accrued. Whereas if uncertainty is mitigated earlier and learning takes place incrementally from the outset, this can lead to reduced risks in the final product and provide valuable feedback to the project sponsor to direct and redirect the project throughout the development effort towards a better result.
source: Alistair Cockburn, 2014
Risk and uncertainty is particularly high for new products, products that are new to the market, and applications that haven’t been implemented before. And new technologies are forming the foundation of many new applications that have never been developed before. That is where the interaction of technology uncertainty and requirements uncertainty is at its highest level.
How to think about uncertainty? One way in which a project team can think about uncertainty is in relation to the perceived level of complexity of a technology undertaking.
The Stacey complexity model can be used to illustrate the level of complexity in relation to two dimensions:
- REQUIREMENTS: How well-understood and well-defined are the project’s goals and requirements?
- TECHNOLOGY: How well-understood is the technology and solution for solving the problem?
Source: Charles Cobb, 2014-18
Agile projects may help address what Daniel Kahneman calls the planning fallacy. People and groups, susceptible to the planning fallacy, overestimate the benefits and underestimate risks and costs of delivering the product. This can be seen as a form of wishful thinking. People want to be successful. Project leaders can be prone to optimism bias. Teams may also fail to take into consideration outside influences and the level of complexity of all the system of systems within an organisation.
The 12th annual state of agile survey, conducted in the second half of 2017, provided a snapshot on the state of play of agile practices in software organisations around the world. It concluded that while agile adoption is growing within organisations, only 12% of respondents said that their organisations have a high level of competency in agile practices.
An agile methodology, such as scrum, involve a more adaptive approachwith an appropriate use of exploration. An agile project can be organised to reduce uncertainty incrementally as the project progresses. However it is critical to have very skilled engineers and team members who understand when it is better to use a traditional, tradition-agile hybrid or agile model. A scrum project still will produce requirements and plans, however with the assumption that details of those plans and requirements will be articulated as the project teams and product owner learns more about the design of the product during incremental process of building in structured sprints and learning periods (retrospectives). Plan-driven approaches to software projects use methods to handle large products and teams, and highly critical products. These type of projects create highly detailed plans upfront that are suitable for highly stable environments. The approaches can be combined which could be the topic of another discussion. Agile projects have an incremental approach to mitigating uncertainty as the project progresses and facilitates disciplined learning that Cockburn advocates. When there is a high level of requirements uncertainty and changes in the early stages of a project, an agile, adaptive approach is what may be needed so long as there is a mature level of competency in fundamental software engineering practices.
References:
Boehm, Barry, & Turner Richard, Using Risk to Balance Agile and Plan-driven Methods, IEEE computer society (June 2003)
Cobb, Charles Understanding Agile at a deeper level: management of uncertainty, High Impact Project management (2014-2018)
Cockburn, Alistair, Disciplined learning: the successor to risk management, Cross Talk Jul/Aug 2014
https://en.wikipedia.org/wiki/James_Dyson
Kahneman, Daniel Thinking Fast and Slow, 2011
Kenneth Rubin, Essential Scrum: A Practical guide to the Most Popular Agile Process
S. Humphrey A Discipline for Software Engineering, Addison-Wesley, 1995
robjauer 