Architecture issues discovered late in a system’s lifecycle and are inherent to its design are more expensive (or may even be impossible) to fix. Often this is because issues discovered later in development stem from early decisions that have far-reaching software consequences and require major modifications. Identifying those issues that are architectural risks early in design can result in significant cost savings over the life of the system. In this blog post, adapted from a recently published report, we propose an approach that draws from Agile Architecture Risk Management (AARM) and Continuous Risk Management (CRM) processes to create a practice for evaluating software architecture risks during development. By weighing the tradeoffs between design pattern attributes and quality attributes, the development team can identify architectural risks early and assess the system impacts of design decisions made during software development.
Meeting the Needs of the Warfighter
Many Department of War (DoW) projects aimed at developing custom software capabilities are now completed using Agile practices. However, most Agile practices do not specifically mention activities related to software design or software architecture components during development. A key attribute in Agile practices is speed of development. Often, this speed comes at the cost of fully evaluating design decisions with respect to architecture impact on quality attributes in the form of risk.
Proposals exist for how to apply software architecture practices to Agile software development projects (i.e., Agile architecture) and how to apply software architecture in Agile software development projects as a design risk mitigation strategy. In this blog post, we propose an approach that involves a continuous review of design decisions during development to better understand their impact on the architecture as risks against the quality attributes.
The process we propose consists of the following activities:
- continuous design risk evaluation with respect to quality attributes
- use of a Minimally Viable Architecture process
- dedication of design decision review early in the agile sprint
- a review of development team concerns as possible impacts to architecture risk
Applying an architecture methodology helps support risk mitigation for software development because it involves making early, high-level decisions that prevent costly mistakes, security vulnerabilities, and performance issues. Being aware of these attributes and how they are impacted by design decisions can later affect the Agile development practices used by development teams. Examples of these types of decisions include identifying user stories that emphasize quality attributes and may require further elaboration and determining how to measure the design decision acceptability of rapidly developed products for the customer. Understanding the clear connection between Agile software design activities and software architecture quality attributes will help architects, designers, program managers, and developers understand where better practices can be applied.
Architecture risk is defined as the software system’s inherent inability to promote stakeholder goals (i.e., quality attributes). In this blog, when we talk about risk, we are speaking of uncertainty of consequences associated with engineering choices and commitments.
Abbie Redmon defines quality attributes this way:
In general, quality attributes are the system properties that make a solution viable for the environment where the solution developed from the requirements is expected to operate. These properties are instantiated when software patterns are used to meet the demands of a desired architecture. Understanding the impact of software design on software architecture can be challenging, as most developers use familiar design patterns to produce a solution with speed as the goal. We are proposing that specific design decisions can impact the architecture by developing risks that impact how the solution meets quality attributes. The process changes identified in this blog will help alleviate this project concern.
Continuous Risk Management
Continuous Risk Management (CRM) collects concerns (i.e., uncertainties) from individual developers or the development team during Agile ceremonies. As appropriate, the concerns can be added to either the product backlog or sprint (i.e. iteration) backlog as a spike or task. Concerns are normally collected during
- a sprint planning meeting
- a backlog refinement meeting
- sprint reviews
Dorofee, A. et al. Continuous Risk Management Guidebook.
Our approach focuses on the potential for quality attribute issues identified during sprints to evolve into architecture risks. These risks have the potential to prevent the final software product from fulfilling the stakeholder’s requirements which are identified as systemic properties (i.e., quality attributes).
Design Change During Incremental Development
The approach outlined here and in our report echoes an earlier proposal by Cristiano Gomes and Rodrigo Pinheiro. In a 2022 blog post on architecture risk management for Thoughtworks, Gomes and Pinheiro begin by quoting the following principles from the Agile Manifesto:
- Continuous attention to technical excellence and good design increases agility.
- The best architectures, requirements and designs emerge from self-organized teams.
Architecture is instantiated using design patterns. Design patterns can have an impact on the architecture patterns meeting the quality attributes that drive the solution.
Architectural patterns show what a solution must do to address reoccurring requirements. Design patterns accumulate the best practices and experiences that software professionals have used over the years to solve general problems.
The quoted principles from the Agile Manifesto imply that design decisions evolve into a “good design” and therefore a “good architecture.” This implication seems to ignore the fact that design decisions can impact the system of system’s software architecture, which is being designed to meet the quality attributes for the solution. The development team, which usually includes design expertise, should use, maintain, and manage the software architecture to help make design decisions that ensure the system meets the customer’s needs. Development design decisions can impact system-level architecture risk.
In developing our approach, we assumed that when a development team is engaged, the lead developers have some notional idea of a high-level architecture that can address the system’s requirements. Many details of the implementation and how interactions will be instantiated are left to be resolved during development. These unresolved details can result in key risks to the system architecture, which must be mitigated.
AARM and aspects of CRM can be used to evaluate software design risk. Mark Richards uses a matrix worksheet that shows the tradeoffs made between design patterns and quality attributes to help the development team better identify architectural risks during system development. This tradeoff matrix enables the software designers and later software developers to consider the impacts of pattern choices on quality attributes. The matrix also helps users evaluate design choices early in the development process instead of discovering issues later in the software lifecycle when they are more expensive (or even impossible) to fix.
Use of an early risk identification method, such as the one proposed here by AARM, can help identify risks to the architecture that meet quality attributes. This early practice in development can help prevent issues rather than waiting until before production, when they can endanger system properties and mission needs.
The AARM process, as detailed in Figure 2 below, describes one method to instantiate Agile development risk management techniques.
AARM’s key concept is to consider software architecture risk during the development process and not leave it for later, when the system is done, or as an afterthought. It also enables the team to document an architecture hand-in-hand with the system architecture rather than only representing what the team has already delivered.
A development process using AARM assumes the completion of broader design steps minimum viable architecture (MVA) before development, which results in the existing architecture instantiation. Capturing design decisions in the MVA allows for future teams to understand the why rather than a singular focus on the how in the code. This helps reduce information loss over the lifecycle of the software.
Selection of a specific design pattern can impact the quality attribute that an architectural decision has been made to support. As illustrated in the figure below, one example is “Architecture Monday” where the development team can dedicate an early period during the sprint to cover AARM issues. These decisions on pattern choice are continuously made as the product evolves into an agile release. This proposal suggests an “engineering architecture risk evaluation day” to the agile sprint in which risks are identified and reviewed as decision are being made. This tactical use of continuous risk management throughout the development process helps avoid the strategic risk failure of impacts to software properties.
More formal design review methods can be used at planning intervals (e.g., system release points) to ensure that the developing system hasn’t significantly deviated from its original software architecture. If it has, then developers check the updated architecture to confirm it still supports the established quality attributes, mission drivers, and requirements. If it doesn’t, then the team may have discovered a high-level risk that must be addressed by the lead architects.
While concerns can and will surface, the activities described in the next section assume that one or more of the processes described earlier have already been implemented, and periodic reviews are taking place.
Comparing Architectural Patterns
Developers should understand how the decisions that make up the architecture that will affect key detailed software design patterns. As the software system architecture is instantiated in code, developers must consider constraints such as the system’s operating environment, governance, and mission demands (among other factors). The need to identify software architecture risk impact, which can result from software design decisions, development issues, and sustainment, becomes critical to ensuring the project doesn’t accumulate technical debt and that the system can support its desired quality attributes that the architecture depends on. There appears to be a lack of any formal process that captures risks to the architecture during Agile development; instead, there is merely a general assumption that architectural risks will be addressed during development. Many processes are available to assist with a software system’s initial software architecture and design. These are examples that teams might employee as part of regular practices to better understand what drives the architecture and can identify areas of risk:
However, the risks associated with meeting the quality attributes are not usually stated. This is because the high-level software architectures abstract away details that can uncover risks, and often developers make decisions during system development that cause the design to deviate from its original architecture. The reasons for these deviations can include accommodating the schedule for expediency, not referencing the original architecture to ensure correct progress, choosing convenient technology opportunities to “improve” the solution, and third-party software choices that drive architecture choices and add effective supply chain risk management. Also, developers often do not consider the risks and impacts of making software changes to the system during sustainment, which leads to accumulating architectural variance and technical debt. Technical debt reduces code quality, slows development, and increases the risk of errors.
What is missing from the CRM diagram in Figure 1 is the ability to understand where some technical risks (i.e., concerns) originate. What choices or what concerns in the proposed solutions (i.e., architectural patterns) affect the key quality attributes that the customer needs in the system? As stated in the report on Minimally Viable Architecture (MVA), the scenarios are needed to present key quality attributes that the user needs to support the mission. Developers should use an identification mechanism regularly during project sprints to help with this concern called the Pattern/Quality Attribute Matrix. Having an architecture storming day (“Architecture Monday”) during each sprint is also recommended. The matrix will also begin to help represent the system’s architecture early in the process so it can be used later, during the system’s sustainment, to know why choices were made during design.
If the software architecture risks can significantly impact the ability to meet the desired quality attributes, the mitigation is to enhance the current design to compensate. Using the Pattern/Quality Attribute Matrix regularly during development allows that the team can determine the priority of the system’s desired quality attributes and then conduct a tradeoff analysis among the attributes provided by the proposed design patterns. The table shows how particular architectural patterns have strengths and weaknesses in supporting certain quality attributes.
Table 1: These quality attributes are taken from the Business Technology Architecture Body of Knowledge (BOK). These architecture patterns are found in the Gang of Four design pattern documentation.
To illustrate how to interpret the pattern/quality attribute matrix, it helps to look at a specific architectural pattern as shown in the figure above. If a team uses the Observer/Pub-sub design pattern, the systemic properties of compatibility and modifiability are easier to attain (shown in green), while reliability is more difficult (shown in red). Pluses ‘+’ and Minuses ‘-’ might also be used to identify the enhancement or degradation of attributes. See Table 1 for comparisons of example system attributes (i.e., strategic goals) versus architecture patterns (i.e., system properties).
Some development methods consider the system’s architecture, but none found in the literature looked specifically at software architectural risk with respect to quality attributes.
When to Identify Architectural Risks
Analysis of architectural risks can take place throughout the software development lifecycle including:
- during the original development of the architecture
- during the construction, development, and review of the design during sprint planning
- when changes are required by real-world (e.g., hardware) constraints
- when code is refactored during construction
- during maintenance/sustainment of the code
- when enhancements are made to the code due to emerging requirements
- when new hardware is adopted, which possibly introduces new constraints
- when any analysis tool returns a metric not consistent with what has been seen before, such as increasing complexity or coupling in the code, as these point to possible architectural changes
Teams should conduct some architectural risk analyses to meet quality attributes during sprint planning, sprint reviews, and backlog refinement. Lattanze recommends scheduling sprints at least every three weeks (many groups conduct them every two weeks) to ensure that these analyses are given the attention they deserve. He also recommends conducting a sprint just for refactoring (with a focus on architecture), whether making the code align with the architecture or deciding that the architecture needs to be modified. Of course, the latter takes more time and carries more risk, since changing the software architecture may require considering new tradeoffs.
The Importance of Identifying Architecture Risk Issues Early
A system’s architecture encompasses a risk mitigation strategy. In this post, we propose a process for adding architecture risk identification into the Agile methodology. Using this approach, developers can avoid discovering design issues later in a system’s lifecycle. A good concluding abstraction of architecture is the following excerpt from Software Architecture in Practice from authors Len Bass, Paul Clements, and Rick Kazman:
[A] software architecture must abstract away some information from the system … and yet provide enough information to be a basis for analysis, decision making, and hence risk reduction.
This is a reminder that abstraction removes complexity so that a problem can be more easily solved, but it also removes some of the “conditions” that can contribute to architecture risk.
I was recently involved with a project that exhibited technical debt as part of the architecture. The architectural choices highlighted the 80 percent solution (i.e., identity provider) delaying the remaining 20 percent until later. The solution for meeting deadlines on the project created technical debt that was not be resolvable in the future. To ensure support for the 80 percent, we chose to use an authoritative source of truth for architecture that was identified as sufficient to cover a large enough group of users to meet future identity requirements needs. Later it was discovered that the 80 percent solution didn’t cover the 20 percent and didn’t cover many of the detailed needs. A complete architectural change may be needed to resolve the issue.
Through our blog post and our accompanying report, our aim is to ensure that the impact of these conditions on architecture risk are understood. Drawing from AARM and CRM, the proposed process enables developers to use this architecture information to not only evaluate software system development risk but also identify design risk issues early that can impact the architecture. This early identification enables developers to better recognize architectural risks during system development and assess the impacts of design decisions on the architecture made during software development.
Don’t assume that design choices are one size fits all. Understand the key quality attributes the design pattern emphasizes and those quality attributes that the pattern deters and how they ultimately affect the architecture you are trying to implement.
