Iterative development is arguably the cornerstone of agile methods, promoting incremental design and delivery of software components. Whereas the benefits can be very significant when the method is properly used with development strategies, iterative development, when conditions and qualifications remain implicit, often becomes dogma igniting irrational controversies.
To begin with, the difference between software engineering and its industrial cousins must be underlined: compared to manufacturing processes, software development flows come with few, if any, physical constraints, and that fluidity is arguably a key success factor of iterative development.
Yet, software components don’t stay on virtual shelves but are embedded in actual products or used to run actual devices or systems, and that where frictions are to come forth.
That put software development at the junction between two perspectives:
- The business perspective faces the real world, where frictions may, and usually will arise. It is said to be synchronic because objectives, constraints, and risks change along time, and software components must be continuously and consistently adjusted to changing requirements defined within time-frames set externally.
- The engineering perspective faces the digital world, which is essentially friction-less. It is said to be diachronic because once rooted into requirements, the design and implementation of system artifacts can live on their own time-span.
- In between the architecture perspective is meant to be as invariant as core business concerns and corporate identities.
That should be the basis of iterative layouts.
Iterative software development is best defined by applying the control loop paradigm to software engineering objectives:
- Scope and invariants are defined with regard to the business (synchronic) perspective
- Increments are defined with regard to the engineering (diachronic) perspective.
- Exit conditions are defined with regard to business, functional, and technical requirements.
Each iteration is then to repeat the same set of different activities, carried jointly by a single team regrouping different skills and representing a collective authority and responsibility.
Defined with this layout, iterative processes are not off-the-cuff solutions to be applied to any circumstances but require specific conditions to be met.
Regarding scope and invariants (business, synchronic perspective), work units should coincide with the perimeter of symbolic representations, which means that iterative loops should not cross organizational units (stakeholder) or functional domain. That will also ensure the synchronization of changes in contexts and systems.
Regarding increments (engineering, diachronic perspective), iterations must be carried out independently of external circumstances, and products delivered continuously providing they meet exit conditions.
When these conditions cannot be achieved projects are to be fragmented according to domains (business perspective) or activities (engineering perspective).
Based upon a basic definition of tasks (requirements, development, deployment, architecture and quality) and the associated milestones, some archetypes of iteration patterns can be identified.
Global iterations repeat the full development cycle, hence combining the authority of different organizational units, eg requirements, development, deployment.
For a given technical architecture, iterations are associated to the development and deployment of incremental requirements.
- Invariants: backbone requirements of business objects and processes whose identity and consistency define the functional architecture.
- Increments: new functionalities or implementations.
- Exit condition: no more requirements
Local iterations repeat parts of the development cycle and are meant to be managed under the authority of a single organizational unit.
For a given set of requirements, iterations are associated to the development (analysis, design, code) of subsets of functionalities.
- Invariants: Set of requirements.
- Increments: new system functionalities.
- Exit condition: components tested and ready to be deployed.
Overlapping (or simultaneous) iterations repeat only parts of the development cycle, hence combining the authority of different organizational units independently of actual deliveries. As a corollary, development teams may be working on new subsets of requirements with previous ones still not be accepted. Based upon MDA model layers, one can identify two main archetypes of such iterative processes:
- Domain driven iterations: analysis and design are merged into a single loop in order to derive full functional specifications (PIM) from given uses case (CIM). That organization is recommended when a functional architecture has to be defined in order to support a representative or critical use case.
- Platform driven iterations: design and implementation are merged into a single loop in order to develop full implementation packages (PSM) from functional specifications (PIM). That organization is recommended when development is outsourced.
Overlapping iterations clearly bring in a higher level of complexity, as perimeters, authorities, and tests may have to be managed independently; and since organizational procedures are no more available as abutments, iterations are to be defined directly on model contents and development flows.
Iterations & Architecture
Except for standalone applications, iterations must be bound by invariants set along architectural tiers: interactions are developed for frozen applications, themselves developed for frozen persistency tiers. As noted above, model layers and architecture tiers don’t necessarily coincide which can make iterative development difficult to manage.
Among development patterns, two are especially representative: domain driven and platform driven projects.
Domain Driven Iterations
Domain driven iterations add features and/or operations to a set business objects and/or use cases. With CIM/PIM overlapping iterations, the critical assumption is that the architecture of symbolic representations (semantics and identification mechanisms) are not affected from within the loop. Using the Rent a Cart example, iterations can add a new option for scythed chariots (with an extended use case) providing they are identified as other carts and rented using the same process.
Hence the control structure of iterative modeling:
- Invariants: A set of persistency and/or execution units.
- Increments: new features and/or operations.
- Exit condition: no more additional requests .
On a broader perspective, overlapping iterations must take into account the different lifespans of CIM artifacts:
- Business opportunities are by nature diverse and volatile. At symbolic level they are represented by applications. Since they are directly used by agents, domain driven iterations (possibly with prototyping) often bring significant benefits.
- Business domains are meant to secure a long-term basis for business development. In that case domain driven iterations may add to objects descriptions without affecting their semantics or identification mechanisms.
- Business organizations are the basis of corporate identities and valuation. In that case domain driven iterations may add to objects descriptions across domains.
Altogether, overlapping iterations should entail only local changes and as such should not introduce new architectural constraints. Yet, only a thorough definition of invariants may guarantee model validity, in particular for external consistency.
Functionalities Driven Iterations
Functionalities driven iterations add functional variants to use cases without affecting existing ones, the assumption being that business domains and rules remain unchanged.
Platform Driven Iterations
Platform driven iterations add features and/or operations to system components. With PIM/PSM overlapping iterations, the critical assumption is that the mapping between business contexts and their system counterpart is not affected from within the loop. Using the Rent a Cart example, iterations can add new features to the symbolic description of scythed chariots (with corresponding changes to interfaces and messages), and modify their implementation accordingly, providing the continuity and consistency of actual business objects representations are maintained.
Iterative development, executable models, & model engineering
All in all, iterative development should always be set within a clearly defined development strategy, and take into account the logical tools supporting iterations, namely: patterns of representation (CIM/PIM) and implementation (PIM/PSM) on one hand, model transformation mechanisms one the other hand.
A particular attention should be given here to the meaning of “executable models”:
- Weak versions add some action semantics in order to support prototyping
- Language based solutions, e.g executable UML, use pseudo language like OCL or state machines and enable full code-generation.
- Domain based solutions use profiles and domain specific languages.
Set on a broader perspective, models can be either descriptive or prescriptive, but cannot be both. Hence, an executable model is either a simulation of a business process (prescriptive) or an application prototype (descriptive), prescriptive models being extensional (concerns are selective) whereas descriptive ones being intentional (everything must be designed).
In any case, executable models are models of computation and therefore useless for the organizational (CIMs) and architectural (PIMs) aspects. Besides engineering aspects, they can only be introduced once responsibilities and functional architectures are explicitly set. Then, model contents and execution platforms must be subject to distinct quality management. From that point of view, patterns and model translators, even if set independently of model contents and transformation tools, have contractual limitations and usually cannot to be used across organizational units.
Agile as a Postscript to Postmodernism
Funnily, Agile practice of iteration can be seen as a healthy correction to postmodernist theorizing. For authors like Derrida the iterability of the written mark entails that the synchronic and diachronic associations of the written sign are both present and divorced from each other so that the written sign can possibly carry an idea while not carrying an idea. Applied specifically to textual requirements that would mean an infinity of contexts and would explain the difficulty of securing systems requirements upfront. Fortunately practical (aka agile) iterations have only to deal with two contexts, business and systems; and that can be achieved if a clear distinction is maintained between the semantics of requirements and design.
- Models Transformation & Agile Development
- Agile Delivery & Dynamic Programming
- Agile Architectures: Versatility meets Plasticity
- Agile Collaboration & Social Creativity
- Agile between Space & Time
- The Scope of Agile Principles
- Agile vs Waterfall: Right vs Left Brain ?
- How to Mind a Tree Story
- From Stories to Models
- Agile Business Analysis: From Wonders to Logic
- Use Cases are Agile Tools
- Agile & Models
- Agile Falls
- Thinking about Practices
- Spaces, Paths, Paces (Part 1)
- Spaces, Paths, Paces (Part 2)
- Tests in Driving seats
- Projects as non-zero sum games