Iterative Processes

Context

Iterative development is arguably the cornerstone of agile methods, promoting incremental design and delivery of software components. Whereas the benefits can be very significant when the method is properly used with development strategies, iterative development, when conditions and qualifications remain implicit, often becomes dogma igniting irrational controversies.

Sysiphus Iterations (Dluho)

Objectives

First, iterative processes must be set in context, namely the dual perspective of system engineering:

• The business perspective is synchronic: since objectives, constraints, and risks change along time, system requirements must be continuously and consistently anchored to business  contexts.
• The engineering perspective is diachronic: once rooted into requirements, the design and implementation of  system artifacts can live on their own time-span.
• In between the architecture perspective is meant to be as invariant as core business concerns and corporate identities.
  

  Iterations may be anchored to business context or system artifacts.

Then, we must consider iterative development within the control loop paradigm and identify the rationales supporting the definition of invariants and exit condition, of which:

• Scope: the same activity is repeated until a given set of specifications (invariant) is satisfied (exit).
• Responsibility: different activities are repeated by a given organizational unit (invariant) realizes its objective (exit).
• Quality: the same activity is repeated until a given set of constraints and measures (invariant) is satisfied (exit).

At the least, the demarcation lines of development work units should match the principles used to organize system components, which means that iterative loops should not cross organizational units (stakeholder) or functional domain. Based upon an uncontroversial organization (requirements, development, deployment, architecture and quality) and the associated milestones, one can  distinguish between basic iteration patterns.

Global Iterations

Global iterations repeat the full development cycle, hence combining the authority of different organizational units, eg requirements, development, deployment.

Global iterations repeat the full cycle.

For a given technical architecture, iterations are associated to the development and deployment of incremental requirements.

• Invariants: backbone requirements of business objects and processes whose identity and consistency define the functional architecture.
• Increments: new functionalities or implementations.
• Exit condition: no more requirements

Local Iterations

Local iterations repeat parts of the development cycle and are meant to be managed under the authority of a single organizational unit.

Local iterations are performed under a single authority.

For a given set of requirements, iterations are associated to the development (analysis, design, code) of subsets of functionalities.

• Invariants: Set of requirements.
• Increments: new system functionalities.
• Exit condition: components tested and ready to be deployed.

Overlapping Iterations

Overlapping (or simultaneous) iterations repeat only parts of the development cycle, hence combining the authority of different organizational units independently of actual deliveries. As a corollary, development teams may be working on new subsets of requirements with previous one still not be accepted. Based upon MDA model layers, one can identify two main archetypes of such iterative processes:

• Domain driven iterations: analysis and design are merged into a single loop in order to derive full functional specifications (PIM) from given uses case (CIM). That organization is recommended when a  functional architecture has to be defined in order to support a representative or critical use case.
• Platform driven iterations: design and implementation are merged into a single loop in order to develop full implementation packages (PSM) from functional specifications (PIM). That organization is recommended when development is outsourced.

Overlapping iterations dissociate organization and deployment.

Overlapping iterations clearly bring in a higher level of complexity, as perimeters, authorities, and tests may have to be managed independently; and since organizational procedures are no more available as abutments, iterations are to be defined directly on model contents and development flows.

Iterations & Architecture

Except for standalone applications, iterations must be bound by invariants set along architectural tiers: interactions are developed for frozen applications, themselves developed for frozen persistency tiers. As noted above, model layers and architecture tiers don’t necessarily coincide which can makes iterative development difficult to manage.

Iterative Development & Architecture Tiers

Among development patterns, two are especially representative:  domain driven and platform driven projects.

Domain Driven Iterations

Domain driven iterations add features and/or operations to a set business objects and/or use cases. With CIM/PIM overlapping iterations, the critical assumption is that the architecture of symbolic representations (semantics and identification mechanisms) are not affected from within the loop. Using the Rent a Cart example, iterations can add a new option for scythed chariots (with an extended use case) providing they are identified as other carts and rented using the same process.

Domain driven iterations add features and/or operations to persistency and/or execution units.

Hence the control structure of iterative modelling:

• Invariants: A set of persistency and/or execution units.
• Increments: new features and/or operations.
• Exit condition: no more additional requests .

On a broader perspective, overlapping iterations must take into account the different lifespans of CIM artifacts:

• Business opportunities are by nature diverse and volatile. At symbolic level they are represented by applications. Since they are directly used by agents, domain driven iterations (possibly with prototyping) often bring significant benefits.
• Business domains are meant to secure a long term basis for business development. In that case domain driven iterations may add to objects descriptions without affecting their semantics or identification mechanisms.
• Business organizations are the basis of corporate identities and valuation. In that case domain driven iterations may add to objects descriptions across domains.

Altogether, overlapping iterations should entail only local changes and as such should not introduce new architectural constraints. Yet, only a thorough definition of invariants may guarantee model validity, in particular for external consistency.

Functionalities Driven Iterations

Functionalities driven iterations add functional variants to use cases without affecting existing ones, the assumption being that business domains and rules remain unchanged.

Iterations within Use Cases add execution paths

Platform Driven Iterations

Platform driven iterations add features and/or operations to system components. With PIM/PSM overlapping iterations, the critical assumption is that the mapping between business contexts and their system counterpart is not affected from within the loop. Using the Rent a Cart example, iterations can add new features to the symbolic description of scythed chariots (with corresponding changes to interfaces and messages), and modify their implementation accordingly, providing the continuity and consistency of actual business objects representations are maintained.

Overlapping platform driven iterations: both PIMs & PSMs are modified within the loop but the continuity of representations is maintained.

Iterative development, executable models, & model engineering

All in all, iterative development should always be set within a clearly defined development strategy, and take into account the logical tools supporting iterations, namely: patterns of representation (CIM/PIM) and implementation (PIM/PSM) on one hand, model transformation mechanisms one the other hand.

Model interpretation shouldn't cross organizational units, code-generation may.

A particular attention should be given here to the meaning of “executable models”:

• Weak versions add some action semantics in order to support prototyping
• Language based solutions, e.g executable UML, use pseudo language like OCL or state machines and enable full code-generation.
• Domain based solutions use profiles and domain specific languages.

Set on a broader perspective, models can be either descriptive or prescriptive, but cannot be both. Hence, an executable model is either a simulation of a business process (prescriptive) or an application prototype (descriptive), prescriptive models being extensional (concerns are selective) whereas descriptive ones being intentional (everything must be designed).

In any case, executable models are models of computation and therefore useless for the organizational (CIMs) and architectural (PIMs) aspects. Besides engineering aspects, they can only be introduced once responsibilities and functional architectures are explicitly set. Then, model contents and execution platforms must be subject to distinct quality management. From that point of view, patterns and model translators, even if set independently of model contents and transformation tools,  have contractual limitations and usually cannot to be used across organizational units.