Iterative Processes


Iterative development, together with users’ stories, is arguably the cornerstone of agile methods as, in contrast to phased approaches, it combines incremental design and delivery of software components.

Circles of Concerns (Paulo Vinluan)

But iterative processes are not limited to agile and can be usefully employed by different development strategies insofar that:

  • The same activity (simple or combined) carried repetitively by the same organizational unit sharing responsibility.
  • A set of conditions to be met.
  • An exit condition (simple or combined) to be checked at each iteration.

Software Development

To begin with, the specificity of software engineering must be underlined: compared to manufacturing processes, software ones come with few, if any, physical constraints, and that fluidity clearly help out iterative development.

Nonetheless, software components don’t stay on virtual shelves but are embedded in actual products or used to run actual devices or systems, and that where frictions are to come forth.

That put software development at the junction between two perspectives:

  • The business perspective faces the real world, where frictions may, and usually will arise. It is said to be synchronic because objectives, constraints, and risks change along time, and software components must be continuously and consistently adjusted to changing requirements defined within time-frames set externally.
  • The engineering perspective faces the digital world, which is essentially friction-less. It is said to be diachronic because once rooted into requirements, the design and implementation of  system artifacts can live on their own time-span.

Iterations may be anchored to business context or system artifacts.

In between architectures are meant to be both lubricant and glue, the former to smooth the alignment of business and systems, the latter to maintain the continuity and consistency of core business concerns and corporate identities. That twofold objective is to be the basis of iterative layouts.

Iterative Software Development

Iterative software development is best defined by applying the control loop paradigm to software engineering objectives:

  • Scope and invariants are defined with regard to the business (synchronic) perspective.
  • Increments are defined with regard to the engineering (diachronic) perspective.
  • Exit conditions are defined with regard to business, functional, and technical requirements.

Each iteration is then to repeat the same set of different activities, carried jointly by a single team regrouping different skills and representing a collective authority and responsibility.

Defined with this layout, iterative processes are not off-the-cuff solutions to be applied to any circumstances but require specific conditions to be met.

Regarding scope and invariants (business, synchronic perspective), work units should coincide with the perimeter of symbolic representations, which means that iterative loops should not cross organizational units (stakeholder) or functional domain. That will also ensure the synchronization of changes in contexts and systems.

Regarding increments (engineering, diachronic perspective), iterations must be carried out independently of external circumstances, and products delivered continuously providing they meet exit conditions.

When these conditions cannot be achieved projects are to be fragmented according to domains (business perspective) or activities (engineering perspective).

Based upon a basic definition of tasks (requirements, development, deployment, architecture and quality) and the associated milestones, some archetypes of iteration patterns can be identified.

Global Iterations

Global iterations repeat the full development cycle, hence combining the authority of different organizational units, e.g requirements, development, deployment.

Global iterations repeat the full cycle.

For a given technical architecture, iterations are associated to the development and deployment of incremental requirements.

  • Invariants: backbone requirements of business objects and processes whose identity and consistency define the functional architecture.
  • Increments: new functionalities or implementations.
  • Exit condition: no more requirements

That’s the blueprint behind agile development approaches.

Local Iterations

Local iterations repeat parts of the development cycle and are meant to be managed under the authority of a single organizational unit.

For a given set of requirements, iterations are associated to the development (analysis, design, code) of subsets of functionalities.

  • Invariants: Set of requirements.
  • Increments: new system functionalities.
  • Exit condition: components tested and ready to be deployed.

That blueprint is typically used for outsourced projects.

Overlapping Iterations

Overlapping (or simultaneous) iterations repeat only parts of the development cycle, hence combining the authority of different organizational units independently of actual deliveries. As a corollary, development teams may be working on new subsets of requirements with previous ones still not be accepted. Based upon MDA model layers, one can identify two main archetypes of such iterative processes:

  • Domain driven iterations: analysis and design are merged into a single loop in order to derive full functional specifications (PIM) from given uses case (CIM). That organization is recommended when a  functional architecture has to be defined in order to support a representative or critical use case.
  • Platform driven iterations: design and implementation are merged into a single loop in order to develop full implementation packages (PSM) from functional specifications (PIM). That organization is recommended when development is outsourced.

Overlapping iterations dissociate organization and deployment.

Overlapping iterations clearly bring in a higher level of complexity, as perimeters, authorities, and tests may have to be managed independently; and since organizational procedures are no more available as abutments, iterations are to be defined directly on model contents and development flows, in other words on shared features.

Iterations & Architecture Capabilities

Except for standalone applications, iterations must be bound by invariants set along architectural tiers: interactions are developed for given business logic (application layer), and applications are developed for given domains (persistency layer). As noted above, model layers and architecture tiers don’t necessarily coincide which can make iterative development difficult to manage.

Iterative Development & Architecture Tiers

That difficulty can be significantly reduced by anchoring iterations to homogeneous architecture capabilities, e.g Zachman’s framework:

  • Who: enterprise roles, system users, platform entry points.
  • What: business objects, symbolic representations, objects implementation.
  • How: business logic, system applications, software components.
  • When: processes synchronization, communication architecture, communication mechanisms.
  • Where: business sites, systems locations, platform resources.

Enterprise, functional (systems), and technical (platforms) capabilities

On that basis iterations can be neatly circumscribed to architecture capabilities, e.g:

  • Business and organization: domain driven iterations.
  • Systems functionalities: application driven iterations.
  • Platforms: migrations.

Domain driven iterations across requirements, functional features, and implementation.

Domain Driven Iterations

Domain driven iterations add features and/or operations to a set business objects and/or use cases. With CIM/PIM overlapping iterations, the critical assumption is that the architecture of symbolic representations (semantics and identification mechanisms) are not affected from within the loop. Using the Rent a Cart example, iterations can add a new option for scythed chariots (with an extended use case) providing they are identified as other carts and rented using the same process.

Domain driven iterations add features and/or operations to persistency and/or execution units.

Hence the control structure of iterative modeling:

On a broader perspective, overlapping iterations must take into account the different lifespans of CIM artifacts:

  • Business opportunities are by nature diverse and volatile. At symbolic level they are represented by applications. Since they are directly used by agents, domain driven iterations (possibly with prototyping) often bring significant benefits.
  • Business domains are meant to secure a long-term basis for business development. In that case domain driven iterations may add to objects descriptions without affecting their semantics or identification mechanisms.
  • Business organizations are the basis of corporate identities and valuation. In that case domain driven iterations may add to objects descriptions across domains.

Altogether, overlapping iterations should entail only local changes and as such should not introduce new architectural constraints. Yet, only a thorough definition of invariants may guarantee model validity, in particular for external consistency.

Application Driven Iterations

Application driven iterations add functional variants to use cases without affecting existing ones, the assumption being that business domains and rules remain unchanged.

Iterations within Use Cases add execution paths

Platform Driven Iterations

Platform driven iterations add features and/or operations directly to system components. With PIM/PSM overlapping iterations, the critical assumption is that the mapping between business contexts and their system counterpart is not affected from within the loop. Using the Rent a Cart example, iterations can add new features to the symbolic description of scythed chariots (with corresponding changes to interfaces and messages), and modify their implementation accordingly, providing the continuity and consistency of actual business objects representations are maintained.

Overlapping platform driven iterations: both PIMs & PSMs are modified within the loop but the continuity of representations is maintained.

Iterative development, executable models, & model engineering

All in all, iterative development should always be set within a clearly defined development strategy, and take into account the logical tools supporting iterations, namely: patterns of representation (CIM/PIM) and implementation (PIM/PSM) on one hand, model transformation mechanisms one the other hand.

Model interpretation shouldn’t cross organizational units, code-generation may.

A particular attention should be given here to the meaning of “executable models”:

  • Weak versions add some action semantics in order to support prototyping
  • Language based solutions, e.g executable UML, use pseudo language like OCL or state machines and enable full code-generation.
  • Domain based solutions use profiles and domain specific languages.

Set on a broader perspective, models can be either descriptive or prescriptive, but cannot be both. Hence, an executable model is either a simulation of a business process (prescriptive) or an application prototype (descriptive), prescriptive models being extensional (concerns are selective) whereas descriptive ones being intentional (everything must be designed).

In any case, executable models are models of computation and therefore useless for the organizational (CIMs) and architectural (PIMs) aspects. Besides engineering aspects, they can only be introduced once responsibilities and functional architectures are explicitly set. Then, model contents and execution platforms must be subject to distinct quality management. From that point of view, patterns and model translators, even if set independently of model contents and transformation tools,  have contractual limitations and usually cannot to be used across organizational units.

PS: Agile as a Postscript to Postmodernism

Funnily, Agile practice of iteration can be seen as a healthy correction to postmodernist theorizing. For authors like Derrida the iterability of the written mark entails that the synchronic and diachronic associations of the written sign are both present and divorced from each other so that the written sign can possibly carry an idea while not carrying an idea. Applied specifically to textual requirements that would mean an infinity of contexts and would explain the difficulty of securing systems requirements upfront. Fortunately practical (aka agile) iterations have only to deal with two contexts, business and systems; and that can be achieved if a clear distinction is maintained between the semantics of requirements and design.

Further Reading

External Links


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: