Specifications QA

If quality is the probability that nothing will go amiss, quality assurance should focus on preventing the risks. For that purpose checking specifications is probably one of the most effective policy.


Better Prevent than Amend


Whatever the target (requirements, specifications, models, source code, documents, etc), QM should be supported by four pillars:

  • Understanding: deliverables must be formulated with languages and standards of the parties involved.
  • Completeness: deliverables or tasks must include a predefined and finite set of managed elements to be checked at completion.
  • Consistency: managed elements must be non ambiguous, reliable, and not contradictory. External consistency should therefore entails traceability between problems and solutions.
  • Correctness: reliability, usability, maintainability, etc.

Quality Perspectives

Depending on development approach, quality management will go for continuous (agile projects) or staged (phased projects) assessment of deliverables.

Phased Validation

As phased approaches usually rely on differentiated models, quality should be assessed with regard to their purpose, as exemplified by the MDA framework with platform independent models (PIMs), computation independent (CIMs) and platform specific (PSMs) ones:


Models must be assessed with regard to their purpose

  • External correctness and completeness: How to verify that all the relevant individuals and features are taken into account. That can only be achieved empirically by building models open to falsification.
  • Consistency: How to verify that the symbolic descriptions (categories and connectors) are complete, coherent and non redundant across models and abstraction levels. That can be formally verified.
  • Internal correctness and completeness (alignment) : How to verify that current and required business processes are to be seamlessly and effectively supported by systems architectures.

Continuous Validation

By contrast, iterative approaches rely on continuous deliveries, which means that quality is part and parcel of development, the idea being that QA is a built-in property of deliverables. Yet, as continuous development by itself doesn’t guarantee continuous quality, differentiated tests are needed if regression is to be avoided.

To begin with, some initial assessment of requirements capture is required before any resources are committed to development. Depending on the specificity and formal properties of domain language, this stage may be routine (non specific languages) or automated (formal languages).

Reviewing the quality of specifications is a prerequisite.

Then, quality checks should tally the prioritization of developments according to requirements taxonomy:

  1. Pure business requirements (business objects and logic)
  2. Functional requirements (bound to users interactions with systems), to be further refined for non shared (local), shared but transient (transactions), and shared and persistent (domains).
  3. Quality of service.

Tests could then be threaded with development iterations:

Unit tests are internal; they deal with technical aspects of development units disregarding functional requirements.

Unit tests can be performed in isolation

Integration tests are internal; they deal with the design of deployment units (aka modules) disregarding functional requirements.

Integration tests check if the whole works properly whatever its use.

Function tests are internal and deal with system design regarding functional requirements.

Function tests check uses without users

Performance tests are external and deal with system design regarding non functional requirements.

Performance tests check for resources

Acceptance tests are external and deal with system design regarding all requirements set in test environment.

Acceptance tests check with users within protected environments

Installation tests are external and deal with the specific resources and procedures associated with setting the product in its operational context.

Installation tests check how to set products in their operational context.

Operational tests are external and deal with system functionalities in actual environment.

Operational tests check functionalities in actual environments

Finally, usability tests deal with ergonomy and maintenance.

Maintenance, ergonomy, and evolution.

Set along that perspective, quality assurance is not to be contingent on development model.

Quality Driven Development

While the ill-fated Waterfall development model has demonstrated the hazards of quality set as an afterthought, agile solution remains partial as it doesn’t deal explicitly with the problem of regression across layers.

That shortcoming could be amended with work units defined according to their validation requirements. And that could be done through phased development as well as agile prioritization.


%d bloggers like this: