Archive for the ‘Documentation’ Category

Agile Collaboration & Social Creativity

February 22, 2016

Open-plan offices and social networks are often seen as significant factors of collaboration and innovation, breeding and nurturing the creativity of knowledge workers, weaving their ideas into webs of truths, and molding their minds into some collective intelligence.

Brains need some breathing space

Open-plan offices, collaboration, and knowledge workers creativity

Yet, as creativity comes with agility, knowledge workflows should give brains enough breathing space lest they get more pressure than pasture.

Collaboration & Thinking Flows

Collaboration is a means to an end. To be of any use exchanges have to be fed with renewed ideas and assumptions, triggering arguments and adjustments, and opening new perspectives. If not they may burn themselves out with hollow considerations blurring clues and expectations, clogging the channels, and finally stemming the thinking flows.

Taking example from lean manufacturing, the first objective should be to streamline knowledge workflows as to eliminate swirling pools of squabbles, drain stagnant puddles of stale thoughts, and gear collaboration to flowing knowledge streams. As illustrated by flood irrigation, the first step is to identify basin levels.

Dunbar Numbers & Collaboration Basins

Studying the grooming habits of social primates, psychologist Robin Dunbar came to the conclusion that the size of social circles that individuals of a living species can maintain is set by the size of brain’s neocortex. Further studies have confirmed Dunbar’s findings, with the corresponding sizes for humans set around 10 for trusted personal groups and 150 for untried social ones. As it happens, and not by chance, those numbers seem to coincide with actual observations: the former for personal and direct collaboration, the latter for social and mediated collaboration.

Based on that understanding, the objective would be to organize knowledge workflows across two primary basins:

  • On-site and face-to-face collaboration with trusted co-workers. Corresponding interactions would be driven by personal dispositions and attitudes.
  • On-line and networked collaboration with workers, trusted or otherwise. Corresponding interactions would be based on shared interests and past exchanges.

Knowledge Workflows

The aim of knowledge workflows is to process data into information and put it to use. That is to be achieved by combining different kinds of tasks, in particular:

  • Data and information management: build the symbolic descriptions of contexts, concerns, and means.
  • Objectives management: based on a set of symbolic descriptions, identify and refine opportunities together with the ways to realize them.
  • Tasks management: allocate rights and responsibilities across organizations and collaboration frames, public and shallow or personal and deep.
  • Flows management: monitor and manage actual flows, publish arguments and propositions, consolidate decisions, …

Taking into account constraints and dependencies between the tasks, the aims would be to balance creativity and automation while eliminating superfluous intermediate products (like documents or models) or activities (e.g unfocused meetings).

With regard to dependencies, KM tasks are often intertwined and cannot be carried out sequentially; moreover, as illustrated by the impact of “creative accounting” on accounted activities, their overlapping is not frozen but subject to feedback, changes and adjustments.

With regard to automation, three groups are to be considered: the first requires only raw processing power and can be fully automated; the second also involves some intelligence that may be provided by smart systems; and the third calls for decision-making that can only be done by human agents entitled by the organization.

At first sight some lessons could be drawn from lean manufacturing, yet, since knowledge processes are not subject to hardware constraints, agile approaches should provide a more informative reference.

Iterative Knowledge Processing

A simple preliminary step is to check the applicability of agile principles by replacing “software” by “knowledge”. Assuming that ground is secured, the core undertaking is to consider what would become of cycles and iterations when applied to knowledge processing:

  • Cycle invariants: tasks would be iterated on given sets of symbolic descriptions applied to the state of affairs (contexts, concerns, and means).
  • Iterations content: based on those descriptions data would be processed into information, changes would be monitored, and possibilities explored.
  • Exit condition: cycles would complete with decisions committing changes in the state of affairs that would also entail adjustments or changes in symbolic descriptions.

That scheme meets three of the basic tenets of the agile paradigm, i.e open scope (unknowns cannot be set in advance), continuity of delivery (invariants are defined and managed by knowledge workers), and users in driving seats (through exit conditions). Yet it still doesn’t deal with creativity and the benefits of collaboration for knowledge workers.

Thinking Space & Pace

The scope of creativity in processes is neatly circumscribed by the nature of flows, i.e the possibility to insert knowledge during the processing: external for material flows (e.g in manufacturing), internal for symbolic flows (e.g in software engineering and knowledge processing).

Yet, whereas both software engineering and knowledge processes come with some built-in capability to redefined their symbolic flows on-the-fly, they don’t grant the same room to creativity. Contrary to software engineering projects which have to close their perspectives on the delivery of working products, knowledge processes are meant to keep them open to new understandings and opportunities. For the former creativity is the means to an end, for the latter it’s the end in itself, with collaboration as means.

Such opposite perspectives have direct consequences for two basic agile collaboration mechanisms: backlog and time-boxing:

  • Backlogs are used to structure and manage the space under exploration. But contrary to software processes whose space is focused and structured by users’ needs, knowledge processes are supposed to play on workers’ creativity to expand and redefine the range under consideration.
  • Time-boxes are used to synchronize tasks. But with creativity entering the fray, neither space granularity or thinking pace can be set in advance and coerced into single-sized boxes. In that case individuals must remain in full control of the contents and stride of their thinking streams.

It ensues that when creativity is the primary success factor standard agile collaboration mechanisms are falling short and intelligent collaboration schemes are to be introduced.

Creativity & Collaboration Tiers

The synchronization of creative activities has to deal with conflicting objectives:

  • On one hand the mental maps of knowledge workers and the stream of their thoughts have to be dynamically aligned.
  • On the other hand unsolicited face-to-face interactions or instant communications may significantly impair the course of creative thinking.

When activities, e.g software engineering, can be streamlined towards the delivery of clearly defined outcomes, backlogs and time-boxes can be used to harness workers’ creativity. When that’s not the case more sophisticated collaboration mechanisms are needed.

Assuming that mediated collaboration has a limited impact on thinking creativity (emails don’t have to be answered, or even presented, instantly), the objective is to steer knowledge workflows across a two-tiered collaboration framework: one personal and direct between knowledge workers, the other social and mediated through enterprise or institutional networks.

On the first tier knowledge workers would manage their thinking flows (content and tempo) independently, initiating or accepting personal collaboration (either through physical contact or some kind of instant messaging) depending on their respective “state of mind”.

The second tier would be for social collaboration and would be expected to replace backlogs and time-boxing. Proceeding from the first to the second tier would be conditioned by workers’ needs and expectations, triggered on their own initiative or following prompts.

From Personal to Collective Thinking

The challenging issue is obviously to define and implement the mechanisms governing the exchanges between collaboration tiers, e.g:

  • How to keep tabs on topics and contents to be safeguarded.
  • How to mediate (i.e filter and time) the solicitations and contribution issued by the social tier.
  • How to assess the solicitations and contribution issued by individuals.
  • How to assess and manage knowledge deemed to remain proprietary.
  • How to identify and manage knowledge workers personal and social circles.

Whereas such issues are customary tackled by various AI systems (knowledge management, decision-making, multi-players games, etc), taken as a whole they bring up the question of the relationship between personal and collective thinking, and as a corollary, the role of organization in nurturing corporate innovation.

Further Readings

EA Documentation: Taking Words for Systems

May 18, 2014

In so many words

Given the clear-cut and unambiguous nature of software, how to explain the plethora of  “standard” definitions pertaining to systems, not to mention enterprises, architectures ?

Documents and architectures, which grows on the other (Gilles Barbier).

Documents and Systems: which ones nurture the others (Gilles Barbier).

Tentative answers can be found with reference to the core functions documents are meant to support: instrument of governance, medium of exchange, and content storage.

Instrument of Governance: the letter of the law

The primary role of documents is to support the continuity of corporate identity and activities with regard to their regulatory and business environments. Along that perspective documents are to receive legal tender for the definitions of parties (collective or individuals), roles, and contracts. Such documents are meant to support the letter of the law, whether set at government, industry, or corporate level. When set at corporate level that letter may be used to assess the capability and maturity of architectures, organizations, and processes. Whatever the level, and given their role for legal tender or assessment, those documents have to rely on formal textual definitions, possibly supplemented with models.

Medium of Exchange: the spirit of the law

Independently of their formal role, documents are used as medium of exchange, across corporate entities as well as internally between their organizational units. When freed from legal or governance duties, such documents don’t have to carry authorized or frozen interpretations and assorted meanings can be discussed and consolidated in line with the spirit of the law. That makes room for model-based documents standing on their own, with textual definitions possibly set in the background. Given the importance of direct discussions in the interpretation of their contents, documents used as medium of (immediate) exchange should not be confused with those used as means of storage (exchange along time).

Means of Storage: letter only

Whatever their customary functions, documents can be used to store contents to be reinstated at a later stage. In that case, and contrary to direct (aka immediate) exchange, interpretations cannot be consolidated through discussion but have to stand on the letter of the documents themselves. When set by regulatory or organizational processes, canonical interpretations can be retrieved from primary contexts, concerns, or pragmatics. But things can be more problematic when storage is performed for its own purpose, without formal reference context. That can be illustrated by legacy applications with binary code can be accompanied by self-documented source code, source with documentation, source with requirements, generated source with models, etc.

Documentation and Enterprise Architecture

Assuming that the governance of structured social organizations must be supported by comprehensive documentation, documents must be seen as a necessary and intrinsic component of enterprise architectures and their design should be aligned on concerns and capabilities.

As noted above, each of the basic functionalities comes with specific constraints; as a consequence a sound documentation policy should not mix functionalities. On that basis, documents should be defined by mapping purposes with users across enterprise architecture layers:

  • With regard to corporate environment, documentation requirements are set by legal constraints, directly (regulations and contracts) or indirectly (customary framework for transactions, traceability and audit).
  • With regard to organization, documents have to met two different objectives. As a medium of exchange they are meant to support the collaboration between organizational units, both at business level (processes) and across architecture levels. As an instrument of governance they are used to assess architecture capabilities and processes performances. Documents supporting those objectives are best kept separate if negative side effects are to be avoided.
  • With regard to systems functionalities, documents can be introduced for procurements (governance), development (exchange), and change (storage).
  • Within systems, the objective is to support operational deployment and maintenance of software components.
Documents’ purposes and users

Documents’ purposes and users

The next step will be to integrate documents pertaining to actual environments and organization (brown background) with those targeting symbolic artifacts (blue background).

Models are used to describe actual or symbolic objects and behaviors

Models are used to describe actual or symbolic objects and behaviors

That could be achieved with MBE/MDA approaches.

Further readings


Making the Rules

August 19, 2013

While models are statements about the categories of things that may exist in business domains, rules are statements about business facts, i.e instances of those categories. Yet, given that requirements usually don’t come with categories fully and consistently defined upfront, the way rules are expressed can only be settled through requirements analysis.


Business concerns and the rules of game (H. Bosch)

As a consequence, from requirements lumps to well-formed expressions of business logic, rules will get through some metamorphosis. That raises two questions:  is it possible to map those transformations, and, if it’s the case, what kind of criteria should govern the making of the rules.


The footprint of a rule is made of the categories of facts to be considered (aka rule domain), and categories of facts possibly affected (aka rule co-domain).

As far as systems are concerned, the first thing to do is to distinguish between actual contexts and symbolic representations. A naive understanding would assume rules to belong to either actual or symbolic realms. Given that the objective of modeling is to decide how the former should be represented by the latter, the opposite is to be expected and dealt with using three categories of rules, one for each realm and the third set across the divide:

  • Rules targeting actual contexts. They can be checked through sensors or applied by actuators. Since rules enforcement cannot be guaranteed on non symbolic artifacts, some rules will have to monitor infringements and proscribed configurations. Example: “Cars should be checked on return from each rental, and on transfer between branches.”
  • Rules targeting symbolic representations. Their enforcement is supposedly under the full control of system components. Example: “A car with accumulated mileage greater than 5000 since its last service must be scheduled for service.”
  • Rules defining how changes in actual contexts should impact symbolic representations: what is to be considered, where it should be observed, when it should be recorded, how it should be processed, who is to be authorized. Example: ” Customers’ requests at branches for cars of particular models should be consolidated every day.”

That classification should be applied as soon as possible because the third category will determine the impact of requirements on architecture capabilities.

Syntax and Semantics

Identifying footprints from requirements may be complex but the outcome isn’t contingent on modeling options. That’s not the case for syntactic constructs used to define observed and targeted facts.

Rules can be built along four basic constructs, depending on the way domains (categories of instances to be valued) and co-domains (categories of instances possibly affected) are considered:

  • Partitions are expressions used to classify facts of a given category.
  • Constraints (backward rules) are conditions to be checked on facts: [domain].
  • Pull rules (static forward) are expressions used to modify facts: co-domain =  [domain].
  • Push rules (dynamic forward) are expressions used to trigger the modification of facts: [domain]  > co-domain.
Pull vs Push Rule Management

Pull vs Push Rule Management

Domains and co-domains are defined by rule semantics; they can be homogeneous (single category) or heterogeneous (joint categories).

Homogeneous rules target single categories and are therefore easier to be anchored and documented in models:

  • Physical objects: monitoring, computation rules, events triggering rules.
  • Symbolic objects: integrity constraints, derivation rules, inference or cascading updates.
  • Activities: local conditions on data, processing of data flows, sequencing of operations.
  • Processes: local conditions on state values, derived states, stimulus or triggering of transitions.
Homogeneous Footprints

Footprints: Syntax and Semantics

Since homogeneous expressions are the rules’ building blocs, factoring them out should be the first step in the process of rules formulation.

In principle, rules targeting different categories of facts are nothing more than well-formed expressions combining homogeneous ones. In practice, because they mix different kinds of artifacts, the way they are built is bound to significantly bear on architecture capabilities.

Homogeneous Footprints

Before combining simple expressions into complex ones, the first step is to circumscribe their scope depending on the anchor used to identify facts that have to be considered (domain) or possibly modified (co-domain):

Where to anchor rules statements

Where to anchor rules statements

  • Features: domain and co-domain are limited to attributes or operations.
  • Object footprint: domain and co-domain are set within the limit of a uniquely identified instance (#), including composites and aggregates.
  • Connections: domain and co-domain are set by the connections between instances identified independently.
  • Collections: domain and co-domain are set between sets of instances and individuals ones, including subsets defined by partitions.
  • Containers: domain and co-domain are set for whole systems.

Those constructs are purely syntactic and can be used indifferently for all statements, actual or symbolic:

Footprints: Range of artifacts to be valued and possibly affected.

Footprints: Range of artifacts to be valued and possibly affected.

While minimizing the scope of simple (homogeneous) rules is arguably a straightforward routine, alternative options may have to be considered for the transformation of joint (heterogeneous) statements, e.g when rules about functional dependencies may be attached either to (1) persistent representation of objects and associations or, (2) business applications.

Heterogeneous (joint) Footprints

Footprints set across different categories will usually make room for alternative modeling options. As those options will determine the way rules will be executed, they will bear differently on architecture capabilities. Hence the need to organize joint rules depending on their impact.

A preliminary approach would be to try to regroup rules along requirements taxonomy:

  • Business requirements: rules set at enterprise level that can be managed independently of the architecture.
  • Functionalities: rules set at system level whose support depends on architecture capabilities.
  • Quality of service: rules set at system level whose support depends on functional and technical architectures.
  • Operational constraints: rules set at platform level whose support depends on technical capabilities.
Rules do not necessarily fit into clear requirements taxonomy

Rules do not necessarily fit into clear requirements taxonomy

While that classification may work fine for homogeneous rules (a), it may fall short for mixed ones, functional (b) or not (c). For instance:

  • “Gold Customers with requests for cars of particular models should be given an immediate answer.”
  • “Technical problems affecting security on checked cars must be notified immediately.”

Given that such rules interweave business, functional, and non functional requirements, their formulation should reflect how priorities are to be sorted out between different levels of requirements.

On a broader perspective, if rule transformation is to be carried out, there will be more than syntax and semantics to consider because almost every requirement can be expressed as a rule, often with alternative options. As a consequence, the modeling policies (aka pragmatic) governing the making of rules should be set explicitly.

Pragmatic: Sorting out mixed rules

Taking into account that functional requirements describe how systems are meant to support  business processes, some rules are bound to mix functional and business concerns. When that’s the case, preferences will have to be set with regard to:

  • Events vs Data (a): should system behavior be driven by changes in business context (as signaled by events from users, devices, or other systems), or by changes in symbolic representations (a).
  • Activities vs Data: should system behavior be governed by planned activities, or by the states of business objects (b).
  • Activities vs Events: should system behavior be governed by planned activities, or driven by changes in business context (c).
Pragmatics: How to fit Rules and Architecture Capabilities

Pragmatic: How to fit Rules and Architecture Capabilities


The same questions arise for rules mixing functional requirements, quality of service, and operational constraints (d), e.g:

  • How to apportion response time constraints between endpoints, communication architecture, and applications.
  • How to apportion reliability constraints between application software and resources at location .
  • How to apportion confidentiality constraints between entry points, communication architecture, and locations.

How those questions are to be answered, in particular at what level, should be decided within a comprehensive and consistent enterprise architecture.

Decision Points, Architecture, and the Making of Rules

Beyond requirements specifics, rules should befit organizational and architectural options set at enterprise level. That should be the aim of requirements analysis and rules transformation.

While rules description combines syntactic constructs (constraints, computation, triggers, etc) and semantics (objects, events, activities, role, etc), their transformation is to be governed by pragmatic, more precisely the modeling options reflecting the architecture framework. For instance, aiming at a Model/View/Controller framework, rules would be organized along three layers:

  • Model:  constraints, derivations, inferences and cascading updates that can be applied on shared and persistent instances independently of applications.
  • Views: constraints and processing rules that can be applied on non shared and transient instances executed at entry points.
  • Controller: processing and control rules that can be applied on shared and transient instances independently of entry points.

Obviously, pragmatic can go further than simple MVC layers and take into account application life-cycle (e.g along MDA’s CIMs, PIMs, and PSMs), and collaboration mechanisms (e.g activity- vs event-driven). For that purpose the making of rules should follow three guidelines:

  1. Partitions and constraints should be used to anchor rules to decision points clearly set with regard to architecture (e.g MVC) and model layers (e.g MDA). For instance: the decision point for the type of reservation belongs to computation Independent business logic, and classifications of customers and cars belong to computation Independent business domains.
  2. Backward constructs should be used for rules pertaining to business logic independently of processes execution. For instance: how to process a reservation (a).
  3. Forward constructs should be used to compare alternative execution models. For instance: activity driven through control flows (b), or event-driven through event notification (c).
"Gold Customers with requests for cars of particular models should be given an immediate answer."

“Gold Customers with requests for cars of particular models should be given an immediate answer.”

Not by chance, applying those guidelines will neatly map rules to standard diagrams: class diagrams (partitions and integrity constraints), use cases (extension points), activity diagrams (backward rules for branches and functions), business processes (backward rules for gateways) and state charts (forward rules for transitions).

Further Reading

External Links