Archive for the ‘Software Engineering’ Category

Focus: Business Analyst Booklet

November 6, 2017

Objective

Business analysts stand between unbounded and moving business landscapes on one hand, distinctive and steady enterprise organization and culture on the other hand.

How to align enterprise resources and business opportunities (Patrick Zachmann)

Assuming that BAs’ primary concern is to keep ahead of the competition, framing business undertakings into universal guidelines could be counterproductive. By contrast, harnessing together versatile business processes and reliable systems architectures will clearly enhance business agility; hence the benefits of lining up enterprise architects’ and business analysts’ conceptual toolboxes:

  1. Concepts : eight exclusive and unambiguous definitions provide the conceptual building blocks.
  2. Models: how the concepts are used to consolidate business requirements and convey them to enterprise architects and software engineers.
  3. Processes: how to harness organization and business objectives and align applications with business value.
  4. Architectures: how to contrive along time the continuity and consistency of business concepts and objectives, and their congruence with systems capabilities.
  5. Governance: assessment of business value and risks.

On that basis, the objective here is not to detail BAs’ tasks or methods but to focus on core issues to be addressed by business analysts.

Concepts

Whereas systems architecture is not their primary concern, business analysts should nonetheless share the same modeling paradigm:

  • Analysis models for business environments and objectives.
  • Design models for the architecture of systems and the specification of components.

Business objects and processes must be consistently identified (#) across business and system realms.

It is worth to remind that the distinction between descriptive (aka analysis) and prescriptive (aka design) models is not arbitrary but based on logic principles: the former are extensional as they classify actual instances of business objects and activities; in contrast, the latter are intensional as they define the features and behaviors of required system artifacts.

The distinction also brings organizational benefits as it tallies with BAs’ responsibility regarding the consistency and continuity of identities and semantics of actual objects and processes (business extensions) and their symbolic counterparts (system intensions):

Relevant categories at architecture level can be neatly and unambiguously defined.

  • Actual containers represent address spaces or time frames; symbolic ones represent authorities governing symbolic representations. System are actual realizations of symbolic containers managing symbolic artifacts.
  • Actual objects (passive or active) have physical identities; symbolic objects have social identities; messages are symbolic objects identified within communications. Power-types (²) are used to partition objects.
  • Roles (aka actors) are parts played by active entities (people, devices, or other systems) in activities (BPM), or, if it’s the case, when interacting with systems (UML’s actors). Not to be confounded with agents meant to be identified independently of their behavior.
  • Events are changes in the state of business objects, processes, or expectations.
  • Activities are symbolic descriptions of operations and flows (data and control) independently of supporting systems; execution states (aka modes) are operational descriptions of activities with regard to processes’ control and execution. Power-types (²) are used to partition execution paths.

While business analysts should only be tasked with the continuous and consistent mapping of business individuals to their system surrogates, and not with their implementations, that cannot be achieved without a full and unambiguous specification of the variants and abstractions for the business objects and processes to be represented.

Languages & Models

Being in charge of requirements, business analysts can be seen as the gate-keepers of the whole engineering process. To begin with, and depending on the nature of domains, BAs can capture requirements using formal (e.g for scientific domains), specific, or natural languages. Then, requirements analysis can be carried out:

  • Iteratively in unison with development and in collaboration with software engineers (agile approach). In that case models are not necessary as requirements are expressed in natural language (users’ stories), possibly combined with domain specific languages (DSLs) for development.
  • As phased undertakings carried out independently, using a dedicated modeling language (e.g BPMN).
  • As phased undertakings carried out jointly with system analysts using a general purpose modeling language (e.g UML).

Three ways to deal with requirements analysis: business oriented and phased (BPMN), system oriented and phased (use cases), or business driven and iterative (users’ stories).

These schemes are therefore best understood as tools whose employ may overlap or be combined:

  • BPMN and UML activity diagrams have much in common.
  • Class diagram can complement BPMN for business objects, and State diagrams for processes control.
  • Use cases can be seen as describing the part of users’ stories to be supported by systems.

How BAs will employ them is to depend on business processes and projects’ objectives.

Business & Development Processes

The responsibility of BAs is about business processes, the choice of development model being left to project managers; hence the need for business analysts to be familiar with basic options:

  • Agile: business analysts collaborate with software engineers in project teams and share responsibilities from requirements to delivery.
  • Phased: roles and responsibilities are defined specifically with regard to development tasks.

With agile schemes BAs share roles and responsibilities all along, with phased ones roles and responsibilities are defined with regard to tasks.

Agile or phased, the contribution of business analysts can be defined around three core issues, corresponding to three typical modus operandi:

  • Concepts associated to business objects and activities that are to be represented. Assuming that conceptual models are meant to be stable and shared across processes, they should be under the responsibility of business analysts independently of applications.
  • Actors (users, devices, or systems) and activities. Insofar as the impact on organization and system functional features can be localized (users interfaces) or circumscribed (business rules), business analysts can collaborate and share responsibility with software engineers all along an iterative process. Otherwise (changes in organization or business functions) business analysts will have to consolidate their work with enterprise architects.
  • Processes execution. Often labelled as non functional capabilities, they essentially deal with the different aspects of user’s experience and the synchronization of changes in business environments and supporting systems. For that purpose business analysts will have to check requirements against systems capabilities.

Business analysts core concerns and MO: conceptual model, activities, and processes.

While these issues are often interwoven, sorting them out can help to match development models with projects objectives and scope: agile for projects facing business users, phased for the ones dealing with architectures; that will also help to characterize the role of BAs depending on focus: business processes (BPM, use cases, users’ stories), functional architecture (services, conceptual models), or quality of services.

Business Analysis & Systems Architectures

When considering business opportunities, business analysts have to define requirements’ footprint with regard to system capabilities:

  • Confined: applications can be developed in collaboration with software engineers from users’ stories to code, without modeling. Assuming agile conditions about shared ownership and continuous delivery are met, that would be the default option.
  • Distributed: some modeling is needed for communication and consolidation purposes. But business processes modeling languages like BPMN make no distinction between processes’ details and the shared features of supporting systems. That puts a challenging toll on business analysts (complexity, ambiguity) with limited benefits (no easy mapping to system functions).

A primary concern for business analysts should therefore to frame projects accordingly: self-contained and business driven on one hand, shared and architecture driven on the other hand, with use cases set in between if and when necessary. For that purpose shared concerns will have to be clearly identified; taking BPMN for example:

BPEA_ArchiProc

Separation of concerns: architecture backbone and processes’ details

  • Containers for physical (locations) and logical (organizations and domains) objects have no BPMN explicit equivalents.
  • Active objects have no BPMN explicit equivalent.
  • Swimlanes and pool tally with roles (aka actors)
  • Data stores tally with entities (persistent representation of business objects).
  • Tasks, transactions, and sub-processes can be translated as activities description and processes execution.

Given backbones shared with enterprise architects, the next step is to flesh them out with specific details. Depending on methods and tools, that can be done using a domain specific language (DSL) with direct implementation, or through a generic subset of BPMN that could be unambiguously mapped to design constructs, for instance:

  • Anchors (#): instances (objects or activities) directly and consistently identified across businesses and system.
  • Collections (*): set of individuals with shared features.
  • Features: attributes or operations without identity of their own.
  • Structures (diamond): composition (black) for individual components (objects or activities) whose life-cycle is bound to their owner, i.e they have no identity of their own; aggregation (white) for components identified independently but used in the context of their owner.
  • Connectors: associate individuals; their semantics is set by context: communication channel, reference, data or control flow, transition. They can bear identification (#).
  • Power-types (2): define subsets of individuals objects or activities. Depending on context and modeling language, power-types correspond to classifications, extension points, gateways, branch and joins, etc.
  • Inheritance (triangle): contrary to structure and functional connectors that deal with instances, inheritance connectors are used to describe relationships between descriptors. Strong inheritance (black) is the counterpart of composition (inheritance of structural features), and weak inheritance (white) the counterpart of aggregation (inheritance functional features).

Separation of concerns: architecture backbone and anchors details

Using the same set of well accepted and unambiguous logical constructs for both objects and behaviors can greatly enhance the consistency of analysis models as well as their traceability to designs.

Business Analysis & Knowledge Architecture

As noted above, while business analysts may have to consolidate functional requirements or check the feasibility of non functional ones with enterprise architects, they should take responsibility for conceptual models, and more generally for enterprise knowledge architecture. Taking a leaf from Davis, Shrobe, and Szolovits, that will cover:

  1. Surrogates: description of symbolic counterparts (aka) of actual objects, events and relationships.
  2. Ontological commitments: statements about the categories of things that may exist in the domain under consideration.
  3. Fragmentary theory of intelligent reasoning: model of what the things can do or can be done with.
  4. Medium for efficient computation: knowledge understandable by computers.
  5. Medium for human expression: communication between specific domain experts on one hand, generic knowledge managers on the other hand.

Putting apart users interfaces (point 5), two typical approaches can be considered:

  • Domain Driven Design (DDD), which deals with domains representation and computation from a system perspective (point 4).
  • Ontologies, which put the focus on knowledge oriented languages independently of computation (points 1-3).

Besides their simplex orientation, both fall short of business analysts needs, the former being too technical, the latter too open-ended. Instead, a conceptual framework should combine bounded domains with a compact and unambiguous knowledge oriented language.

As it happens, mapping the symbolic footprint of business domains and knowledge into systems may be dictated by the generalization of networked environments and digital business flows. Along that reasoning, BAs will have to deal with knowledge from domains as well process perspectives.

With regard to domains, a distinction should be maintained between institutional (external, statutory), business specific (external, agreed), and enterprise specific (internal).

vvv

A conceptual approach to domain layers: institutional, business specific (e.g HR management) and enterprise specific (e.g supply, sales).

With regard to processes, knowledge must be understood as the dynamic and multi-faceted outcome of data analytics, production systems, and decision-making. Taking a (revised) leaf of Zachman’s framework, business and operational objectives would be reset as to cross architecture layers instead of being aligned. Using a pentagonal representation of enterprise architecture, Zachman’s sixth column (“Why” ) would be rounded as an outer range.

Knowledge: timely and multi-faceted information put to use

Such tightened integration of business processes and IT systems can be decisive in getting a competitive edge, as illustrated by the OOAD (Observation, Orientation, Decision, Action) loop, a real-time decision-making paradigm originally developed by Colonel John Boyd for USAF fighter jets:

  • Observation: operational processes must provide accurate and up-to-date analysis of business contexts as well as feedback.
  • Orientation: transparency of functional architecture is to support business positioning and the adjustment of business objectives.
  • Decision: versatility and plasticity of applications are to facilitate change of tactical options..
  • Action: integration of business, engineering, and operational processes are to ensure just-in-time business moves.

BAs must consider the benefits of systems integration for decision-making.

On a broader perspective the integration of data analytics, production systems, and knowledge management is becoming a key success factor for governance.

Governance: Metrics, Quality, & Risks

As gate-keepers, business analysts have to rank projects with regard to business value, risks, and return on investment. Assuming that business value is set independently of supporting systems, projects’ assessment and ranking should be set according to the nature of problems:

  • Intrinsic business size and complexity: requirements can be estimated from individuals (objects and activities), features, relationships, and partitions.
  • Supporting systems functionalities: intrinsic business metrics are to be combined with what is expected from supporting systems: processes and transactions, triggering events, users and devices interfaces, etc.
  • Business and functional measurements can then be weighted by non-functional (aka Quality of Service) requirements.

Assessment should be aligned with problems: business, supporting systems, operations.

If returns on investment (ROI) and risks are to be assessed consistently and decision-making carried out accordingly, value, costs, quality, and hazards have to be set within the same framework, in particular for quality and risks management:

  • Business environment: risks are external and quality is to check for timely and relevant analysis models.
  • Engineering:  risks are internal and quality is to focus on processes maturity.
  • Technologies: risks are external and quality is to address versatility, plasticity, and effectiveness of solutions.

To conclude, whereas business risks remain the primary concern of business analysts, the fusion of business and systems processes means that they can no longer ignore engineering pitfalls and the importance of quality for risks management.

Further Reading

Advertisements

On The Holistic Nature of MBSE

October 1, 2017

Preamble

Interestingly, variants of MBSE/MDSE acronyms put the focus on the duality of the concept, software on one side, systems on the other.

MBSE is by nature a two-faced endeavor (Sand Painting Navajo Rug)

As that duality operates for models, systems, and organizations, MBSE offers a holistic view on enterprise architecture.

Models and Software

Models are symbolic representations of actual contexts in line with specific purposes: requirements analysis, simulation, software design, etc. Software is a subset of models characterized by target (computer code) and language (executable instructions). Based on that understanding, MBSE should not be limited to DSLs silos and code generation but employed to bring together and manage the whole range of concerns and artifacts.

Systems and Applications

The hapless track record of Waterfall and the parallel ascent of Agile have clouded the grounds for phased development processes. But whereas agile schemes are the default option when applications can be developed independently, external dependencies prevent their scaling up to system level. That’s when system engineering takes precedence on applications development, with MBSE introduced to manage shared models and support collaboration between teams.

Organization and Projects

As epitomized by agile development models, projects can be driven by specific business needs or shared architecture capabilities. Whereas the former are best carried out iteratively by autonomous teams sharing skills and responsibility, the latter entail collaboration between organizational units along time. MBSE provides the link between standalone projects, phased processes, and enterprise organization.

MBSE provides a holistic view of organisations and systems.

By providing a holistic view of changes in organizations, systems, and software, MBSE should be a key component of enterprise architecture.

Further Reading

 

EA’s Merry-go-round

June 14, 2017

Preamble

All too often EA is planned as a big bang project to be carried out step by step until completion. That understanding is misguided as it confuses EA with IT systems and implies that enterprises could change their architectures as if they were apparel.

EA is a never-ending endeavor (Robert Doisneau)

But enterprise architecture is part and parcel of enterprises, a combination of culture, organization, and systems; whatever the changes, they must keep the continuity, integrity, and consistency of the whole.

Capabilities

Compared to usual projects, architectural ones are not meant to address specific business needs but architecture capabilities that may or may not be specific to business functions. Taking a leaf from the Zachman Framework, those capabilities can be organized around five pillars supporting enterprise, systems, and platform architectures:

  • Who: enterprise roles, system users, platform entry points.
  • What: business objects, symbolic representations, objects implementation.
  • How: business logic, system applications, software components.
  • When: processes synchronization, communication architecture, communication mechanisms.
  • Where: business sites, systems locations, platform resources.

These capabilities are set across architecture layers and support business, engineering, and operational processes.

Enterprise architecture capabilities

Enterprise architects are to continuously assess and improve these capabilities with regard to current weaknesses (organizational bottlenecks, technical debt) or future developments (new business, M&A, new technologies).

Work Units

Given the increased dependencies between business, engineering, and operations, defining EA workflows in terms of work units defined bottom-up from capabilities is to provide clear benefits with regard to EA versatility and plasticity.

Contrary to top-down (aka activity based) ones, bottom-up schemes don’t rely on one-fits-all procedures; as a consequence work units can be directly defined by capabilities and therefore mapped to engineering workshops:

Iterative development of architecture capabilities across workshops

Moreover, dependency constraints can be directly defined as declarative assertions attached to capabilities and managed dynamically instead of having to be hard-wired into phased processes.

That approach is to ensure two agile conditions critical for the development of architectural features:

  • Shared ownership: lest the whole enterprise be paralyzed by decision-making procedures, work units must be carried out under the sole responsibility of project teams.
  • Continuous delivery: architecture driven developments are by nature transverse but the delivery of building blocs cannot be put off by the decision of all parties concerned; instead it should be decoupled from integration.

Enterprise architecture projects could then be organized as a merry-go-round of capabilities-based work units to be set up, developed, and delivered according to needs and time-frames.

Time Frames

Enterprise architecture is about governance more than engineering. As such it has to ensure continuity and consistency between business objectives and strategies on one side, engineering resources and projects on the other side.

Assuming that capability-based work units will do the job for internal dependencies (application contents and engineering), the problem is to deal with external ones (business objectives and enterprise organization) without introducing phased processes. Beyond differences in monikers, such dependencies can generally be classified along three reasoned categories:

  • Operational: whatever can be observed and acted upon within a given envelope of assets and capabilities.
  • Tactical: whatever can be observed and acted upon by adjusting assets, resources and organization without altering the business plans and anticipations.
  • Strategic: decisions regarding assets, resources and organization contingent on anticipations regarding business environments.

The role of enterprise architects will then to manage the deployment of updated architecture capabilities according to their respective time-frames.

Portfolio Management

As noted before, EA workflows by nature can seldom be carried out in isolation as they are meant to deal with functional features across business domains. Instead, a portfolio of architecture (as opposed to development) work units should be managed according to their time-frame, the nature of their objective, and the kind of models to be used:

EA portfolio

  • Strategic features affect the concepts defining business objectives and processes. The corresponding business objects and processes are primarily defined with descriptive models; changes will have cascading effects for engineering and operations.
  • Tactical features affect the definition of artifacts, logical or physical. The corresponding engineering processes are primarily defined with prescriptive models; changes are to affect operational features but not the strategic ones.
  • Operational features affect the deployment of resources, logical or physical. The corresponding processes are primarily defined with predictive models derived from descriptive ones; changes are not meant to affect strategic or tactical features.

Architectural projects could then be managed as a dynamic backlog of self-contained work units continuously added (a) or delivered (b).

EA projects: a merry-go-round of work units.

That would bring together agile development processes and enterprise architecture.

Further Reading

Views, Models, & Architectures

May 27, 2017

Preamble

Views can take different meanings, from windows opening on specific data contexts (e.g DB relational theory), to assortments of diagrams dedicated to particular concerns (e.g UML).

Fortunato Depero tunnels

Deconstructing the Universe along Contexts and Concerns (Depero Fortunato)

Models for their part have also been understood as views, on DB contents as well as systems’ architecture and components, the difference being on the focus put on engineering. Due to their association with phased processes, models has been relegated to a back-burner by agile approaches; yet it may resurface in terms of granularity with model-based engineering frameworks.

Views & Architectures

As far as systems engineering is concerned, understandings of views usually refer to Philippe Kruchten’s “4+1” View Model of Software Architecture” :

  • Logical view: design of software artifacts.
  • Process view: captures the concurrency and synchronization aspects.
  • Physical view: describes the mapping(s) of software artifacts onto hardware.
  • Development view: describes the static organization of software artifacts in development environments.

A fifth is added for use cases describing the interactions between systems and business environments.

Whereas these views have been originally defined with regard to UML diagrams, they may stand on their own meanings and merits, and be assessed or amended as such.

Apart from labeling differences, there isn’t much to argue about use cases (for requirements), process (for operations), and physical (for deployment) views; each can be directly associated to well identified parts of systems engineering that are to be carried out independently of organizations, architectures or methods.

Logical and development views raise more questions because they imply a distinction between design and implementation. That implicit assumption induces two kinds of limitations:

  • They introduce a strong bias toward phased approaches, in contrast to agile development models that combine requirements, development and acceptance into iterations.
  • They classify development processes with regard to predefined activities, overlooking a more critical taxonomy based on objectives, architectures and life-cycles: user driven and short-term (applications ) vs data-based and long-term (business functions).

These flaws can be corrected if logical and development views are redefined respectively as functional and application views, the former targeting business objects and functions, the latter business logic and users’ interfaces.

Architecture based views

Architecture based views

That make views congruent with architecture levels and consequently with engineering workshops. More importantly, since workshops make possible the alignment of products with work units, they are a much better fit to model-based engineering and a shift from procedural to declarative paradigm.

Model-based Systems Engineering & Granularity

At least in theory, model-based systems engineering (MBSE) should free developers from one-fits-all procedural schemes and support iterative as well as declarative approaches. In practice that would require matching tasks with outcomes, which could be done if responsibilities on the former can be aligned with models granularity of the latter.

With coarse-grained phased schemes like MDA’s CIM/PIM/PSM (a), dependencies between tasks would have to be managed with regard to a significantly finer artifacts’ granularity.

Managing changes at architecture (a) or application (b) level.

Managing changes at architecture (a) or application (b) level.

For agile schemes, assuming conditions on shared ownership and continuous deliveries are met, projects would put locks on “models” at both ends (users’ stories and deliveries) of development cycles (b), with backlogs items defining engineering granularity.

Backlogs mechanism can be used to manage customized granularity and hierarchical dependencies across model layers

From the enterprise perspective it would be possible to unify the management of changes in architectures across layers and responsibilities: business concepts and organization, functional architecture, and systems capabilities:

EAGovern_ConcFoncCapa

Functional architecture as symbolic bridge between business needs and system capabilities.

From the engineering perspective it would be possible to unify the management of changes in artifacts at the appropriate level of granularity: static and explicit using milestones (phased), dynamic and implicit using backlogs (agile).

Fine grained model based frameworks could support phased as well as agile development solutions

Such a declarative repository would greatly enhance exchanges and integration across projects  and help to align heterogeneous processes independently of the methodologies used.

Further Reading

External Links

Beans must be Counted, one way And the other

May 2, 2017

Preamble

Conversations across software engineering forums sometimes reveal unexpected views, as it’s the case for the benefits of accountability.

Counting Paper Beans (Pieter Brueghel the Younger)

One would assume that competition would impel enterprises to scrutiny with regard to resources employed and product outcomes, pushing for the assessment of internal activities based on some agreed metrics. And yet, now and again, software development is viewed as a boutique occupation, if not an art pursuit, carried out by creative craftsmen for enlightened if demanding patrons; a vocation too distinctive to be gauged by common yardsticks.

Difficulties of Oversight

Setting apart creative delusions, the assessment of software development is effectively confronted with rational as well as practical obstacles.

To begin with rationality, and unlike traditional products, there is no market pricing mechanism that could match software development costs with customers’ value. As a consequence business stakeholders and systems engineers prefer to play safe and keep their respective assessments on the opposed banks of the customer/provider divide.

As for the practicality of assessments, the choice is between idiosyncratic approaches (e.g users’ points) and reasoned ones (essentially function points). The former ones being by nature specific and subject to changes in business opportunities, whereas the latter ones are being plagued by implementation plights that make them both costly and unreliable.

Yet, the diluting of IT systems in business environments is making that conundrum irrelevant: the fusing of business processes and supporting software is blanketing the discontinuities between business value and development costs.

Perils of Oversight

Given the digital integration between systems and business environments and the part played by software in production, marketing and operations, enterprises can no longer ignore the economics of software development.

As far as enterprises are concerned, economics use prices for two key purposes, external and internal.

With regard to their business environment, enterprises need metrics to price the resources they could buy and the products they could sell; their competitive edge fully depends on the thoroughness and accuracy of both.

With regard to their internal governance, enterprises need metrics to gauge the efficiency of their factors and the maturity of their processes, and allocate resources accordingly. That internal assessment is the basis of their versatility and plasticity:

  • Confronted to continuous, frequent, and often abrupt changes in business environments, enterprise must be able to adapt their activities without having to change its architectures. That cannot be achieved without timely and accurate assessments of the way their resources are put to use.
  • Conversely, enterprises may have to change their architectures without affecting their performances; that cannot be achieved without a comprehensive and accurate assessments of alternative options, organizational as well as technical.

To summarize, the spread and intricacy of software footprint over both sides of the crumbling fences between enterprise systems and business environments makes software economics a necessary component of enterprises governance, so a tally of software beans should not be an option.

Further Reading

Deep Blind Testing

March 21, 2017

Preamble

Tests are meant to ensure that nothing will go amiss. Assuming that expected hazards can be duly dealt with beforehand, the challenge is to guard against unexpected ones.

Unexpected Outcome (Ariel Schlesinger)

That would require the scripting of every possible outcomes in an unlimited range of unknown circumstances, and that’s where Deep Learning may help.

What to Look For

As Donald Rumsfeld once famously said, there are things that we know we don’t know, and things we don’t know we don’t know; hence the need of setting things apart depending on what can be known and how, and build the scripts accordingly:

  • Business requirements: tests can be designed with respect to explicit specifications; yet some room should also be left for changes in business circumstances.
  • Functional requirements: assuming business requirements are satisfied, the part played by supporting systems can be comprehensively tested with respect to well-defined boundaries and operations.
  • Quality of service: assuming business and functional requirements are satisfied, tests will have to check how human interfaces and resources are to cope with users behaviors and expectations which, by nature, cannot be fully anticipated.
  • Technical requirements: assuming business and functional requirements are satisfied as well as users’ expectations for service, deployment, maintenance, and operations are to be tested with regard to feasibility and costs.

Automated testing has to take into account these differences between scope and nature, from bounded and defined specifications to boundless, fuzzy and changing circumstances.

Automated Software Testing

Automated software testing encompasses two basic components: first the design of test cases (events, operations, and circumstances), then their scripted execution. Leading frameworks already integrate most of the latter together with the parts of the former targeting technical aspects like graphical user interfaces or system APIs. Artificial intelligence (AI) and machine learning (ML) have also been tried for automated test generation, yet with a scope limited by dependency on explicit knowledge, and consequently by the need of some “manual” teaching. That hurdle may be overcame by the deep learning ability to get direct (aka automated) access to implicit knowledge.

Reconnaissance: Known Knowns

Systems are designed artifacts, with the corollary that their components are fully defined and their behavior predictable. The design of technical test cases can therefore be derived from what is known of software and systems architectures, the former for test units, the latter for integration and acceptance tests. Deep learning could then mine recorded log-files in order to identify critical cases’ events and circumstances.

Exploration: Known Unknowns

Assuming that applications must be tested for use during their expected shelf life, some uncertainty has to be factored in for future business circumstances. Yet, assuming applications are designed to meet specific business objectives, such hypothetical circumstances should remain within known boundaries. In that context deep learning could be applied to exploration as well as policies:

  • Compared to technical test cases that can rely on the content of systems log-files, business and functional ones have to look outside and mine raw data from business environments.
  • In return, the relevancy of observations can be assessed with regard to business objectives, improved, and feed the policy module in charge of defining test cases.

Blind Errands: Unknown Unknowns

Even with functional and technical capabilities well-tested and secured, quality of service may remain contingent on human quirks: instinctive or erratic behaviors that could thwart the best designed handrails. On one hand, and due to their very nature, such hazards are not to be easily forestalled by reasoned test cases; but on the other hand they don’t take place in a void but within known functional circumstances. Given that porosity of functional and cognitive layers, the validity of functional test cases may be compromised by unfathomable cognitive associations, and that could open the door to unmanageable regression. Enter deep learning and its ability to extract knowledge from insignificance.

Compared to business and functional test cases, hazards are not directly related to business activities. As a consequence, the learning process cannot be guided by business and functional test cases but has to chart unpredictable human behaviors. As it happens, that kind of learning combining random simulation with automated reinforcement is what makes the specificity of deep learning.

From Non-regression to Self-improvement

As a conclusion, if non-regression is to be the cornerstone of quality management, test cases are to be set along clear swim-lanes: business logic (independently of systems), supporting systems functionalities (for shared applications), users interfaces (for non shared interactions). Then, since test cases are also run across swim-lanes, it opens the door to feedback, e.g unit test cases reassessed directly from business rules independently of systems functionalities, or functional test cases reassessed from users’ behaviors.

Considering that well-defined objectives, sound feedback mechanisms, and the availability of massive data from systems logs (internal) and business environment (external) are the main pillars of deep learning technologies, their combination in integrated frameworks could result in a qualitative leap toward self-improving automated test cases.

Further Reading

 

Focus: Analysis vs Design

January 4, 2017

Preamble

Definitions should never turn into wages of words as they should only be judged on their purpose and utility, with  such assessment best achieved by comparing and adjusting the meaning of neighboring concepts with regard to tasks at hand.

GChirico_prodigal-son

Analysis & Design as Duet (Giorgio de Chirico)

That approach can be applied to the terms “analysis” and “design” as used in systems engineering.

What: Logic & Engineering

Whatever the idiosyncrasies and fuzziness of business concerns and contexts, at the end of the day business and functional requirements of supporting systems will have to be coerced into the uncompromising logic of computers. Assuming that analysis and design are set along that path, they could be characterized accordingly.

As a matter of fact, a fact all too often ignored, a formal basis can be used to distinguish between analysis and design models, the former for the consolidation of requirements across business domains and enterprise organization, the latter for systems and software designs:

  • Business analysis models are descriptive (aka extensional); they try to put actual objects, events, and processes into categories.
  • System engineering models are prescriptive (aka intensional); they define what is expected of systems components and how to develop them.

Squaring Logic with Engineering

As a confirmation of its validity, that classification along the logic basis of models can be neatly crossed with engineering concerns:

  • Applications: engineering deals with the realization of business needs expressed as use cases or users’ stories. Engineering units are self-contained with specific life-spans, and may consequently be developed on a continuous basis.
  • Architectures: engineering deals with supporting assets at enterprise level. Engineering units are associated with shared functionalities without specific life-spans, with their development subject to some phasing constraints.

That taxonomy can be used to square the understanding of analysis, designs, and architectures.

Where: Business unit or Corporate

Reversing the perspective from content to context, the formal basis of analysis and design can also be crossed with their organizational framework:

  • Analysis is to be carried out locally within business units.
  • Designs are to be set both locally for applications, and at enterprise level for architectures.

Organizational dependencies will determine the roles, responsibilities, and time-frames associated with analysis and design.

Who: Analysts, Architects, Engineers

Contents and contexts are to determine the skills and responsibility for stakeholders, architects, analysts and engineers. On that account:

  • Analysis should be the shared responsibility of business and system analysts.
  • Designs would be solely under the authority of architects and engineers.

The possibility for agents to collaborate and share responsibility will determine the time-frames of analysis and design .

When: Continuous or Discrete

As far as project management is concerned, time is the crux of the matter: paraphrasing Einstein, the only reason for processes [time] is so that everything doesn’t happen at once. Hence the importance of characterizing analysis and design according to the nature of their time-scale:

  • At application level analysis and design can be carried out iteratively along a continuously time-scale.
  • At enterprise level the analysis of business objectives and the design of architectures will require milestones set along discrete time-scales.

The combination of organizational and timing constraints will determine analysis and design modus operandi.

How: Agile or Phased

Finally, the distinction between analysis and design will depend on the software engineering MO, as epitomized by the agile vs phased debate:

  • The agile development model combines analysis, design, and development into a single activity carried out iteratively. It is arguably the option of choice providing the two conditions about shared ownership and continuous delivery can be met.
  • Phased development models may rely on different arrangements but most will include a distinction between requirements analysis and software design.

That makes for an obvious conclusion: whether analysis and design are phased or carried out collaboratively, understanding their purpose and nature is a key success factor for systems and software engineering.

PS: Darwin vs Turing

As pointed out by Daniel Dennett (“From Bacteria to Bach, and Back“), the meaning of analysis and design can be neatly rooted in the theoretical foundations of evolution and computer science.

Darwin built his model of Natural Selection bottom up from the analysis of actual live beings. Roundly refuting the hypothesis of some “intelligent designer”, Darwin’s work epitomizes how ontologies built from observations (aka analysis models) can account for the origin, structure and behaviors of individuals.

Conversely, Turing’s thesis of computation is built top-down from established formal principles to software artifacts. In that case prior logical ontologies are applied to design models meant to realize intended capabilities.

Further Reading

iStar and the Requirements Conundrum

December 12, 2016

Synopsis

Whenever software engineering problems are looked at, the blame is generally put on requirements, with each side of the business/system divide holding the other responsible.

rockwell_runaway

iStar modeling put the focus on communication (N. Rockwell)

The iStar approach tries to tackle the problem with a conceptual language focused on interactions between business processes and supporting systems.

Dilemma

Conceptual approaches to requirements try to breach the dilemma between phased and agile development schemes: the former takes for granted that requirements can be fully and definitively set upfront; the latter takes a more pragmatic path and tries to reconcile business and system analysts through direct and continuous collaboration.

Setting apart frictions between specific methods, the benefits of agile principles and practices are now well-recognized, contingent on the limits of agile scope. Summarily, agile development is at its best when requirements capture and analysis can be weaved with development and tests. The question remains of what happens when requirements are to be dealt with separately.

The iStar’s answer shares with agile a focus on collaboration and doesn’t take side for business (e.g users’ stories) or systems (e.g use cases). Instead, iStar modeling language is meant to support a conceptual description of interactions between business processes and supporting systems in terms of actors’ goals and commitments, and the associated dependencies.

Actors & Goals

The defining aspect of the iStar modeling approach is to replace one-sided perspectives (business or system) by a systemic one focused on the interactions between agents. The interactive part of a requirement will therefore comprise three basic items:

  • A primary actor trigger an interaction in order to meet some goal; e.g a car owner want his car repaired.
  • Secondary actors may be involved during the ensuing exchanges: e.g body shop, appraiser, insurance company.
  • Functions to be performed: actual task; e.g appraise damages; qualification (soft goal), e.g fair appraisal; and resources, e.g premium payment.
Actors & dependencies

Actors & Dependencies

Dependencies Semantics

The factual description of interactions is both detailed and enriched by elements set within a broader scope:

  • Goal (strong) dependency: assertions about actual state of affairs: object, activity, or expectations.
  • Soft-goal dependency: assertions about expected outcomes.
  • Task dependency: organizational, functional, or technical constraints pertaining to the execution of activities.
  • Resource dependency: constraints or conditions on the availability of inputs, actual or symbolic.

It would be tempting to generalize the strong/soft distinction to dependencies as to make use of modal logic, strong dependencies associated with deontic rules, soft dependencies with alethic ones. That would .

iStar & Caminao

Since iStar modeling categories are directly aligned with UML Use Cases, they can easily mapped to core Caminao stereotypes for actors, objects, events, and activities.

Actors & dependencies

iStar with Caminao Stereotypes

Interestingly, the iStar strong/soft distinction could translate to the actual/symbolic one which constitute the conceptual backbone of the Caminao paradigm.

Assessment

From the business perspective, iStar must be credited with two critical tenets:

  • The focus on interactions between agents is essential for business and system analysts to collaborate. Such benefits appear clearly for the definition of primary and secondary roles (aka actors), intents (business) and capabilities (supporting environments).
  • The distinction between strong and soft goals, even if the logical basis remains unexploited.

Yet, the system perspective lacks a functional dimension, e.g:

  • Architecture levels (enterprise and organization, systems and functionalities, platforms and technologies) are not taken into consideration, nor the nature of capabilities, e.g strategic and operational.
  • The strong/soft dependencies distinction is not explicitly associated with systems capabilities.

On the whole these pros and cons reflect iStar’s declared intent on conceptual modeling; as a corollary these flaws mark also the limits of conceptual modeling when it is detached from the symbolic description of supporting systems functionalities.

Nonetheless, as illustrated by the research quoted below, iStar remains a sound basis for the specification of interactions between users and systems, either as use cases or users’ stories.

Further Reading

External Links

Business Agility vs Systems Entropy

November 28, 2016

Synopsis

As already noted, the seamless integration of business processes and IT systems may bring new relevancy to the OOAD (Observation, Orientation, Decision, Action) loop, a real-time decision-making paradigm originally developed by Colonel John Boyd for USAF fighter jets.

Agility: Orientation (Lazlo Moholo-Nagy)

Agility & Orientation (Lazlo Moholo-Nagy)

Of particular interest for today’s business operational decision-making is the orientation step, i.e the actual positioning of actors and the associated cognitive representations; the point being to use AI deep learning capabilities to surmise opponents plans and misdirect their anticipations. That new dimension and its focus on information brings back cybernetics as a tool for enterprise governance.

In the Loop: OOAD & Information Processing

Whatever the topic (engineering, business, or architecture), the concept of agility cannot be understood without defining some supporting context. For OODA that would include: territories (markets) for observations (data); maps for orientation (analytics); business objectives for decisions; and supporting systems for action.

OODA loop and its actual (red) and symbolic (blue) contexts.

OODA loop and its actual (red) and symbolic (blue) contexts.

One step further, contexts may be readily matched with systems description:

  • Business contexts (territories) for observations.
  • Models of business objects (maps) for orientation.
  • Business logic (objectives) for decisions.
  • Business processes (supporting systems) for action.
ccc

The OODA loop and System Perspectives

That provides a unified description of the different aspects of business agility, from the OODA loop and operations to architectures and engineering.

Architectures & Business Agility

Once the contexts are identified, agility in the OODA loop will depend on architecture consistency, plasticity, and versatility.

Architecture consistency (left) is supposed to be achieved by systems engineering out of the OODA loop:

  • Technical architecture: alignment of actual systems and territories (red) so that actions and observations can be kept congruent.
  • Software architecture: alignment of symbolic maps and objectives (blue) so that orientation and decisions can be continuously adjusted.

Functional architecture (right) is to bridge the gap between technical and software architectures and provides for operational coupling.

Business Agility: systems architectures and business operations

Business Agility: systems architectures and business operations

Operational coupling depends on functional architecture and is carried on within the OODA loop. The challenge is to change tack on-the-fly with minimum frictions between actual and symbolic contexts, i.e:

  • Discrepancies between business objects (maps and orientation) and business contexts (territories and observation).
  • Departure between business logic (objectives and decisions) and business processes (systems and actions)

When positive, operational coupling associates business agility with its architecture counterpart, namely plasticity and versatility; when negative, it suffers from frictions, or what cybernetics calls entropy.

Systems & Entropy

Taking a leaf from thermodynamics, cybernetics defines entropy as a measure of the (supposedly negative) variation in the value of the information supporting the control of viable systems.

With regard to corporate governance and operational decision-making, entropy arises from faults between environments and symbolic surrogates, either for objects (misleading orientations from actual observations) or activities (unforeseen consequences of decisions when carried out as actions).

So long as architectures and operations were set along different time-frames (e.g strategic and tactical), cybernetics were of limited relevancy. But the seamless integration of data analytics, operational decision-making, and IT supporting systems puts a new light on the role of entropy, as illustrated by Boyd’s OODA and its orientation component.

Orientation & Agility

While much has been written about how data analytics and operational decision-making can be neatly and easily fitted in the OODA paradigm, a particular attention is to be paid to orientation.

As noted before, the concept of Orientation comes with a twofold meaning, actual and symbolic:

  • Actual: the positioning of an agent with regard to external (e.g spacial) coordinates, possibly qualified with the agent’s abilities to observe, move, or act.
  • Symbolic: the positioning of an agent with regard to his own internal (e.g beliefs or aims) references, possibly mixed with the known or presumed orientation of other agents, opponents or associates.

That dual understanding underlines the importance of symbolic representations in getting competitive edges, either directly through accurate and up-to-date orientation, or indirectly by inducing opponents’ disorientation.

Agility vs Entropy

Competition in networked digital markets is carried out at enterprise gates, which puts the OODA loop at the nexus of information flows. As a corollary, what is at stake is not limited to immediate business gains but extends to corporate knowledge and enterprise governance; translated into cybernetics parlance, a competitive edge would depend on enterprise ability to export entropy, that is to decrease confusion and disorder inside, and increase it outside.

Working on that assumption, one should first characterize the flows of information to be considered:

  • Territories and observations: identification of business objects and events, collection and analysis of associated data.
  • Maps and orientations: structured and consistent description of business domains.
  • Objectives and decisions: structured and consistent description of business activities and rules.
  • Systems and actions: business processes and capabilities of supporting systems.
cccc

Static assessment of technical and software architectures for respectively observation and decision

Then, a static assessment of information flows would start with the standing of technical and software architecture with regard to competition:

  • Technical architecture: how the alignment of operations and resources facilitate actions and observations.
  • Software architecture: how the combined descriptions of business objects and logic facilitate orientation and decision.

A dynamic assessment would be carried out within the OODA loop and deal with the role of functional architecture in support of operational coupling:

  • How the mapping of territories’ identities and features help observation and orientation.
  • How decision-making and the realization of business objectives are supported by processes’ designs.
ccccc

Dynamic assessment of decision-making and the realization of business objectives’ as supported by processes’ designs.

Assuming a corporate cousin of  Maxwell’s demon with deep learning capabilities standing at the gates in its OODA loop, his job would be to analyze the flows and discover ways to decrease internal complexity (i.e enterprise representations) and increase external one (i.e competitors’ representations).

Further Readings

Business Agility & the OODA Loop

November 21, 2016

Preamble

The OOAD (Observation, Orientation, Decision, Action) loop is a real-time decision-making paradigm developed in the sixties by Colonel John Boyd from his experience as fighter pilot and military strategist.

(Moholy Nagy)

How to get inside opponent’s loop (Lazlo Moholy-Nagy)

The relevancy of OODA for today’s operational decision-making comes from the seamless integration of IT systems with business operations and the resulting merits of agile development processes.

Business: End of Discrete Time-Frames

Business governance was used to be phased: analyze the market, select opportunities, build capabilities, launch operations. No more. With the melting of the fences between actual and symbolic realms, periodic transitional events have lost most of their relevancy. Deprived of discrete and robust time-frames, the weaving of observed facts with business plans has to be managed on the fly. Success now comes from continuous readiness, quicker tempo, and the ability to operate inside adversaries’ time-scales, for defense (force competitor out of favorable position) as well as offense (get a competitive edge). Hence the reference to dogfights.

Dogfights & Agile Primacy

John Boyd train of thoughts started with the observation that, despite the apparent superiority of the soviet Mig 15 on US F-86 during the Korea war, US fighters stood their ground. From that factual observation it took Boyd’s comprehensive engineering work to demonstrate that as far as dogfights were concerned fast transients between maneuvers (aka agility) was more important than technical capabilities. Pushed up Pentagon’s reluctant ladders by Boyd’s sturdy determination, that conclusion have had wide-ranging consequences in the design of USAF fighters and pilots formation for the following generations. Its influence also spread to management, even if theories’ turnover is much faster there, and shelf-life much shorter.

Nowadays, with the accelerated integration of business processes with IT systems, agility is making a comeback from the software engineering corner. Reflecting business and IT convergence, principles like iterative development, just-in-time delivery, and lean processes, all epitomized by the agile software development model, are progressively mingling into business practices with strong resemblances to dogfights; and the resemblances are not only symbolic.

IT Systems & Business Competition

While some similarities between dogfights and business competition may seem metaphorical, one critical aspect is all too real, namely the increasing importance of supporting machines, IT systems or fighter jets.

Basically, IT systems, like fighters’ electronics, are tasked to observe environments, analyse changes in relation to position and objectives, and support decision-making. But today’s systems go further with two qualitative leaps:

  • The seamless integration of physical and symbolic flows let systems manage some overlapping between supporting decisions and carrying out actions.
  • Due to their artificial intelligence capabilities, systems can learn on-the-job and improve their performances in real-time feedback loops.

When combined, these two trends have drastic impact on the way machines can support human activities in real-time competitive situations. More to the point, they bring new light on business agility.

Business Agility

As illustrated by the radical transformation of fighter cockpits, the merging of analog and digital flows leaves little room for human mediation: data must be processed into information and presented instantly along two critical dimensions, one for decision-making, the other for information life-cycle:

  • Man/Machine interfaces have to materialize the merging of actual and symbolic realms as to support just-in-time decision-making.
  • The replacement of phased selected updates of environment data by continuous changes in raw and massive data means that the status of information has to be incorporated with the information itself, yet without impairing decision-making.

Beyond obvious differences between dogfights and business competition, that double exigence is to characterize business agility:

  1. Instant understanding of changes in business opportunities (Observation) .
  2. Simultaneous assessment of the reliability and shelf-life of pertaining information with regard to current positions and operations (Orientation).
  3. Weighting of options with regard to enterprise capabilities and broader objectives (Decision).
  4. Carrying out of decisions within the relevant time-span (Action).

That understanding of business agility is to be compared with its development and architecture cousins. Yet it doesn’t seem to add much to data analytics and operational decision-making. That is until the concept of orientation is reassessed.

Agility & Orientation: Task vs Tack

To begin with basics, the concept of Orientation comes with a twofold meaning, actual and symbolic:

  • Actual: a position with regard to external (e.g spacial) coordinates, possibly qualified with abilities to observe, move, or act.
  • Symbolic: a position with regard to internal (e.g beliefs or aims) references, possibly mixed with known or presumed orientation of other agents, opponents or associates.

When business is considered, data analytics is supposed to deal comprehensively and accurately with markets’ actual orientations. But the symbolic facet is left largely unexplored.

Boyd’s contribution is to bring together both aspects and combine them into actual practice, namely how to foretell the tack of your opponents from their actual tracks as well as their surmised plans, while fooling them about your own moves, actual or planned.

Such ambitions once out of reach, can now be fulfilled due to the combination of big data, artificial intelligence, and the exponential growth on computing power.

Further Readings