Ergonomy, Fingertips Errors & Automated Testing


When interacting with systems, users do things they aren’t supposed to do and walk along irrelevant, even unthinkable, paths that can put tests designers at a loss. This apparent chink between users’ conscious self and their fingertips can be explained by the way humans assess situations and make decisions. Curtailing it is the aim of ergonomics.

Errors at fingerstips (Rembrandt)

Anatomy of Errors: from brain to fingers (Rembrandt)

Taking a leaf from A. Tversky and D. Kahneman (who received the 2002 Nobel Price in Economics), decision-making relies on two cognitive mechanisms:

  1. The first one “operates automatically and quickly, with little or no effort and no sense of voluntary control”. It’s put in use when actual situations must be assessed and decisions taken rapidly if not instantly.
  2. The second one “allocates attention to the effortful mental activities that demand it, including complex computations”. It’s put in use when situations can be assessed with regard to past experience in order to support informed decisions making.

That distinction can be directly applied to users’ behaviors interacting with systems:

  1. Intuitive behavior: decisions are taken on the basis of the visual context and options as presented by users interfaces before taking into account underlying business contents and logic.
  2. Rational behavior: decisions are taken on the basis of business contents and logic disregarding supporting systems interfaces.

Set in context, that distinction can be put in parallel (but not confused) with the one between domain and functional requirements, the former dealing rationally with business objects and logic, the latter putting the former to use through interactions with supporting systems.

Functional requirements describe the part played by supporting systems

Functional requirements describe the part played by supporting systems

Assuming that business logic should not be contingent on supporting systems interfaces, the best option would be to test its implementation independently of users interactions; moreover, tests targeting intuitive behaviors (i.e not directly based on domain specific contents), could then be generated automatically.

Looking for Errors

Given that testing is meant to find flaws in deliverables, tests are certainly more effective when designers know what they are looking for.

For that purpose phased approaches rely on sequences of differentiated tests dealing successively with programming (unit tests), functional requirements (integration tests), and business requirements (acceptance tests).  The unfortunate downside of those policies is that the most wide-ranging flaws are the last to be looked for, with the risk of being found after cascading and costly consequences for functionalities and programs.

Phased and Iterative approaches to tests

Phased and Iterative approaches to tests

Conversely, agile approaches follow iterative policies, with each development cycle combining the definition, programming, and tests of software products. When properly implemented those policies significantly improve the early detection and correction of errors whatever their origin. Yet, since there is no explicit management of intermediate outcomes, it’s difficult to differentiate the tests according the kind of errors to look for, e.g faulty business rules implementation or flawed user interface.

Architecture driven approaches may provide an answer, with requirements unambiguously sorted out depending on their architectural footprint: business contents or system functionalities. As a corollary, tests could also be designed along the same lines, targeting business rationale or human behavior.

Errors in Mirrors

Acceptance tests being performed with regard to requirements, they should be designed along requirements taxonomy, respectively for business logic, users’ interactions, quality of services, and components implementation. Being aligned on requirements, those tests can be neatly defined with regard to closed sets of specifications, functional or otherwise.

Functional tests have to expect the unexpected

Functional tests have to expect the unexpected

But that’s not the case for users’ interactions because people behaviors are not fully predictable; hence, while tests can be systematically designed with regard to the set of users’ actions framed by business and functional requirements, there is no way to comprehensively and unambiguously check for all and every possible behavioral contingencies. That will make for three levels of functional tests:

  1. Implementation of business logic: tests should be designed directly from business requirements, independently of interactions with users.
  2. Implementation of scenarii: while interactions are defined in reference to business logic, their validation should focus on the presentation of contents and dialog control.
  3. Users exceptions: in addition to inputs validity, already checked with business logic, and users’ actions, supposedly secured by interaction scenarii, it is necessary to check that unexpected behaviors have been properly considered .
How to check that unexpected behaviors have been properly considered ?

How to check that unexpected behaviors have been properly considered ?

In other words, functional tests will have to look simultaneously for errors in software (defined with regard to a finite set of requirements), and for users’ mistakes (set in an open range of behaviors). As if tests designers were to mirror users errors in order to look for software ones. So, assuming that errors in business logic and interactions have been considered, what should still be checked, and how ?

Fingertips Errors

When faced with choices, users bank on mental maps combining graphical and business layers, with the implicit assumption that maps’ contexts and concerns are kept up to date. Those maps combine three communication mechanisms:

  • Languages, natural or specific, use syntax and semantics to define business contents, logic, and operations.
  • Icons use similarity for the visual representation of business operations or functional primitives (e.g create, delete, etc).
  • Signals use proximity to draw users’ attention to predefined events (e.g sounds for operations completion or incoming emails).

While language-based interactions are supposedly fully covered by business and functional tests, icons and signals make room for “fingertips” reactions which cannot be directly framed within business logic or functional scenarii, and therefore cannot be comprehensively checked for erroneous behaviors.

Icons and signal based communication can trigger unexpected behaviors.

Icons and signal based communication can trigger unexpected behaviors.

Yet, if instinctive reactions preclude rational considerations, decisions may be swayed by analogies and associations before being informed by the relevant business contents. To prevent that risk, test scenarii built on business logic and functional interactions should be extended in order to take into account the intuitive aspects of users’ behaviors.

Mental Maps & Automated Tests

As noted above, mental maps are built on three layers, one deep (language semantics) and two shallow (icons and signals). While the shallow layers are supposed to reference the deep one, icons and signals may induce instinctive behaviors independently of the referenced business logic. Those behaviors can be triggered by two kinds of mechanisms:

  • Analogy: users will look for similarities and familiar configurations.
  • Proximity: users will look for continuity with regard to scope and operations.

Clearly, lapses in such behaviors will normally escape tests designed for business and functional requirements; yet, by being driven by self-contained mechanisms, intuitive behaviors can be checked independently of references to business contents. And that may open the door to automated tests generation.

With regard to similarities, tests should look for possible confusion between:

  • Objects with common representation but specific features (inheritance).
  • Operations with shared semantics but different scope (polymorphism).
  • Sequences with shared operations but different timing .

With regard to proximity, tests should look for possible confusion between:

  • Objects and their parts, or between their parts (structural proximity).
  • Operations usually associated into the same activity (functional proximity).
  • Operations usually executed successively (chronological proximity).

Scripts for such tests could be generated through pattern-matching and run by wizard applications.

Further Reading

External Links


One Response to “Ergonomy, Fingertips Errors & Automated Testing”

  1. Max J. Pucher Says:

    Let me point out that the speed with which decisions are taken by humans does not change that they are purely made emotionally. That is close to a scientific fact as we can know. People with emotional deficiency syndrome can no longer make decisions, while they can contemplate about them at length. (A. Damasio

    All decisions come in a snap through emotional pattern matching but we have been trained to question those snap decisions and apply reason and logic. That is partially a huge fallacy, but neither do we have the knowledge to apply logic, nor do we have the ability to influence the world through our actions so accurately so that the logic we assume will apply will actually be relevant. (Gigerenzer

    Even once we reworked a decision it is still an emotional one. We replace the snap decision with one that is driven by the fear of being wrong or causing someone to get hurt.

    That is partially one of the main problems of people and management today that much of their decision making is driven by fear only. We are constantly badgered with information about all the bad things that happen to people who decide wrongly. It ahs become a neurosis so that even young children already need tranquilizers to make it through the day.

    If any decision you should take causes you fear then my suggestion is to not take that decision at all. Bi and predictive analytics do not help at all because their models are still made by humans and might be wrong. It is an illusion to think that mathermatical rigor turns fiction into fact. There is no such thing.

    Yes, user interfaces and application testing must consider that humans have a certain way to approach new things they see and the best thing that you can give them is familiarity. But you must also give them understanding. Help them to feel good about the decision that they need to take. That is a lot more important than a cool GUI.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: