For automation for quality (a superset of/fix for what is often called “test automation”) MetaAutomation provides trustworthy detail and transparency on what the SUT is doing and how fast to all roles that care about quality, including the executive suite.
Techniques that use keywords (e.g., Cucumber/Gherkin) are a step in the right direction but they don’t go nearly far enough:
- Keywords must be implemented, and the implementation is hidden to anyone that is not in the code. The implementation can change, too; I have done this myself, just to make a keyword reusable.
- The runtime behavior behind the keywords is even more hidden; one must step through with a debugger to see what’s really happening. Cucumber claims to be self-documenting, but really it’s only the keywords that document themselves.
- Cucumber claims to be self-documenting, but really, it is only the keywords that document themselves. Whether the keywords do anything at all is hidden in source code of the keyword implementation.
- Complexity gets in the way of transparency, because a parser is needed (Cucumber), a special language (Gherkin), and keywords and verbs, each of which must be implemented separately.
The Hierarchical Steps pattern is a natural pattern that we all use in our daily lives, and when applied to automation used to measure and report quality, it leads to results much better than Cucumber/Gherkin for self-documentation, complete and discoverable detail, and trustworthiness as well:
- The entire procedure of driving the SUT forms an ordered tree hierarchy, with each step in context of all the other steps, including a parent, children, and peer steps.
- Each step documents itself at runtime with a name that reflects its business-facing function
- At the leaf nodes of the hierarchy, the atomic steps – each with no more than one point of failure -- that drive the SUT are each self-documenting with a name that reflects the business-facing function of that step.
Best of all, for the executives,
- Any person in any role that is interested in what the SUT is doing, and how fast, can look at any check on the intranet and drill down as desired from the top-level business-facing description of the whole check (at the root node) to the atomic steps driving the SUT (at the leaf nodes).
Normally, you wouldn’t expect the executives to spend their time doing this, but the fact that the option is available to them is an indication of the transparency of the system available to the entire team.
The new trustworthiness comes because the hierarchy that represents the check’s runtime behavior extends all the way down to the atomic steps driving the SUT. Those small steps will rarely change, and if they do, the business-facing name of the step can change as well to reflect the change at runtime.
Traditionally, if someone outside of the QA role wanted to know what a check did exactly, one would ask someone in the QA role. It would be word-of-mouth, assuming one knew the correct person to ask.
MetaAutomation shows how to digitize this process: make it more explicit, more detailed, more accurate, more easily discoverable, and best of all – measurable! Now it can be managed better, all the way up the report chain to the executive suite. The productivity of the QA team is more clear and more understandable.
Also, exactly what the SUT does and how fast it does it (as driven by the QA team) is very detailed, clear and available to analysis, which helps with SOX compliance and similar processes, including valuing a company’s software assets.
More information on SUT behavior and timing, …
More trustworthy information.
More accessible information.
More structured information available for analysis and presentation.
Quality management with BDD/Cucumber/Gherkin makes a small step in the right direction, but MetaAutomation shows how to go all the way towards knowledge, trustworthiness, clarity, and accessibility into how the SUT is driven and how it responds. Communication about and knowledge of the software product is much better then possible before.