From requirement to business process: a better way to define SAP test coverage

The question

Ask a programme how testing is going and you will usually be told a number of test cases, a percentage executed, and a pass rate. All three are real. None of them answers the question a business sponsor is actually asking, which is whether the things the business needs to do will work.

The gap is structural, not cultural. A test-case count measures the work the test team has created. Coverage is a claim about the enterprise. Those are different objects, and no amount of the first automatically produces the second.

What we see in programmes

Two programmes can have identical test-case totals and completely different levels of real coverage.

The first has four thousand test cases derived from a functional specification library, where roughly one test case exists per specification, each one exercising the main path. Coverage looks excellent. The specifications are covered. Whether the business is covered is a separate and unanswered question.

The second has fifteen hundred test cases, but they are organised by business process, and each critical process carries its main path plus its material variants plus its common exceptions. Fewer cases, substantially more coverage of the thing that matters.

The first programme will report better numbers. The second will have a better go-live.

The reason this happens is that test-case count is easy to measure and coverage is not. Nobody sets out to build the first programme. It is what you get when the only available metric is a total.

A better way to think about it

Coverage becomes meaningful when it is expressed as a chain, and the chain has five links.

Requirement. What the business asked for, in the business’s words.

Process. Where that requirement lives — which end-to-end business process carries it. Requirements do not exist in isolation; they modify processes.

Variant. The materially different ways that process runs. Different approval thresholds. Different document types. Different plants, company codes, vendor categories, material types. A variant is not a nice-to-have; frequently it is the requirement.

Scenario. The test that exercises a given variant of a given process, end to end, with data that can actually support it.

Evidence. What was produced when it ran, retained in a form somebody else can examine.

When those five links are connected, a coverage statement becomes something you can defend: this requirement is represented in these processes, across these variants, exercised by these scenarios, with this evidence. When any link is missing, you have located a specific gap rather than a vague unease.

Coverage is not a percentage. It is a chain, and it is only as strong as the weakest link in it.

What good looks like

Traceability built during design, not reconstructed before go-live. This is the single largest cost difference we see. Linking a test case to a requirement while both are being written costs minutes. Reconstructing the same link eighteen months later, from documents, from memory and from people who have moved on, costs weeks and produces a worse answer. SAP Cloud ALM supports assigning test cases to requirements and user stories and linking functional tests to solution processes precisely so this is a property of the system rather than a retrospective exercise.

Variants enumerated deliberately, and descoped deliberately. Not every variant deserves a test. A programme that has decided, in writing, not to test a rare variant is in a much better position than one that never noticed the variant existed. The first has accepted a risk. The second has an unknown.

Coverage reported per process, with a denominator you can argue with. “Two hundred and eighty of four hundred and twelve in-scope business processes are covered” invites a useful conversation about the remaining hundred and thirty-two. “Ninety-four per cent complete” does not invite anything.

Test data treated as part of coverage. A scenario that cannot be constructed because the data does not exist is not covered, whatever its status says. Testers are resourceful and will test the variant the data permits, in good faith, and record a pass.

Questions leadership should ask

What is our coverage a percentage of? If the denominator is a list of test cases, you are being told about effort.

Can we see one critical requirement traced all the way to evidence? Pick one. The exercise takes ten minutes and tells you a great deal.

Which variants did we decide not to test, and who decided?

Which scenarios are blocked by test data rather than by defects?

If a requirement changed next week, would we know which tests to re-run? This is the practical test of whether traceability exists or is merely asserted.