Before you approve an SAP go‑live, what evidence should be on the table?

The question

Somewhere near the end of every SAP programme there is a meeting where a small number of people decide whether the business goes live. The material in front of them is usually a status pack: execution percentages, a defect count by severity, a RAG rating per workstream, and a recommendation.

That pack answers one question well — how much of the planned work has been completed. It answers a different question poorly, and that second question is the one the meeting is actually for: if we go live on Monday, what could stop the business running, and how would we know?

These are not the same question, and the gap between them is where most go-live surprises live.

What we see in programmes

A high pass rate is a genuinely good sign. It means the team built test cases, executed them, and most of them behaved as expected. Nobody should dismiss that.

But a pass rate is a ratio, and a ratio hides its own denominator. Ninety-four per cent of what? Of the scenarios someone wrote down during design workshops. Which raises the only interesting question about the number: what did the business need that nobody wrote down?

Three patterns recur often enough to be worth naming.

The scope was set from memory rather than usage. Test scenarios get built in workshops, from what people remember doing. Real transaction volumes tell a different story — longer tails, more variants, more exceptions, more custom programs still running quietly at period end.

The standard path was tested and the variants were not. A purchase order gets approved. Fine. But the approval that goes to a substitute, the one above a second threshold, the one that gets rejected and resubmitted — these are separate behaviours, and they are where design intent and configuration most often part company.

The evidence exists but cannot be produced. Somebody tested it. Everyone is sure. There is no screenshot, no result record, no link back to a requirement, and the person who ran it has rolled off the programme. This is not dishonesty; it is what happens when execution is measured on completion and nobody is measured on evidence.

A better way to think about it

The useful reframe is to stop asking the programme to prove it finished, and start asking it to substantiate a small number of claims about the business.

For each business-critical process in scope, leadership should be able to follow one line without leaving the room:

Business requirement → process and design → test scenario → execution result → evidence.

Where that line is complete, there is a basis for confidence. Where it breaks — and it will break in places, on every programme — the break itself is the finding. Not a failure, not blame: a known, located, describable gap that leadership can decide what to do about.

This is a smaller ask than it sounds. Nobody is proposing that every test case in a programme carries a full audit trail. The claim only needs to hold for the processes the business cannot run without, which on most programmes is a few dozen, not a few thousand.

The percentage tells you how much was done. The traceability tells you what it was done to. Only the second one supports a decision.

What good looks like

A pack that supports a go-live decision tends to have five things in it.

Coverage stated against business processes, not test counts. Which critical processes are covered, which are partially covered, which are not covered, and why. A number like “280 of 412 in-scope processes” is more useful than any pass rate, because it can be argued with.

Execution stated honestly, including what is blocked. Passed, failed, blocked and not started are four different states. Collapsing the last three into “in progress” is where optimism enters the pack.

Defects expressed as business impact. A severity rating is an internal engineering judgement. What leadership needs is the sentence underneath it: which process is affected, what happens to it, and whether there is a workaround someone has actually tried.

Dependencies that are still open. Environment availability, test data, interface partners, third-party readiness, cutover rehearsal outcomes. These are frequently the real constraint and they are frequently outside the testing report entirely.

Evidence that can be produced on request. Not attached to the pack — nobody reads it — but retrievable. The test of a good pack is whether someone could ask “show me” about any line in it and get an answer in a few minutes.

SAP’s own tooling assumes this shape. SAP Cloud ALM links functional test cases to solution processes, requirements and user stories precisely so that traceability is a property of the system rather than something reconstructed by hand at the end. Programmes that use that structure from the start find this conversation much easier than programmes that try to assemble it in the last three weeks.

Questions leadership should ask

None of these are gotchas. All of them are answerable, and a well-run programme will enjoy answering them.

Which business processes are we saying we are confident in, and how many are there? If the answer is a percentage rather than a list, ask for the list.

Where did the test scope come from? Workshops, prior programmes, production usage data, or some mix. All are legitimate. The answer tells you where the blind spots are likely to be.

Which critical processes have variants we did not test, and was that a decision or an oversight? A deliberate decision to descope a rare variant is fine, and should be visible.

What is still open that is not a defect? Blocked scenarios, missing test data, an interface partner who has not confirmed. These rarely appear in a defect count.

If we asked for the evidence behind any one of these results, how long would it take? The answer to this question tells you more about programme quality than the pass rate does.

The point of the exercise is not to delay the date. Dates move for commercial reasons, rarely for testing ones, and a good assessment respects that. The point is that the people accepting the risk should be able to describe the risk they are accepting.