Five questions worth answering before SAP go‑live

The question a green dashboard answers

A green test dashboard is a real achievement. Someone wrote the cases, someone executed them, and most of them behaved as expected. None of that deserves to be dismissed.

It answers one question well: how much of the planned work is complete. It cannot answer the question underneath it, which is whether the plan was the right size.

There is a documented illustration of the gap. When MillerCoors sued HCL over its SAP implementation, the complaint records that “final testing of the implementation indicated there were 8 critical severity defects and 47 defects of high severity”. The programme went live on 2 November 2015 anyway, and thousands of further defects surfaced in post-launch support. The testing had not failed to find anything. The finding was reported, read, and outranked.

That is the pattern worth guarding against. Difficult go-live decisions are rarely made in ignorance. They are made with a number in front of them that was never designed to carry the weight being put on it.

Five questions go somewhere a pass percentage cannot. All five are answerable by any well-run programme, and none of them require an outside party to ask.

01 — Can every critical business requirement be traced to a test that was actually executed, with evidence of the result?

Not whether a test case exists. Whether the programme can connect the business requirement to the executed test, the result, the defect history and the evidence behind it — in that order, without a reconstruction exercise.

Where test management is centralised this is a property of the system rather than an act of archaeology. SAP Cloud ALM, for example, links functional tests to solution processes, requirements and user stories, rates each test action rather than each test case, and requires comments carrying evidence and the test data used. If your repository cannot answer at that level, the question is not whether traceability exists but where it was allowed to stop.

The reason this matters is visible in the National Grid complaint against Wipro. When Deloitte re-ran the original test scripts after go-live, paragraph 85 records that it “uncovered twice the number of defects in the areas of procurement and supply chain”, and found “an $8,000 receipt that was recorded in the system as an $880 million receipt”. The scripts had run. They had passed. What they had not done was check the result against what the business expected the result to be.

A practical version of this question: pick one critical requirement and follow it end to end in front of the room. Ten minutes tells you a great deal about how the other several thousand are built.

02 — Did we test the way the business actually operates — variants, interfaces, scheduled processing, and with real roles?

The distinction that exposes most coverage gaps is vertical against horizontal. Vertical testing validates within a module or domain. Horizontal end-to-end testing, as TMAP defines it, covers the complete business process including legacy systems, third parties, overarching modules and authorisations — and is run with full role-based authorisations in place, which is precisely the part most programmes defer.

Three things tend to be missing from a scope that looks complete on paper.

The variants. A happy-path order can pass while the materially different ways that order actually runs remain untested. The productive conversation is with the process owner rather than the specification: what are the versions of this that behave differently? The answer is usually four to eight, and it usually surprises the test team.

The interface legs. A realistic chain runs from a Fiori screen through a transformation layer, into an IDoc, into a posting, out to a partner, and back as an acknowledgement. Testing the screen validates none of the rest of it. A green transaction is not proof that the business event completed.

The roles. Authorisation defects concentrate in the first week after go-live for a structural reason: roles are built around job functions, not tasks. A user who asks for read-only display of a purchase requisition is commonly given a role that also carries requisition creation, purchase order creation and change, release, vendor master maintenance, invoice verification and the payment run — three procure-to-pay segregation-of-duties conflicts nobody requested. Testing with administrator access finds none of this.

Worth knowing before you rely on after-the-fact diagnosis: SU53 displays a maximum of 100 failed authorisation checks per user and only for the last three hours, and the older ST01 trace requires the trace and the traced transaction to be running on the same application server. In a load-balanced production landscape that is the difference between reproducing an authorisation defect in week one and not.

03 — Did we test with data that behaves like the data the business will actually meet?

Successful execution against convenient test data does not prove the same process behaves correctly across realistic combinations of master, transactional and organisational data. Test environments tend to hold clean, recently created records, which is exactly the data least likely to reveal a problem.

The technical form of this is stricter than most test data strategies admit: production-like datasets with masked personal data, while maintaining referential integrity across customers, materials, plants and pricing conditions. Masking that quietly breaks the customer–material–plant–pricing chain narrows what the test exercises without changing what the test reports.

And a scenario that could not be constructed because the data did not exist is not covered, whatever its status says. That distinction is worth making explicit in the pack rather than leaving it inside a tester’s memory.

The same logic applies to how you decide scope in the first place. SAP’s own guidance on identifying which custom code is genuinely used is to collect production usage for six to eighteen months, and that the window “should include at least one year-end closing”. The transferable principle is broader than custom code: any scope derived from less than a full annual cycle of observed behaviour is structurally blind to annual processes.

04 — Has the cutover itself been rehearsed, and do we know when the rollback deadline actually falls?

SAP Activate scopes cutover narrowly and deliberately: “a set of activities which are performed on the last weekend before go-live”. Treating that weekend as the readiness event, rather than as the culmination of one, is a recognisable failure shape.

A cutover rehearsal — dress rehearsal, mock cutover, dry run, the names vary — exists to measure five things: timing, dependencies, sequence, correctness and team readiness. Only one of those five is correctness. Most programmes rehearse for correctness and discover the other four in production.

The second half of the question is the one most often left implicit. SAP’s own cutover guidance places the go/no-go decision point so that there is “sufficient time for a restore / recovery back to the legacy systems”. Which means the latest moment you can responsibly decide is set by the rollback window, not by when the defect list is finished. If nobody in the room can state that time, the decision is being made without knowing whether it is still reversible.

Two supporting questions belong here. Has the final data migration been rehearsed against the production timeline, rather than against a comfortable one? And has critical data been validated and signed off by business data owners rather than by the technical team that loaded it? Reconciliation across systems, not load success, is the real gate. Asda’s own major incident reporting during its systems separation logged mismatches of ten and eleven million items in successive months between its warehouse management system and SAP closing stock reports — a discrepancy visible only to somebody reconciling the two, and invisible to either system on its own.

05 — If we approved go-live today, could we show the evidence — and who is accountable for saying no?

Go-live confidence should rest on traceability, execution evidence, defect status and coverage of the critical business processes. A practical test of whether it does: pick any result in the pack at random and ask how long it would take to produce what sits behind it.

Two things are worth settling before the meeting rather than during it.

Severity, in business language. SAP’s own ladder, from SAP Note 67739, is Very High, High, Medium and Low, and every step is defined by effect on normal business processes rather than by technical component. Very High means core business processes are seriously affected with no workaround. A programme using looser definitions is deviating from the vendor’s own standard, and hypercare loses control quickly when every ticket is labelled critical.

Who owns the no. The most uncomfortable question in the litigated cases is not whether anyone knew. It is who was accountable for saying so. Zimmer Biomet’s complaint against Deloitte alleges it was advised to proceed with cutover without showstopper risks being flagged. Whatever the merits of that particular case, the governance question it raises is a fair one to ask of your own programme before the fact: if we are not ready, whose job is it to tell this committee, and does that person have anything to lose by doing it?

Alongside the pack, the most useful artefact is often the shortest: a list of what is knowingly not ready, what will be done about it, and who accepted it. Programmes are rarely damaged by known gaps. They are damaged by gaps that were known to somebody and not to the person deciding.

Five questions. One purpose: know what was actually tested, and what was actually decided, before calling anything ready.

Two dates worth putting in the plan

SAP’s own GoingLive Check is not optional in any meaningful sense and it carries hard timing. The Analysis Session runs “a minimum four weeks before your start of production”. The Verification Session runs approximately four to six weeks after. Those are two fixed points a steering committee can hold a plan against, and they are frequently discovered late.

Using these with your own team

These work perfectly well without us. Ask them in a normal governance meeting, give the team notice, and treat gaps as information rather than as failures. A gap found six weeks out is a planning input. The same gap found in week one after go-live is an incident, and it arrives with an audience.