TestmarshalMarshal the evidence.

Writing a test sign-off report your release board will accept

Most sign-off reports get argued with because the numbers can't be checked. Here's how Azure Test Plans actually counts things, why the same test plan produces three different pass rates, and what belongs in a GO / NO-GO recommendation that survives contact with an auditor.

Verified against Microsoft documentation, August 2026 · About 12 minutes

A sign-off report has one job: let someone who was not in the room decide whether to release. It fails when the reader cannot check the number, and it fails badly when they check it and get a different answer.

Both failures usually come from the same root cause, and it isn't dishonesty. It's that Azure Test Plans can legitimately produce several different pass rates from the same data, and almost nobody says which one they used.

Test case, test point, test result — the three things people confuse

Start here, because every downstream problem is a version of this one.

"Test cases by themselves aren't executable. When you add a test case to a test suite, you generate test points. A test point is a unique combination of a test case, test suite, configuration, and tester."

Microsoft's example: a test case called Test sign in functionality with two configurations — Edge and Chrome — generates two test points. Each runs independently. Each produces its own results. The Execute tab shows the latest result per point.

Why this produces three different pass rates

Take one test plan at the end of a cycle. All three of these are defensible, and they will not agree.

DenominatorWhat it answersHow it distorts
Passed test points ÷ total test points"Are we done testing?"Multiplies with configurations. Two browsers doubles your denominator overnight
Passed results ÷ total results"How much of our execution effort succeeded?"A flaky test re-run five times contributes five rows. Fail-then-pass shows as both
Distinct cases passed ÷ distinct cases"How much functionality is verified?"A case that passed on Chrome and failed on Edge has no honest single value

And Azure DevOps itself switches between these depending on which chart you build. From the Test Plans FAQ, verbatim:

"For test case charts, if a test case gets added to multiple test suites in a plan, then the test only gets counted once. For test result charts, each instance of a test that is run is counted for each of the test suites separately."

Same plan. Two charts. Different totals. Both correct. If your report shows one and your release manager opens the other, you will spend the meeting explaining arithmetic instead of discussing risk.

The fix is one sentence

State the denominator beside the number. "94.2% — 1,551 of 1,646 test points in the active suite as of 09:00, excluding 41 deferred to 24.4." Nobody argues with a number that shows its working. This costs you one line and removes an entire category of meeting.

The Progress Report's limitations, in Microsoft's own words

The built-in Progress Report is the obvious source for a sign-off number. It is also documented with an unusually candid list of caveats, and several of them will change your figure.

None of these make the Progress Report useless. They make it something you have to explain, which is not the same as a sign-off report.

The question you cannot answer retrospectively

This is the one that turns into an audit finding, and it deserves to be understood before you need it.

You cannot ask Azure Test Plans what was in a test suite on a past date. Microsoft states it directly in the Progress Report documentation:

"Report data corresponds to the current test suite hierarchy in the selected test plans. Hierarchy history isn't stored."

The consequences compound:

The nearest thing to a mechanism is the Analytics OData entity TestPointHistorySnapshot, which holds one row per test point per calendar day with TestSuiteId, TestCaseId, TestConfigurationId and the last outcome for that day. Filtering to a past DateSK gets you close. Caveats: all Test entity sets are preview-only (v3.0-preview / v4.0-preview — there is no GA OData surface for Test Plans data), snapshot entities are expensive enough that Microsoft explicitly tells you to aggregate rather than pull rows, and the documentation does not say whether rows survive the deletion of a test point. Do not build a control on an assumption there.

The practical answer: capture forward. If you have an evidence obligation, snapshot the suite and test-point list on a schedule and store it yourself. Nothing you do after the fact will reconstruct it.

The retention landmine

One more thing that moves numbers on its own. When retention policy deletes test runs and results, the related test points' outcome is reset to Active — and Microsoft notes the visible effect: "progress reports may reflect decreased run rates."

So a cycle you signed off at 94% can, a year later, report a lower run rate. Nothing changed. Retention ran. If your sign-off report is the artefact of record, it needs to be a file you keep, not a link to a live report that will quietly disagree with itself later.

What belongs in the report

A structure that survives both a release board and an auditor. Nine sections, none of them long.

1. The verdict, first

GO, NO-GO, or GO WITH CONDITIONS, in the first line. Not on page four. If you have conditions, number them and name an owner for each. A release board that has to read three pages to find your recommendation will form its own before it gets there.

2. Exit criteria, evaluated

Each criterion as a row: the criterion, the threshold, the actual, and met/not met. Criteria that cannot be evaluated mechanically — "quality is acceptable" — are not exit criteria, they are opinions, and they will be argued with. Fix them before the cycle, not during sign-off.

3. The numbers, with denominators

Every figure states its population and its as-at time. Say which unit you counted — test points, results, or distinct cases. If you're using the Progress Report, note the ~15 minute lag.

4. What is blocked, and what that hides

Blocked tests are the most under-reported risk in test management, precisely because they're excluded from both Passed% and Failed%. List them, with the reason and the owner. A 94% pass rate with 60 blocked tests is not a 94% pass rate.

5. Scope changes during the cycle

What was added, what was removed, what was deferred, who approved it. This is the section that turns a defensible report into an audit-proof one, and it is the section nobody writes — because, as covered above, the tool won't tell you. You have to have been capturing it.

6. Open defects by severity, with the release decision on each

Not a raw bug list. For each one that isn't fixed: are we accepting it, deferring it, or mitigating it, and who decided.

7. Coverage against requirements

A traceability view, and — more usefully — the requirements with no linked tests. The gaps are the finding, not the coverage.

8. Evidence reference

Where the artefacts are, in a form the reader can actually open. A link to dev.azure.com is not evidence for someone without a licence, which is most of your audience — see our guide on sharing results without a licence for what each access level really allows.

9. Who was asked to sign, and who responded

Names, dates, and — importantly — who was asked and stayed silent. Silence is not approval, and recording it protects you.

One habit worth adopting

Write the report so that a reasonable person who disagrees with your verdict can still find every number you used and check it. That is the actual test. A report nobody can check is a report nobody can trust, and the fastest way to lose a release board is to be right in a way they can't verify.

A note on step-level evidence

If your sign-off is going anywhere near an audit, turn on the Actual Result field before the cycle starts. It went generally available in July 2026, it captures what actually happened at each step alongside Pass/Fail, and it is off by default. It cannot be applied retrospectively — a cycle run without it has no step-level factual record, and no amount of reporting will invent one. We cover the configuration and its two governance traps in Exporting test evidence from Azure Test Plans.

The short version

Sign-off packs that show their working

Testmarshal evaluates exit criteria against live Azure Test Plans data and produces a GO / NO-GO recommendation with the reasoning attached — every figure printed beside its definition and denominator, scope changes surfaced automatically, evidence embedded in the file. In development; early access is open.

No spam, and you can leave whenever you like.

Keep reading
How to share Azure Test Plans results with someone who has no licence Exporting test evidence from Azure Test Plans

Sources