How do I audit a client backlog?

Audit a client backlog by scoring every story against the same explicit criteria, then reporting the aggregate picture before any individual example. The assessment that holds up in front of a product owner is full-coverage and criteria-based; the one that gets argued with is a sample of stories the consultant happened to dislike.

This guide is the method end to end for a consultancy, coaching practice, or training firm: what to ask the client for, why sampling undermines your authority, what to evaluate in each story, and how to package the findings so they become a baseline for the engagement instead of a debate about your taste.

Why baseline before you coach?

Because without a baseline there is nothing to compare the end of the engagement against. Most coaching and transformation work is judged on how people feel about it: a retrospective, a survey, a sponsor’s impression. A scored backlog gives you a measure of the work product itself on day one. When the engagement ends, you score again, and the change in the backlog is the change you made. The baseline is also your authority in the room: it lets you say “the backlog scores here” instead of “I think the stories are weak.”

What should I ask the client for?

One file: a full export of the active backlog as CSV or Excel. Every major tracker produces one in minutes, and asking for an export instead of tracker access keeps the assessment scoped, avoids a security review, and gives you a frozen snapshot to score. Ask for four columns plus one that clients forget:

• Summary, description, acceptance criteria: the text you’re evaluating. Only the summary is strictly required, but criteria coverage is usually where the findings live.
• A stable ID column (the issue key): this is what lets the post-engagement round match stories to the baseline, so you can show improvement instead of two unrelated snapshots.
• Epic and author, if available: they unlock quality-by-epic and quality-by-author views. Keep authors anonymized in anything the client sees; the author view is for deciding where coaching helps, not for a performance review.

Should I sample the backlog or score all of it?

Score all of it. A sampled assessment invites the response that ends the conversation: “you picked the bad ones.” Full coverage turns the conversation from anecdotes into distribution (“38% of the backlog scores below the readiness bar, and it’s concentrated in the checkout epic”), which nobody can dismiss as cherry-picking. Manual review is why consultants sample; a senior reviewer at 10 minutes per story clears maybe 40 stories a day. Automated INVEST scoring removes that constraint: backlogs of 100,000+ stories are in scope, so coverage stops being the bottleneck and your time goes into judgment and coaching instead.

What am I looking for in each story?

Six qualities, applied identically to every story: whether it’s independent of hidden cross-team work, negotiable rather than a disguised implementation order, clearly valuable, estimable from the text alone, small enough for a sprint, and testable. In practice, client backlogs fail in recognizable patterns: acceptance criteria that exist but can’t be tested (“works correctly on mobile”), stories that bundle three features under one summary (“Improve the checkout experience”), and dependencies that live in the author’s head instead of an issue link. The finding you deliver is the pattern plus the count, not a complaint about any one story, and the pattern is what tells you which workshop to run first.

How do I turn findings into a deliverable?

Build the report aggregate-first: quality posture up front, the per-dimension diagnosis, the score distribution, and quality by epic where the data supports it. Illustrate with a small, capped list of the lowest-scoring stories and one or two “what good looks like” examples from their own backlog, and attach the complete per-story dataset as a spreadsheet so nothing looks hidden. Write the executive summary yourself; it’s the one page the sponsor will actually read. Close with the plan the findings justify: which teams, which dimensions, which workshop first. This structure is also how you stay diplomatic: aggregates critique the backlog, not the people who wrote it.

Vindex for Consultancies produces exactly this deliverable: it scores the full export, gives you a worst-first review workspace to exclude junk and add your own notes, and builds the report white-labeled under your brand and methodology, with the full dataset attached. The report builder reference lists every section and branding option.

How do I prove the engagement worked?

Re-score after the engagement. The baseline is a snapshot; the second round is evidence. Keep the ID column stable between exports so stories match across rounds, then show the movement: the share of stories above the readiness bar, the dimensions that improved, the teams and epics that still lag. Round-over-round improvement is both the proof the coaching changed the work and the natural opening for the follow-on: the lagging epic or team becomes the next engagement. If you deliver hard findings to a team whose product owner is in the room, presenting findings to a client covers the readout itself.