by Don Mills
In an earlier paper on “Reviews In Agile Projects”, I made a case for conducting requirements reviews in Agile, even if you are so Agile that there is no documentation at all to review (except code), and I offered some practical tools to help make Agile reviews work.
This was worthwhile, because reviews of what people write, and (perhaps even more importantly) what people say, when requirements are being laid down are amazingly cheap and effective ways of killing off bugs before they even have time to hatch from their eggs. This is often encapsulated in the 1 : 10 : 100 ratio, though as the following diagram shows, those figures are only round-number approximations:
Summarising eight different studies, the piles of coins represent the relative costs of finding and fixing a requirements bug at three different stages. The sterling costs are arbitrary, but their proportions represent the findings of the studies: 13 to 30 times in testing, 40 to 90 times in production.
This essay follows a similar theme to its predecessor, although the principles I’ll express here are just as valid for “non-Agile” reviews, like fully formal inspections. The tips (like the diagram) are based on decades of other peoples’ research.
1. It’s really about bug prevention
This doesn’t just mean that finding and fixing bugs in requirements prevents them leaking into designs and code. It means that the most important use of reviews is to teach the creators of software work products (written or oral) (a) what kinds of mistake they make, and (b) how to stop making them.
People are pretty habitual in their error-patterns, both collectively and, to a greater extent, individually. For this reason, if the work product is going to be relatively large (more than half a page, say), review it early, before its creator’s got very far, in order to make a quick kill on some systematic bugs. Don’t wait till the product’s “finished”—that’s a test-last approach to reviews. Be Agile and test first!
All participants in a good review learn something from it, if only about the work product; but the creator of the work product (“the author” for short) should learn most. This is “grass roots” process improvement.
2. Reviewers are the author’s “quality assistants”
I got this term from Tom Gilb’s Inspection course. The role of the reviewers is to help the author be the best author they can be. You do this by identifying as many genuine issues in the work product as you can—not trivia such as typos or poor style (unless they directly affect the meaning and/or use of the product).
Looking for “inconsistencies” is a helpful way to think about it: inconsistencies within the product itself; inconsistencies between the product and other sources of information; inconsistencies between the product and standards (including checklists like the “Elephant’s Child” and “SCUTA” heuristics); and inconsistencies between what the product says, and what you know or believe to be true, necessary, or appropriate. Of course, you may be wrong—that’s why we call them “issues” rather than “defects”.
Depending on the nature of the review (which has to be set out up front, preferably in consultation with the author), you may also be asked to help by suggesting and discussing potential improvements to the product, such as a clearer way to express an ambiguous requirement. Always be aware, though, that giving such advice unsought is likely to create resentment, and giving it at all often leads to arguments.
3. Keep it non-threatening
The need for this is one of the main reasons that formal training is required for formal reviews like inspection. Briefly, though, what are some of the things that may make a review “threatening” to the participants, apart from un-asked-for advice?
- Managers present in a staff-level review. Managers should review management-level documents like plans, schedules, and budgets (they absolutely should!). But many studies show that their presence in a review team for a non-management product (requirements, designs, code, tests …) tends to inhibit defect discovery because the reviewers don’t want to admit to not understanding something—it makes both them and their colleague the author look bad. “Peer reviews” are conducted by people at the same sort of organisational level as the author. (Just as well Agile teams don’t have managers …)
- Reviewing the author, not the product. This is precisely what an author might fear if there were a manager on the team. Respect the fact that the author was doing their best in the circumstances (if you’ve got managers, you can blame them for the latter!). Avoid even hinting that you’re judging their capabilities. Make every effort to be genuinely helpful.
- Reacting offensively to criticism. This is a warning for authors: take the advice of your colleagues seriously, they’re trying to help! On the other hand, you don’t have to slavishly accept everything they say (another reason reviewers report “issues” rather than “defects”). Keep your Thinking Cap on, and try to see their point of view, particularly if they’re going to have to work with the result. If you disagree strongly, keep your temper (and you too, reviewers!), and suggest coming back to it later.
- Going on and on and on and … . One of the ways Agile processes deal with any tendency to drag things out is Timeboxing. Standups are typically timeboxed to 15 minutes. Iterations are timeboxed to whatever was agreed during Release Planning. If an activity isn’t complete at the end of the timebox, it has to be transferred to a future timebox. This helps people prioritise what’s important to the tops of their lists. Review activities should be timeboxed at one or two hours, depending on the scale of the material to be reviewed, but not more than two hours at a time. People just can’t concentrate longer.
Boris Beizer wrote, in Software Testing Techniques, “The proper use of testing is not to find bugs, but to prevent them.” Ron Radice (trained by Michael Fagin as the world’s first Inspection Leader) wrote, “The most important thing to realise about inspections is that finding and fixing the bugs in today’s document is a free side-benefit of the author’s learning never to make the same mistakes again”.
Constantly striving for excellence is a major theme of Agile development. Reviews are an important mechanism by which, if you go into them with the right spirit, you can help one another learn, and learn from one another.
Be preventative, be quality-focussed, be non-threatening, but above all, concentrate on training one another, learning from one another, and improving the ways you create, review, test, and use one another’s work products.