by Marnie Hutcheson.
Paperback, pp 408
I knew that Marnie had a new book published, and as I was unable to find any details, I looked on the internet, and searched for her, using ‘google’. This revealed someone who was part of the USA teamsters squad, for horses and carriages. Had I got the wrong person? NO. Marnie has led an interesting life, and was both a performing classical dancer, and a ballet teacher, both of which are mentioned in the book.
The book is in three parts, with chapters 1 – 5 covering Backgrounds and Concepts, the real meat of the book in the following three chapters, concerned with the Test Inventory and how to create it, and chapters 9 – 14 on tools and techniques. If you have NEVER read a testing book, or do not own any, I would suggest that this book is not for you; in spite of the title, this is not a beginner’s book. I found parts of it were hard going, but easier on the second reading. I suppose reading a testing book twice makes me a sad individual.
Marnie lists four key problems for software testers. Whilst there is some truth in this, it seems to me to bit a rather gloomy summary. Perhaps I have just worked on projects with some very good testers, and don’t find things so gloomy.
Changes in the way software is developed have eroded the value of testing.
Time-to-market is now a key factor.
There are fewer trained testers using methods and metrics
Most important software quality improvements have come from standards.
The sub-title for this book is Methods and Metrics, and the author appears to be a born measurer; if she were given 10 minutes to live, I guess she would measure it to see if the figure were to be correct, or only 9′ 55″. The discussion on metrics, and the need to measure appropriate figures is one of the strong points of the book. Measuring so that an answer can be given is contrasted with the ‘I feel lucky’ approach, where the usual answer is ‘I have tested IT’ (where ‘it’ is a program, a system, etc). She also stresses the need to keep records from one project to another, and to continually refine what and how you measure, in the light of experience. Marnie was one of the pioneers of ‘S’ curve analysis, and in measurement discussions, talks of the Defect Removal Effectiveness metric (usually known in Europe as the Defect Detection Percentage [DDP] ).
The introduction to the key topic in the book did not really work for me (the key topic, by the way, is the Most Important test [MIT] principle). In the discussions about what should happen, and what does happen, it is almost as if reality is set up as a straw man, to be rescued, a-la John Wayne and the US Cavalry, by MITs. This part of the book is again better on a second reading. In the ideal world, by the way, a plan is presented to management, and this is used to negotiate the resources necessary to conduct a comprehensive testing effort. The reality can be very different: testing is trapped in the (usually too small) space between the end of development and the ship date. This gives the classical testing squeeze.
It is not possible in a few short words to do justice to the MIT principle. For this reason alone, buy the book. There are three stages to MITs:
Build your test inventory (by consensus). This defines what is to be tested.
Undertake a risk analysis (using a 5-fold ranking: Severity, Probability, Cost [to the business], Cost for Customer Services [in dealing with all the errors and service desk calls], and whether the feature or attribute being tested is an absolute requirement) of each item in the inventory.
Use the average from the rankings to determine the % of the identified test cases are to be run.
All in all, I would recommend you buy the book, subject to reading it twice! It is rooted in practical examples. There is a very nice analogy of the different testing stages (unit test, system integration test) being like settling ponds in the filtration of water. This little cameo is probably only one short paragraph. Finally, this is the first time in book form for Marnie’s big two ideas: ‘S’ curves and MITs.
Reviewed by Peter Morgan, Principal Test Consultant