I have heard UAT described as the final frontier or last staging post before a system goes ‘live’ which indeed is as good a description as any. If we take this view, then we have a good starting point in understanding why a business invests in this phase of testing. Go ‘live’ means real business users will undertake activities with the ‘tested’ software. Whilst software testing is important (and must be done) it is only a part of a solution which involves IT Systems, People and Business Processes ; in fact the successful integration of all three is critical to a successful ‘go live’ of the system, deployment and subsequent maintenance.
Successful integration of all three areas is critical to successful UAT. Often businesses recognise that the software needs to be tested placing too strong a focus on removing as many software bugs as possible, they also recognise and that staff in the business will use the software in their daily work but sometimes see these separately rather than integrated in the testing.
UAT testing is a level of testing by itself and should involve real users. Some organisations have what I call light bulb moments when they see the importance of seconding the actual users into testing, such users can be expert in the business process and have detailed knowledge on how, for example, a supply chain system is used to configure , approve , pick and pack , ship and invoice complex orders for say a telecoms network.
The missing ingredients for the real users are the knowledge, skills and training in formal test techniques which is an essential for effective, economical and efficient UAT.
I have come across organisations that have limited involvement of the actual users of the system prior to going live. These companies do test the software, with trained software testers but hit problems once the system goes live as real users don’t follow scripts or test cases which may be somewhat artificial; the system may not align with the process. Sometimes the development view is that ‘the users aren’t operating the system correctly’ which fails to see that the system is part of a business initiative aimed at achieved a business goal. For these folk (IT Testers) the realisation is that software testing at the lower levels should be done but to help ensure that what is delivered is ‘test ready’ for UAT. Acceptance ultimately can only really be deemed successful if the business benefits are realised, this principle can be a point of learning or light bulb moment for those whose work has largely been in the IT area. The role of the IT tester is essential in ensuring the system is fit for UAT testing having been tested as the component, integration and system levels to an acceptable quality level.
Real users are not trained testers, so may have difficulty in structuring scenarios, running tests, characterising problems to write bug reports and recognising the need for regression testing (all things that a formally trained tester will know about). It is possible that they are only loaned to a project for a couple of hours a week and the UAT is not their highest priority. Those organisations that tend to have a smoother ‘go live’ and deployment phase are those who second staff to UAT almost on a full time basis and recognise the lack formal training can be addressed with a short course. Often just laying out a test process for them and the concept of a test plan, where what’s in scope and out of scope is clarified, describing the concept of acceptance criteria and agreed quality levels (i.e. the system won’t be bug free but still should be fit for purpose) can assist in the structuring of the tests as well as build confidence for these users to ask and challenge assumptions. Techniques learned such as Decision Table Testing make for more efficient UAT where there are many ‘business rules’. Simply learning what should and should not be in a bug report provides these users with a consistent and systematic way of characterising problems as well as highlighting that the bug (or incident) reports are critical and one of the most valuable and visible ‘deliverables’ produced by the User Acceptance Testing team.
From an employer and employee perspective, it’s quite clear that organising training for business users as well as testers equips the staff to do the job in the best way they can, provides the knowledge and subsequent skill set for future UAT and more likely to result in a successful go live, deployment and maintenance phase.
Steve Helsby, Senior Trainer