by Praveen Prakash
In our series on Testing concepts and fundamentals, this blog post is going to address an important testing practice called User Acceptance Testing and the UAT test plan.
User Acceptance Testing or UAT Testing is an inherent part of all Software Testing – irrespective of methodology. Whether you’re using Agile development methodologies or sticking it out with Waterfall, any software product that you build needs to undergo User Acceptance.
By definition, UAT is the point at which a user accepts that the product meets requirements and design standards. Therefore, UAT is conducted after all development and functional testing (read System Testing) is complete, and just before the product is pushed forward to the release or implementation phase. I’m sure you know all this.
Coming as it does at such a critical juncture of the project delivery cycle, it is imperative that you get all the support you need to plan and execute UAT successfully. And with next to no slippages or issues.
It is possible to get UAT right – when you’ve done your utmost to plan ahead and plan well. A true reflection of your efforts to adequately prepare for the critical UAT phase of your projects is in how well your UAT test plan is received, accepted, and approved. The success of the plan is also directly proportionate to the success of your team’s testing efforts – they’re going to (more or less) execute the plan after all.
What is User Acceptance Testing?
To the uninitiated, UAT represents essentially the last frontier for Testing to catch any unseemly bugs prior to a product release to customers. The idea is that you won’t have the chance to rectify any glaring defects with your product beyond this phase in the project.
UAT is also the point at which the ‘user’ accepts the product as meeting all requirements. By user, we mean those that commissioned the product in the first place. These may be people that directly use the product, or who expect their customers to use it.
UAT Test Plan
UAT represents the cycle of testing traditionally performed by users themselves to ascertain that the product works as expected, and meets all the requirements that they provided at the beginning of the project.
The actual testing can be conducted completely by users themselves. This would require a certain amount of dedicated resource allocation from the sponsoring user team.
But this isn’t always possible. Due to any number of reasons, you may not always have adequate user time available to complete a thorough UAT cycle. What is increasingly common, therefore, is that the users appoint a dedicated team of testers that they outsource UAT testing effort.
Key enablers for building a rock-solid UAT test plan
On Agile projects, things can get a bit convoluted
Does your Agile project have development sprints, but dedicated test cycles to help polish the product before release? With experience, I can tell you 60% to 80% of all Agile projects run some level of dedicated System Testing and User Acceptance Testing cycles to catch the unseemly bugs that need to be caught before deploying the product to customers.
Why is this necessary? Because, when you’re working Sprints, you don’t really have the time to ensure 100% quality in your product. Especially if it is new product development, Sprints primarily focus on building the core product features as quickly as possible to get these rights before worrying about release quality.
So there’s usually going to be a fair bit of testing that is left to the pre-release prep so the Scrum team can focus on the 80% (of the famous 80-20 rule) first. And however, you clothe it – call it Sprint, call it a dedicated Test Cycle, call it just ‘Testing’ or delineate System Testing from User Acceptance, use actual users or delegate testers – you need UAT at the end of such projects.
“There can be no confusion about ‘what is the requirement?’ when your testers and developers are arguing about the validity of a defect.”
So coming to the point about Signed off or Approved requirements, even on Agile projects, you can build a repository of signed-off requirements. When a user story has been absorbed into a sprint for delivery, essentially that particular story is signed off.
The Product Owner has provided their requirements for the Sprint – albeit in the form of a ‘story’ and ‘acceptance criteria’. And these requirements are maintained centrally (insert your favorite Agile Requirements management tool) to track delivery.
These are, then, your signed-off and approved requirements.
Why is sign-off or approval important?
User Acceptance Testing basically seeks to ensure the product meets the requirements of those who pay for it to be built. When the people that pay specify their requirements clearly and are willing to approve or sign off, you then get a tangible set of base requirements to compare and test the product against.
Having signed off requirements therefore will help you build a plan to meet those exact requirements. There can be no confusion about ‘what is the requirement?’ when your testers and developers are arguing about the validity of a defect.
Test scenarios MUST be reviewed and signed off by the User or Product Owner
We have established that end-users should really be testing the product, and signing off User Acceptance Testing. We have also established that this doesn’t commonly happen – at least, users don’t directly perform UAT testing, and if they do, users don’t necessarily complete all UAT testing. They mostly recruit specialist UAT testers to do their job.
The thing with specialist testers, however, is they don’t always understand everything about the product to be tested. This can be because they are new to the product, line of business or even simply that they don’t have the same view that the business sponsors and corresponding end users have about the business needs the product is expected to fulfill, target audience, design standards and a host of other requirements.
So how do you help a UAT tester be effective? By defining the boundaries and parameters that they are expected to test, of course.
And who better validate that the testing scope covers all that needs to be covered? To give you the answer, it is the same people that define the requirements in the first place – the same people that will use the product being built.
Get the product owner or your users to review test scenarios in scope for your testing in detail. Provide them with a walkthrough if you have to. On Waterfall projects, this is a daunting task – especially if your project is large and complex. On Agile, when you lock in requirements and corresponding test scenarios/cases within each sprint, you make your job that much easier.
On Agile again, when you have a dedicated UAT cycle after the Sprints, it is still a good idea to build your Test scope iteratively through the Sprints. This way, you can be ready when the UAT cycle begins and you won’t have to try and build this humongous test case database all at once.
UAT Test Plan Template
UAT templates are much more important, as they help clients have an overview of reports and input templates.
Entry & Exit Criteria
With so much going on in a project, especially if you are working on a high profile one, it’s often not clear to everyone in your project (not just your testers but everyone) what are the minimum criteria that need to be met before Testing can begin, and before Testing can end.
Why is this important?
Entry and Exit criteria for Testing cycles define the absolute minimum that the UAT manager will accept (on behalf of the sponsor) before they can begin testing the product and before they will recommend sign-off of Testing for release.
Entry criteria could be that the requirements are signed off, the Test plan is signed off, test scenarios and test cases reviewed approved by key stakeholders, etc. Exit criteria to move the product into release could be that 100% of all test cases have been executed, 100% of all critical and high defects have been fixed, 90% of all medium defects are fixed, 95% of all test cases have passed, etc.
When you have Entry and Exit Criteria defined on your UAT test plan and agreed by all key stakeholders (including your engineers), you then have a clear path to define the progress of Testing when you’re in full swing. And there will be no surprises.
