User acceptance testing (UAT) is one of the many flavours of testing that has emerged over the last twenty years or so.
The emergence of UAT has provided thought and guidance on the business analyst role in UAT.
As the software or product development life cycle has become more sophisticated and rigorous (or, in the case of agile, more lightweight), the testing phase has been broken down into many different aspects and styles of testing.
This article explains the business analysts role in UAT. It will describe the history, how it fits into the project life cycle, the objectives of UAT and how the business analyst can be involved.
Table of Contents
What is User Acceptance Testing (UAT)?
User Acceptance Testing (UAT) is when actual users evaluate the software or product to ensure that it can perform necessary operations in realistic scenarios, according to requirements. UAT is usually the last testing stage of a software project or product.
User Acceptance Testing (UAT) History
User acceptance testing (UAT) has emerged as more user involvement is both expected and required. User means the business or subject matter experts (SME’s) or, usually, the actual end user of the product rather than the manager who may have been more involved in the project.
The business expects to accept the product rather than simply assuming the final product will be fit for purpose. This additional involvement is reflected in the software or product life cycle in all stages prior to user acceptance testing with a particular focus during the analysis phase when the requirements are agreed.
The benefit is that there are no surprises when the product is released and any surprises can be addressed before the product is released to the wider user community. The downside is greater involvement of user time but this can be managed and the benefits offset the disadvantages.
User acceptance testing (UAT) has emerged to focus and address the perspectives of:
- User – users who will use the solution are rarely engaged in the project that produces it. This is true even in Agile projects, where the product owner (who is very involved) is only one person, and other users are more removed. UAT allows users to see what has been produced as well as provide feedback before it is deployed.
- Acceptance – no system or product is perfect, but we can make sure it’s up to standard before releasing it. The final users of the system or product – those who aren’t involved with building it- must be the ones to accept or reject what we’ve made. User acceptance testing is their opportunity to give feedback about what does and doesn’t work for them.
- Testing – attempts to define ahead of time what the system or product will be and do is difficult and whilst there are many tools and techniques to help the process – it is good practice to demo and test the product with real-world customers and user testing.
User Acceptance Testing (UAT) Objectives
UAT’s prime purpose is to demonstrate that the system is fit for use in the business. It is not the same as system testing which should prove (within time and resource constraints) that the system behaves in the way that was described in the requirements specification.
Also, the objectives for user acceptance testing (UAT) should be to design and conduct test scenarios with a view to identifying where a product fails to meet the needs of the business stakeholders and users. The objectives should also include supporting the business user community in ensuring that a product is of sufficient level of quality to be accepted for deployment.
UAT is, to some degree, a public relations exercise as end users tend to be involved as well as stakeholders and they will (hopefully) gain some confidence in the quality of the system, understand how to use it and, very importantly, start to see its benefits.
There are many other forms of testing which, for example, prove the code works as designed at a fine grained level (unit testing) or prove the various systems interact as expected (integration testing).
User Acceptance Testing (UAT) Principles
Although it is not as exhaustive as system testing, it is equally rigorous in what is tested and documented and formal acceptance criteria and process.
Project objectives, business benefits and critical success factors should all be used to help define acceptance criteria
Users and business stakeholders should be involved in decision making and planning throughout UAT. They will become natural ‘champions’ for the new system if they are fully involved and consulted.
It is critical that the business are fully engaged in the planning and design of the user acceptance test and the production of acceptance criteria. This is one of the last hurdles before the product is released into the world and the understanding of the product and problems uncovered at this stage are vital in making the decision to ‘go live’ with the product.
Poor user acceptance testing can cause significant problems for the business.
User Acceptance Testing (UAT) Testing Context
The V model is a common representation of the project life cycle (from a testing perspective). It should be read at the start of the V (top left) with user requirements, follow the V down to code and back up to Acceptance testing (top right). Acceptance testing can be seen to have a direct correspondence with the user requirements which should inform the structure and content of the user acceptance tests.
User Acceptance Testing (UAT) Planning
Some time should be invested in selecting appropriate scenarios for inclusion as it will not be possible nor appropriate to test all scenarios. The following should be considered when deciding these scenarios:
- Existing requirement documentation;
- Critical success factors;
- CSF’s will not all be provable in UAT as some may only be proven over a period of time;
- Business benefits;
- Most business benefits will also not be provable as they are likely to involve a cost saving or revenue generation which can only be measured over a period of time;
- Typical scenarios;
- Normal or happy path scenarios should be a central part of UAT;
- High risk scenarios;
- Scenarios that are recognised to drive profit, can trigger cost savings or have regulatory impact should be candidates for UAT.
The planning should include identifying stakeholders with concerns relevant to a proposed business change, to analyse stakeholders levels of power and interest to help ensure that the right people are brought together to help with the user acceptance testing of the solution.
We need to consider not only the end users we determined during requirements elicitation
and analysis, but also the requirements themselves when identifying UAT testers. User acceptance testing should include examining all of the system’s functions for acceptability, so it’s a good idea to go over the requirements and double-check that there is a user who will check each one during UAT.
User Acceptance Testing (UAT) Documentation
It is important for stakeholder or users to maintain logs of UAT tests and their findings with records that are retrievable and clear.
A UAT test report documentation should include the following kinds of data:
- ID of the UAT test;
- UAT test case name or scenario;
- UAT test case steps that need to be followed;
- Business requirement being tested;
- Acceptance criteria that should be met for the test to pass or fail;
- Date executed;
- Pass/Fail.
Business Analyst Role in User Acceptance Testing (UAT)
The UAT business analyst will be involved in user acceptance testing to a degree, depending on the project.
At a minimum, he/she will provide guidance to system behaviour and identify scenarios which would be suitable to test. They may be requested to review the test scripts and objectives and, in some cases, may actually coordinate and manage the execution of the user acceptance testing.
The business analyst role in UAT, should do no more than co-ordination as it is critical that the business has ownership of the tests and, hence, believe that passing business acceptance testing genuinely means the product is fit for purpose.
This will depend on the company culture, the specific project and whether there are any resourcing issues.
As a service value proposition the business analyst should seek to collaborate with stakeholders to support business acceptance of the solution.
The service activities for business acceptance testing and hence the business analyst role in testing in UAT should include:
- Agreeing scope for the testing activity;
- Defining test scenarios and test cases;
- Provide support to stakeholders when testing for business acceptance.
The business analyst can help facilitate UAT including arranging the schedule, environment and who will be involved in UAT.
Conclusion – Business Analyst Role in UAT
This article discussed the history of UAT and the context of UAT in the project lifecycle and where and how business analyst role can support UAT.
The article also explained how the business analyst can be involved with the planning for user acceptance testing and the principles that should be followed.
The business analyst role in UAT is an important one and a business analyst is not just on a project to do requirements but should be involved through the project or product lifecycle.