“This troublesome user acceptance testing got to be easier. It is really a pain” said of my colleague quoting an IT manager responsible for release management and user acceptance testing. Major problem according the IT manager was getting the “users” to do their job – testing a defect fix, feature enhancement or a new release. Users are always busy with their “day job” and for most of them doing testing is a low priority activity. I have heard many similar stories regarding this important aspect of “end user” responsibility.
In traditional IT organizations, user acceptance testing is a formal stage in the testing life cycle where business users will test the proposed changes to production software applications. IT organization that delivers the software to meet the needs of business users, require a (formal) approval of proposed changes that come in terms of defect fixes, enhancements and new releases. Since it is business users that ask for changes for their existing and fund the development/testing work – they will have the final say on “acceptance” of proposed changes to production applications.
Primary purpose of user acceptance testing is check (cursory- final) proposed changes to the production software by those users that asked for the changes so that any last minutes changes and surprises are avoided. User acceptance test is the last gate before software goes Live so that it provides last opportunity for all involved (both IT and business) to make corrections if required. Depending upon the nature of business supported, nature of software application, nature of changes proposed UAT may last for few days to few weeks. Software that fails in UAT is typically assumed to be insufficiently tested and is variably returned to IT for fixes and more testing.
The problem of UAT
Business users who hold the responsibility of approving software updates to production systems, typically are not testers and as such do not come with testing skills. Depending upon the nature of software updates, IT team will ask business users to carry out testing to check the features of new system. Most of the user acceptance testing tends to be repetitive and that is what drives “real users” crazy. For development team (IT), their work is not complete until the piece of code is user tested and accepted. Hence they literally chase user group to perform UAT with users complaining about the time crunch and their “important” business work getting impacted. This creates a situation where both parties (IT and business) want UAT to be somehow completed so that each can carry out their business as usual. Hence UAT often gets “fixed” with the consent of both parties having stakes in the activity.
Forms of fixing UAT
Between Business and IT, UAT can be fixed in number of ways – some of these are reasonably business driven models given the constraints.
1. UAT will be performed by the members from IT team where business only reviews the results of the test. If the results are OK then UAT is ought to have been completed
2. UAT will be performed by a third party, such as someone from support staff. Business will review the results and accept the proposed changes if results are OK
3. Business will provide a prescribed set of test scripts that can either be used by anyone IT staff or support staff. Results will be verified by the Business.
4. UAT will be done like a demo to the users where IT staff will execute some pre-approved test scenarios related to proposed changes.
5. IT team will train the business in new features proposed to be introduced. Business users after training, test (use) the proposed software and accept the software
Why fixing user acceptance testing is bad thing?
Why bother to UAT after all? For IT, more often than not, it is a formality to be completed before they push the code to production.
In my opinion, it is the spirit and purpose of UAT that gets compromised. Typically, in spite of all best efforts, the depth and frequency of interactions between IT and business throughout the project remains low. When business users do not participate with full spirit in UAT, lots of things go unnoticed into production. This might results users (non participating ones especially) getting surprised when they see the product.
What has been your experience? Do you bother if you see your UAT is fixed?
Shrini
Leave a Reply