We’re in the planning stages of a project to replace several of our mid-range servers with fancy-schmancy new servers. They will run the latest version of the operating system and database, with a new hardware architecture. Needless to say, we’re excited about the boost in power. But we face some issues – for example, several of the applications have a lot of custom code. Several others are no longer supported by the vendors who created them; in two cases, because they’re no longer in business. What’s worse, we don’t have the source code for these “orphaned” applications. We’re also changing the IP addresses of these servers as we consolidate them, which means we have to deal with coordinating the change with multiple internal and third-party interfaces, and over a thousand time clocks. New hardware, new system software, new IP addresses –sobering, eh?
So I was on the phone with the technology team project manager who has been charged with putting the plan together, along with one of my project managers, who ran the last minor version OS upgrade, and several other folks. We were talking about test scripts, and communicating with the users, when he asked, “Can we get the users to do the testing? Just get them to conduct UAT?”
If you’ve been in this business for more than a little while, you’ve put together a test plan. The programmers do unit testing, another team does integration testing, somebody else handles load testing, and the QA team does system testing. And at the end, you bring in representatives of the user community to do acceptance testing. Except, it isn’t really “testing.” All that other stuff is testing, in that the people exercising the system are trying every logical path through the software, to find defects so they can be fixed. Prior to user acceptance testing, enough defects have been removed so that an operator can go for a long time without finding an issue. Quality is at the level required, and it’s time to turn it over to the people who will use it, so they can inspect it. Because in user acceptance testing, the emphasis is on “acceptance,” not “testing.” They’re doing the software equivalent of opening the box and checking what’s inside against the packing list. Does it have all the features they asked for? Are there any issues with the user interface? Is everything accessible to the people who are supposed to see it, and inaccessible to everyone else? Are wait times tolerable? Sure, they have a “day in the life” script to follow, but it’s there to structure the activity, not to exercise every path through the code. Because it isn’t about testing, it’s about signing off on the deliverable. It’s about acknowledging receipt.
So I replied, “UAT is not testing. We’ll get them to run their scripts, but only after we’ve completed unit testing on the custom code, integration testing, and QA has done a complete system test.” I explained that the only response we wanted to get from the users is, you didn’t break anything, and we’re OK with the change. Because they long ago went through the exercise of acknowledging receipt, and have gone on to “bond” with the applications that appear on their screens. We aren’t going to do anything that would change their experience as users; we’ll succeed only if they don’t notice the difference beyond a somewhat faster response time when they complete a transaction. And the way to meet their expectations is to test the Hell out of it, before they ever see it, and remove the defects, resolve the conflicts, and otherwise make the system as good as it needs to be. In this case, UAT is just an alternate spelling for PR – public relations. It’s about change management, not testing.