Ui test automation tools are snake oil

However, few people have ever seen real UI test automation. Try answering these questions:. These are simple questions, yet very few can answer them well. The reason is simple: UI test automation is extremely hard. The following scenario has happened over and over again: I visited a team and asked about their testing situation.

We talked about unit tests, exploratory testing, the works, etc. Then, I asked about automated end-to-end testing. They pointed at a machine in the corner. That poor machine has had an installation of some highly-priced per seat testing tool or an open source one , and the chair in front of it was empty.

We walked over, swept the dust away from the keyboard, and loaded up the tool. Then, we glanced through their set of test scripts and tried to run them. I may change my mind again. Real UI test automation is that software teams run full regression automated tests many times a day.

If they all pass, software is ready to ship to production. However, most software teams have never reached Step 3. Functional testing is mostly the same: test manually. Without automated regression tests, the whole CI promise release high quality software faster falls short. The test execution features in traditional CI servers are for unit testing, which is quite different from UI test execution. Comparing to unit tests, UI tests are.

Besides, it often requires a team effort to detect UI test failures from a build,. The only way to test them is through the UI.

UI testing can provide the initial safety net, which later on can be enhanced or replaced with API-based unit tests. In his post, Mr. Well, let your conscience be your guide. Feathers for it, is a guide on how to test the UI layer in isolation which I will be doing soon. My intention is to create a tool that generates clean Java code, as if it was hand-crafted, to overcome to what I think are the problems with this technique.

Overall, I liked and enjoyed Mr. Feathers article. It has pretty good points and observations. I wish he backed up his ideas with more details and examples, to sound less emotional and more objective. The problem is in the human factor, not in the tool. Ideally, vendors in general should set the right expectations about their products, and customers should have enough knowledge to avoid being taken advantage of. At least, some of us are trying to do the right thing :. Michael Feathers posted a thoughtful piece describing the general problems of UI testing tools and the industry in general.

Automation tool vendors, as with almost every tool vendor out there, are eager to solve perceived software development problems. Unfortunately, these tools usually only address symptoms of larger problems.

With UI testing tools, this is no different. When we started out authoring UI tests for a large MVC application, we had a few large goals in mind.