If you never heard of it before: automated acceptance testing is a form of testing where you describe with code what the interactions of a user with the application would be. You basically specify how the GUI should look like and behave, so that after every change in your codebase you can run this specification and check if the changes caused some sort of regression. For Web applications this automation is usually performed using a tool called Selenium WebDriver, which is a driver that provides an API for interacting with the browser through code.

From the first look acceptance testing seems amazing, you can replace with code all that job of manually opening a web application, checking if the buttons are in the proper place, if submitting a form creates a certain entity, if clicking the delete button really deletes the thing, etc. Having written and maintained a lot of these tests over the years I can say that yes, they can provide real value for your app development, avoiding both functionality and interface regressions and also saving a lot of time by removing the need for manual tests, but in practice there are some caveats.

It’s not simple to run the tests and they are slow

With unit tests things are usually as simple as setting up the application database and then running the testing framework of your choice. Also they tend to be pretty fast, or at least as fast as the code being tested.

For acceptances there’s a lot more effort involved. You need your application to be running behind a proper web server, a browser, the selenium WebDriver and maybe a testing framework. After that you can run your tests, the problem is that this can take some long time depending on their size and complexity, an acceptance testing suite for a small project can easily take hours to run if you try to test every single feature of the application.

They tend to break. A lot.

In applications under heavy development the HTML an CSS of the pages tend to be continuously changed all the time. Maybe it’s a DIV tag that was replaced by a HEADER or somebody decided to change the name of a class or there are new elements in the page or there were some responsive CSS fixes that rearranged the layout, the list goes on. The problem with that is that selenium acceptance tests heavily rely on CSS and XPATH selectors and these can be quite fragile as even structural changes that don’t affect the look and behavior of a page can invalidate selectors (changing tags or class names for instance).

Should I avoid them like the plague?

No, as I wrote before, they do provide real value, the thing is that having acceptance tests shouldn’t be an automatic decision as having unit ones. Before deciding that you want them you need to make sure that you understand that writing and maintaining those tests will probably take a lot of time from your team, so be sure that they will have this time. I’ve been in situations where there was this huge acceptance testing suite, but more than half of it was failing and there wasn’t available time for anybody to fix it, so they went on just ignoring the results, making the entire suite useless as any new failure was seen as “normal”. But I also experienced projects where we had really stable acceptance tests that would catch a lot of regressions over time, hence I still like them.

Some general tips on writting your acceptance test suites

• Have a CI system automatically running the acceptance tests because nobody is going to be continuously doing this, at least on their free will!
• Vagrant, docker or whatever else that can provide a “ready to run” environment on local machines will save tons of time, especially with people that don’t have previous experience with these types of tests.
• Stick to CSS selectors as much as you can, XPATH is more powerful but also less readable and (I think) the majority of web developers are not proficient in it, so use it only as you last resource (when you need to match text or select parent elements for instance).
• Avoid interacting with complex widgets like Datepickers, interactive Charts or stuff that is heavy on JavaScript.
• Only add them for pages or functionalities that have been stable for some time, you don’t want to waste time having a test that breaks every day because of something like the designers changing the page constantly.
• Avoid overly specific selectors, .className #id is much more resilient to changes than div.className > p#id.
• TDD doesn’t make much sense here.