Standing the test of time

In this blog post we’ve proposed a set of rules for creating low maintenance, well-written tests.

Rule #1: Understand the value of testing

Systems have users, and those users expect the system they’re using to do something.

So why test the intricacies of how something works, when all you care about is a set of rules? Rules define how the system behaves when a user interacts with it. Or more specifically, the output of a system for any given input.

For example, the driver of a car does not care that when they press the accelerator pedal, fuel is put into the engine and that (simply put) turns the wheels. They only care that on pressing the accelerator, their car moves forwards.

Let’s say for a second, however, that as part of your test, you checked that the accelerator puts fuel into the engine. Although seemingly innocent on the surface, this has tied the test to our implementation. The driver is now unable to switch out a fossil fuel engine for an electric one without first modifying the test.

Now imagine the perfect test suite - you’ve covered your potential input and outputs and now someone new is joining your project. How are you going to bring them up to speed? Documentation of course. And tests are documentation - describing in detail what your system is supposed to do. They also have the added benefit of demonstrating that it really does it too.

Now, if you set these assertions to run any time a change is made to the system, you have a reliable place to go and check for red lights, lessening the dependency on manual smoke checks to ‘ensure everything is still working’.

Rule #2: Adapt your language to that of the end user

A developer is rarely forced to look beyond the scope of their implementation, so change your testing language to make that happen.

Stop thinking in terms of ‘should do this’ as a test description and start using a language such as Gherkin or triple AAA (Arrange, Act, Assert). This will force the perspective of an end user, and who better to write your tests than the person who will actually be using the product.

A Gherkin example (known as a scenario) using the driving example from before:

Given - The state at the start of the test. A scenario might have lots of different pieces to its initial state. For this, you can follow up with ‘and statements’. In the example above, this is ‘And I have turned on the engine’

When - The action/input to the system. What are you going to do? - it is strongly recommended there is only one of these. Having more than one ‘when’ step is usually a sign that your scenario can be split into multiple scenarios.

Then - The expected output from the system given the state and action you performed. You might expect a system to do multiple things when you perform certain actions. You can also follow up ‘then’ statements with ‘and’ statements. In the example above, this is ‘And the speedometer shows the increase in speed’.

Rule #3: Look forwards, not backwards

Don’t worry about your previous tests. Just change your test style going forwards. If you find spare time here and there, come back and rewrite an old test. Just be careful not to try and force the original test into Gherkin. Write a scenario that covers the original input and outputs, but that covers larger ground. Sometimes one scenario can replace multiple unstructured tests.

In the case of the car example, imagine the following unstructured tests:

• It should put fuel in the engine when the accelerator is pressed and the engine is on
• The camshaft should turn when the fuel is burnt
• The wheels should turn when the camshaft does
• The speedometer shows the speed of the car
• All four of these tests can be replaced by the one scenario we wrote before*.

Rule #4: Don't get tied down by tooling

Being tied to tooling is bad. Try and have multiple extendable tools in your arsenal with the following properties (if they are applicable):

• They can parse the language you write your tests in (in our example this was Gherkin)
• Cross browser/platform compatibility
• Tools that are extendable
• Tools that show coverage of what you’ve already tested and its health

If you can obtain all of these things, you’ll end up with a test environment that provides you with confidence and support.

In summary

Adopt a common language by describing the behaviour you want to see. It’s all about outcomes. Have confidence that automatable tests regularly monitor for the achievement of outcomes, even if the manner by which it does it, changes.