Musings on BDD

This month at the London Python Dojo I gave a talk on Behaviour Driven Development. People were a lot more interested than I expected, so I thought I’d share my thoughts more widely on here. The main motivation behind this talk was to dispel some of the myths around BDD and make it more accessible. You can view the slides on google drive.

What is BDD?
The main points I made could be summarised as:

  • BDD is a methodology, not a tool or a process
  • As a methodology it’s a collection of best practices
  • You don’t have to take the whole thing for it to be useful
  • As the name should make clear – it’s about focussing on behaviour not implementation.

For a more thorough introduction I recommend that you read the slides with the speaker notes.

What I found more interesting though was the Q&A, which lasted longer than the slides themselves. There appeared to be a lot of misconceptions about testing in general. This came out because BDD encourages you to look at behaviour at the user level. This is important as it helps focus your tests and give them tangible meaning. The leap of comprehension though is that when you test at different levels you are concerned with different users.

At a unit level your user is a software developer who will be using that unit of code. The most important thing at this level is the behaviour to this developer, who could be you, or could be someone else.

At an integration level your user is another service or unit of code. Here you are concerned with the interface that is provided to these other services and code units. When this second piece of code interfaces with the first, how does it expect it to behave?

At a system level your user is probably a person, or possibly a script. At this level you should really be using the user interface, be it a GUI or text. This is the level at which there was least confusion, as it should be fairly obvious that the behaviour your interested in here is that of the system as a whole when the user attempts to interact with it.

BDD in Practice

One of the questions I was asked was one about speed – as behaviour is only the first word, it is important to be able to use this to drive your development. Indeed system level tests shall rarely be as fast to execute as your unit tests. This does not however destroy the value of having these tests, and with a little work, in most cases, system and integration tests do not have to be as slow and unpredictable as you might have experienced. This comes down to the proper application of mocking.

The most useful way I have heard of explaining this is from a talk by Simon Stewart where he talks about having small, medium and large tests. A small test lives entirely in memory, is single threaded, doesn’t sleep, and doesn’t access system properties. They should ideally be entirely self contained and able to be run in parallel without any restrictions on ordering, and each one should take less than a minute to run.

A medium test is given much more free reign. Its only restriction is that it cannot touch anything not running on localhost, i.e. any network operations should run on loopback. It should take no more than five minutes. A large test has none of these restrictions.

From this it should be clear that your small tests are almost certainly unit tests of some description. They will act entirely on one unit of code, and any other units of code they interact with should be mocked out with the behaviour they expect to consume. Because they run so fast, you can afford to set these up to run on every commit, and wait for the results before you continue to write code that corrects any errors.

Your large tests are going to be full end-to-end system tests, but might also include integrations with third party services, such as AWS. These are going to be slow. You might choose to use your continuous integration server to run these, or you might run them nightly. You might also choose to include your manual tests in this category, and I find it a perfectly acceptable, or in fact desirable, first step into BDD to have your behaviour specifications executed by a manual tester.

Your medium tests therefore might be assumed to be integration tests, and in some cases they will be, however by mocking out everything non-local which your user interface interacts with, you can make these medium tests too, and if you have a behaviour specification for the units this interacts with, you should have no less confidence in the results of this test. It is true that there is value in having end-to-end tests, but this can give you a much earlier indication of any issues your changes may have introduced or resolved on the system level. With up to a five minute execution time you’re probably not going to wait for them to run before you carry on coding, but you should definitely wait before merging your work to somewhere it will affect others.

Leave a Reply