Another day, another bunch of tests not really coded to be rerunnable.
A smell: in order to fix tests you have to run some scripts to clean / setup database or environment. It’s wrong. Change the test, fix it so it doesn’t depend on any clean state. Make running test a pleasant experience – let the other devs run it from their favorite IDEs as many times as they want, let them see a green bar whatever state application has or whatever data it has.
Now the question is: to clean up on start, on finish or both? Well… I don’t really care. Just make it rerunnable.
Hold on… actually, I prefer not to clean up after test… Let’s think about some tests hitting database: Why not to clean up after every test?
- I quickly find issues with other tests if they fail because there’s been some data left from other tests (e.g. it means that tests are not independent and rely on successful execution of clean-up logic in some other place).
- I have some data after execution of single test – that sometimes helps in debugging, seeing what happened (especially when using languages/environments where debugging is a pain)
- I can run test and have some test data left for demoing functionalities
- Less coding & smaller code base (nobody, I say nobody can beat this argument =)