"There's not much point in spinning a yarn if your audience keeps losing the thread"
P.K. Shaw
Who are you writing tests for?
The boss?
That's a terrible reason for doing pretty much anything. Anyway, you're not writing tests for the boss.
I don't mean, who asked you to write the tests. I mean who are you writing them for? The boss might have a vested interest in ensuring the tests are written, but that's not your target audience.
Ok. I'm writing them for myself
Really? Have you been reading self-help books again? I've warned you about that.
Next you're going to tell me that you're testing "in the now"....
Alright, I give up. Who am I writing the tests for?
It's not a trick question. Who is actually going to read your tests?
Well, since you put like that ... no-one else reads my tests
Really? Never?
Well, I suppose if they start failing ...then someone would look at them
Yes....
... but it might be years before that happens
Is the penny dropping yet?
...And I don't know who that will be. I might be working on something else or even a different company
That's who you are writing the tests for.
But, I just said ... I don't know who that will be
Nevertheless, that's who you are writing your tests for.
The unknown tester?
That has a certain ring to it. Yes, you're writing your test for the tester or testers that will follow you. You will probably never meet him/her, but that's your audience.
Ok, glad we cleared that up... is there a point here somewhere?
Yes, several.
1) The lifetime of your tests should match the lifetime of the system under test
2) The system under test might be around for 5, 10, 15 years
3) You probably won't be....
Naturally your test is running on a regular basis over this period. Maybe even every night.
Every time it runs; it passes. And then one day... boom ...the test fails
It did its job. It caught a problem
Correction... it did half of its job. Identifying the fact that there is a problem is a job half done; the crucial task of identifying what the problem is remains.
And the task of interpreting that failing test falls to our "unknown tester".
What's that got to do with me? Your point is that I won't be around...
Your job is now. You need to design the test now so that when it fails in the future it is a) clear to others what the expected behaviour was and b) strongly indicate what has likely to have gone wrong.
That sounds laudable. But what if I don't? I won't be around when it goes wrong
Then you'll be like the majority of testers. Just want the tests to pass so they can move on to something else ...
Ok. I accept I should aim higher. What can I do about it?
The first step, and the biggest step, is to recognise that you are writing something that will be read by someone else, at some point in the future, when you are not around.
Step one is therefore to know your audience.
What can you reasonably expect this future "you" to know? What might they not know, that you take for granted? How are you conveying that knowledge to them?
You mean documentation?
Possibly, but I'm thinking more of test design. Are you structuring your tests for root cause?
I don't even know what that means....
I'll take that as a "no", then.
When one test fails, often many tests will fail; one underlying bug manifests itself in many ways.
The question to ask yourself is, does the way I have structured my tests help highlight the underlying bug; the root cause?
If you have a test that checks multiple outcomes in one go, then it is not structured for root cause. When it fails, who is to know what has really gone wrong?
You saved yourself five minutes checking everything in one go, but will cost the unknown tester many, many wasted hours trying to unpick what has happened.
ok. So I could do a better job of that...
And will our unknown tester understand the intent of your tests? If your tests boldly assert, say, that you expect some price to be £123.45..... what is our friend going to make of things when one day the actual answer is £127.95?
What would you do?
Sometimes I think that industry standard procedure is to comment out the failing test....
Too true.
But you want to do a better job....Where did that number come from? Why should it be that number and not something else?
If you don't know, then you're not a tester. We're supposed to know what the system should be doing.
Assuming you do know why that's the right answer, how is the future tester suppose to attain your level of understanding?
You can help them by putting yourself in their position. Knowing your audience.
Can you give me an example?
Maybe the reason the expected answer is £123.45 is because that's what you get if you multiply a volume by a unit price, or if you add some other prices together. If that's the case, then write that in your test. Don't just jump straight to the answer. Structure for root cause and make clear where the expected answer came from and help the poor soul out.
If you were reading someone else's tests, which of these two would you prefer to be looking at when the boss want's to know what's broken?
Version 1)
blah blah.
AND the total price is £123.45
Version 2)
blah blah
AND the unit cost is £90
AND the tax rate is 20%
AND the postage rate is £15.45
AND the total price is £ (90 * 1.2) + 15.45
It turns out that the reason the price is different is because the system has calculated the tax rate incorrectly and that manifests as an incorrect price. The root cause is the tax rate. The test was structured for root cause.
I wish people had thought of me when they wrote their tests. I've got stuff failing and I have no idea why
Have you thought of just commenting them out?