Steve Fenton

<< June | July | August >>

Monday, 2nd July 2012

Why do you use unit tests? To get an understanding of the problem before you code the solution? To drive the design of your code? To make sure your code works when you refactor it? To help other developers understand the intention of your code?

Maybe you just do it to get a good number in your code coverage tools?

If you want your tests to give you the most value for the investment you make in writing them, following the "test one thing" rule can really help. Here is a real life example of a test that tests too many things:

[Test]
public void ValidateServerName()
{
    Assert.AreEqual(false, detailValidation.ValidateServerName(""));
    Assert.AreEqual(true, detailValidation.ValidateServerName("abcd0"));
    Assert.AreEqual(true, detailValidation.ValidateServerName("test14"));
    Assert.AreEqual(true, detailValidation.ValidateServerName("nottoolong1111"));
    Assert.AreEqual(false, detailValidation.ValidateServerName("toolong111toolong111toolong111toolong1114321"));
    Assert.AreEqual(false, detailValidation.ValidateServerName("-!~//$^^"));
    Assert.AreEqual(true, detailValidation.ValidateServerName("blahblahblah.my"));
    Assert.AreEqual(true, detailValidation.ValidateServerName("blah0blah-blah.my"));
    Assert.AreEqual(false, detailValidation.ValidateServerName("-blahblahblah.my"));
}

It doesn't help that the items expected to pass and the items expected to fail are interspersed with each other. It also doesn't help that the developer used Assert.AreEqual instead of Assert.IsTrue and Assert.IsFalse but even with those changes this is still a poor test.

If you are thinking that maybe test cases would help, I'm sorry to say you are still wrong.

The problem here is that you don't know what is being tested in each of these cases and if one of these scenarios breaks, you cannot quickly tell what type of validation is causing the problem.

If you look closer at the tests, you'll see that there is validation of the length of the server name, but are the boundaries properly tested? (i.e. if the minimum length is 5 characters and the maximum is 20 characters, are there tests for 4, 6, 20 and 21?) You can also gauge that although special characters are mostly not allowed, some are allowed - but which ones?

The aim of the unit testing game is not to write the least number of tests, but to write as many tests as you need to demonstrate the intent of the code.

I would rather see the following tests in place of the above example:

[Test]
public void ValidateServerWithBelowMinimumLengthPasswordExpectFalse()
{
    const string server = "a";
    var result = _target.ValidateServer(server);
    Assert.IsFalse(result);
}
[Test]
public void ValidateServerithMinimumLengthPasswordExpectTrue()
{
    const string server = "aB";
    var result = _target.ValidateServer(server);
    Assert.IsTrue(result);
}
[Test]
public void ValidateServerWithMaximumLengthPasswordExpectTrue()
{
    const string server = "aaaaaaaa10aaaaaaaa20aaaaaaaa30aaaaaaaa40...";
    var result = _target.ValidateServer(server);
    Assert.IsTrue(result);
}
[Test]
public void ValidateServerWithOverMaximumLengthPasswordExpectFalse()
{
    const string server = "aaaaaaaa10aaaaaaaa20aaaaaaaa30aaaaaaaa40....";
    var result = _target.ValidateServer(server);
    Assert.IsFalse(result);
}
[Test]
public void ValidateServerWithValidSpecialCharactersExpectTrue()
{
    const string server = "a.b";
    var result = _target.ValidateServer(server);
    Assert.IsTrue(result);
}
[Test]
public void ValidateServerWithInvalidSpecialCharactersExpectFalse()
{
    const string server = "a!b";
    var result = _target.ValidateServer(server);
    Assert.IsFalse(result);
}