Over-testing: the data-driven edition

Learning to test at the right level of abstraction is tough. A common mistake involves the abuse of data-driven testing tools. Data-driven testing tools, like TestCase in NUnit, are convenient and powerful. You can write a lot of tests really fast, but if you're not careful, you can end up repeating yourself.

For example, suppose we define the following (woefully oversimplified) utility function to test if a string is a valid email address:

public static bool IsValidEmail(string email)
{
    return Regex.IsMatch(email, @"\w+@\w+\.com");
}

The test suite for this function should be complete. A data-driven approach works really well here:

[TestCase("a@test.com")]
[TestCase("1@test.com")]
//... lots more
public void TestValidEmail(string email)
{
    Assert.That(IsValidEmail(email));
}
[TestCase(null)]
[TestCase("not an email address")]
//... and so on
public void TestInvalidEmail(string email)
{
    Assert.That(IsValidEmail(email), Is.False);
}

Now, suppose we use this method in some application logic:

public void SendConfirmation(string message, string email)
{
    if (!IsValidEmail(email))
    {
        throw new ArgumentException("email");
    }
    SendEmail(message, email);
}

Awesome, let's test this bad boy... er, I mean here's the test we wrote before writing the method:

[TestCase("a@test.com")]
[TestCase("1@test.com")]
//... lots more
public void TestSendEmailIfValid(string email)
{
    SendConfirmation("test", email);
    //Assert sends email...
}
[TestCase(null)]
[TestCase("not an email address")]
//... and so on
public void TestSendEmailThrowsIfInvalid(string email)
{
    SendConfirmation("test", email);
    //Assert throws exception...
}

Okay, stop... do we really need to re-test every single valid and invalid email address? Data-driven tests make this easy, but that doesn't make it right. In this function, there are really only two paths that matter: an invalid email address throws an exception, and sending an email successfully. Let's try again:

public void TestSendEmailIfValid(string email)
{
    SendConfirmation("test", "validemail@test.com");
    //Assert sends email...
}
public void TestSendEmailThrowsIfInvalid()
{
    SendConfirmation("test", "not an email address");
    //Assert throws exception...
}

Mockists might even take this so far as to stub out the IsValidEmail function for the purposes of testing this function, but for this simple case, it might be overkill since IsValidEmail has no external dependencies to begin with.

In the real world, this kind of mistake is harder to spot. Generally, if you have an explosion of TestCase attributes covering all kinds of permutations, consider revisiting the code under test. There may be some opportunities to simplify both your code and your test suite.

JavaScript - apply invocation pattern part 2

In my last post I promised a real-world example of the apply invocation pattern. The truth is it was getting late, and I didn't feel like writing anymore. However, one of my coworkers called me out, most likely because he's a JavaScript ninja and wants to put me in my place. In an effort to deny him the satisfaction, I'll steal an example from people who are smarter than I.

If you use jQuery, you've probably used the each function at some point. The each function works something like this (what follows is a vast oversimplification, see here for the actual implementation):

jQuery.extend({
  each: function(array, callback) {
    for (var i = 0; i < array.length; i++) {
      var value = array[i];

      //Apply callback with "this" as the current value
      callback.apply(value, [i, value]);
    }
  }
});

Now we can use the each function like this:

var sentence = '';
var words = ['hello', ' ', 'world', '!'];
jQuery.each(words, function() {
  sentence += this;
});
//sentence == 'hello world!'

Things your parents never told you about JavaScript - Apply invocation pattern

So, yeah, it's been a while... okay, it's been more than a while. I'm sure my 8 followers missed me. Anyway, I recently had the opportunity to learn a whole mess of new stuff, and I wanted to start writing some of it down. I had a 3 week fling with Ruby, followed by a longer affair with JavaScript. I was a static language guy for the last 3 years, so all this dynamic nonsense had my head spinning... and you know what?

I liked it.

I always thought of JavaScript development as painful; a sort of necessary evil. Turns out there's an obscenely powerful language there, if you go looking for it (incidentally, start by looking here). Among those powerful features are first-class functions. For example, suppose I run the following code:

var greeter = {
  name: 'Mike',
  greet: function() {
    return 'hello ' + this.name;
  }
};

greeter.greet(); // -> 'hello Mike'

Nothing shocking there, right? Well, I can actually reach into the greeter object and steal the greet function:

var greet = greeter.greet;
greet(); // -> 'hello '

Okay, kind of cool, but not quite the expected result. Where did my name go? When you grab a function that references this and invoke it, you need to tell JavaScript what context you want to execute it in. The built-in JavaScript Function object has a method called apply which lets you set the value of this (it also lets you pass an argument array if you want to):

var bob = {
  name: 'Bob'
};

greet.apply(bob); // -> 'hello Bob'

This is called the apply invocation pattern. What can you do with it? That'll have to wait for another post...

Re-throwing exceptions in C#

What is the difference between the following two blocks of code?

// #1
void HandleException()
{
    try
    {
        //Do something...
    }
    catch (Exception e)
    {
        throw e;
    }
}

// #2
void HandleException()
{
    try
    {
        //Do something...
    }
    catch (Exception e)
    {
        throw;
    }
}

Both blocks catch and re-throw an exception. The first block re-throws the caught exception, the second block does not specify anything to throw. The main difference between these blocks becomes apparent when the exception is logged higher up in the call stack. The stack trace for the first code block will be missing anything that came before the throw. The second code block will contain the full stack trace. This can be the difference between life and death when trying to diagnose an issue in production.

Castle Windsor: Order Matters

I've been learning a lot about Castle Windsor lately. I ran into something that surprised me yesterday. Suppose you have the following class hierarchy:

interface Service
{
}

class ServiceA : Service
{
}

class ServiceB : Service
{
}

What do you think the output from this code will be?

var container = new WindsorContainer();

container.Register(Component
    .For<Service>()
    .ImplementedBy<ServiceA>());
container.Register(Component
    .For<Service>()
    .ImplementedBy<ServiceB>()
    .Named("serviceb"));

var service = container.Resolve<Service>();

Console.WriteLine(service.GetType().Name);

If you said ServiceA, then you were right! Now, what about the output from this code?

var container = new WindsorContainer();

container.Register(Component
    .For<Service>()
    .ImplementedBy<ServiceB>()
    .Named("serviceb"));
container.Register(Component
    .For<Service>()
    .ImplementedBy<ServiceA>());

var service = container.Resolve<Service>();

Console.WriteLine(service.GetType().Name);

If you said ServiceB, then maybe the title of this post gave the answer away. I expected the named service to be a special case. In other words, I thought asking for a Service implementation would always give me ServiceA, unless I specifically asked for ServiceB by name.

ASP.NET MVC XML Model Binder

Update 12/14/2011
This is one of my most highly viewed posts. Unfortunately, this post applies to version 1 of ASP.NET MVC (it might also work with version 2 with some small modifications). If that is what you are looking for, then please read on! Otherwise, for more relevant coverage of this topic I recommend Jimmy Bogard's post

I needed a way to receive raw XML in my action methods, rather than form-encoded values. Model binding makes it a piece of cake:



In my action method all I have to do is apply the model binder attribute:



Now the edit action receives an XML string. Even better, I can get a strongly typed object if it is serializable:



Now my action method looks like this:



The model binder attribute is a little too verbose for my taste, so I wrote a custom model binder attribute:



Finally my action method is nice and readable:

"Good Enough" Test Automation

If you are practicing Agile and you want to stay releasable at the end of every iteration, you must automate your tests. If you do not automate, one of two things will happen:

1. Your velocity will decrease each sprint as you need to reserve more and more time for regression testing.
2. You will not run (or will forget to run) regression tests, and you run the risk of breaking something unintentionally. Best-case, your testers will catch any problems before release. Worst-case, your customers will find problems after release.

Now, here's the fun part: you have 2-3 weeks to build and test something potentially releasable. How can you possibly squeeze test automation into that time frame? The same way you attack that next killer feature: start small and iterate.

A few days ago I was testing some changes to a multi-step online signup process. The current code base is, shall we say, less than testable at the unit level. As a result, all testing for this feature has been manual in the past. After a few manual test runs, introducing some automated tests at the browser level seemed like a good idea. I immediately thought about continuous integration, but quickly realized dealing with all the necessary data setup and tear-down was going to take several days to get right. All I needed at that moment was a script to fill out the forms and click through the whole process. I coded a short Watir script, ran it a few times, and visually verified the process still worked end-to-end.

If all you need is a script to do the boring, repetitive parts of testing for you, write the script. If the script is useful, you'll keep coming back to it, and eventually get it running under continuous integration. Just take the first step, choose a tool, and start automating.