Handling bugs is never an easy task. Some are not even meant to be logged while some can not be fixed without blowing up the whole system. Some of the books I've read described great ways to deal with defects. Most suggest to record them to a DTS, write some reproduction steps and wait for the programmer team to fix it. This approach works well and gives a good numbers to generate trends. I might be a bit obsessed with automation and TDD though, so I've started using something else.

What is a bug in this context?

A bug is a piece of behavior that doesn't work as expected from the user's point of view. The key points in the definition above are that we're only talking about defects that can be reproduced from the user interface only. Why? Because no one cares about the unseen.

So you've caught a bug!

Great! You probably have some notes about how to recreate the defect. You are aware of what it should be doing instead of what it is doing now. You'll probably record it to a bug tracker, but try this:

Create an automated test!

All levels of testing require inputs and expectations. Even the smallest micro-test that drive core methods rely on expected output from a known input. In the current scenario you'll have steps as inputs and you have expectations how your user interface should respond to those. For a primitive demonstration lets look at the following bug report:

Description:

A logged in user of the system cannot change the registered e-mail address!

Steps to reproduce:

1. Open the site.
2. Log into the site.
3. Click the "Account Settings" link.
4. Change the e-mail address via the settings form.
5. Click "Save" button.

Expected behavior:

The pre-populated e-mail field should contain the new e-mail.

Current behavior:

The new e-mail address is not saved, the old one is visible after saving the form.

In your automated test script you should be able to easily re-create the steps above. The interesting part are the expectations. The just enough assertions to consider the bug fixed depends on the complexity of the error. In our case it's a pretty straight-forward conclusion: assert that the e-mail field after submitting the data holds the e-mail address we've entered before. Having test like these is pretty nice if you make them part of your regression suite. It helps you target error prone parts of your system and if such test case is present you can be sure you'll be notified if the bug resurrects.

But how to handle these before the bug is fixed?

Create a test suite for bugs. With TestNG you can easily manage test groups. I prefer having a group named Bugs and one called Regression. The latter contains only bugs that are fixed. The tests in the former are executed via a nightly run for example. Each run will report the number of tests failed only in this case it will mean: the number of bugs still alive! If you hook these up with a CI server you can get nice little trends showing the number of living bugs over time. It works for us! Leave your thoughts if you have tried this approach, I'm extremely curious!

How many bugs live in your system?