|
|
|
|
|
|
|
|
|
|
|
|
||
|
More times than I can count, I have heard someone say "We're trying for XX percent code coverage with our unit tests". If XX is 100% are you guaranteed to have no defects? The first thing that comes to my mind when people start rambing on about unit tests is: What about integration and system testing? There are a number of blog entries out there that talk about the difference between the types of testing. It suffices to say that pure unit testing, even at 100% code coverage, is only going to reveal a small percentage of an application's defects. So what are you actually trying to do? Test for a reason -- don't just test to test. You can get into a cycle where you're writing unit test apon unit test and still releasing applications that are defect riddled. First you need to define a quality metric (a way to measure the quality of the application). "As few bugs as possible" isn't it! Ask yourself questions such as: What's a bug or defect? and How is the priority and severity of a defect determined? (e.g. Are spelling errors bugs? They may not be to a developer, but they may be show stoppers to a client.) Everyone wants to jump in and test up the wazoo but the problem is, if you test the wrong things and you don't know what a bug or defect is then your testing is potentially wasting time. Let's put this another way: let's say you have 80% coverage with all of your tests. What's preventing that 20% that you're not testing from containing 90% of the defects and 100% of the P1 show-stoppers?
Your goal is to identify the cross product of the areas of highest quality risks and the areas that are most important to the user of the application. These areas must have enough testing to ensure that the desired level of quality is met. For example, what if the 20% that you did not test just so happens to be the login page? The client cannot log into the application! So what's the point of testing or even writing the rest of the app? This is an obvious overtrivialization as the login page is an obvious element to test. But in reality, the show stoppers commonly fall into a trivial category. A number of projects that I've performed triage in the past suffered from this problem to an alarming degree. Months of work and testing when into components of the product that, at the end of the day, were not high on the clients list and were overshadowed by trivial defects. Plan ahead. Identify what determines quality in your application. Test effectively. |
| Comments | ||||||||||||||||||||
|
|
Unless otherwise expressly stated, all original material of whatever nature created by Rob Grzywinski and included in this weblog and any related pages, including the weblog's archives, is licensed under a Creative Commons License. |