Unreproducible bugs can easily make a tester’s existence painful. Too often testers find a bug, then report it and get the answer “This is not a bug – I can’t reproduce it.” Still, the bug may be there, awaiting its next victim.
Unreproducible bugs can be very much expensive because of their increased lifetime and investigation time. They may as well have a quite damaging effect on your product perception in case the users reporting such bugs are ignored. Therefore, testers should do better to prevent them.
Here are 10 things testers can do to reduce the number of unreproducible bugs even now.
- Regularly do stress testing of your system unless you want to run across some unexpected failures when it is heavily loaded.
- Test timeouts. Make tests which fake or mock dependencies for timeout code testing. If the timeout code works wrong, it can cause a bug appearing only under certain conditions of the system.
- Test with optimized and debug builds. It happens that a good debug build works right, but your system fails in some strange ways as soon as optimized.
- Test with constrained resources. It’s a good practice to simulate the reduced network bandwidth as well as minimize the number of data centers, threads, machines, processes, available disk space and memory.
- Test for longevity. There are some bugs that require a long time to expose themselves. For instance, persistent data can become corrupt over some time.
- Utilize dynamic analysis tools such as memory debuggers regularly. They help identify numerous categories of unreproducible threading and memory issues.
- Perform fuzz testing to check your system’s hardiness at the time of enduring bad data.
- Always test the error handling code. Usually this is best achieved by mocking and faking the component which triggers the error.
- Remember to test concurrent data access when it’s the system’s feature. And if it’s not, verify that your system rejects it.
- Save the test logs for later potential analysis.
These seemingly obvious and not so obvious software testing guidelines will help you reduce the chances of unreproducible bugs to occur in your project.