Tips on writing bug reports

From Lazarus wiki
Revision as of 15:52, 3 July 2010 by Michael (talk | contribs) (Why you should spend more time on bug reports)

Why you should spend more time on bug reports

First of all, thank you for using Free Pascal and Lazarus, and thank you for your help.

That is what reporting bugs is: helping the project further. On most open-source projects your bug reports are handled directly by an experienced developer and not by some help desk employee who can spend a lot of time on the bug report to make it perfect, before it goes to a real developer. So spend some time on your reports so that:

  • For the project: More bugs are solved each week
  • For the submitter of bugs: His bugs are fixed sooner
  • For the developers: They can do more for the project in the same amount of time

The time spend in making a good bug report is not lost, it is time invested by you in the open source project you are using. So please take some time to read this carefully and think about this tips when sending in bugs.

The four most important things when writing bug reports

The four most important things you have to think of when submitting bugs are:

  • 1. report only one bug in each report,
  • 2. explain what the problem is,
  • 3. do not mix the explanation of the problem with solutions, thoughts and other guesses

and finally

  • 4. consider the people who are reading the report as complete computer-nimbo's.

This sounds easy but when you scroll though the bug reports you soon conclude that a lot of reporters have problems with it. Submitting one bug in each report also means that you can not add a comment somewhere in the report that slightly introduces another bug. This way the developer can not close the bug when it is in fact fixed, because he needs a reminder of that remark. But each time the bug is scanned, that remark has to be identified and checked. Very time-consuming. And never, really never, submit lists of bugs in one report. If we wanted to have a list of bugs within one document, we would have used a wiki page where everybody could add bugs on the bottom. Also when the bugs are related, or when one bug must be fixed first before the second one can be fixed, do not post them within one report. You can add relations between bugs to show that and how they are related.

The most difficult part of creating a bug report, is explaining what the problem is. Before writing the report down, think about this. What is actually the problem? In general it can only be two things: you get an error message or exception you didn't expect. Or you get a result you did not expect. So describe the error or the result you got, and the result you expected. If the result or expected result is missing, realise that you did not describe any problem at all. Also something like "the memory of this variable is freed twice" is not a problem at all! The problem is that this may lead to an exception (access violation) or memory corruption, leading to other errors. Or, it can slow-down something because the second free is redundant. A way to check if what you describe is really the problem, is to add an example. An example to show that memory is freed twice is difficult. A program that raises the exception clearly shows the problem. Adding compilable, (not only parts but a complete program) and as small as possible examples is always a good thing. But do not make the fault to forget to explain the problem within the report because you've added an example. It makes a quick-scan of the bug-report nearly impossible.

Another big problem with writing down what the actual problem is, is that people tend to think that the developers are interested in the solution of the problem and not the problem itself. Well, without a problem there is no solution so the developers have to guess what the problem is, which is always a bad idea. Also a lot of times the solution is the wrong one and if the problem is missing this makes the bug-report useless. There is also the risk of steering the developer into the wrong direction which can really slow down the problem solving. So, as a general rule: no not add thoughts and guesses to the bug report. But if you really insist on doing so, do this separately from the explanation of the problem. So you can not 'forget' to explain the problem and the risk you steer the developer into the wrong direction is at a minimum. Even if you do have the solution or even a patch: first the explanation of the problem, then add a newline, start a new paragraph, and only after that give your probable solution, patch, thoughts or guesses.

Finally, write your bug report as if the developers you are writing it for are completely clueless. In fact they are clueless, since if they knew about the problem it probably would have been fixed. So describe the problem as you would subscribe it to someone who is calling the internet-provider helpdesk complaining that 'the internet is broken'. Never assume that the developer will know what you mean, so you can omit it. For example, telling that TSomeClass.Method always crashes is not enough. You may think that the developer knows that you have to create the class using TSomeClass.Create, before calling the method. But maybe you did miss that there is also a constructor called TSomeClass.CreateNew. What if crash doesn't happen if you use CreateNew, so that the developer can not reproduce your problem? Explain step by step how you got to the problem, so the developer can reproduce it.

Also do not forget this when adding examples. Try to give a complete example that is compilable. When you only give a procedure it could be that we miss that you have added some defines on top of your program. Or that you forgot to add those. So always send in complete examples. On the other hand, remove everything from the example that is unnecessary to show your problem. Maybe that by removing parts that seemed irrelevant at first even solve your problem. So you get a better understanding of what the problem actually is.

So now you know you have to write one bug report at a time, explain the problem and do not mix it up with guesses and random thoughts. And that the people who are reading bug reports should be considered not that smart. Although this are the most important parts, I'm not completely finished. Some last tips:

General tips

Give the bug-report a good name

A good name tells what the problem is. It should contain the following: 1) On which class/procedure/etc the problem is 2) What the problematic behaviour is 3) What you were doing when the problem happend Always mention 1. If you can not tell 2 and 3 in a short title, mention only one of them. Do not add anything else. If the three points above are included, that's always enough. So do not add the solution if you think you know it. It can point the developers into the wrong direction.

Fill in the extra information about OS, used versions etc

Always fill in at least which version of Free Pascal and Lazarus you used. (This are two different programs, so you have to give two version numbers) If you have used a snapshot give the explicit revision numbers or date of the snapshot. Also if you think that it doesn't matter, report it. What you think is not important, facts are important.

Also mention your operating system and version and your processor-type.