Difference between revisions of "Tips on writing bug reports"

From Lazarus wiki
Jump to navigationJump to search
m
(Layout, removed duplication (repeating stuff won't help people that don't read and only discourage other people) , emphasis, grammar)
Line 2: Line 2:
 
First of all, thank you for using Free Pascal and Lazarus, and thank you for your help.  
 
First of all, thank you for using Free Pascal and Lazarus, and thank you for your help.  
  
That is what reporting bugs is: helping the project further. Bug reports submitted to most open-source projects are handled directly by an experienced developer, as opposed to by a tech support employee who can spend a lot of time on the bug report to make it perfect before it goes to a real developer. Therefore, please spend some time on your reports so that:
+
That is what reporting bugs is: helping the project further.  
 +
 
 +
As happens with most open-source projects, bug reports submitted to FPC & Lazarus are handled directly by an experienced developer.
 +
This is opposed to what happens in many closed-source software: where tech support employees are paid to can spend a lot of time on the bug report to make it perfect before it goes to a real developer.  
 +
 
 +
Therefore, please spend some time on your reports so that:
  
 
* For the project: more bugs are solved each week
 
* For the project: more bugs are solved each week
Line 8: Line 13:
 
* For the project's developers: they can do more for the project in the same amount of time
 
* For the project's developers: they can do more for the project in the same amount of time
  
Time spent on making a good bug report is not wasted. On the contrary, it is time invested by you in the open source project you are using. So please take some time to read this carefully and think about these tips when reporting bugs.
+
Time spent on making a good bug report is not wasted. On the contrary, it is time invested by you in the open source project you are using. So please take some time to read this carefully and think about these tips when reporting bugs.
  
 
= The four most important things when writing bug reports =
 
= The four most important things when writing bug reports =
Line 19: Line 24:
 
* 4. '''be complete and always include a fully compilable example if possible'''
 
* 4. '''be complete and always include a fully compilable example if possible'''
  
This sounds easy, but when you scroll though the bug reports it soon becomes apparent that it is easy to sin against these guidelines.  
+
This sounds easy, but when you scroll though the bug reports it soon becomes apparent that it is easy to deviate from these guidelines.  
  
 
==Report only one bug in each report==
 
==Report only one bug in each report==
  
Only one bug per report also means that you should not add comments mentioning other potential bugs. The reason is that other the developer cannot close the bug once it is fixed, because he needs a reminder of that remark. However, each time the bug is scanned that remark has to be identified and checked, which is very time-consuming. When two separate but related bugs are submitted, we can however relations between the bugs in the bug reporting tool so that the context can be easily looked up. Feel free to mention the numbers of related bugs in your comments.
+
Only one bug per report also means that you should not add comments mentioning other potential bugs. The reason is that the developer cannot close the bug once it is fixed, because he needs a reminder of that remark/other issue. However, each time the bug is scanned that remark has to be identified and checked, which is very time-consuming.  
 +
 
 +
When two separate but related bugs are submitted, we can however add relations between the bugs in the bug reporting tool so that the context can be easily looked up. Feel free to mention the numbers of related bugs in your comments.
  
 
Obviously, submitting a whole list of bugs in one report is completely out of the question.
 
Obviously, submitting a whole list of bugs in one report is completely out of the question.
Line 29: Line 36:
 
==Clearly explain what the problem is==
 
==Clearly explain what the problem is==
  
Explaining the problem is the most difficult part of creating a bug report. Before writing the report down, think it through: what is actually the problem? In general it can only be one of two things: you get an error message or exception you didn't expect, or you get a result you did not expect. Therefore describe the error or the result you got, but make sure to also mention the result that you expected.
+
Explaining the problem is the most difficult part of creating a bug report. Before writing the report down, think it through: what is actually the problem? In general it can only be one of two things: you get an error message or exception you didn't expect, or you get a result you did not expect. Therefore make sure to mention the '''result that you expected''' as well as describe '''the error or the result you actually got'''.
  
If the result or expected result is missing, realise that you did not describe the problem at all. Also something like "the memory of this variable is freed twice" is not a problem at all! The problem is that this may lead to an exception (access violation) or memory corruption, leading to other errors. Or, it can slow-down something because the second free is redundant. A way to check if what you describe is really the problem, is to add an example. An example to show that memory is freed twice is difficult. A program that raises the exception clearly shows the problem. Adding compilable,  (not only parts but a complete program) and as small as possible examples is always a good thing. But do not make the fault to forget to explain the problem within the report because you've added an example. It makes a quick-scan of the bug-report nearly impossible.
+
If the result or expected result is missing, realise that you did not describe the problem at all. Also something like "the memory of this variable is freed twice" is not a problem at all!  
 +
 
 +
The problem is that this may lead to an exception (access violation) or memory corruption, leading to other errors. Or, it can slow something down because the second free is redundant.  
 +
 
 +
A way to check if what you describe is really the problem, is to add an example. An example to show that memory is freed twice is difficult. A program that raises the exception clearly shows the problem. See below for more information on examples.
  
 
==Do not mix the explanation of the problem with solutions, thoughts and other guesses==
 
==Do not mix the explanation of the problem with solutions, thoughts and other guesses==
  
Another big problem with writing down what the actual problem is, is that people tend to think that the developers are interested in the solution of the problem and not the problem itself. Well, without a problem there is no solution so the developers have to guess what the problem is, which is always a bad idea. Also a lot of times the solution is the wrong one and if the problem is missing this makes the bug-report useless. There is also the risk of steering the developer into the wrong direction which can really slow down the problem solving. So, as a general rule: '''no not add thoughts and guesses to the bug report'''. But if you really insist on doing so, do this separately from the explanation of the problem. So you can not 'forget' to explain the problem and the risk you steer the developer into the wrong direction is at a minimum. Even if you do have the solution or even a patch: first the explanation of the problem, then add a newline, start a new paragraph, and only after that give your probable solution, patch, thoughts or guesses.  
+
Another big problem with writing down what the actual problem is, is that people tend to think that the developers are only interested in the solution of the problem and not the problem itself. Well, without a problem there is no solution so the developers have to guess what the problem is, which is always a bad idea.  
 +
 
 +
Also a lot of times the solution you kindly thought of is wrong. If the problem is missing this makes the bug report useless. There is also the risk of steering the developer into the wrong direction which can really slow down the problem solving.  
 +
 
 +
So, as a general rule: '''do not add thoughts and guesses to the bug report'''. But if you really insist on doing so, do this separately from the explanation of the problem (e.g. in the Additional Information section in the bug tracker). This also applies if you have a patch (which you are of course more than welcome to attach in the bug report).
 +
 
  
 
==Be complete and always include a fully compilable example if possible==
 
==Be complete and always include a fully compilable example if possible==
  
Finally, ''write your bug report as if the developers you are writing it for are completely clueless''. In fact they are clueless, since if they knew about the problem it probably would have been fixed. So describe the problem as you would subscribe it to someone who is calling the internet-provider helpdesk complaining that 'the internet is broken'. Never assume that the developer will know what you mean, so you can omit it. For example, telling that TSomeClass.Method always crashes is not enough. You may think that the developer knows that you have to create the class using TSomeClass.Create, before calling the method. But maybe you did miss that there is also a constructor called TSomeClass.CreateNew. What if crash doesn't happen if you use CreateNew, so that the developer can not reproduce your problem? Explain step by step how you got to the problem, so the developer can reproduce it.
+
Regarding completeness: ''write your bug report as if the developers you are writing it for are completely clueless''.  
 +
 
 +
In fact they are clueless, since if they knew about the problem it probably would have been fixed. So describe the problem as you would subscribe it to someone who is calling the internet provider helpdesk complaining that 'the internet is broken'.  
 +
 
 +
Never assume that the developer will know what you mean, so you can omit it. For example, telling that TSomeClass.Method always crashes is not enough. You may think that the developer knows that you have to create the class using TSomeClass.Create, before calling the method. But maybe you did miss that there is also a constructor called TSomeClass.CreateNew. What if the crash doesn't happen if you use CreateNew, so that the developer can not reproduce your problem? Explain step by step how you got to the problem, so the developer can reproduce it.
  
Also do not forget this when adding examples. Try to give a complete example that is compilable. When you only give a procedure it could be that we miss that you have added some defines on top of your program. Or that you forgot to add those. So always send in complete examples. On the other hand, remove everything from the example that is unnecessary to show your problem. Maybe that by removing parts that seemed irrelevant at first even solve your problem. So you get a better understanding of what the problem actually is.
+
Adding '''compilable examples''' (not only snippets/some code listing but a complete program) that are as small as possible is always a good thing. Not only do these help see what the problem is, developers may also adapt these to (regression) test cases that can be run automatically, so it can be checked if the bug ever surfaces again.
 +
 
 +
Try to give a complete example that is compilable. When you only give a procedure it could be that we miss that you have added some defines on top of your program. Or that you forgot to add those. So always send in complete examples.
 +
 
 +
On the other hand, remove everything from the example that is unnecessary to show your problem. Maybe that removing parts that seemed irrelevant at first even solves your problem. So you get a better understanding of what the problem actually is.
  
 
= General tips =
 
= General tips =
  
So now you know you have to write one bug report at a time, explain the problem and do not mix it up with guesses and random thoughts. And that the people who are reading bug reports should be considered as being not that smart. Although those are the most important parts, below you can find a few more tips.
+
So now you know you have to write one bug report at a time, explain the problem and do not mix it up with guesses and random thoughts. And that the people who are reading bug reports should be considered as being not that smart. Although those were the most important things, below you can find a few more tips.
  
== Give the bug-report a good name ==
+
== Give the bug report a good name ==
  
 
A good name tells what the problem is. It should contain the following:
 
A good name tells what the problem is. It should contain the following:
Line 57: Line 81:
 
== Fill in the extra information about OS, used versions etc ==
 
== Fill in the extra information about OS, used versions etc ==
  
Always fill in at least which version of Free Pascal and Lazarus you used (these are two different programs, so you have to give two version numbers). If you have used a snapshot, give the exact revision numbers or date of the snapshot. Even if you think that it doesn't matter, report it. This information always matters. Also mention your operating system, its version and the architecture you are using (x86, PowerPC, ARM, ...).
+
Always fill in at least which version of Free Pascal and Lazarus you used (these are two different programs, so you have to give two version numbers). If you have used a snapshot, give the exact revision numbers or date of the snapshot. Even if you think that it doesn't matter, report it. This information always matters. Also mention your operating system (Windows, Linux, OSX,..), its version and the architecture you are using (x86, PowerPC, ARM, ...).
 +
 
 +
== See also ==
 +
* [How do I create a bug report] General information on using the bug tracker
 +
* [Creating A Patch] If you have modified the source code to implement a solution, this article helps you to add it to your bug report in the most efficient way, so that developers can add it to the main code as fast as possible
 +
 
 +
[[category:Debugging]]

Revision as of 11:57, 27 November 2012

Why you should spend more time on bug reports

First of all, thank you for using Free Pascal and Lazarus, and thank you for your help.

That is what reporting bugs is: helping the project further.

As happens with most open-source projects, bug reports submitted to FPC & Lazarus are handled directly by an experienced developer. This is opposed to what happens in many closed-source software: where tech support employees are paid to can spend a lot of time on the bug report to make it perfect before it goes to a real developer.

Therefore, please spend some time on your reports so that:

  • For the project: more bugs are solved each week
  • For the submitters of the bugs: their bugs are fixed sooner
  • For the project's developers: they can do more for the project in the same amount of time

Time spent on making a good bug report is not wasted. On the contrary, it is time invested by you in the open source project you are using. So please take some time to read this carefully and think about these tips when reporting bugs.

The four most important things when writing bug reports

The four most important things you have to think of when submitting bugs are:

  • 1. report only one bug in each report,
  • 2. clearly explain what the problem is,
  • 3. do not mix the explanation of the problem with solutions, thoughts and other guesses

and finally

  • 4. be complete and always include a fully compilable example if possible

This sounds easy, but when you scroll though the bug reports it soon becomes apparent that it is easy to deviate from these guidelines.

Report only one bug in each report

Only one bug per report also means that you should not add comments mentioning other potential bugs. The reason is that the developer cannot close the bug once it is fixed, because he needs a reminder of that remark/other issue. However, each time the bug is scanned that remark has to be identified and checked, which is very time-consuming.

When two separate but related bugs are submitted, we can however add relations between the bugs in the bug reporting tool so that the context can be easily looked up. Feel free to mention the numbers of related bugs in your comments.

Obviously, submitting a whole list of bugs in one report is completely out of the question.

Clearly explain what the problem is

Explaining the problem is the most difficult part of creating a bug report. Before writing the report down, think it through: what is actually the problem? In general it can only be one of two things: you get an error message or exception you didn't expect, or you get a result you did not expect. Therefore make sure to mention the result that you expected as well as describe the error or the result you actually got.

If the result or expected result is missing, realise that you did not describe the problem at all. Also something like "the memory of this variable is freed twice" is not a problem at all!

The problem is that this may lead to an exception (access violation) or memory corruption, leading to other errors. Or, it can slow something down because the second free is redundant.

A way to check if what you describe is really the problem, is to add an example. An example to show that memory is freed twice is difficult. A program that raises the exception clearly shows the problem. See below for more information on examples.

Do not mix the explanation of the problem with solutions, thoughts and other guesses

Another big problem with writing down what the actual problem is, is that people tend to think that the developers are only interested in the solution of the problem and not the problem itself. Well, without a problem there is no solution so the developers have to guess what the problem is, which is always a bad idea.

Also a lot of times the solution you kindly thought of is wrong. If the problem is missing this makes the bug report useless. There is also the risk of steering the developer into the wrong direction which can really slow down the problem solving.

So, as a general rule: do not add thoughts and guesses to the bug report. But if you really insist on doing so, do this separately from the explanation of the problem (e.g. in the Additional Information section in the bug tracker). This also applies if you have a patch (which you are of course more than welcome to attach in the bug report).


Be complete and always include a fully compilable example if possible

Regarding completeness: write your bug report as if the developers you are writing it for are completely clueless.

In fact they are clueless, since if they knew about the problem it probably would have been fixed. So describe the problem as you would subscribe it to someone who is calling the internet provider helpdesk complaining that 'the internet is broken'.

Never assume that the developer will know what you mean, so you can omit it. For example, telling that TSomeClass.Method always crashes is not enough. You may think that the developer knows that you have to create the class using TSomeClass.Create, before calling the method. But maybe you did miss that there is also a constructor called TSomeClass.CreateNew. What if the crash doesn't happen if you use CreateNew, so that the developer can not reproduce your problem? Explain step by step how you got to the problem, so the developer can reproduce it.

Adding compilable examples (not only snippets/some code listing but a complete program) that are as small as possible is always a good thing. Not only do these help see what the problem is, developers may also adapt these to (regression) test cases that can be run automatically, so it can be checked if the bug ever surfaces again.

Try to give a complete example that is compilable. When you only give a procedure it could be that we miss that you have added some defines on top of your program. Or that you forgot to add those. So always send in complete examples.

On the other hand, remove everything from the example that is unnecessary to show your problem. Maybe that removing parts that seemed irrelevant at first even solves your problem. So you get a better understanding of what the problem actually is.

General tips

So now you know you have to write one bug report at a time, explain the problem and do not mix it up with guesses and random thoughts. And that the people who are reading bug reports should be considered as being not that smart. Although those were the most important things, below you can find a few more tips.

Give the bug report a good name

A good name tells what the problem is. It should contain the following: 1) With which class/procedure/etc the problem is 2) What the problematic behaviour is 3) What you were doing when the problem happend Always mention 1. If you can not tell 2 and 3 in a short title, mention only one of them. Do not add anything else. If the three points above are included, that's always enough. So do not add the solution if you think you know it. It can point the developers into the wrong direction.

Fill in the extra information about OS, used versions etc

Always fill in at least which version of Free Pascal and Lazarus you used (these are two different programs, so you have to give two version numbers). If you have used a snapshot, give the exact revision numbers or date of the snapshot. Even if you think that it doesn't matter, report it. This information always matters. Also mention your operating system (Windows, Linux, OSX,..), its version and the architecture you are using (x86, PowerPC, ARM, ...).

See also

  • [How do I create a bug report] General information on using the bug tracker
  • [Creating A Patch] If you have modified the source code to implement a solution, this article helps you to add it to your bug report in the most efficient way, so that developers can add it to the main code as fast as possible