15 Empirical research in software engineering

15.1 Problems with empirical research

Because it is important to base your technology and business decisions on fact, not opinion it is recommended that you ask the following questions about each claim you hear:

• Is it based on empirical evaluation rather than intuition and advocasy?

If it is supported by empirical evaluation:

• does it have good experimental design?
• is it a toy situation or a real situation?
• are the measurements appropriate to goals of the experiment?
• was the experiment run long enough?

15.2 Investigating products

Based on the Examples 15.1 and 15.2 on maintenance the following conclusions can be made:

• establish hypothesis
• rather collect data than use historical data
• perform appropriate analysis

Examples 15.3-15.7 are examples of research trying to use models to forecast faults. Based on these we may conclude:

• the hypothesis should be clearly defined
• define a measure for the independent variable
• the investigation should be general enough for the results to be transferable
• there should be some underlying explanatory theory of behaviour governing the investigation