Part I Concepts
10.3 Analyse Your Data
algorithms do, however, seem to be significant for all points above 60,000 tuples.
In summary, this means that we can say that these experiments indicate that the new algorithm is advantageous for 70,000–100,000 tuples, but that neither an advantage nor disadvantage was found for smaller numbers of tuples.
It is of vital importance, before attempting to draw any conclusions from data obtained through experiments or simulations, that you apply a suitable test for statistical significance. Failure to do so means that you run the risk of drawing incorrect conclusions, and – of course – that your examiner rejects the conclusions you propose, since they have not been shown to be significant. Which significance test fits your data will depend on the experimental set up and the type of data you have generated, and this is too large an issue to discuss in detail in this book. You are therefore strongly advised to consult the statistics literature and to seek the advice of your supervisor on this important issue.
could be used as comparison criteria. You may even have come across already existing and well-used comparison criteria. If you have several potential comparison criteria, you need to justify why you include some of them in your set of criteria, and why you exclude the other ones. The result of this objective is your set of comparison criteria, which include different factors that will be evaluated against the collected data.
In the third objective, you should “compare the previous work with respect to the comparison criteria”. In order to fulfil this objective you need to evaluate the comparison criteria against the collected (and categorised) data items. The result of this objective may be to ascertain to what degree your collected data (i.e. the previous work) support the different factors that are part of your comparison criteria. For example, you may find that the earlier work was weak in terms of security issues, whereas recent work is strong regarding security issues.
10.3.2 Theory Oriented Projects
In a project with a strong theoretical emphasis, you may have extended or compared existing theoretical models, without testing them in practice. Let us consider the aim and objectives of the first example project in Sect. 5.2.2. The aim of this project was to “extend an already existing theory or model”. We assume that you have already collected, read and presented the relevant data (i.e. the existing theory or model).
According to the first objective, you should “identify the details of the extension”. In order to fulfil this objective you need to analyse the collected data (e.g. details of possi- ble extensions). In this example, you would identify which features should be included in the extension. You will then need to justify why some features are more relevant than others. The result of this objective is your detailed description of the extension.
In the second objective, you should “introduce the extension to the original theo- retical model”. In order to fulfil this objective you need to first introduce the extension to the original model. It further implies that you need to verify that the extension is correctly introduced. The result of this objective is a new theoretical model com- posed of the original theoretical model and the extension.
In the third objective, you should “compare the original theoretical model with the extended version”. In order to fulfil this objective you need to compare the two mod- els against some comparison criteria. The result of this objective may be to ascertain whether the new model has better support for some features than the old model.
10.3.3 Applied Projects
In an applied project, you may have conducted experiments and gained experience from them. Let us consider the aim and objectives of the example project in Sect. 5.2.3. The aim of this project was to “gain experience from applying a theoretical model”.
In the first objective, you should “set up a simulator for caching of web data”.
In order to satisfy this objective you need to design and implement the simulator. It further implies that you need to justify that the simulator is valid and correct with respect to existing baselines. The result of this objective is a simulator that can be used in your further experiments.
In the second objective, you should “implement the new algorithm”. In order to satisfy this objective you need to implement the algorithm by the aid of the simula- tor. You need to justify that you have implemented the algorithm correctly in the simulator. The result of this objective is an implementation of the new algorithm in the simulator.
In the third objective, you should “test and analyze the new algorithm”. This includes setting up relevant test scenarios and analyze them. Investigate the outcome of each scenario. Can you, for each scenario, explain the result of the experiment?
Furthermore, you should see if you can identify any patterns in the results of the experiments. It may be possible to make a categorisation of the results of the experi- ments. The result of this objective is your explanation and categorisation of the results obtained from the experiments.
In the forth objective, you should “identify refinements to the design of the algorithm”. In order to satisfy this objective you need to investigate the result of the third objective. Based on what you found in the third objective, can you suggest any refinements to the design of the algorithm? The result of this objective is your recommendation for refinements to the design of the algorithm.
10.3.4 A Comparison of Theory and Practice
As mentioned earlier, projects that combine theory and practice usually contrast the theory with current practice in companies or organisations. Let us consider the aim and objectives of the example project in Sect. 5.2.4. The aim of this project was to
“contrast the current theory relating to a particular subject with how companies and organisations support it in practice”.
In the first objective, you should “select companies or organisations” to be inves- tigated. You will also need to provide justifications for why certain companies or organisations are not part of your investigation. The result of this objective is your list of companies and organisations that should be investigated.
In the second objective, you should “select comparison criteria”. In order to fulfil this objective you need to select features of a particular subject and justify your investigation of them. This is needed in order to be able to do a comparison in objective number five. The result of this objective is your set of comparison criteria.
In the third objective, you should “investigate the details of the theory of the particular subject”. This implies that you evaluate the theory against the compari- son criteria. The result of this objective is to ascertain to what degree the theory supports the comparison criteria.
In the fourth objective, you should “investigate how companies or organisations work with the particular subject in practice”. In order to fulfil this objective you
10.3 Analyse Your Data 85
need to evaluate how the particular subject is supported in practice against the comparison criteria. The result of this objective is to ascertain to what degree the comparison criteria are supported in practice.
In the fifth objective, you should “compare the results obtained from the second and third objectives”. In order to fulfil this objective you need to evaluate the results obtained from the third and fourth objective. Are there any similarities or differences?
Can you explain the reasons for the similarities and differences? The result of this objective is your identification of similarities and differences between the theory and how companies and organisations support the subject in practice.