Chapter 5 Conclusion
5.3 Contribution of the thesis
The contribution of the research work has been divided into two broad categories.
First, the findings of the research investigation would be discussed in the light of the published literature in “test mode” affect studies and then practical implication of the findings would be discussed for the user interface designers. A detailed argument elaborating the research findings from the perspective of the Culture variable is presented in Appendix 4, page 239.
5.3.1 Theoretical contribution
When the results of the five experiments are viewed from the perspective of the past research literature, it highlights the following:
1. Past research literature in scrolling did not find a significant difference in performance (test scores) between groups with and without scrolling (Hewson et al., 2007;
Whitelock, 2009). In our study also, there was no significant effect of scrolling and without scrolling CBA presentation format on performance but the current study highlights that the subjective evaluation of these presentation formats are different for the two different student communities. While Indian students prefer without scrolling interface, American students on the other hand prefer with scroll interface. Now, given the case that the students’ were experts, the effects of the interface may be pronounced for novice students. Moreover, from the instructional efficiency point of view it was found that without scroll is preferred by Indian students more because it invoked less cognitive load in comparison to the American students who felt without scrolling CBA presentation format invoked high cognitive load. Moreover, the study conducted by Hewson et al., 2007 and Whitelock, 2009 had not carried out a cross cultural investigation.
In this thesis a cross cultural investigation was carried out and reported.
2. Navigation is important in a hypermedia based learning environment. It has been reported that navigational tools increase navigation efficiency, reduce the feeling of being lost and improve learning performance (Allison& Hammond, 1989; Dee-Lucas & Larkin, 1995;
McDonald & Stevenson, 1999; Puntambekar et al., 2003). But it may also impede the cognitive flexibility of the learners by imposing a simplified structure over the existing knowledge structure (Gall & Hannafin, 1994) and as such may not compel the learners to fully interact with
the hypertext environment ( Jonassen, 1986). The current investigation suggests that students in computer based assessment environment behave very differently than students in other components in hypermedia based learning environment. The results of the research investigation highlight that students in CBA do not get affected due to the navigational mechanisms namely step or vertical menu.
3. Past research investigations on item review and presentation found that performance are largely unaffected by whether examinees had the option to review, examinees prefer to have the option to review available (Leeson, 2006). But the current study highlighted that when customization features are used for facilitating item review and presentation students’
performance get affected.
4. White space (Screen density): The published research literature has limited findings and few studies investigated this feature. Moreover, they studied the effect of white space in comprehension based test item layout but almost none reported about the effect of white space in MCQ (Multiple Choice Question) format and how graphical elements embedded in the GUI to aid the students in a test format and thus reducing the white space may affect the students.
The current study investigates the issue of white space from this perspective and found that contrary to the effect of white space in comprehension based computer tests, in MCQ format based computer tests students’ do not get affected due to the reduction or increase in white space.
5.3.2 Practical implications of the findings
The findings of the investigations highlight important design heuristics for GUI designers of CBA. While scrolling is preferred more by Americans than Indians, for designing a CBA system the designers should note that Indian students prefer single item without scroll presentation. Screen density does not seem to affect students in a CBA environment; hence visual aids in the GUI may assist students’ performance. Indian students prefer simple, non- customizable interfaces whereas Americans prefer customized interfaces. These heuristics are helpful guides to the GUI designers in their efforts to minimize the effect of interface features on students’ performance.
5.3.3 Summary of the contribution of the thesis
The summary of the contribution of the thesis are listed below:
1. The thesis provides answers to the theoretical contradictions of the research literature and highlights the effects of interface features on students’ performance and other
Figure 5.1. A CBA GUI design example for the Indian students based on the research
findings of the thesis.
1 & 4 – Both step based and vertical menu based navigational mechanisms are used.
2. Single Item representation without scrolling 3. Maximum white space
Figure 5.2. A CBA GUI design example for the US students based on the research
findings of the thesis.
1 – Interactive elements for helping the students during assessment with reduced white space.
2 & 3 - Multiple Item representation with scrolling.
sensitive variables such as subjective cognitive load and perceived usability of the CBA system empirically.
2. It provides a verified frame of heuristics to Graphical user interface designers of computer based assessment systems to design the presentation format of test items both in
Indian as well as American cultural context. A CBA GUI design example incorporating the research findings has been illustrated in the figures 5.1 and 5.2.
3. It highlights the “presentation format effect” of various GUI features in a CBA system. Till now only “test mode effect” was known. This study for the very first time empirically investigated and documented the “presentation format effect” in addition.
4. This study provides the first cross cultural analysis of the effect of presentation format on test performance and the subjective evaluation of the CBA system by the students belonging to different cultural backgrounds (India & USA).