The results of the survey informed the questions for the focus group interviews. The following sections describe the participant demographics and report on the results of the interviews, based on the identified usability and UX constructs (see Figure 3-14).
6.6.1. Focus Group demographics
Five focus group interviews were conducted with a total of 32 teachers from 8 different faculties of one private school, as depicted in Table 6-16.
177
Table 6-16: Focus group demographics
Faculty Number in
faculty Number participated in
survey Number at focus group
Commerce, Business
and IT 9 4 1 x Accounting
2 x Business Studies 1 x IT
Humanities and Arts 8 5 2 x Geography
1 x Theory of Knowledge 1 x Music
1 x Art Languages (English,
Afrikaans, French) 17 7 2 x English
3 x Afrikaans 2 x French
Mathematics 11 8 7 x Mathematics
1 x Mathematical Literacy
Science 12 8 3 x Life Sciences
5 x Physical Sciences
Many of the initial respondents to the survey were unable to attend the focus group interviews owing to time constraints. The survey was carried out at the end of the first term and the focus group interviews were conducted at the beginning of the second term. In this time, one teacher had left the school. Two teachers simply forgot to attend. One teacher who had accepted the invitation tested positive for COVID and was therefore unable to join the focus group.
6.6.2. Focus group results
The results obtained from the focus group interviews are reported on in the following sections, based on the pragmatic and hedonic qualities that affect adoption decisions. Pragmatic qualities are discussed in terms of the usability constructs of ease of use, flexibility, and learnability.
Hedonic qualities are discussed in terms of aesthetics, collaboration and the novelty and stimulation of the eModeration system.
6.6.2.1. Ease of use
When asked about their overall experience of the prototype, most participants immediately commented on how easy it was to use, as is evidenced by the comments: “I loved it; I found it very easy to use” (H1) and “As long as things are clearly labelled, and you know where to click. I found the instructions useful; I knew exactly where to go and the user interface was quite intuitive in
178
terms of its design, it made sense, it wasn’t frightening, it was logical. It seemed familiar even though it was completely new” (C1).
Ease of use was the most common response across all focus group interviews in all subject areas.
6.6.2.2. Flexibility
The results concerning the flexibility of the system are reported by considering the subject-specific requirements, changing curricula, the ability to access the system on portable devices, the ability to annotate using additional input devices, and the file types.
Considering the subject-specific requirements, practical subjects had very different requirements from other subjects. Subject-specific requirements influenced how teachers viewed the interface and how they approached the moderation process. The size of the moderating window was an important consideration for some subjects. For instance, L2 indicated that the system was not flexible due to the “Size of window; when you have a diagram and you have questions on the diagram, then having to go up and down for moderation. Questions are very long”. The ability to correct or include additional solutions was an issue for subjects such as music where annotations on specifically lined sheets were necessary, as is evidenced by the comment: “just personally with a very practical subject I found it very difficult to do on the screen. That was one thing that I grappled with” (H3).
Considering changing curricula, teachers believed that additional functionality would enhance the flexibility of the system to adapt to possible changes in the curriculum, as is evidenced by the comment: “if there is a change in syllabus or change in content, then it becomes necessary to, e.g., add diagrams. The application should be able to adapt to these changing requirements” (M8).
Considering access on portable devices, participants wanted to easily view the document on a smaller screen and wanted access to documents from devices such as iPads, as is evidenced by the following comments: “The iPad is one of the devices that you could use this on with a stylus and a pencil so it makes it more accessible to a wider group of people” (H2).
179
Considering the ability to annotate using additional input devices, participants indicated the need to annotate documents using an Apple Pencil (“could not use an iPad or pencil” (I1)), and to easily scribble using touch screen functionality (“Handwriting input – make that more accessible like an iPad or touchscreen laptop so that we can scribble more efficiently” (M6)).
Handwriting-type functionality was a recurrent theme amongst all faculties, with participants from Mathematics emphasizing the ability to “scribble”. Additionally, participants wanted specific typesetting for mathematical annotations, as is evidenced by the comment: “Maths typesetting – we can type and make comments, but the typesetting is not there” (M7).
Considering file types, participants indicated that they wanted the facility to upload documents of different file types so that documents were easier to edit, as is evidenced by the comments:
“it needs to incorporate more file types. Maybe even the ability to upload movie clips at a later stage like for I don’t know, children performing you know to incorporate more practical elements to subjects” (H5) and “Because it is a pdf, I can’t adjust it on the pdf. Any changes made on the document must be transferred to the original document” (I1).
6.6.2.3. Learnability
Considering the learnability of the system, all participants indicated that the system was easy to learn (“It wasn’t loaded with so many things that it became overwhelming” (M6)), easy to navigate (“I liked the idea that you go from one to the other. It is top down, it’s not like you go from this menu to that menu …, you just go”) (M8), and that they were able to easily complete the moderation “Because I am technologically challenged, as long as I could follow, I’m perfectly fine. It did that” (C2).
6.6.3. Factors influencing adoption decisions
The factors affecting decisions to adopt an eModeration system centred around the utility of the system, the ease of use and learnability, user characteristics, time, environmental impact, stakeholder buy-in, and the nature of assessments.
Participants were unanimous that utility would be the most important factor affecting their decision to adopt an eModeration system, as is evidenced by the responses “Utility is most important” (C1) and “Less is more. It fulfilled the function that you wanted” (M6).