5.3.2. Results from group activities: Workshop One
5.3.2.3. Requirements of an eModeration system
As explicated in Section 2.5.1.6, the stance taken in this study was to define requirements as the functional and design expectations of an eModeration system. Accordingly, the responses to questions pertaining to the need for an eModeration system, the improvements and functionality to include, the features of an eModeration system, and the requirements are presented together.
The dominant themes pertain to the facilitation of the moderation process, communication, and hardware and system requirements.
5.3.2.3.1. Facilitation of moderation
A key reason identified for using an eModeration system was to facilitate the process of regional and national moderation of student portfolios. Examples of participant responses included: “it makes it easy to get submissions” (C1) and “to simplify the process of portfolio moderation” (B3).
The recurring focus on convenience emphasized the importance of an eModeration system in facilitating moderation processes, thus easing the load on teachers and saving them time. The key
151
themes concerning the facilitation of moderation pertain to tracking, alerts and notifications, cost savings, and file management.
With respect to tracking, responses highlighted a need to track changes, track the moderation process, and track whether deadlines had been met. Considering tracked changes, the advantages to a digital moderation process included “Can see time/date + who made changes” (A4) and that tracking would “Show development of moderation process”.
Tracking the different versions of moderation documents would allow the moderator to
“Work on latest version-so no version errors” (A4). Concerning deadlines being met, participants mentioned a “stage by stage analysis” (B1) of completion of the moderation process and “Tracked deadlines” (D1) which would alert teachers and moderators of deadlines not being met. An emphasis on tracking the process demonstrates the need for a system that allows teachers and moderators to ensure that they are working on the most current version that contains an audit trail of the edits made.
The inclusion of notifications and alerts was the most frequent response to questions pertaining to functionality to include. Notifications were articulated in the context of alerting the moderator and assessment body if deadlines were not met; notifications when documents are uploaded, edited, and/or deleted; and when feedback from the moderation process is ready. Participant A1 indicated the need to “alert (the) teacher if required SBA and PAT requirements are not submitted within the required timeframe”, while participant A2 indicated that the person uploading the file should be alerted when the file is uploaded.
Notably, cost-savings was a common response articulated in terms of the benefits arising from a reduction in printing costs, the cost of transport, and saving of paper. Digital moderation eliminates the cost of sending hard copies “via courier” (C1), saves the time taken to print “loads of pages”, and saved paper. Digital moderation of portfolios provides teachers with the added benefit of saving valuable time from having to “travel to moderation venues set up by assessment bodies” (C2) to complete the moderation process.
Considering file management, facilitation of moderation pertains to centralized file storage and the related advantage of file sharing, ease of access, and having an organized file structure.
152
With reference to the need for centralization of documents, responses included “Easy, quick referencing”, “Things are available in one place”, and that centralization provides a
“Collective organization of data”. Centralized storage enables the synchronization and cross-referencing of moderated documents. This finding confirms the views of Booth and Rennie (2015) and Newhouse and Tarricone (2016) that eModeration ensures that moderated script(s) are electronically available for future reference. Considering file sharing as a result of centralized storage, 46.7% of the participants focused on the advantages of easily sharing and accessing files from a central location, as evidenced by the comment “can easily share a Onedrive folder which I have instant access to see what is in Onedrive folder” (D2).
Considering ease of access and flexibility, participants indicated that accessing files via a link to online storage provided flexibility and made “the organization of portfolio(s) easier” (C1).
Concerning an organized file storage, the dominant theme was to “Develop an e-portal where papers are uploaded” (C2) with participant D1 indicating that an “organizational setup” needed to be a “priority”. Participants further elaborated that they would like to have an organized structure with “pre-defined organized folders” (A1). Organization was also mentioned in the context of organizing the different stages of the moderation process according to priority levels.
5.3.2.3.2. Communication
Responses around communication were articulated in terms of the ease of communication, instantaneous feedback, and the provisioning of functionality for comments and communication between stakeholders of a digital moderation system. Concerning ease of communication, participants indicated that a digital moderation system facilitated communication as evidenced by D3, commenting that “Communication is quick and response time even quicker”. Notably, three of the four participants in group two mentioned the need for instant or “instantaneous” feedback as a reason for using an eModeration system. The focus on instantaneous feedback and ease of communication is consistent with previous findings that eModeration provides faster feedback
153
(Booth & Rennie, 2015; Newhouse & Tarricone, 2016), which resonates with the focus of this study.
Concerning comments and communication, a focal point was a database of customizable, pre- defined comments to enable the moderator to easily comment on assessed documents. Related features indicated by participants in Groups two and three were the ability to leave voice notes and the need for a live collaborative space within which moderators and teachers could chat with each other.
5.3.2.3.3. Hardware and system requirements
“Robust hardware specifications” and the need either for a larger screen or multiple screens were identified as requirements. Additionally, Internet connectivity, bandwidth, and platform independence were identified as requirements of an eModeration system. Participants emphasized the need for a stable Internet connection and questioned the effect of load shedding on connectivity. Responses to system requirements centred around the design of the interface, security, and help functionality, as described below.
For interface design, three of the four groups included user friendliness, ease of use, and intuitive design/interface amongst the required design elements. Participant D1 indicated that it would be “nice” to have “more features” while ensuring that the system is “user friendly”. However, D1 did not elaborate on what additional features would be nice to have. While participant A2 indicated that relevant documents should be available, no clarity was provided as to what documents were referred to. In contrast, participant C2 indicated the need for report writing and analysis features.
Security was the highest ranked theme by one group, and was identified by all groups on the idea web template. One group also required authentication for multiple users by means of secure email addresses and passwords. Confidentiality of individual school and student work was another important aspect of security.
Help: Participants highlighted the need for help functionality, mentioning a “ticket/help system” (C1), a “functional help system”, and “FAQs”.
154
Other features included a mobile application with cloud storage integration, platform independence, a checklist, report generation, format conversions, and a calendar with notifications. Two participants expanded on features for automatic changes to the mark allocation on the memo whenever the mark allocation on the question paper changed, and the self-generation of analysis grids. This study focused on moderation of assessments. Features related to creating an assessment, such as automatic mark allocation changes and self- generation of analysis grids, were not included in the final analysis.