User experience frameworks - The moderator may ask schools to courier files to their school or

Model 2: The moderator may ask schools to courier files to their school or a central venue

3.6. User experience frameworks

Table 3-3: Mapping of IS Success constructs to eModeration Requirements Construct System

quality Information

quality Service

quality Use User satisfaction Net benefit Definition

(DeLone &

McLean, 2003; Petter &

McLean, 2009

The desired features of the system.

Qualities of the output offered by the IS.

Support of users by the IS department of the organization.

Anticipated future utilization of an IS.

Measures user approval of an IS and its output.

The effect the IS has on an individual or organization.

Mapping to identified eModeration requirements

(Rajamany, 2020)

Ease of Use Availability Reliability Response Time

Compatibility Flexibility Complexity

Accuracy Timeliness Completeness Output quality Security Legibility Data currency

Technical Support

Job effects The usefulness of system features and functions

Satisfaction with specific functions Ease of Learning Ease of Use Confidence Task Performance Output Quality

Productivity Job Effects Efficiency

The identified requirements of an eModeration system correlate to the IS Success constructs.

Additionally, from the mapping of the eModeration requirements to the Use construct, use correlates to the usefulness of the system. The D&M model is relevant in evaluating an eModeration system as it considers the interdependence of factors that contribute to the success of an IS.

3.6.1. Framework 1: User Experience Research Framework

Mahlke (2007) proposes a research framework that conceptualises user experience as comprising instrumental and non-instrumental quality perceptions, together with users’

emotional reactions to better understand how people experience technology (see Figure 3-8).

The system properties, user characteristics, and context or task parameters impact the three central components of instrumental qualities, non-instrumental qualities, and emotional user reactions.

Figure 3-8: User Experience Research Framework (Mahlke, 2007, p. 27)

Instrumental qualities refer to the ease of use of a system, which encompasses features such as the effectiveness of its functionality. Non-instrumental qualities address user needs that extend beyond the efficient achievement of tasks to incorporate emotional reactions to the appearance and experience of the system (Mahlke, 2007; Mahlke & Thüring, 2007).

The components of the User Experience Research Framework are tabulated in Table 3-4.

Table 3-4: User Experience Research Framework Framework: User Experience Research

Framework eModeration System

Reference: Thüring and Mahlke (2007, p. 27);

Mahlke (2007) Applicability Justification

Domain: Portable digital audio players Components Constructs Variables

Instrumental qualities Usefulness Utility yes Exclusionary and inclusionary criteria

for these constructs are based on the underlying theoretical foundation informed by literature, as discussed in Section 3.5.2.

Efficiency yes

Usability Controllability no Effectiveness yes Helpfulness yes Learnability yes

Non-instrumental qualities

Aesthetic

aspects Visual aesthetics yes The evaluation of an eModeration system is based on the functional usability of the system and on overall satisfaction with the functionality. The visual aesthetics will contribute to the overall satisfaction that the user feels while using the system. The focus of this study is on usability; hence, the other variables are not applicable.

Haptic quality no Acoustic quality no

Symbolic

aspects Associative

symbolics no Not applicable to the eModeration

context as teachers and moderators will communicate digitally.

Communicative

symbolics no

Motivational

aspects No variables are

identified. Yes

Emotional user reactions

Subjective

feelings yes A positive experience can create a

sense of satisfaction, while a negative experience can lead to feelings of mistrust or dissatisfaction.

Motor

expressions no Although emotions are important, it is not necessary to measure motor expressions or physiological reactions as the eModeration context does not elicit extreme emotions.

Physiological

reactions no

Cognitive

appraisals no

Behavioural

tendencies no

3.6.2. Framework 2: M-health User Experience Framework

Ouma (2013) describes a user experience framework for a mobile health application specific to the healthcare sector in South Africa (see Figure 3-9).

Figure 3-9: M-health user experience framework (Ouma, 2013, p.271)

The three main components of the M- health User Experience Framework comprises three domains:

 Mobile user experience components are comprised of user characteristics, user expectations, emotions, motivation, experience, user roles, and goals.

 M-health technology requirements impact the use of m-health applications. The users’

engagement with the application are influenced by aspects such as the user’s pragmatic goals, the available software and hardware, the infrastructure, and usability and interoperability considerations.

 Domain requirements are essential to improve the user experience. These requirements include supporting the vision and mission of the health department, stakeholders implementing the application, m-health policies, and meeting the needs of various services provided by different levels of hospitals.

The components of the M-health user experience framework are tabulated in Table 3-5.

Table 3-5: Applicability of M-health components to an eModeration system Framework: M-health User Experience

Framework eModeration system

Reference: Ouma (2013)

Applicability Justification Domain: Public health System

Mobile User Experience Components

Mobile User User

characteristics yes Although the context is the mobile environment, user characteristics and goals apply to the eModeration domain and may affect the user experience of the system.

User goals yes

Resources

available yes Mental resources determine the time taken to learn how to use the system.

User expectations yes Determine user satisfaction.

Diversity of users yes Tasks that moderators perform are different to those that teachers need to perform.

Users' emotions yes A positive experience can create a sense of satisfaction, while a negative experience can lead to feelings of mistrust or dissatisfaction.

Users' motivation yes The usefulness of the eModeration system will determine how motivated the user is to use the system.

User experience yes User experience and prior knowledge will impact their interaction with an eModeration system.

Knowledge yes

Mobile context Spatial context no Teachers’ jobs do not require them to be on the move. Mobility is therefore not a consideration.

Mental context no

Task context no

Infrastructural

context no

Social context no

Content Informative no An eModeration system does not provide

content, but enables the production of content in the form of a moderator report. It is not an inherent function of the system nor a requirement for moderation.

Useful yes The onscreen information that is provided

should allow the user to easily navigate the eModeration environment.

Design Visual

presentation yes A consistent interface is necessary for users to learn to use the system easily.

Layout no The eModeration context is not inherently a

mobile application domain that has specific requirements to ensure readability due to the smaller screen size of a portable device.

Visual flow no

Marketing Awareness no There will not be a need to market the product in the domain of application.

M- healt

h th

Application Type of m-health

application no

Framework: M-health User Experience

Framework eModeration system

Reference: Ouma (2013)

Applicability Justification Domain: Public health System

Design guidelines

for applications no Specific to the mobile technology environment;

therefore not applicable to an eModeration system.

Mobile Device Hardware no

Software issues no M-health

infrastructure E-health

infrastructure no Specific to the mobile environment, which does not apply to the eModeration context.

Cellular networks no Wireless networks no Mobile business

operations no

Digital

technology Interoperability no

Privacy issues yes The eModeration system should maintain the security and privacy of users.

Interoperability

issues yes eModeration systems should promote the sharing

of information.

Usability issues yes An eModeration application should be easy to learn, easy to use, and free from errors.

Domain Requirements

M-health vision

and mission no Domain-specific requirements for the mobile health system do not apply to the eModeration environment.

M-health

stakeholders no

M-health policies no

M-health needs no

Funding issues no

Research no

Political will no

Level of hospitals no

Stewardship no

Cultural aspects no

3.6.3. Framework 3: User Evaluation Framework for eModeration

The User Evaluation Framework (see Figure 3-10) consists of three levels, as explained below:

 Environment level: identifies “users” and “organisation” (Van Staden, 2017, p. 354) as constructs, with users having defined roles and responsibilities;

 eModeration requirements level: specifies the processes involved in moderation. This level ensures that secure access is provided and that processes are in place to upload documents for moderation, track the moderation process, download moderation reports, and provide feedback to assessors; and

 eModeration user experience construct level: identifies instrumental and non- instrumental qualities that work together with the Environment and eModeration requirements levels to affect the user experience.

Figure 3-10: User experience evaluation framework for eModeration (Van Staden, 2017, p.357)

The components of the User Evaluation Framework for eModeration are depicted in Table 3-6.

Table 3-6: Applicability of User Evaluation Framework to the eModeration system Framework: User Evaluation Framework

eModeration system Reference: Van Staden (2017) Applicability Justification

Domain: Academic institutions

Constructs

Environment level Users Roles yes The roles and responsibilities of teachers and moderators differ in an eModeration system, and the process followed is dependent on which role the user adopts.

Responsibilities yes

Organization

Higher education institutions no

The environmental context of this study is the secondary school environment, that is, an academic institution.

Academic institutions

yes

eM ode

atio Process Access platform yes

Framework: User Evaluation Framework

eModeration system Reference: Van Staden (2017) Applicability Justification

Domain: Academic institutions

Constructs

Uploading or

downloading yes

The eModeration system would provide the platform to upload and download documents for moderation.

eModeration

Network

infrastructure yes

Moderators and teachers should have access to the necessary service, support, and technology resources.

Service quality yes

Support yes

Security yes

Devices yes

Technology yes

Resources yes

eModeration UX construct level

Instrumental qualities

Navigation yes Moderators and teachers must be able to easily navigate the system, which allows them to effectively complete their respective tasks in the least possible time with satisfaction.

Effectiveness yes

Feedback yes

Efficiency yes

Satisfaction yes

Context yes Identified in the task requirements.

Content yes

Being able to easily access appropriate content is important for efficiency and effectiveness.

Visibility of the

system yes

Ensures that the navigation links are unambiguous.

Error prevention yes Users need to be able to easily recover from any errors that occur whilst using the system.

User control yes

The eModeration system provides the user control over the processes that need to be carried out.

Non-instrumental qualities

Overall

experience yes

Ensures that the overall experience of the user is positive.

Source quality no Incorporated into information quality.

Personalisation yes Interface can be personalized to suit different roles and preferences.

Cross-platform yes

Moderators and teachers should be able to access the system via different devices and platforms.

Context-aware

service no

Identified in the task requirements.

3.6.4. Applicability of existing UX frameworks to eModeration

The constructs of Mahlke and Thüring's (2007) User Experience Research Framework applies to an eModeration system, which also consists of the interaction between the user, the task, and the system. The user experience is an outcome of the interaction between the instrumental and

non-instrumental quality perceptions of the user and the emotional user reactions arising from this interaction (Mahlke, 2007). As with an m-health application, the user and technology are important user experience components of an eModeration system.

The eModeration user experience framework proposed by Van Staden (2017) is most applicable to this study as the context of use is similar. The difference is that the evaluation framework developed in this study was tested in the private secondary school environment rather than the HEI environment.

While the three frameworks discussed are based on systems that have already been deployed, this thesis used a prototype specifically created for the domain in which the evaluation framework will be used.

3.6.5. User experience constructs

The perception of a product is based on the individual values of the user and the context of use, therefore the perception of user experience is highly subjective (Jetter & Gerken, 2007).

Adopting Hassenzahl et al.’s (2010) description of UX as a dynamic, context-dependent, and subjective account of human–technology interaction, the influence of the context, user, and the system on UX is depicted in Figure 3-11.

Figure 3-11: Influence of context, user, and system on user experience (Van Staden, 2017, p. 196)

3.6.6. Definition of UX for this study

In this study, UX refers to the user experience related to the usage of a system as recommended by Law et al. (2009). Hassenzahl and Tractinsky (2006) categorize UX components as a consequence of the user’s internal state, the characteristics of the system, and the context within which the interaction occurs. Roto (2006), Roto and Kaasinen (2008) and Tan et al. (2013) identify attributes related to each component, as depicted in Table 3-7.

Table 3-7: UX attributes

In line with Hassenzahl's (2007) view that pragmatics is much more influenced by tasks than hedonics, is and Tan et al's. (2013) explanation of context as inclusive of the physical, social, temporal, and task contexts; the context is described in terms of the task requirements (see Table 3-9) which, together with the user characteristics, impacts on the user. The task is therefore a fundamental aspect of user-centred analysis and evaluation techniques (Hassenzahl

& Tractinsky, 2006). The interaction between the user, the eModeration system, and the context is depicted in Figure 3-12.

Figure 3-12: Interaction between the user, the system, and the task in an eModeration system

Desmet and Hekkert (2007) maintain that user experience is shaped by user characteristics (e.g., personality, skills, etc.) as well as those of the system (e.g., aesthetics). Users’ perceptual and cognitive processes, for example, remembering and understanding, contribute to this

UX Component System Context User

Attributes

Product Physical context User needs

Object Social context Mental resources

Service Temporal context Physical resources Infrastructure Task context Emotional state

Complexity Experience

Purpose Expectations

Usability Functionality

experience, while the context in which the interaction takes place influences the experience. It is therefore important to consider system properties, user characteristics, and the usage situation when designing and evaluating interactive systems (Mahlke, 2007). The context is described as a UX component that includes systems and objects that are not part of the system, but that affect the UX of the system (Roto, 2006). Roto (2006) further outlines the following aspects of the context:

 The physical context refers to the tangible physical surroundings that affect the user, for instance the outside temperature.

 The social context refers to the expectations from, and influences of other people and/or the readiness of, the user to share in social situations.

 The temporal context refers to the time dedicated to the use of the system given context restrictions.

 The task context refers to the role of the system in fulfilling the higher-level goals of users.

An eModeration system is a tool to deliver a good UX. Conceptualising the system as an agent (Adie, 2011) in the process of online moderation expands the inquiry to include an investigation of how the technology interacts and affects the moderation process (task). The user surroundings, social influences, and time dedicated to system use are not relevant to the eModeration context, nor are these contexts included in the inherent functionality of the system.

Thus, based on Roto's (2006) definitions, the physical, social, and temporal contexts do not apply to an eModeration system. In line with Roto's (2006) suggestion that it is beneficial to carefully consider the task context in a specific use case scenario, the context has been replaced by task in this study (see Figure 3-13).

Figure 3-13: UX Requirements (adapted from Van Staden, 2017, p. 196)

Against the background of definitions (see Table 2-6) and frameworks (Section 3.7) associated with user experience discussed in the preceding sections, the following high-level components are included in a theoretical framework for eModeration in this study:

 eModeration system components (system);

 eModeration user components (user); and

 eModeration task components (task).

Dalam dokumen A framework for evaluating the user experience of digital moderation systems in the South African secondary school environment (Halaman 94-105)