Chapter 3 Research Methodology
3.12 Research methodology
In t his section, t he f ollowing a spects w ill be de scribed, na mely us er t esting i n s ection 3.11.1, observation in section 3.11.2 and interview in section 3.11.3.
3.12.1 User testing
For t his s tudy 10 s tudents f rom t he U niversity of K waZulu-Natal ( Howard College and Pietermaritzburg campuses) ranging from blind to visually impaired volunteered to participate in this research. The researcher used the disability unit at the Howard and Pietermaritzburg campus to obtain the blind and visually impaired students’ contact details. A purposive sampling method will be us ed. T he da ta c ollection pr ocess w ill be c onducted b y us ing 3 r esearch m ethods:
unstructured interviews (Kvale, 1996), user testing (Dumas and Reddish, 1999) and observation together with the ‘think aloud method’ (Baron, 2000). Because part of the data will be collected through observation, participants were requested to ‘think aloud’ while they perform the required web-based tasks. This t echnique will al low t he r esearcher t o understand uncertainties and difficulties encountered whilst performing tasks. The 10 pa rticipants were involved in the user testing of 4 di fferent websites in an effort to gain a greater understanding of web accessibility problems encountered by blind and visually impaired people.
The bl ind a nd visually i mpaired individuals tha t pa rticipated in the us er te sting accessed the selected websites using screen readers and screen magnification technologies ( accessibility tools). The websites involved were Google (South Africa), OLS, Woolworths and Computicket.
These websites were chosen because the researcher wanted to test if South African websites used accessibility design guidelines. The web-based tasks participants were required to perform were as follows: search for information using Google on how to make a kite; view and save lectures slides on O LS which is an academic w ebsite; pe rform onl ine s hopping on t he W oolworths website and buy a bus ticket from the Computicket website to any place in South Africa. The user t est w as pe rformed i ndividually; each pa rticipant pe rformed t he t ask alone. T his w as because the researcher did not want the participants to communicate with one another or to ask each other for help when one could not perform a t ask. Also it was easier for the researcher to keep track of the time of completion of a task and to observe an individual performing the task
alone rather than watching two or more individuals perform a task. It would have been confusing to make notes and monitor more than one individual at a time.
The Google task was ch osen because it is popular amongst students. The majority of students make us e of Google to s earch for inf ormation w hether it is a n assignment or f or r esearch purposes. It is the student’s personal preference to use Google for academic purposes. Since this website i s us ed m ost of ten i t w ould be a ppropriate t o c heck i f Google follows a ccessibility guidelines.
The OLS task was chos en because students at UKZN make use of this educational web-based course management system on a daily basis to retrieve their lecture material and other academic information. Students are compelled to use OLS. Also the researcher wanted to investigate if this website falls within the W3C web design guidelines.
Woolworths is a very popular store that is available online and has a physical existence. The aim of this task was to check if Woolworths provides the same services online that it normally does at their stores and to determine if the online shopping facility is easy to perform for blind and visually impaired individuals and if it is user friendly.
The C omputicket t ask was c hosen be cause m any of the s tudents do not live in Durban a nd therefore require transportation to get back home during the midterm breaks. Computicket was used t o determine i f i t provides a t icketing s ystem t hat i s fast and easy f or bl ind and vi sually impaired individuals to use in order to purchase a bus ticket.
OLS and Google will fall under the category academic.
3.12.2 Observation
Observation w as us ed w ith t he ‘ think a loud pr otocol’. T his m ethod w as chosen t o e nable t he researcher to study the participants whilst performing the user test. The researcher would be able to notice c ertain thi ngs f rom the us er te st tha t the pa rticipant m ay not e xpress dur ing t he interview such as any frustrations or problems they may experience whilst engaging in the tasks.
This w ill a llow t he r esearcher t o ge t a be tter unde rstanding of t he t asks t hat w ould no t be conveyed in the interview.
3.12.3 Interview
An interview was chosen to get a better understanding of the tasks performed during the user test for ex ample pr oblems t hat m ay h ave be en experienced, task satisfaction, etc. The r esearcher formulated a l ist of questions t o be asked during t he i nterview, the i nterview s chedule can be found i n t he Addendum A . T he i nterview begins with de mographic q uestions s uch a s a ge, gender, race, t ype of di sability, e tc. T hese que stions a re a sked i n or der t o get ba ckground information about the participants in the study. Thereafter questions relating to the participant’s Internet use are asked such as number of years of Internet use, frequency of Internet usage, what they use the Internet for, etc. These questions aim to get an understanding of the participant’s Internet usage and experience. After that, questions based on the tasks in the user test are asked such as problems experienced and how long it took for the completion of a task, etc. The purpose of these questions is to tie it up w ith the data retrieved from the observation in order to get a better understanding of the user test findings.
A pilot study was conducted to ensure that the questions formulated would yield the results that the researcher is looking for and to determine if a blind or visually impaired individual would be able to perform the user test tasks. The pilot study conducted comprised ofthree individuals, two blind students and one visually impaired student at the Pietermaritzburg campus.
3.12.4 The case study protocol
The case study protocol is both the instrument, with which the case study will be conducted, and the general rules and procedures with which the work is carried out (Lubbe, 1998). At the centre of the case study protocol is the interview questions that reflect the actual enquiry (Lubbe, 1998).
The que stions a re reminders t o t he i nvestigator r egarding t he i nformation t hat s hould be collected. It is expected that each question will initiate a discussion on t he issue at hand. The interview questions and rationale behind each question is laid out in Appendix. The sections to follow will give a detailed discussion on the case study research method.
The type of case study the researcher will use is a com bination of explanatory and descriptive research.
3.12.5 Recording tasks
A c ombination of t echniques namely a udio r ecording a nd not e-taking w ere us ed t o r ecord observations of participants performing tasks during user testing and with post-task interviews.
The researcher did not give any instructions to the participants while they were performing the respective tasks. However when the participants showed signs of stress, the researcher provided a "hint" to enable the user to continue. According to Nielsen (1993), (cited by Craven 2003) this type of intervention is necessary in certain circumstances, as is prompting a user to ensure the transcription is accur ate. Post-task interview questions were asked in order to gather data of a more qualitative nature. This enabled the researcher to get greater feedback from the participant as to why they performed the given task in a particular manner during the user test.
3.12.6 Data Analysis
Data will be collected from the interview, user test and observation with the think aloud protocol by ana lysing pa rticipants’ interactions w ith websites by us ing c ontent a nd c orrespondence analysis.