ANALYSIS OF QRIS USER EXPERIENCE USING THE USER EXPERIENCE QUESTIONNAIRE (UEQ) METHOD
Tira Siya Fajar Rahayu1)*, Muhammad Fikry Aransyah2)
1, 2)Program Studi Administrasi Bisnis, Universitas Mulawarman
Samarinda, Kalimantan Timur
e-mail: [email protected]1), [email protected]2)
*e-mail korespondensi : [email protected]
ABSTRACT
Digital wallets are the most preferred payment method by Indonesian digital society compared to other payment methods, namely cash payments and bank transfers. Over time, several digital wallet platforms began to create QR codes according to their respective platforms. With each platform having its own QR Code, this can make it difficult for users to make payments.
Therefore, Bank Indonesia, which is tasked with regulating and maintaining the smooth running of the payment system, regulates the QR Code to be standardized in accordance with the International Europe Mastercard Visa standard. A product is said to be successful if the product is able to meet user needs to increase user satisfaction. In order for the product to meet the standards, it is necessary to evaluate the quality of the product. One of the evaluations that must be carried out is an evaluation of the User Experience. The research utilized purposive sampling, a form of non-probability sampling technique, with the requirement that the individuals selected as participants were residents of Samarinda who had utilized QRIS as a payment method. The analysis of the data was performed using the UEQ Data Analysis Tool. The findings revealed that the user experience, which encompasses aspects such as attractiveness, perspicuity, efficiency, dependability, stimulation, and novelty, had a favorable impact on the satisfaction of QRIS users in Samarinda City.
Keyword: QRIS, UEQ, User Experience
I. INTRODUCTION
he advancement of the fourth industrial revolution has given rise to the era of Society 5.0, resulting in significant changes in people's daily routines. The primary determinant that enables Indonesians to embrace the industrial revolution and Society 5.0 is technology's ability to satisfy their requirements effortlessly[1]. The financial sector, in particular, has undergone a transformational disruption due to the emergence of Financial Technology (Fintech), which is rooted in technological applications [2].
According to a report titled "Consistency That Leads: 2023 E-Wallet Industry Outlook" presented by marketing firm InsightAsia, recent research indicates that digital wallets are the preferred mode of payment among the digital populace in Indonesia, surpassing cash payments and bank transfers [3]. Over time, various digital wallet applications began to create QR codes according to their respective platforms. With each platform having its own QR Code, this can make it difficult for users to make payments. Therefore, Bank Indonesia, which is tasked with regulating and maintaining the smooth running of the payment system, regulates the QR Code to be standardized according to the International Europe Mastercard Visa standard [4]. This standardization is done to make it easier, faster and safer, and able to support interoperability and interconnection between countries and between instruments. Another goal of making QRIS is to encourage economic efficiency, accelerate inclusive finance, and advance MSMEs [5].
A product is said to be successful if the product is able to fill user needs to increase user satisfaction [6]. In order for the product obtained to meet the standards, it is necessary to evaluate the quality of the product, one of the evaluations that must be carried out is an evaluation of the User Experience [7]. User Experience refers to the perception and reaction of individuals who utilize a product, system, or service (as per ISO 9241-210). To assess the level of user experience with QRIS, researchers employ the User Experience Questionnaire (UEQ) method, which aims to elicit direct feedback from users regarding a product [8]. UEQ displays extensive and complete knowledge about user experience, starting with the classic usability aspects to the user experience aspects. In addition, there are analysis tools that facilitate the interpretation of results easily, accurately, and free of charge [9].
I. LITERATUREREVIEW A. Human Computer Interaction (HCI)
The scientific discipline of Human Computer Interaction (HCI) focuses on the interplay betwixt computer technology and users, with the objective of developing and evaluating technology that can effectively aid users in
T
fulfilling their requirements, resulting in user satisfaction. HCI is a multidisciplinary field that encompasses computer science, behavioral science (such as psychology, sociology, and anthropology), cognitive science, ergonomics, and design principles. HCI involves multiple components that interact with one another to create an optimal user experience, including users, tasks oriented toward specific objectives, interfaces, and contexts [10].
B. User Experience
The notion of user experience is subjective in nature, although it comprises distinct quality aspects associated with the user's psychological requirements for their interaction with a product [11]. User experience pertains to an individual's impressions, perceptions, and responses when utilizing a product, system, or service, which
encompasses both practical and emotional quality aspects [12].
C. User Satisfaction
Consumer satisfaction refers to an individual's emotional state of contentment or dissatisfaction resulting from a comparison between the actual performance of a product and the anticipated level of performance [13]. The degree of customer satisfaction measures the extent to which the usability experience of a product meets or surpasses the value expected by the purchaser [14]. There are three distinct types of user satisfaction evaluations: Positive disconfirmation, where the product's performance exceeds expectations; Simple confirmation, where performance aligns with expectations; and Negative disconfirmation, where performance falls short of expectations.
D. User Experience Questionnaire (UEQ)
The User Experience Questionnaire (UEQ) was originally created in 2005 by Laugwitz, Schrepp, and Held in the German language. It is currently accessible in over 30 languages and can be used free of charge without the need for a license. UEQ has advantages over other user experience measurement methods. UEQ offers comprehensive knowledge of user experience, including classic usability factors and user experience factors. In addition, there are analysis tools that facilitate the interpretation of results easily, accurately, and free of charge [9] The main purpose of using the UEQ is to capture users' impressions directly about a product. The UEQ consists of six measurement scales consisting of 26 question items [8], which are as follows:
1) Attractiveness, which pertains to the overall perception of the product by the user.
2) Perspicuity, refers to the degree to which the user can quickly and easily understand how to use a product.
3) Efficiency, which measures the user's ability to complete a task accurately and efficiently without exerting too much effort.
4) Dependability, namely how much control the user feels over the interactions carried out.
5) Stimulation, which evaluates the level of motivation or excitement experienced by users when using a product.
6) Novelty, which gauges the degree of inventiveness and innovation in the product that can attract user attention.
UEQ is divided into three categories, namely attractiveness, pragmatic quality, and hedonic quality, which are further divided among the six scales mentioned earlier. Since the pure degree dimension of UEQ deals with the user's assessment of attractiveness, the component (attractiveness) plays an important role. The user's impression of the technical elements that concentrate on achieving the goal (performing the task) is the pragmatic quality aspect. The hedonic quality aspect is the user's assessment of non-technical features but is not focused on achieving the goal (performing the task).
Figure. 1. UEQ Scale Structure
II. RESEARCHMETHODS A. Type of Research
The methodology utilized by the author in this study is quantitative descriptive research. Quantitative research involves the systematic and contextual exploration of symptoms by gathering data from a natural setting, with the researcher serving as a primary instrument. This type of research is typically descriptive and employs inductive analysis. Descriptive research, on the other hand, strives to methodically and precisely reveal symptoms, facts, or events about the attributes of a specific population or area. In this type of research, there is no need to establish or elucidate relationships, or to test hypotheses [15]
B. Population and Sample
The research utilized purposive sampling, a form of non-probability sampling technique. The study focused on selecting respondents who reside in Samarinda and have used QRIS as a payment system.
The size of the population in this study comprises users who utilize QRIS. However, the exact number of individuals in the population is undetermined. The Lemeshow Method is utilized to determine the sample size of this research since the population size is unknown. The calculation is done as follows with an error of 10%.
n = 𝑍2𝑃 (1−𝑃)
𝑑2 (1)
therefore n = 1,962×0,5×(1−0,5)
0,12 = 96,04 ≈ 100 (2) Consequently, the research was conducted using 100 samples.
C. Methods of Data Collection
This study employs questionnaires and literature reviews as the primary methods for data collection.
1) Questionnnaire
The researchers utilized a survey method by disseminating questionnaires to QRIS users with the purpose of gathering the information needed. The questionnaire was disseminated using various social media platforms such as Twitter Whatsapp, and Instagram, which can be filled out through Google Forms.
2) Literature Study
The literature review is a research method employed in this study that involves examining multiple sources, such as theories, books, existing studies, and websites that provide online services, pertaining to the research topic and subject matter. The information gathered through the review serves as the foundation for the intended research.
D. Data Analysis Techniques
User Experience Questionnaire (UEQ) method was utilized for data analysis in this study, which involved both demographic analysis and statistical analysis.
1) Demographic Analysis
In demographic analysis, respondent data is collected based on gender, age, education, and domicile.
2) Statistical Analysis
In the statistical analysis, validity, reliability and UEQ measurement were tested. The validity test was carried out using SPSS version 23, while the reliability test was carried out using UEQ Data Analysis Tool version 10.
Pearson correlation values for each indicator in each variable to carry out the validity test, while the Cronbach alpha (∝) value of each research variable was examined to carry out the reliability test.
III. RESULTANDDISCUSSION A. Respondent Characteristics
The distribution of this questionnaire managed to get up to 100 respondents. Based on the responses obtained, the characteristics of respondents can be classified based on gender, age, education level, domicile, length of use of QRIS.
1) Gender
Based on Table 1, the majority of respondents were female, namely 70 respondents, while male amounted to 30 respondents.
TABLE1.GENDEROFQRISINSAMARINDACITY
Gender Total Male 30 Female 70
2) Age
Based on Table 2, respondents aged 15-24 years were 89 respondents, and respondents aged 25-44 years were 11 respondents.
TABLE2.AGEOFQRISUSERSINSAMARINDACITY Age Total
15-24 years 89 25-44 years 11
3) Education
Based on Table 3, respondents who have education equal to or below high school equivalent are 25 respondents, respondents with Diploma education are 4 respondents, with S1 education as many as 70 respondents, and S2 as many as 1 respondent.
TABLE3.EDUCATIONOFQRISUSERSINSAMARINDACITY Education Total
< SMA/Sederajat 25
Diploma 4
S1 70
S2 1
4) Domicile
According to Table 4, most respondents in this study reside in Samarinda Ulu, which consists of 24 participants.
In contrast, there were 9 respondents from Samarinda Utara, 3 from Sungai Pinang, 3 from Samarinda Seberang, 20 from Loa Janan Ilir, 4 from Samarinda Ilir, 20 from Sungai Kunjang, 24 from Samarinda Ulu, 9 from Samarinda Kota, 1 from Sambutan, and 7 from Palaran.
TABLE4.DOMICILEOFQRISUSERSINSAMARINDACITY Domicile Total
Samarinda Utara 9 Sungai Pinang 3 Samarinda Seberang 3 Loa Janan Ilir 20 Samarinda Ilir 4 Sungai Kunjang 20
Samarinda Ulu 24 Samarinda Kota 9
Sambutan 1
Palaran 7
5) Length of Use
According to Table 5, a greater proportion of respondents have been using QRIS for more than a year, with 40 respondents falling in this category. On the other hand, 37 respondents have used QRIS for less than a year, and 23 respondents have used it for exactly one year.
TABLE5.LENGTHOFUSEOFQRISINSAMARINDACITY Length of Use Total
< 1 year 37
1 year 23
> 1 year 40
B. Validity and Reliability Test
Before conducting data analysis, the author began to pre-treatment because it was realized that there were several respondents who filled out the questionnaire without seriousness. Suspicious data was detected, namely 7 respondents, so that data of 93 respondents were used.
In this study, the validity test used 93 respondents with 26 question items with a confidence level of 5% so that the Rhitung must be more than the Rtabel, namely 0,2039.
TABLE6.VALIDITYTESTRESULTS
Item Rhitung Rtabel Description Item Rhitung Rtabel Description 1 0,742 0,2039 Valid 14 0,772 0,2039 Valid 2 0,701 0,2039 Valid 15 0,737 0,2039 Valid 3 0,777 0,2039 Valid 16 0,819 0,2039 Valid 4 0,851 0,2039 Valid 17 0,794 0,2039 Valid 5 0,795 0,2039 Valid 18 0,834 0,2039 Valid 6 0,870 0,2039 Valid 19 0,877 0,2039 Valid 7 0,870 0,2039 Valid 20 0,869 0,2039 Valid 8 0,727 0,2039 Valid 21 0,830 0,2039 Valid 9 0,871 0,2039 Valid 22 0,888 0,2039 Valid 10 0,697 0,2039 Valid 23 0,777 0,2039 Valid 11 0,822 0,2039 Valid 24 0,789 0,2039 Valid 12 0,776 0,2039 Valid 25 0,799 0,2039 Valid 13 0,790 0,2039 Valid 26 0,819 0,2039 Valid
The study evaluated the reliability of the data by measuring the Cronbach's Alpha value for each variable. The data was analyzed using the User Experience Questionnaire Tool (UEQ), with a Cronbach's Alpha value of 0,6 or higher indicating high reliability.
TABLE7.RELIABILITYTESTRESULTS
Attractiveness Perspicuity Efficiency Dependability Stimulation Novelty
Items Corr Items Corr Items Corr Items Corr Items Corr Items Corr
1, 12 0,41 2, 4 0,42 9, 20 0,77 8, 11 0,49 5, 6 0,53 3, 10 0,48
1, 14 0,74 2, 13 0,59 9, 22 0,84 8, 17 0,33 5, 7 0,63 3, 15 0,34
1, 16 0,73 2, 21 0,49 9, 23 0,46 8, 19 0,45 5, 18 0,53 3, 26 0,59
1, 24 0,4 4, 13 0,53 20, 22 0,83 11, 17 0,52 6, 7 0,8 10, 15 0,24
1, 25 0,5 4 , 21 0,57 20, 23 0,48 11, 19 0,72 6, 18 0,64 10, 26 0,34
12, 14 0,36 13, 21 0,54 22, 23 0,48 17, 19 0,69 7, 18 0,56 15, 26 0,62
12, 16 0,45 Average 0,52 Average 0,64 Average 0,53 Average 0,61 Average 0,43
12, 24 0,61 Alpha 0,81 Alpha 0,88 Alpha 0,82 Alpha 0,86 Alpha 0,75
12, 25 0,54 Conf. Int.
Alpha (5%)
0,74 Conf. Int.
Alpha (5%)
0,83 Conf. Int.
Alpha (5%)
0,75 Conf. Int.
Alpha (5%)
0,81 Conf. Int.
Alpha (5%) 0,66
14, 16 0,86 0,87 0,91 0,87 0,9 0,82
14, 24 0,48 14, 25 0,48 16, 24 0,49 16, 25 0,56 24, 25 0,58 Average 0,55 Alpha 0,88 Conf. Int.
Alpha (5%) 0,83 0,91
C. Measurement of UEQ
The questionnaire data inputted in the UEQ Version 10 tool is transformed into initial value data to determine the negative and positive values of each item with the following details.
TABLE8.DATATRANSFORMATIONOFUEQVALUE
1 2 3 4 5 6 7
Annoying O O O O O O O Enjoyable
-3 -2 -1 0 1 2 3
After transforming the data, the mean, variance, and standard deviation were computed, and the data was grouped according to six variables, which are attractiveness, perspicuity, efficiency, dependability, stimulation, and novelty.
36
TABLE9.MEAN,VARIANCE,ANDSTANDARDDEVIATIONRESULTS
Item Mean Variance Std. Dev. No. Left Right Scale
1 2,3 0,8 0,9 93 annoying enjoyable Attractiveness
2 2,5 0,6 0,8 93 not understandable understandable Perspicuity
3 1,7 2,3 1,5 93 creative dull Novelty
4 2,0 2,9 1,7 93 easy to learn difficult to learn Perspicuity
5 2,3 1,2 1,1 93 valuable inferior Stimulation
6 1,9 1,0 1,0 93 boring exciting Stimulation
7 2,1 1,1 1,0 93 not interesting interesting Stimulation
8 1,6 1,8 1,3 93 unpredictable predictable Dependability
9 2,3 1,1 1,1 93 fast slow Efficiency
10 1,2 3,2 1,8 93 inventive conventional Novelty
11 2,1 0,9 1,0 93 obstructive supportive Dependability
12 1,7 3,0 1,7 93 good bad Attractiveness
13 2,4 1,0 1,0 93 complicated easy Perspicuity
14 2,0 1,1 1,1 93 unlikable pleasing Attractiveness
15 1,1 3,4 1,8 93 usual leading edge Novelty
16 2,2 1,0 1,0 93 unpleasant pleasant Attractiveness
17 2,1 1,6 1,3 93 secure not secure Dependability
18 1,5 1,8 1,4 93 motivating demotivating Stimulation
19 1,9 1,3 1,2 93 meets expectations does not meet expectations Dependability
20 2,3 1,0 1,0 93 inefficient efficient Efficiency
21 2,2 1,9 1,4 93 clear confusing Perspicuity
22 2,3 0,9 1,0 93 impractical practical Efficiency
23 1,8 2,5 1,6 93 organized cluttered Efficiency
24 1,5 1,9 1,4 93 attractive unattractive Attractiveness
25 2,1 2,2 1,5 93 friendly unfriendly Attractiveness
26 1,8 1,9 1,4 93 conservative innovative Novelty
The average score of all variable items will be classified on each variable based on the value index in the table below:
TABLE10.AVERAGERATINGINDEXONTHEQUESTIONNAIRE Averange Score Description
< -0,8 Negative Value –0,8 and 0,8 Normal Value
>0,8 Positive Value
Table 11 displays the mean values for each variable, where the values of attractiveness, perspicuity, efficiency, dependability, stimulation, and novelty all surpass 0,8. Consequently, the results suggest that the users of QRIS in Samarinda City have a favorable perception of their user experience.
TABLE11.AVERAGERESULTANDVARIATIONOFUEQSCALE UEQ Scales (Mean and Variance)
Attractiveness 1,975 0,96 Perspicuity 2,266 0,96 Efficiency 2,177 0,95 Dependability 1,933 0,89
37
Stimulation 1,927 0,91 Novelty 1,454 1,50
Figure. 2. Averange of UEQ Scale
Following the computation of the mean value for each variable, it is necessary to compare the obtained results with a benchmark dataset to assess the evaluated product's quality compared to other products. The benchmark dataset for User Experience Questionnaire (UEQ) classifies products into five categories according to their scores relative to other products in the dataset. The first category is Excellent, which includes products that fall within the top 10% of products with the best results. The second category is Good, which indicates that 10% of the products in the dataset have higher scores, while the other 75% have lower scores. The third category is Above Average, which includes products that score higher than 25% of the products in the dataset, while the other 50% have lower scores. The fourth category is Below Average, which includes products that score higher than 50% of the products in the dataset, while the other 25% have lower scores. The final category is Bad, which includes products that fall within the bottom 25% of products with the worst results.
The mean values of the variables for the QRIS research were computed as follows: 1,97 for attractiveness; 2,27 for perspicuity; 2,18 for efficiency; 1,93 for dependability; 1,93 for stimulation; and 1,45 for novelty. These findings suggest that QRIS performed remarkably well in the categories of attractiveness, perspicuity, efficiency, dependability, and stimulation, placing it within the top 10% of product results in the data set. However, the novelty variable was rated good, indicating that 10% of products in the data set obtained higher scores. In order to assess the quality of the QRIS product in comparison to others, the benchmark data set was used to compare the obtained values.
TABLE12.UEQQRISBENCHMARKRESULTS
Scale Mean Comparisson to benchmark Description
Attractiveness 1,97 Excellent The top 10% of products with the best results
Perspicuity 2,27 Excellent The top 10% of products with the best results
Efficiency 2,18 Excellent The top 10% of products with the best results
Dependability 1,93 Excellent The top 10% of products with the best results
Stimulation 1,93 Excellent The top 10% of products with the best results
Novelty 1,45 Good 10% of the products in the dataset have higher scores, while the other 75% have lower scores
Figure. 3. UEQ QRIS Benchmark Result
38
IV. CONCLUSION
The research findings and discussion lead to the conclusion that the user experience (attractiveness, perspicuity, efficiency, dependability, stimulation, and novelty) has a positive impact on QRIS user satisfaction in Samarinda City. The comparison of QRIS products with 452 products in the UEQ benchmark dataset indicates that the variables of attractiveness, perspicuity, efficiency, dependability, and stimulation are in the excellent category, while the novelty variable falls in the good category.
REFERENCE
[1] H. A. Mumtaha and H. A. Khoiri, “Analisis Dampak Perkembangan Revolusi Industri 4.0 dan Society 5.0 Pada Perilaku Masyarakat Ekonomi (E-Commerce),” JURNAL PILAR TEKNOLOGI Jurnal Ilmiah Ilmu Ilmu Teknik, vol. 4, no. 2, 2019.
[2] Otoritas Jasa Keuangan, “FAQ Fintech Lending,” Otoritas Jasa Keuangan, 2017. https://ojk.go.id/id/kanal/iknb/data-dan- statistik/direktori/fintech/Documents/FAQ%20Fintech%20Lending.pdf (accessed Nov. 10, 2022).
[3] Khoirul Anam, “Riset InsightAsia: e-Wallet GoPay Paling Banyak Digunakan,” CNBC Indonesia, Jakarta, Nov. 2022.
[4] A. C. Sari and H. H. Adinugraha, “Implementation of QRIS-Based Payments Towards the Digitalization of Indonesian MSMEs,” EKONOMIKA SYARIAH : Journal of Economic Studies, vol. 5, no. 2, Jan. 2022, doi: 10.30983/es.v5i2.5027.
[5] E. G. M. Nainggolan, B. T. F. Silalahi, and E. M. Sinaga, “Analisis Kepuasan Gen Z Dalam Menggunakan Qris Di Kota Pematangsiantar,” MANAJEMEN: Jurnal Ekonomi, vol. 4, no. 1, pp. 24–32, 2022.
[6] A. L. T. Adinegoro, R. I. Rokhmawati, and H. M. Az-Zahra, “Analisis Pengalaman Pengguna pada Website E-commerce Dengan Menggunakan Usability Testing dan User Experience Questionnaire (UEQ)(Studi pada Lazada. co. id, Blibli. com dan JD. id),” Jurnal Pengembangan Teknologi Informasi dan Ilmu Komputer e-ISSN, vol. 2548, p. 964X, 2018.
[7] S. Prasetyaningsih and W. P. Ramadhani, “Analisa User Experience pada TFME Interactive Learning Media Menggunakan User Experience Questionnaire,” Jurnal Integrasi, vol. 13, no. 2, pp. 147–157, 2021.
[8] Dr. Martin Schrepp, User Experience Questionnaire Handbook, 8th ed. 2019.
[9] H. B. Santoso, M. Schrepp, R. Isal, A. Y. Utomo, and B. Priyogi, “Measuring user experience of the student-centered e- learning environment.,” Journal of Educators Online, vol. 13, no. 1, pp. 58–79, 2016.
[10] Vijay Kanade, “What Is HCI (Human-Computer Interaction)? Meaning, Importance, Examples, and Goals,” Jul. 2022.
https://www.spiceworks.com/tech/artificial-intelligence/articles/what-is-hci/ (accessed Mar. 17, 2023).
[11] M. Schrepp, A. Hinderks, and J. Thomaschewski, “Construction of a Benchmark for the User Experience Questionnaire (UEQ),” International Journal of Interactive Multimedia and Artificial Intelligence, vol. 4, no. 4, p. 40, 2017, doi:
10.9781/ijimai.2017.445.
[12] Al Ghifari Hartzani, “Evaluasi User Experience Pada Dompet Digital Ovo Menggunakan User Experience Questionnaire UEQ ,” Universitas Islam Negeri Syarif Hidayatullah , Jakarta, 2021.
[13] P. Kotler and K. L. Keller, “Marketing management (15th global ed.),” England: Pearson, pp. 803–829, 2016.
[14] A. Abd Razaka, M. F. Shamsudinb, and R. M. Abdul, “The influence of atmospheric experience on Theme Park Tourist’s satisfaction and loyalty in Malaysia,” 2020.
[15] Hardani et al., Metode Penelitian Kualitatif & Kuantitatif, 1st ed. CV. Pustaka Ilmu, 2020.
