SIMPLIFY XGAMEFLOW TO BE A MORE EFFICIENT
MEASUREMENT TOOL OF PLAYER SATISFACTION
Jasson Prestiliano
Faculty of Information Technology Satya Wacana Christian UniversityJl. Diponegoro 52-60 Salatiga 50711, Central Java, Indonesia
Andeka Rocky Tanaamah
Faculty of Information Technology Satya Wacana Christian UniversityJl. Diponegoro 52-60 Salatiga 50711, Central Java, Indonesia
ABSTRACT
XGameFlow is one of the measurement tools for game player satisfaction. It includes nine criteria. The criteria are concentration, goal clarity, challenge, player’s skill or knowledge improvement, control or automation, feedback, immersion, social interaction and persuasion. However, the questionnaire to measure satisfaction consists of about 58 questions that should be answered by the player. This comprehensive survey takes a long to complete. In this research, we attempted to reduce the number of questions by combining similar questions and eliminating redundant questions, so the measuring tools are more efficient and effective. The simplified questionnaire will be tested and compared with the full questionnaire to determine its validity.
KEYWORDS
Game Enjoyment, XGameFlow, Measurement Tools, Simplify.
1.
INTRODUCTION
When a person decides to “play” a game, it can be assumed that they want to have an enjoyable time. If it is enjoyable, the person will continue to play. A well-designed game can bring enjoyment and satisfaction to players who spend time playing games (Korhonnen, Montola&Arrasvuori, 2009). To design a good game, especially a computer game, a game designer should play many games to identify which aspects of the game are most enjoyable (Crawford, 2003).
Games players enjoy playing are ones that capture the player’s attention and provide the opportunity to feel a sense of satisfaction when reaching a goal. Players usually wish to experience a certain measure of entertainment when they play games. Some players feel excitement when they complete assigned tasks in the game. Additionally, some players are attracted to the storyline, graphics, music and the method of play or the accessories for playing (Yannakakis&Hallam, 2008).
The problem is how we can make sure that the player enjoys the game? Enjoyment or satisfaction is a measure of quality. How can we measure this quantitatively in order to know if the player is satisfied with their experience?
2.
PAPER BODY
2.1 XGameFlow Criteria
XGameFlow is a model that is provided to measure the enjoyment and satisfaction experienced by a game player after playing a game. Figure 1 shows the enjoyment criteria included in XGameFlow.
Figure 1 XGameFlow Criteria (Prestiliano, Manongga& Lin, 2011)
These criteria are:
1. Games should require concentration and the player should be able to concentrate on the game. The player should give his or her attention to the game. This is the essence of concentration.
2. Games should have clear goals, so the player knows the aim of the game. The clarity of the goal should be explained in the game. The information could be directly or indirectly expressed. Often games have intermediate goals and then a larger final goal.
3. For the challenge, the game should be challenging and match the player’s skill level. The level of difficulty should be adequate and should increase gradually. However, it should remain balanced and should meet the player’s skill level. If the challenge is sufficient, the player will continue to play and will not feel bored or become frustrated (Sweetser & Wyeth, 2005).
4. Player’s skill or knowledge improvement is a criterion which states that the game can improve the knowledge or skill of the player (Fu, Su & Yu, 2008). In this criterion, players will be taught
something they don’t know before.
5. For control or automation, players should feel a sense of controlling the game. The game will automatically follow the instructions that are given. The player should find that the game can give them a sense of control through the role, the object and over the game. Moreover, the game can give them some warning when the player is going too far off track and it will cause an error that stops the player from continuing the game.
6. Players should receive appropriate feedback at the appropriate time. They should be given immediate notice about success, failure, events and statuses. The interface also can provide information about something that was or is happening in the game (Sweetser & Wyeth, 2005). 7. Immersion is when a player feels drawn into the game and feels a deep connection to the game. The
player should feel involved with the game. The player can feel that the music, graphics and the story are drawing them into the world of the game. They can recall it even when they are not playing the game (Scheible, Tuulos & Ojala, 2007).
8. Social interaction is when a game can support and create opportunities for social interaction.
Communities are built inside and outside of the game. The player interacts with other players from different places to discuss the game (Sweetser & Wyeth, 2005).
Gamer Enjoyment Concentration
Control Feedback
Immersion Persuasion
Social Interaction
Skill/ Knowledge Improvement Challenge
9. Persuasion is when a game can affect the mind of the player even though he or she isn’t playing the game anymore. The message of the game can persuade him or her to do something or think in new ways (Prestiliano, Manongga & Lin, 2011).
2.2 XGameFlow Questionnaire Review
These criteria are broken down into several questions which should be answered by game players in order to measure their enjoyment. XGameFlow measures game player’s enjoyment using 58 questions. The game player should answer by marking with a number between 1 and 5. 1 means not at all, 2 means only a few or a little, 3 means fair, 4 means good, and 5 means very good.
If each question takes approximately one minute to answer, an hour is needed to finish this assessment. If a player assesses 10 games, it will take 10 hours. This questionnaire can be simplified by eliminating redundant questions or combining questions with similar meaning.
Table 1 Concentration Criterion Questions Review
No. Criterion Question What to do
1
Concentration
Does the game provide content that grabs and stimulates your attention?
Combining the first, second and third questions into:
“Does the game or the game’s missions provide content that grabs and stimulates your deep attention, so you feel that you should concentrate to finish them?”
2 Do the game tasks or missions make you feel that you should concentrate to finish them?
3 Do most of the gaming activities require your deep attention to complete?
4 Are you being distracted by other things in the game while trying to concentrate on the tasks you need to concentrate on?
Keep it.
5 Do you feel the main tasks are important enough to be completed first rather than the secondary tasks?
Eliminate it because it was covered in the first question.
Table 1 is the reviewing table for the Concentration criterion. There are five questions, but after we reviewed it, the first, second and third question can be combined into one single question, because they talk about same thing. So the first question becomes: “Does the game or the game’s missions provide content that grabs and stimulates your deep attention, so you feel that you should concentrate to finish them?” The fourth question covers different aspects, so we kept it. The fifth question has been covered in the first question, so we eliminated it. Now, the concentration criterion has only two questions.
Table 2shows how the other criteria are reviewed by combining, keeping or eliminating questions as was done in Table 1.
Table 2 Other Criterion Questions Review
No. Criterion Question What to do
1
Goal Clarity
Were the overall game goals presented clearly and at the appropriate time?
Combining the first and second questions into:
“Were the goals of the game presented clearly and at the appropriate time?” 2 Were the intermediate goals presented clearly and
at the appropriate time?
3 Do the intermediate goals support or relate to the overall goals?
Eliminate it because we don’t have to ask about intermediate goals.
4 Do you understand the goals throughout the game?
Keep it.
5 Is the feature to re-open or review the goals presented clearly?
Keep it.
1
Feedback
Do you receive feedback on your progress during the game?
Combining the first and second questions into:
“Do you receive feedback on your progress and actions throughout the game?”
3 Are you notified of new tasks and new events immediately?
Combining third and fourth questions into:
“Do you receive information on your success (or failure) andnotification about new tasks and new events immediately?” 4 Do you receive information on your success (or
failure) on intermediate goals immediately?
5 Do you receive information on your status, such as score, level or any other information related to your status?
Combining fifth and sixth questions into: “Do the game interfaces provide real-time information change about your current status such as score, level or any other information related to your status?” 6 Do the game interfaces give you real-time
information change about your current status? 1
Challenge
Are you enjoying the game without feeling bored or anxious?
Keep it.
2 Is the challenge adequate, neither too difficult nor too easy?
Keep it.
3 Does the game provide ‘‘hints” in text that helps you overcome the challenges?
Combining third, fourth and fifth questions into:
“Does the game provide ‘‘hints”, online support, or video or audio auxiliaries that help you overcome the challenges?” 4 Does the game provide ‘‘online support” that
helps you overcome the challenges?
5 Does the game provide video or audio auxiliaries that help you overcome the challenges?
1
Player Skill/ Knowledge Improvement
Does your skill gradually improve through the course of overcoming the challenges?
Keep it.
2 Are you encouraged by the improvement of your skills?
Eliminate it because it has no relation to the improvement of skill or knowledge. 3 Does the difficulty of the challenges increase as
your skills improved?
Combining third, fourth and fifth questions into:
“Does the difficulty of the challenges increase with appropriate pacing as your skills improved?
4 Does the game provide new challenges with appropriate pacing?
5 Does the game provide different levels of challenges that tailor to different players? 6 Does the challenge make you want to replay the
game when you have finished it?
Keep it.
1
Control/ Autonomy
Do you feel a sense of control the menu (such as start, stop, save, etc.)?
Combining first, second and third questions into:
“Do you feel a sense of control over the menu (such as start, stop, save, etc.) and over actions and interactions between roles or objects?”
2 Do you feel a sense of control over actions and interactions of roles or objects?
3 Do you feel a sense of control over interactions between roles or objects?
4 Does the game not allow players to make errors to a degree that they cannot progress in the game?
Keep it.
5 Does the game support you recovery from errors? Eliminate it because it has no relation with control/autonomy.
6 Do you feel that you can use strategies freely? Keep it.
7 Do you feel a sense of control and impact over the game?
Eliminate it because it was covered in the first question.
8 Do you know the next step in the game? Eliminate it because it has no relation to control/autonomy but concerns goal clarity.
1
Immersion
Do you forget about time passing while playing the game?
Combining first, second and third questions into:
“Do you become unaware of time passing, your surroundings or worries about everyday life while playing the game?”
2 Do you become unaware of your surroundings while playing the game?
3 Do you temporarily forget worries about everyday life while playing the game?
4 Do you experience an altered sense of time? Eliminate it because it was covered in the first question.
5 Do you feel emotionally involved in the game? Keep it.
7 Does the music of the game stir your emotions during play?
Keep it.
8 Do the graphics of the game affect your emotions?
Keep it.
9 Do you enjoy the story of the game? Keep it.
10 Do you memorize some of the aspects of the clearly after playing it?
Keep it.
11 Do you feel curious about how the game will continue when you must stop playing?
Keep it.
1
Social Interaction
Does the game provide the opportunity to cooperate with other players?
Combining first and second questions into:
“Can you cooperate or collaborate with other players to finish the game?” 2 Can you collaborate with other players to finish
the game?
3 Do you feel that you need someone to join you to play the game?
Eliminate it because it was covered in first question.
4 Does the game support social interaction between players (chatting, sharing items, etc)?
Keep it.
5 Does the game support communities within the game?
Combining fifth and sixth questions into: “Does the game support communities within or outside of the game?” 6 Does the game support communities outside the
game? 1
Persuasion
Does the game increase your knowledge and tell
you about something you didn’t know before? Eliminate it because it has no relation to persuasion but relates to player skill/knowledge improvement.
2 Do you try to apply knowledge in the game? Eliminate it because it has no relation to persuasion but relates to player skill/knowledge improvement. 3 Do you want to know more about something in
the game?
Eliminate it because it has no relation to persuasion but with immersion. 4 Does the game motivate you to do something? Keep it.
5 Does the game provide reasons to change your mind about something?
Keep it.
6 Can the game or some of the contents affect some aspects of your life?
Keep it.
2.3 Simplified Questionnaire
Based on Table 1 and Table 2, we can get the result of table reviews. Now, the questionnaire has
about 30 questions and varied for each criterion without losing its content to measure game player’s
enjoyment. Table 3 shows the simplified questionnaire of XGameFlow.
Table 3XGameFlow Simplified Questionnaire
No. Criterion Question
1
Concentration
Does the game or the game’s missions provide content that grabs and stimulates your deep attention, so you feel that you should concentrate to finish them?
2 Are you being distracted by other things in the game while trying to concentrate on the tasks you need to concentrate on?
3
Goal Clarity
Were the goals of the game presented clearly and at the appropriate time?
4 Do you understand the goals throughout the game?
5 Is the feature to re-open or review the goals presented clearly?
6
Feedback
Do you receive feedback on your progress and actions throughout the game?
7 Do you receive information on your success (or failure) and notification about new tasks and new events immediately?
9
Challenge
Are you enjoying the game without feeling bored or anxious?
10 Is the challenge adequate, neither too difficult nor too easy?
11 Does the game provide ‘‘hints”, online support, or video or audio auxiliaries that help you overcome the challenges?
12
Player Skill/ Knowledge Improvement
Does your skill gradually improve through the course of overcoming the challenges?
13 Does the difficulty of the challenges increase with appropriate pacing as your skills improved?
14 Does the challenge make you want to replay the game when you have finished it?
15
Control/ Autonomy
Do you feel a sense of control over the menu (such as start, stop, save, etc.) and over actions and interactions between roles or objects?
16 Does the game not allow players to make errors to a degree that they cannot progress in the game?
17 Do you feel that you can use strategies freely?
18
Immersion
Do you become unaware of time passing, your surroundings or worries about everyday life while playing the game?
19 Do you feel emotionally involved in the game?
20 Does the music of the game stir your emotions during play?
21 Do the graphics of the game affect your emotions?
22 Do you enjoy the story of the game?
23 Do you memorize some of the aspects of the game clearly after playing it?
24 Do you feel curious about how the game will continue when you must stop playing? 25
Social Interaction
Can you cooperate or collaborate with other players to finish the game?
26 Does the game support social interaction between players (chatting, sharing items, etc)?
27 Does the game support communities within or outside the game?
28
Persuasion
Does the game motivate you to do something?
29 Does the game provide reasons to change your mind about something?
30 Can the game or some of the contents affect some aspects of your life?
2.4Testing the Simplified Questionnaire
The process involves calculating the average points of each combined question, deleting the eliminated
questions’ points and after that recalculating the average points for each criterion. As we see on Table 4 (a) and (b) the difference ranged between 0.02 until 0.17. If the whole range of points is 4.00, the difference tolerance percentage is about 0.5% to 4.25% for concentration criterion. Now we will see the calculation results and the comparisonfor all of the criterion in Table 5.
Table 5Difference Before Simplified and After Simplified
Criterion
Before Simplified After Simplified Difference (After – Before) Game 1 Game 2 Game 3 Game 1 Game 2 Game 3 Game 1 Game 2 Game 3
Concentration 3.82 3.96 4.71 3.80 4.13 4.83 -0.02 0.17 0.12
Goal Clarity 3.66 3.56 3.98 3,71 3,85 4,17 0,05 0,29 0,20
Feedback 3.92 3.82 4.50 3,60 3,58 4,35 -0,32 -0,24 -0,15
Challenge 2.68 2.94 3.56 2,81 3,12 3,81 0,13 0,18 0,26
Player Skill 3.07 3.25 4.06 2,93 3,42 4,20 -0,13 0,17 0,14
Control 3.43 3.60 4.28 3,43 3,19 4,10 0,36 -0,06 0,04
Immersion 3.39 3.72 4.36 3,32 3,46 4,08 -0,07 -0,26 -0,28
Social 2.27 2.30 3.00 2,04 2,65 3,02 -0,23 0,35 0,02
Persuasion 2.72 3.35 3.56 2,80 3,14 3,64 0,09 -0,21 0,08
As we see in Table 5, the minimum tolerance is 0.02 and the maximum tolerance is 0.36. We can calculate the tolerance percentage to be between 0.5% and 9%. After testing, the maximum tolerance is still less than 10% or 0.1 of the whole range.
We can say that the simplified questionnaire can be concluded as valid. The conclusion is taken according to the manual calculation from before and after simplified.
3.
CONCLUSION
The simplified questionnaire can make the process of measurementmore efficient and effective. We have 30 questions in the simplified questionnaire compared to 58 questions in the full questionnaire. If one question need requires about one minute to answer, the player only needs half of hour to complete the entire questionnaire.This is incredibly more efficient than the full hour that would be required for the full questionnaire at the same rate of response.
The content of the questionnaire itself is not changed. This can be proven with the testing of the full questionnaire and the simplified questionnaire. The maximum tolerance was about 9% or about 0.36 points. So we can conclude that the simplified questionnaire can be used to replace the full questionnaire of
XGameFlow to measure game player’s enjoyment although in some cases the points may be extremely different.
In this research study, only three games were tested. Further research regarding measurment tools can be conducted using more test participants and more games.
REFERENCES
Crawford, Chris. 2003. Chris Crawford on Game Design. U.S.A: New Riders Publishing.
Fu, Fong-Ling, Su, Rong-Chang and Yu, Seng-Chin. 2008. EGameFlow: A scale to measure learners’ enjoyment of e-learning games. Science Direct. Computers & Education 52 (2009) 101–112.
Prestiliano, Jasson., Manongga, Danny. & Lin, Chiuhsiang Joe. 2011. XGameFlow, Extended Model for Measuring Player’s Enjoyment in Playing Games. 2nd Workshop of EAEFS, Hsinchu, Taiwan. Paper ID: 64.
Scheible, J., Tuulos, V. H. & Ojala, T. 2007. StoryMashup: Design and Evaluation of Novel Interactive Storytelling Game for Mobile and Web Users. MUM’07, December 12-14, 2007, Oulu, Finland.
Sweetser, Penelope and Wyeth, Petra. 2005. GameFlow: A Model for Evaluating Player Enjoyment in Games. ACM Computers in Entertainment, Vol. 3, No. 3, July 2005. Article 3A.
