It can give us the opportunity to understand the user's learning style so that we can design better visualization tools in the future. In this dissertation, I analyze the interaction logs of an educational visualization system, VAiRoma, to investigate how users generate insights through the system. Based on the results, users tried more exploratory interactions in the early stages of their insight generation journey.
Towards the end, they tried to show their understanding of what they learned by making an annotation. In 38% of cases, during the annotation creation process, users canceled to "create an annotation" and went back to read some text information.
Introduction
In general, students attempted more exploratory interactions for their initial steps of an insight generation path. This includes searching for a Wikipedia article, playing with a map and other visualizations in the system. Then, for most of the insight generation path, they mostly read the Wikipedia articles, searched for a new Wikipedia article, read it, or read the already visited Wikipedia article.
By the end of the journey, they mostly read the article, tried to create a note, but “cancel creation” to get more information.
Background and Related Works
Recently, there are some attempts to model user behavior and understand how people interact with visualization systems. User behavior in visual exploration path can be modeled as Markov chain process with transitions between interaction, mental and computational states [16]. This model describes the insight generation process more accurately, as it includes both cognitive and computational states that are essential for gaining insight in exploratory visual analysis.
Another cognitive model is proposed by Chen et al., where human decision making through interaction is modeled as a partially observable Markov decision process (POMDP) [11]. They tried to predict human behavior by solving POMDP using machine learning and achieved a good result. According to some of these research works, some cognitive processes are more likely to co-occur than others [13], and certain patterns of interactions are prone to insights [12].
Other research works have contributed to assist in the analysis of insight provenance by proposing fundamental taxonomies of an insight generation pathway.
Data Collection
Experiment Settings
The VAiRoma System
Log Format
Approaches for Insight Generation Path
Filtering the Paths
After the data preprocessing steps, the total number of recorded interactions is 42,935, with 138 unique interactions. As a result, I removed all Type B paths and kept only the Type A paths for my analysis. To see an overview of the pathways, I first created a histogram of number of interactions in each pathway (Figure 3).
To improve the readability of the graph, I plot only 1120 paths out of 1189, with each path consisting of less than 100 interactions. Since these paths don't have any exploration-related interactions, it's impossible to get insight origins. In the visualization system, there are only three options for the user to read the text: wiki view, tabular view, and group notes view.
A user can read a Wikipedia article from the wiki view and table view or he can read other users' notes from the group notes view. Since the user creates an annotation at the end of the exploration path, I assume there must be some meaningful information for the user while exploring. To achieve more accurate results, the three knowledge sources (eg wiki view, table view, and group notes view) require separate analysis.
Since the wiki view is frequently used compared to the other two knowledge sources, I focused on the wiki view and kept only 560 paths where users read several Wikipedia articles. An interesting pattern is that some of the paths involve the interaction of reading Wikipedia articles, but without opening the article in that path. These paths are used for the rest of this paper when I refer to knowledge generation paths.
The High-Level View of an Insight Generation Path
The "Pen" icon with a red cross indicates that the user has canceled editing an annotation. Finally, the check mark in the green circle indicates that the user has created an annotation. This visualization shows that the annotation process is not just about reading an article and making an annotation.
Although the process of creating notes varies between paths, there are also some common behaviors. For example, most of the users canceled the creation of a note several times (indicated by the "x" .icons in Figure 5). Usually they read the same article again to gather more information to finally create a note.
Although the visualization (Figure 5) gives us detailed information about each path, it is difficult to see the overview of an annotation creation process. Users read several Wikipedia articles one after the other until they feel comfortable making an entry. But even during the process of creating notes, users canceled to create a note and returned to read the same or another article.
For example, only 11 paths have an interaction sequence from "Cancel to edit note" to "Edit note", which is only 2% of all paths. The first is to create an annotation directly after several cycles of selecting and reading Wikipedia articles. Second, users repeatedly selected and read Wikipedia articles, then canceled to "create a note" and then returned to read a new one.
The Role of Visualization Interactions in Insight Generation Path
- Path to First Wikipedia Article Selection
- Paths Between Wikipedia Article Selections
- Path from Last Wikipedia Article Selection to Annotation Creation
The frequency of interaction indicates the total number of occurrences of each interaction before selecting the first Wikipedia article in all 539 paths. Since this is the most frequent interaction, we can guess that the timeline view had a big role in the initial selection of the Wikipedia article. To reveal this feature, I draw another graph in Figure 10, which shows the existence of each interaction in all paths before selecting the first Wikipedia article.
It means that searching for the article from the search box was a big interaction to find the first Wikipedia article. However, users may have preferred to use the search box rather than the timeline view for selecting the first Wikipedia article. To find the path to the selection of the first Wikipedia article, a tree-based visualization technique is used in Figure 11.
After carefully analyzing the tree, I realized that there are 2 common paths leading to the selection of the first Wikipedia article. In this section, I will analyze the interaction sequences from one selection of Wikipedia articles to another. First, reading a Wikipedia article (wiki1 view-. >elapsed time) becomes one of the most frequent interactions.
An interesting pattern is that they did not use the 'search box' much compared to the first selection of Wikipedia articles. To see the common interaction sequences from one selection of Wikipedia articles to another, I visualized the path tree in Figure 16. In this section I will analyze the interaction sequences from the last selection of Wikipedia articles to the creation of annotations.
This can be supported by Figure 19, which shows the existence of each interaction in the sequence from the last selection of a Wikipedia article to the creation of annotations. The tree visualization in Figure 20 shows the main path from the last selection of a Wikipedia article to the creation of an annotation.
Discussion
One interesting pattern is that the frequency of research interactions such as the topic tree view or the timeline view or the search box is almost negligible. This may mean that users have already completed all research interactions in the initial stages and are ready to create a note. Users launched the system through research interactions such as examining and selecting topics in the topic tree view, selecting a period from the timeline view, or searching for Wikipedia articles directly using the search box.
During the article reading state, they tried to create a note several times, but it was canceled. Based on these results, we can argue that the process of generating insights is complex and that users always go back to a previous state to gain more insights. Therefore, the cyclical behavior of the insight generation process should be considered when designing educational visualization systems.
Examples could be providing bookmarks or history saves to make easier access to information already visited. One of the future works could be to study user interaction logs based on different cycles to observe the change of user behavior over time. Another interesting research could be to see a possible effect of different knowledge generation paths on the quality of the final annotation.
Conclusion
Barish, “AnaFe: Visual analysis of image-derived temporal features—focusing on the spleen,” IEEE Transactions on Visualization and Computer Graphics, vol. Lv, “IBBAS: A Visual Analytics System of Large-Scale Traffic Data for Bus Body Advertising,” in International Conference on Parallel and Distributed Computing, Applications and Technologies (PDCAT), 2017. Wu, “SmartAdP: Visual Analytics at Scale wide Taxi,” IEEE Transactions on Visualization and Computer Graphics, vol.
Shorov, "Towards Visual Analytical Tasks for Security Information and Incident Management," in International Conference "Quality Management,Transport and Information Security, Information Technologies," 2017. Donertasli, "Analysis and Prediction of E-Customer Behavior by Mining Clickstream Data," in IEEE International Conference on Big Data, 2015. Ribarsky, “VAiRoma: A Visual Analytics System for Making Sense of Places, Times, and Events in Roman History,” IEEE Transactions on Visualization and Computer Graphics, vol.
Adam, "Peekquence: Visual Analytics for Event Sequence Data," in ACM SIGKDD Workshop on Interactive Data Exploration and Analysis, 2016. Shneiderman, "EventAction: Visual Analytics for Temporal Event Sequence Recommendation," in Proceedings of the IEEE Science Conference on Visual Analytics and Technology, 2016. Howes, “A cognitive model of how people make decisions through interaction with visual displays,” in Proceedings of the ACM CHI Conference on Human Factors in Computing Systems, 2017.
Stasko, “Requirements for Visual Interaction Analysis Systems,” in IEEE VIS Workshop on Interactive Visualization Logging and Interaction Log Visualization, 2016. Zhao, “Unsupervised Clickstream Clustering for User Behavior Analysis,” in Proceedings of the ACM CHI Conference on Human Factors in Computing Systems, 2016. Ribarsky, “Toward Effective Insight Management in Visual Analytics Systems,” in Proceedings of the IEEE Symposium on Pacific Visualization, 2009.
