4.10 Ablation Study
4.10.4 Analyzing clusters
4.10.4.2 Comparing relation-wise clustering performance
In Figure4.11, we compare unsupervised and semi-supervised node clustering strategies in a relation-wise manner. Figure4.11compares the clustering scores obtained from the layer- wise learned node-community membership matrixHr :r∈Rfor the unsupervised method DMGC and semi-supervised SSDCM. Besides the relational layers, we also compare the clustering performance obtained from the MEANcommunity matrixHMEAN= (|R|1 ∑r∈RHr) for the considered methods. This is because we do not employ any effective aggregation strategy for the learned relational clusters. Comparison is made for four candidate datasets – FLICKR, AMAZON, DBLP, and SLAP. We observe that, except for SLAP, SSDCM outperforms DMGC by significant margins in terms of NMI-C scores. However, in SLAP, DMGC is seen to perform competitively in 2 out of 6 relations, but, outperformed by SSDCM in the rest of the relational layers, as well as inHMEAN. We attribute the performance gain of SSDCM over DMGC to our label-informed cluster learning strategy that preserves label-
4.10 Ablation Study 113 correlations beside the local network contexts while logically grouping the nodes in each layer.
Summary
In this study, we propose a semi-supervised framework for representation learning in multi- plex networks. This framework incorporates a unique InfoMax-based learning strategy to maximize the MI between local and contextualized global graph summaries for effective joint modeling of nodes and clusters. Further, we use the cross-layer links to impose further regularization of the embeddings across the various layers of the multiplex graph. Our novel approach, dubbed SSDCM, improves over the state-of-the-art over a wide range of experimental settings and four distinct downstream tasks, namely, classification, clustering, visualization, and similarity search, demonstrating the proposed framework’s overall effec- tiveness. In the future, we hope to extend this work in a couple of ways. First, we hope to improve the scalability of the approach – perhaps by leveraging a graph coarsening and refinement strategy [155] within SSDCM. Second, we propose to see if the ideas we have presented can be generalized for other types of multi-layer graphs (i.e., not just multiplex networks).
Chapter 5
Structure-Aware Network Representation Learning on Heterogeneous Graphs
In this work, we present a novel approach for link prediction on heterogeneous networks – networks that accommodate multiple types of nodes as well as multiple types of relations among the nodes. Specifically, we propose a multi-view network representation learning framework to incorporate structural intuitions from the underlying graph and enrich the relational representations for link prediction. The method relies on the metapath view, the community view, and the subgraph view between a source and target node pair whose linkage is to be predicted. Furthermore, our proposed model leverages a relation-aware attention mechanism to aggregate the candidate contexts in a principled way. Empirically, we demonstrate that the proposed architecture outperforms state-of-the-art transductive and inductive methods in link prediction by a significant margin. A detailed ablation study and attention weight visualizations suggest that the chosen views are complementary and useful to predict links robustly.
5.1 Introduction
Heterogeneous graphs pose unique challenges for the task of Link Prediction (LP), such as — i) node, edge heterogeneity, ii) (under/ over) representations of edges and relational paths due to the type-imbalanced existence of nodes and edges, and, iii) link sparsity. Relying only on the local contexts surrounding a node-pair is often not sufficient to justify the formation of an edge between the end nodes. A rather useful strategy is to analyze various higher-
order associations at multiple scales between the end nodes to evidentially justify the direct interaction between the nodes. Also, distinguishing various graph structure-based cues, understanding their roles, and using them in a principled way can provide more support to the inference algorithm in predicting links, especially in challenging scenarios.
Among the higher-order multi-scale graph structures, metapaths and communities are intuitively useful in characterizing various connectivity patterns between the end nodes.
Metapaths are a schematic representation of paths and indicate various composite semantic associations between two nodes. It has been shown that they are helpful for inference [91, 12,47]. Higher-order associations between the end nodes of a link can also be captured by the community memberships of those nodes. This aids directly in link prediction, especially when the links are sparse.
Existing approaches often rely on Network Representation Learning (NRL) methods that learn network embeddings by capturing the local neighborhood contexts [148,144,156–
158,19,45,47,159,26,48], and subsequently use such representations for link prediction.
An alternative class of methods rely on metapath-aware random-walks [12, 91, 46–48], subgraphs [26], and, metagraphs [49,50] to generate a triplet representation (source, relation, target), subsequently levered for link prediction.
A few works also explore designing expressive Graph Neural Networks (GNNs) [24–26]
by learning only from the common subgraph between the end nodes for various link prediction scenarios. They demonstrate that learning from enclosing-subgraph can approximately encapsulate all the topological features that support the formation of a link between two nodes. However, these approachesdo not learn explicitly from the network structures or differentiate contributions of various unique structural cues. The tree-like message passing mechanism is a well-known bottleneck of GNNs and causes the fusing of various structural information from the neighborhoods.
Finally, the role of communities in predicting links has been investigated in traditional network science research works [160–163]. These methods rely on re-designing topologi- cal link-prediction metrics to include community information obtained from simultaneous community discovery. However, to the best of our knowledge, no work in recent times has considered communities to obtain a global view of an edge (source, relation, target).
Hence, in this work, we explore and incorporate various graph structure-based cues and analyze their ability to mitigate the challenges (refer to the following section) typically exhibited inHeterogeneous Information Networks(HINs) for the task of link prediction.