Motivation

This chapter proposes a multi-view NRL framework based on subgraph, metapath, and community views for link prediction in heterogeneous graphs. In this section, we explain why it is worthwhile to consider relational paths and communities for predicting links besides the popularly used subgraph view. First, we detail on the views considered. Next, we

5.3 Motivation 119

s t

n2

n5

r₃ r2

r₁ r₇

n4 r₅

r₆ n₁ r1

r₂ r₄ n3

r4

r₃

r₃ t'

r_t

Metapath View

s t

n2

n5

e₃ (r₃) e₄ (r₂)

e₁₀ (r₁) e₁₂ (r₇) n4 e8 (r5)

e₁₁ (r₆) n₁

e₁ (r₁)

e₂ (r₂) e6 (r4) n₃ e₅ (r₄)

e₇ (r₃)

e₉ (r₃) e (r_t)

t'

Enclosing Subgraph View

I) II) III)

s t

n₂

n5 n4

n1 n3

rt t'

Community View modularity

based graph-wide collective community-context of

[CLS] A set of sampled metapaths of max length M (>1) between and [PAD]

N-hop enclosing sub-graph between , Negative sample:

Fig. 5.2 Various views to contextualize a relation triple. [Please refer to Section2.3.2for view definitions. Highlights denote different contexts, I) Blue, Purple: Subgraph, II) Cyan, Yellow:

Community, III) Green: Metapaths.]

empirically provide some insights on the structural cues that the considered views offer, supporting the formation of a targeted link.

Figure5.2illustrates the views for an input relational triple(s,r_t,t). In theEnclosing Subgraph View, the common neighborhood of the end nodes is considered, which includes vertex setVst, edge-setEstand the underlying relation semanticsRassociated with the edges.

In theCommunity View, the global connectivity patterns of the end nodes are considered based on the ground-truth modularity of the graph. This view imposes higher-order associations of the candidate nodes to the rest of the nodes in the graph based on the edge-formation characteristics of the underlying graph. In theMetapath View, a set of sampled relational paths that represent the target relationr_tare considered. The aim is to consider various views of the connectivity pattern exhibited between the end nodes.

As illustrated in Figure5.2, the proposed candidate views offer very different cues to predict a relation between a (source, target) node-pair. There has been no systematic study to understand the effectiveness and complementarity of each view, and to devise aggregation strategies for combining the views to predict links. Also, no study so far has explored the community-viewfor the task of link prediction in heterogeneous graphs. We elaborate more on this while discussing the usefulness of the views considered and proposing a novel view aggregation strategy to combine them.

5.3.1 Usefulness of metapaths

In a case study shown in Figure5.3, we plot the co-occurrence characteristics of 1−hop relation and various metapaths (of length>1) between the same set of (source, target) node

(1, 2) (0, 1) (1, 2, 2) (0, 0, 1)

(1, 2, 2, 2) (0, 1, 2, 2) (0, 1, 2)

(0, 0, 0, 1) (0, 0, 1, 2) (2, 2, 2) (2, 2, 2, 2) (2, 2) (0, 0, 0) (0, 0) (0, 0, 0, 0) (4, 2, 2) (3, 1, 2, 2) (3, 1, 2)

(5, 5, 3, 1) (6, 8, 2, 2) (4, 2, 2, 2) (3, 1) (4, 2)

(5, 3, 1) (5, 4, 2, 2) (5, 5, 4) (3, 0, 1)

(5, 5, 4, 2) (5, 4, 2) (5, 4)

(3, 0, 1, 2) (5, 6, 8, 2) (5, 5, 5, 4) (6, 7, 1) (6, 8) (6, 8, 2)

(5, 3, 0, 1) (5, 5, 6, 8) (3, 0, 0, 1) (6, 9, 8)

(5, 3, 1, 2) (6, 7, 1, 2) (6, 7, 0, 1) (5, 6, 8) (5, 6, 9, 8) (5, 5, 3) (6, 7) (6, 7, 0) (3, 0) (3, 0, 0)

(3, 0, 0, 0) (5, 5, 3, 0) (5, 5, 5, 3)

(5, 3) (5, 3, 0, 0) (5, 3, 0) (8, 2, 2) (7, 1, 2)

(7, 1, 2, 2)

(8, 2) (7, 1)

(7, 0, 1) (8, 2, 2, 2) (9, 9, 8)

(9, 9, 8, 2) (7, 0, 1, 2) (9, 8, 2, 2) (7, 0, 0, 1) (9, 7, 1, 2)

(9, 8) (9, 8, 2) (5, 5, 5, 5) (5, 5) (5, 5, 5)(5, 6)

Metapaths

1204385Relations

Co-occurrence of metapaths and the predicted relations

0.0 0.2 0.4 0.6 0.8 1.0

*Various relations are identified with numbers here. Interpretations of the relations are enlisted on the Right. Interpretations of used color-codes — Dark Blue: co-occurrence, Light Blue: no co-occurrence.

Fig. 5.3 Co-occurrence of predicted relations (y-axis) and sampled metapaths (x-axis) in PubMed. [Please refer to Table5.2for dataset description]

pairs for the PubMed dataset. To do this, we explore the HIN using a random walker and slide a dynamic window up to a fixed size over the paths to extract all possible composite relational associations between every node pair. We observe that separate sets of metapaths co-occur with various 1-hop relations, and there is no-overlapping occurrence. For example, complex semantic associations such as —i) similar species causing similar diseases via common genes(9,7,1,2)are found to co-occur with the relationSPECIES-with-DISEASE (r=8),ii) chemicals found in certain species are useful in treating certain diseases that affect genes similar to that of the species’s(6,7,0,1)are found to co-occur with the relation CHEMICAL-with-DISEASE(r=4). Therefore, without looking at the neighborhood node identities, we can predict links with a high probability between a (source, target) node pair by only looking at the relevant higher-order semantic associations that act as logical evidence to deduce the direct relation.

5.3.2 Usefulness of communities

In an interesting case-study shown in Table5.1, we consider the train-level heterogeneous graph of ACM and discover communities of nodes via modularity maximization algo- rithm [61]. Now, given a test-triple s(PAPER),r_t(HAS-TERM

,t(TERM)), we enlist all the participating nodes in the 3−hop surrounding contexts of(s,t)and in the associated meta- paths (up to length−5) between(s,t). We remove these nodes from the node-set comprising the members of the target node’stassociated community. This gives us all the nodes beyond the subgraph and relational path context of(s,t)fromt’s community. Now, we check the col- lective relevance of(s,t)witht’s community members and try to understand whether nodes in this community provide any cues to predict the target relation. To do so, we select nodes (in the second row of Table5.1) fromt’s community, which are highly similar to both test PAPERand TERMnodes (in the first row of Table5.1) based on the cosine-similarity of input node features. We observe that, for the candidate test-triple in the first row relevant terms in the second row —{parameters, probabilistic, robust}; authors with aligned research interests

5.3 Motivation 121 Paper:Adaptive event detection with time-varying poisson processes

Term: Learning

Paper:A bayesian mixture model with linear regression mixing proportions Paper:A new efficient probabilistic model for mining labeled ordered trees Paper:Community evolution in dynamic multi-mode networks

Paper:Fast logistic regression for text categorization with variable-length n-grams

Paper:Reverse testing: an efficient framework to select amongst classifiers under sample selection bias Paper:Statistical entity-topic models

Author:Indrajit Bhattacharya {Machine Learning, Probabilistic Topic Models, Bayesian Nonparametrics, Knowledge Graphs}

Author:Lise Getoor {Machine Learning, Reasoning Under Uncertainty, Graph & Network Analysis, Information Integration}

Author:Petros Drineas {Randomized Algorithms, Numerical Linear Algebra}

Term: parameters, probabilistic, unsupervised, series, statistical, significantly, mining, robust, normal, process, partial, estimation, relevance

Table 5.1 Explainability of formation of a relation(PAPER−−−−−−→^HAS-TERM TERM)influenced by community members in ACM [Please refer to Table5.2for dataset description]

like{Bayesian Nonparametrics, Probabilistic Topic Models, Reasoning Under Uncertainty};

papers on aligned topics{probabilistic model, statistical model, efficient framework} are selected. The retrieved nodes’ similarity with both the test PAPERand TERMnodes provide strong evidences for the direct relation {HAS-TERM} between them.

5.3.3 Effective aggregation of views to contextualize a triple

We also propose a novel attention mechanism to aggregate the (subgraph, metapath, and community) views based on the target relation. Our modeling intuition is based on the obser- vation that neighboring contexts, metapaths, and community associations are more beneficial for predicting links of a certain kind than others. For example, given the task of predicting the subject of a paper in a bibliographic HIN, the view-contexts based on co-authorship information might be more beneficial than the view-contexts involving citation information;

or the community-view of relevant terms for that subject might be more beneficial than complexly correlated metapaths. Again, there are existing challenges [164,165,4] in HINs as explained in Section5.2. Our proposed framework mitigates the mentioned issues by using various complementary views or a combination of them. Section5.9.4empirically establishes the robustness of the proposed framework and explains the contributions of chosen views in predicting the plausibility of relations.