Report on Modeling Discrete Dynamic Topics

(1)

Report on Modeling Discrete Dynamic Topics

Zijing Zhang (Zing)

5/21/2019 Report on Modeling Discrete Dynamic Topics 1

(2)

Issue

How to model topic trend? *

• Medium

• Academic paper

• Blog post

• Tweet

• Data Stream

• Continuous

• Discrete

• Change-detection

• Low delay

(3)

Dataset Input [1]

• Signal Media

• a publicly available dataset

• 985,867 articles

• Average length 405 words

• from September 1 till September 30, 2015.

• removing stop words, URLs, tokens not starting with alphabet letters, punctuation marks, and words which occur less than 5 times.

• NFCWorld Twitter channel

• emerging technologies

• the Twitter API

• 3,374 tweets

• removing stopwords, URLs, hashtag signs, tokens not starting with alphabet letters, punctuation marks, and words occurring less than 3 times.

(4)

Vocabulary *

• Generative Model

A model of joint probability distribution as how data have been generated.

• Dirichlet Distribution

A probability distribution of category probabilities adding up to 1.

• Kalman ﬁlter

A linear quadratic estimator that fuses noisy observable data series to guess the hidden state of variable.

• Logit

The logarithm of the odds p/(1-p)

• Logistic Normal Distribution

A probability distribution with logit as normal distribution

• Bayesian Information Criterion

Model selection criterion based on statistical principle

• Latent variable:

Variable inferred from other observed variables.

• Hidden Markov Model

Model which infers target hidden state of a random variable from another observable random variable's emission sequence.

• Negative Log Likelihood

Surprisal of a model

5/21/2019 Report on Modeling Discrete Dynamic Topics * Interpretation 4

(5)

Latent Dirichlet Allocation [4]

A generative model that categorizes observations with imaginary vocabulary. *

Document is a distribution of topics. * Topic is a distribution of words. *

(6)

Dynamic Topic Model [1]

A generative model that analyzes the topic evolution over continuous time slices from document stream *

• Divides data into diﬀerent time slices by using the document timestamps.

• Models topics of each time slice starting from the ﬁrst one using LDA

• Uses Kalman ﬁlter to compute the evolution of each topic over time.

(7)

Pros & Cons Prior Works

• LDA [2]

• Pros:

• Effective assumptions for topic modeling

• Cons:

• Topic number

• Lacking correlation detection among topics

• DTM [1]

• Pros:

• Connects the same topics over time

• Cons:

• New topic lag. [1]

• Topic continuous. [1]

• Effectiveness of chaining with Dirichlet prior *

(8)

Discrete Dynamic Topic Model

uses BIC to discover the best number of topic chains [1]

(9)

Performance of dDTM

(10)

Result of Signal Media Dataset

(11)

Result of NFCWorld Dataset

(12)

Future [1]

• apply dDTM for analyzing trending topics and compare dDTM against other methodologies developed in this domain

• A comparison of diﬀerent methods for computing HMM state probabilities such as Gaussian mixture models

• the time slice length allocation

• conduct a more in-depth analysis of dDTM which would include

analysis of the topic chains’ quality with respect to human judgment

(13)

Report on Modeling Discrete Dynamic Topics