Recurrent Neural Networks (RNNs) dengan Keras - IR-NLP Lab

Recurrent Neural Networks (RNNs) dengan Keras

Alfan Farizki Wicaksono Fakultas Ilmu Komputer

Universitas Indonesia

RNNs

X1 X2 X3 X4 X5

h1 h2 h3 h4 h5

O1 O2 O3 O4 O5

One of the famous Deep Learning Architectures in the NLP community

U V

W

Parameters are shared along timesteps

A time step

Xt ht Ot

U ϵ R

^HxN

V ϵ R

^2xH

Example: RNNs for POS Tagger

I went to west java

h1 h2 h3 h4 h5

PRP VBD TO JJ NN

Example: RNNs for Named-Entity Recognition

http://dirko.github.io/Bidirectional-LSTMs-with-Keras/

Text Classification : Single-Output RNNs

I went to west java

h1 h2 h3 h4 h5

Label: Travel We only need one single output !

Advantage: The history is unbounded !

Varian dari RNNs

RNNs suffer from vanishing gradient problem.

• LSTMs (Long Short Term Memory)

• GRUs (Gated Recurrent Unit)

Please read the following nice blog post!

http://colah.github.io/posts/2015-08-Understanding-LSTMs/

Perhatikan Penjelasan dari 4 kode berikut

• rnnsingle.py

• maskingrnn.py

• rnnembedding.py

• rnnsinglebatch.py