Characterizations of subregular tree languages

(1)

Characterizations of subregular tree languages

Andreas Maletti

Universität Leipzig, Germany [email protected]

CAALM, Chennai — January 24, 2019

(2)

Constituent Syntax Tree

Syntax tree forWe must bear in mind the Community as a whole

S NP1

PRP We

VP2

MD must

VP₃ VB

bear PP IN in

NP₁ NN mind

NP₂ NP₂

DT the

NN Community

PP IN as

NP₂ DT

a NN whole

(3)

Constituent Syntax Tree

Tree

T_Σ(V)for sets ΣandV is least setT oftreess.t.

1 Variables:V ⊆T

2 Top concatenation:σ(t₁, . . . ,t_k)∈T fork ∈N,σ ∈Σ,t₁, . . . ,t_k ∈T

tree language= set of trees

(4)

Constituent Syntax Tree

Tree

T_Σ(V)for sets ΣandV is least setT oftreess.t.

1 Variables:V ⊆T

2 Top concatenation:σ(t₁, . . . ,t_k)∈T fork ∈N,σ ∈Σ,t₁, . . . ,t_k ∈T

tree language= set of trees

(5)

Constituent Syntax Trees

Syntax tree is not unique

(weights are used for disambiguation)

S NP1

PRP We

VP2

VBD saw

NP2

PRP$

her NN duck

S NP1

PRP We

VP2

VBD saw

S-BAR S NP1

PRP her

VP1

VBP duck

(6)

Parses

Representations enumeration

proof trees of combinatory categorial grammars local tree languages

tree substitution languages regular tree languages

Regular tree language

L ⊆T_Σ(∅)regulariff∃congruence∼=(top-concatenation)onT_Σ(∅)s.t.

1 ∼=has finite index(finitely many equiv. classes)

2 ∼=saturatesL; i.e.L =S

t∈L[t]^∼₌

(7)

Parses

t∈L[t]^∼₌

(8)

Parses

t∈L[t]^∼₌

(9)

Regular Tree Languages

Examples for Σ ={σ, δ, α}:

2 equivalence classes(L andT_Σ(∅)\L)

L={t ∈T_Σ(∅)|t contains odd number ofα}

3 equivalence classes(“noσ”, “someσ, but legal”, illegal) L⁰={t ∈T_Σ(∅)|σ never belowδ}

(10)

Regular Tree Languages

Examples for Σ ={σ, δ, α}:

2 equivalence classes(L andT_Σ(∅)\L)

L={t ∈T_Σ(∅)|t contains odd number ofα}

3 equivalence classes(“noσ”, “someσ, but legal”, illegal) L⁰={t ∈T_Σ(∅)|σ never belowδ}

(11)

Regular tree grammar [Brainerd, 1969]

G = (Q,Σ,I,P)

alphabetQof nonterminals and initial nonterminals I⊆Q alphabet of terminalsΣ

finite set of productionsP ⊆T_Σ(Q)×Q (we writer →qfor productions(r,q))

Example productions VP₃ q5 NP₁ q2

q3 →q4

S NP₁

q1

q4 →q0

S q6 VP₂

q2 q4

→q0

(12)

Regular tree grammar [Brainerd, 1969]

G = (Q,Σ,I,P)

alphabetQof nonterminals and initial nonterminals I⊆Q alphabet of terminalsΣ

finite set of productionsP ⊆T_Σ(Q)×Q (we writer →qfor productions(r,q))

Example productions VP₃ q5 NP₁ q2

q3 →q4

S NP₁

q1

q4 →q0

S q6 VP₂

q2 q4

→q0

(13)

Regular Tree Languages

Derivation semantics and recognized tree language Regular tree grammarG = (Q,Σ,I,P)

for each productionr →q∈P

r

=⇒_G

q

generated tree language

L(G) ={t ∈T_Σ(∅)| ∃q∈I:t ⇒^∗_G q}

(14)

Derivation semantics and recognized tree language Regular tree grammarG = (Q,Σ,I,P)

for each productionr →q∈P

r

=⇒_G

q

L(G) ={t ∈T_Σ(∅)| ∃q∈I:t ⇒^∗_G q}

(15)

Regular Tree Languages

Recall 3 equivalence classes(“noσ”, “someσ, but legal”, illegal) L⁰ ={t ∈T_Σ(∅)|σ never belowδ}

C₁= [α] C₂= [σ(α, α)] C₃ = [δ(σ(α, α), α)]

Productions with nonterminalsC₁,C₂,C₃ α→ C₁ δ(C₁,C₁)→ C₁

σ(C₁,C₁)→ C₂ σ(C₁,C₂)→ C₂ σ(C₂,C₁)→ C₂ σ(C₂,C₂)→ C₂

δ(C₁,C₂)→ C₃ δ(C₁,C₃)→ C₃ δ(C₂,C₁)→ C₃ δ(C₂,C₂)→ C₃ δ(C₂,C₃)→ C₃ δ(C₃,C₁)→ C₃ δ(C₃,C₂)→ C₃ δ(C₃,C₃)→ C₃ σ(C₁,C₃)→ C₃ σ(C₂,C₃)→ C₃ σ(C₃,C₁)→ C₃ σ(C₃,C₂)→ C₃ σ(C₃,C₃)→ C₃

(16)

Recall 3 equivalence classes(“noσ”, “someσ, but legal”, illegal) L⁰ ={t ∈T_Σ(∅)|σ never belowδ}

C₁= [α] C₂= [σ(α, α)] C₃ = [δ(σ(α, α), α)]

Productions with nonterminalsC₁,C₂,C₃ α→ C₁ δ(C₁,C₁)→ C₁

σ(C₁,C₁)→ C₂ σ(C₁,C₂)→ C₂ σ(C₂,C₁)→ C₂ σ(C₂,C₂)→ C₂

δ(C₁,C₂)→ C₃ δ(C₁,C₃)→ C₃ δ(C₂,C₁)→ C₃ δ(C₂,C₂)→ C₃ δ(C₂,C₃)→ C₃ δ(C₃,C₁)→ C₃ δ(C₃,C₂)→ C₃ δ(C₃,C₃)→ C₃ σ(C₁,C₃)→ C₃ σ(C₂,C₃)→ C₃ σ(C₃,C₁)→ C₃ σ(C₃,C₂)→ C₃ σ(C₃,C₃)→ C₃

(17)

Properties 3 simple

3 most expressive class we consider

7 ambiguity,(several explanations for a generated tree) but can be removed

3 closed under all Boolean operations (union/intersection/complement: 3/3/3)

3 all relevant properties decidable(emptiness, inclusion, . . . )

(18)

Regular Tree Languages

Characterizations

finite index congruences regular tree grammars (deterministic)tree automata regular tree expressions

monadic second-order formulas . . .

(19)

Parses

Combinatory Categorial Grammars

Combinators (Compositions) Composition rules of degreek are

ax/c, cy → axy (forward rule)

cy, ax / c → axy (backward rule) withy=|₁c₁|₂· · · |_kc_k

Examples:

C D/E/D / C D/E/D

| {z }

degree 0

D/E/D D/E / C D/E/E / C

| {z }

degree 2

(23)

Combinatory Categorial Grammar (CCG) (Σ,A,k,I,L)

terminal alphabetΣand atomic categoriesA maximal degreek ∈N∪ {∞}of composition rules initial categoriesI ⊆A

lexiconL ⊆Σ× C(A)withC(A)categories overA

Notes:

always all rules up to the given degreek allowed k-CCG= CCG using all composition rules up to degreek

(24)

Combinatory Categorial Grammars

Combinatory Categorial Grammar (CCG) (Σ,A,k,I,L)

terminal alphabetΣand atomic categoriesA maximal degreek ∈N∪ {∞}of composition rules initial categoriesI ⊆A

lexiconL ⊆Σ× C(A)withC(A)categories overA

Notes:

always all rules up to the given degreek allowed k-CCG= CCG using all composition rules up to degreek

(25)

c..

....

..

C c..

C

d..

D/E/D / C D/E/D

d..

... D/E / C D/E/E / C D/E/E

e..

....

. E D/E

e..

....

E D

2-CCG generates string language LwithL ∩c⁺d⁺e⁺ ={cⁱdⁱeⁱ|i≥1}

for initial categories {D}

L(c) ={C}

L(d) ={D/E / C, D/E/D / C} L(e) ={E}

(26)

Combinatory Categorial Grammars

allow (deterministic) relabeling(to allow arbitrary labels) treet min-height bounded byk

if the minimal distance from each node to a leaf is at mostk

Theorem

(Under relabeling)Class of proof trees of 0-CCGs

=class of min-height bounded binary regular tree languages

joint work withMarco Kuhlmann

(27)

Combinatory Categorial Grammars

Theorem

(Under relabeling)Class of proof trees of 1-CCGs (class of binary regular tree languages

Theorem

(Under relabeling^∗)Class of proof trees of∞-CCGs (class of simple context-free tree languages

(28)

Theorem

(Under relabeling)Class of proof trees of 1-CCGs (class of binary regular tree languages

Theorem

(Under relabeling^∗)Class of proof trees of∞-CCGs (class of simple context-free tree languages

(29)

Combinatory Categorial Grammars

ax/(by) byα/c

axα/c c

axα

−→R1 ax/(by)

byα/c c byα axα

c

byα / c ax / (by) axα / c axα

−→R2

c byα / c

byα ax / (by) axα

c

ax/(by) byα / c axα / c axα

−→R3 ax/(by)

c byα / c byα axα

byα/c ax / (by)

axα/c c

axα

−→R4

byα/c c

byα ax / (by) axα

(30)

Properties 3 simple

7 ambiguity(several explanations for each recognized tree) 7 not closed under Boolean operations

(union/intersection/complement: 3/?/7^∗) 3 closed under (non-injective) relabelings

? decidability of membership for subregular classes (0-CCG & 1-CCG) of a regular tree language

(31)

Tree Languages

Representations enumerate trees

Local tree grammar [Gécseg, Steinby 1984] Local tree grammar= finite set of legal branchings (together with a set of root labels)

G = (Σ,I,P)withI ⊆ΣandP ⊆S

k∈NΣ×Σ^k

(32)

Tree Languages

Local tree grammar [Gécseg, Steinby 1984]

Local tree grammar= finite set of legal branchings (together with a set of root labels)

G = (Σ,I,P)withI ⊆ΣandP ⊆S

k∈NΣ×Σ^k

(33)

Local Tree Languages

Example (with root label S)

S→NP₁VP₂ VP₂→MD VP₃ NP₂ →NP₂PP VP₃→VB PP NP₂

MD→must . . .

(34)

Local Tree Languages

MD→must . . .

S NP₁ PRP We

VP₂ MD must

VP₃ VB

bear PP IN in

NP1 NN mind

NP₂ NP2 DT the

NN Community

PP IN as

NP2 DT

a NN whole

(35)

MD→must . . .

S NP₁ PRP We

VP₂ MD must

VP₃ VB

bear PP IN in

NP1 NN mind

NP₂ NP2 DT the

NN Community

PP IN as

NP2 DT

a NN whole

(36)

MD→must . . .

S NP₁ PRP We

VP₂ MD must

VP₃ VB

bear PP IN in

NP1 NN mind

NP₂ NP2 DT the

NN Community

PP IN as

NP2 DT

a NN whole

(37)

Local Tree Languages

MD→must . . .

S NP₁ PRP We

VP₂ MD must

VP₃ VB

bear PP IN in

NP1 NN mind

NP₂ NP2 DT the

NN Community

PP IN as

NP2 DT

a NN whole

(38)

MD→must . . .

S NP₁ PRP We

VP₂ MD must

VP₃ VB

bear PP IN in

NP1 NN mind

NP₂ NP2 DT the

NN Community

PP IN as

NP2 DT

a NN whole

(39)

not closed under union these singletons are local

S NP₂ PRP$

My NN dog

VP₁ VBZ sleeps

S NP₂

DT The

NN candidates

VP₂ VBD scored

ADVP RB well

but their union cannot be local

(as we also generate these trees — overgeneralization)

(40)

Local Tree Languages

not closed under union these singletons are local

S NP₂ DT The

NN candidates

VP₁ VBZ sleeps

S NP₂ PRP$

My NN dog

VP₂ VBD scored

ADVP RB well

but their union cannot be local

(as we also generate these trees — overgeneralization)

(41)

Properties 3 simple

3 no ambiguity(unique explanation for each recognized tree) 7 not closed under Boolean operations

(union/intersection/complement: 7/3/7) 7 not closed under (non-injective) relabelings 3 locality of a regular tree language decidable

(42)

Tree Languages

Tree substitution grammar [Joshi, Schabes 1997] Tree substitution grammar= finite set of legal fragments (together with a set of root labels)

G = (Σ,I,P)withI ⊆Σand finiteP ⊆T_Σ(Σ)

(43)

Tree Languages

Tree substitution grammar [Joshi, Schabes 1997]

Tree substitution grammar= finite set of legal fragments (together with a set of root labels)

G = (Σ,I,P)withI ⊆Σand finiteP ⊆T_Σ(Σ)

(44)

Tree Substitution Languages

Typical fragments [Post 2011]

VP VBD NP CD

PP

S NP PRP

VP

S NP VP

TO VP

Derivation step ξ ⇒_G ζ ξ =c

root(t)

andζ =c t

for some context c and fragmentt ∈P

(45)

Tree Substitution Languages

Tree substitution grammarG = (Σ,I,P) for each fragmentt ∈P with root labelσ

σ

=⇒_G

t

L(G) ={t ∈T_Σ(∅)| ∃σ∈I:σ ⇒^∗_G t}

(46)

Tree substitution grammarG = (Σ,I,P) for each fragmentt ∈P with root labelσ

σ

=⇒_G

t

L(G) ={t ∈T_Σ(∅)| ∃σ∈I:σ ⇒^∗_G t}

(47)

Tree Substitution Languages

Fragments

S NP₁(PRP),VP₂

PRP(We) VP₂ MD,VP₃(VB,PP,NP₂)

MD(must)

Derivation

S NP₁ PRP We

VP₂ MD must

VP₃ VB PP NP2

(48)

Fragments

S NP₁(PRP),VP₂

MD(must)

Derivation

S NP₁ PRP We

VP₂ MD must

VP₃ VB PP NP2

(49)

Fragments

S NP₁(PRP),VP₂

MD(must)

Derivation

S NP₁ PRP We

VP₂ MD must

VP₃ VB PP NP2

(50)

Fragments

S NP₁(PRP),VP₂

MD(must)

Derivation

S NP₁ PRP We

VP₂ MD must

VP₃ VB PP NP2

(51)

Fragments

S NP₁(PRP),VP₂

MD(must)

Derivation

S NP₁ PRP We

VP₂ MD must

VP₃ VB PP NP2

(52)

Tree Substitution Languages

Fragments

S NP₁(PRP),VP₂

MD(must)

Derivation

S NP₁ PRP We

VP₂ MD must

VP₃ VB PP NP2

(53)

Fragments

S NP₁(PRP),VP₂

MD(must)

Derivation

S NP₁ PRP We

VP₂ MD must

VP₃ VB PP NP2

(54)

Fragments

S NP₁(PRP),VP₂

MD(must)

Derivation

S NP₁ PRP We

VP₂ MD must

VP₃ VB PP NP2

(55)

Fragments

S NP₁(PRP),VP₂

MD(must)

Derivation

S NP₁ PRP We

VP₂ MD must

VP₃ VB PP NP2

(56)

Fragments

S NP₁(PRP),VP₂

MD(must)

Derivation

S NP₁ PRP We

VP₂ MD must

VP₃ VB PP NP2

(57)

Tree Substitution Languages

Fragments

S NP₁(PRP),VP₂

MD(must)

Derivation

S NP₁ PRP We

VP₂ MD must

VP₃ VB PP NP2

(58)

Fragments

S NP₁(PRP),VP₂

MD(must)

Derivation

S NP₁ PRP We

VP₂ MD must

VP₃ VB PP NP2

(59)

Fragments

S NP₁(PRP),VP₂

MD(must)

Derivation

S NP₁ PRP We

VP₂ MD must

VP₃ VB PP NP2

(60)

not closed under union

these languages are tree substitution languages individually

S C C C a

a

S C C C b

b

L₁={S(Cⁿ(a),a)|n∈N} L₂={S(Cⁿ(b),b)|n∈N} but their union is not

(exchange subtrees below the indicated cuts)

(61)

Tree Substitution Languages

not closed under union

these languages are tree substitution languages individually

S C C C a

a

S C C C b

b

L₁={S(Cⁿ(a),a)|n∈N} L₂={S(Cⁿ(b),b)|n∈N} but their union is not

(exchange subtrees below the indicated cuts)

(62)

not closed under intersection

these languagesL₁andL₂are tree substitution languages individually forn≥1 and arbitrary x₁, . . . ,xn ∈ {a,b}

S’

x₁ S x1 S

x₂ S x₂ S

x3 S x_n−1 S

xn S xn c

∈L₁

S’

x₁ S x₂ S

x₂ S x3 S

x₃ S xn−1 S

xn−1 S x_n c

∈L₂

but their intersection only contains trees withx₁=x₂=· · ·=x_n and is not a tree substitution language

(63)

Tree Substitution Languages

not closed under intersection

these languagesL₁andL₂are tree substitution languages individually forn≥1 and arbitrary x₁, . . . ,xn ∈ {a,b}

S’

x₁ S x1 S

x₂ S x₂ S

x3 S x_n−1 S

xn S xn c

∈L₁

S’

x₁ S x₂ S

x₂ S x3 S

x₃ S xn−1 S

xn−1 S x_n c

∈L₂

but their intersection only contains trees withx₁=x₂=· · ·=x_n and is not a tree substitution language

(64)

not closed under complement

this languageLis a tree substitution language

S A A A⁰ A⁰ a

∈L

S B B B⁰ B⁰ b

∈L

but its complement is not

(exchange as indicated in red)

(65)

Tree Substitution Languages

not closed under complement

this languageLis a tree substitution language

S A A A⁰ A⁰ a

∈L

S B B B⁰ B⁰ b

∈L

S A A A⁰ A⁰ b

∈/L

S B B A⁰ A⁰ a

∈/L

but its complement is not (exchange as indicated in red)

(66)

Properties 3 simple

3 contain all finite and co-finite tree languages

7 ambiguity(several explanations for a generated tree) 7 not closed under Boolean operations

(union/intersection/complement: 7/7/7)

3 can express many finite-distance dependencies (extended domain of locality)

(67)

Tree Substitution Languages

Open questions

multiple intersections more expressive?

which regular tree languages are tree substitution languages?

relation to local tree languages?

extension to weights application to parsing

Thank you for your attention!

(68)

Open questions

Thank you for your attention!

(69)

Open questions

Thank you for your attention!

(70)

Experiment [Post, Gildea 2009]

grammar size Prec. Recall F₁ local 46k 75.37 70.05 72.61

“spinal” TSG 190k 80.30 78.10 79.18

“minimal subset” TSG 2,560k 76.40 78.29 77.33

(on WSJ Sect. 23)

(71)

Tree Substitution Languages with Latent Variables

Experiment [Shindo et al. 2012]

F1 score

grammar |w| ≤40 full

TSG [Post, Gildea, 2009] 82.6 TSG [Cohnet al., 2010] 85.4 84.7 CFGlv [Collins, 1999] 88.6 88.2 CFGlv [Petrov, Klein, 2007] 90.6 90.1

CFGlv [Petrov, 2010] 91.8

TSGlv (single) 91.6 91.1

TSGlv (multiple) 92.9 92.4

Discriminative Parsers

Carreraset al., 2008 91.1

Charniak, Johnson, 2005 92.0 91.4

Huang, 2008 92.3 91.7