Problems with simple CFGs (1)

Agreement

Swahili nouns and verbs

Wavulana watatu wadogo walianguka.
boys     three  small  fell
'Three small boys fell.'

Viti   vitatu vidogo vilianguka.
chairs three  small  fell
'Three small chairs fell.'

Nyumba   tatu  ndogo zilianguka.
house(s) three small fell
'Three small houses fell.'

Meno  matatu madogo yalianguka.
teeth three  small  fell
'Three small teeth fell.'

Miti  mitatu midogo ilianguka.
trees three  small  fell
'Three small trees fell.'

In a CFG for Swahili, we seem to need separate NP rules and separate S rules for each class of nouns. For English Ss (which are much simpler), we would have
```
S -> NP_sing VP_sing
S -> NP_plur VP_plur
```

Long-distance dependencies
- What did you say you thought Bill wanted to wear?
- The ordinary rules fail to apply here because wear, a transitive verb, does not have an immediately following direct object.
- We seem to need separate rules for each way in which a clause like Bill wanted to wear (something) can be embedded in a question which asks about the "something", rules like the following
  ("TVP-NP" means "transitive VP without its normal NP", "S-NP" means "sentence without an expected NP", and "SCV" means "verb taking a sentence complement" like say)
```
S -> NP VP
S -> WH+N Aux NP TVP-NP
S -> WH+N Aux NP SCV S-NP
```
- There would need to be many of these, they would not permit embedding at an arbitrary level, and they would not show how questions are related to the corresponding statements.
Ambiguity
- Ambiguity in a CFG results from the same terminal or non-terminal appearing on the right-hand sides of multiple rules.
```
S -> NP V NP
NP -> NP S-NP
S-NP -> NP V
NP -> 'police'
V -> 'police'
```
- For sentences like police police police ..., the number of parses is `((2n)!)/((n+1)!n!)`

CFGs with complex categories

HPSG, LFG, etc.: the basic idea
- Non-terminals in rules are replaced by feature structures
- During parsing, unification replaces simple matching to reinforce constraints built into the feature structures in the grammar

Handling Swahili agreement

# Subjects agree with predicates on cls
S -> NP[cls=?c] VP[cls=?c]
# NPs get their cls from their head N; adjectives have to agree with the noun
NP[cls=?c] -> N[cls=?c] | N[cls=?c] Adj[cls=?c]
# VPs get their cls from their head V
VP[cls=?c] -> V[cls=?c]
# A small lexicon
N[cls=2] -> 'wavulana'
N[cls=8] -> 'viti'
N[cls=4] -> 'miti'
Adj[cls=2] -> 'wadogo'
Adj[cls=8] -> 'vidogo'
Adj[cls=4] -> 'midogo'
V[cls=2] -> 'walianguka'
V[cls=8] -> 'vilianguka'
V[cls=4] -> 'ilianguka'

Handling subcategorization

Subcategorization groups words, especially verbs, by what kinds of arguments they take.

In place of atomic categories like TV and IV, we can use a subcat feature to distinguish the different categories.

S -> NP VP
SBar -> Comp S
VP -> V[subcat=intrans]
VP -> V[subcat=trans] NP
VP -> V[subcat=clause] SBar
# Lexicon
V[subcat=intrans] -> 'walk' | 'sing'
V[subcat=trans] -> 'see' | 'like'
V[subcat=claus] -> 'say' | 'claim'
NP -> 'you' | 'cats'
Comp -> 'that'

Handling long-distance dependencies

The slash feature, which has a POS category as its value; /NP means that the constituent with this feature is missing an NP

A set of rules to handle fronting in English; the inv feature represents inversion; "/?x" abbreviates "[slash=?x]".

S[-inv] -> NP VP
S[-inv]/?x -> NP VP/?x
S[-inv] -> NP S/NP
S[-inv] -> Adv[+neg] S[+inv]
S[+inv] -> V[+aux] NP VP
S[+inv]/?x -> V[+aux] NP VP/?x
SBar -> Comp S[-inv]
SBar/?x -> Comp S[-inv]/?x
VP -> V[subcat=intrans, -aux]
VP -> V[subcat=trans, -aux] NP
VP/?x -> V[subcat=trans, -aux] NP/?x
VP -> V[subcat=clause, -aux] SBar
VP/?x -> V[subcat=clause, -aux] SBar/?x
VP -> V[+aux] VP
VP/?x -> V[+aux] VP/?x
# Lexicon
V[subcat=intrans, -aux] -> 'walk' | 'sing'
V[subcat=trans, -aux] -> 'see' | 'like'
V[subcat=clause, -aux] -> 'say' | 'claim'
V[+aux] -> 'do' | 'can'
NP[-wh] -> 'you' | 'cats'
NP[+wh] -> 'who'
Adv[+neg] -> 'rarely' | 'never'
NP/NP ->
Comp -> 'that'

Variations (in HPSG)
- Order is represented separately from constituency relations.
- Words and phrases are typed feature structures organized in a type hierarchy.
Code for this page

Probabilistic context-free grammars

Simple CFGs, with or without categories replaced by feature structures, can't help us prefer one analysis of a sentence over another
Treebanks: corpora annotated with parse trees

Probabilistic (or stochastic) CFGs have a probability associated with each rule: the probability of the right-hand side, given the left-hand side of the rule. Thus the sum of the probabilities for the rules with a given LHS must be 1.

S -> NP VP [.8] | Aux NP VP [.15] | VP [.05]
NP -> Pro [.35] | PN [.3] | Det Nom [.2] | Nom [.15]
Nom -> N [.75] | Nom N [.2] | Nom PP [.05]
VP -> V [.35] | V NP [.2] | V NP PP [.1] | V PP [.15] | V NP NP [.05] | VP PP [.15]
PP -> Prep NP [1.0]
Det -> 'that' [.1] | 'a' [.3] | 'the' [.6]
N -> 'book' [.1]|'flight' [.3]|'meal' [.15]|'money' [.05]|'flights' [.3]|'dinner' [.1]
V -> 'book' [.3] | 'include' [.3] | 'prefer' [.4]
Pro -> 'I' [.4] | 'she' [.05] | 'me' [.15] | 'you' [.4]
PN -> 'Houston' [.6] | 'NWA' [.4]
Aux -> 'does' [.6] | 'can' [.4]
Prep -> 'from' [.3] | 'to' [.3] | 'on' [.2] | 'near' [.15] | 'through' [.05]

The probability of a given analysis of a sentence is the product of the probabilities of all of the rules used in the analysis (assuming statistical independence of all of the rules).
For disambiguation, select the parse for a sentence that has the highest probability. Even better, guide parsing with the probabilities associated with the rules. In the above example, the preferred parse of book the dinner flight is
```
(S
  (VP
    (V book)
    (NP (Det the) (Nom (Nom (N dinner)) (N flight))))) (p=1.62e-06)
```
Learning the probabilities in a PCFG using a treebank
Code for this page

What is syntax for?

Constituency grammars

Problems with simple CFGs (1)

CFGs with complex categories

Probabilistic context-free grammars