CGELBank: CGEL as a Framework for English Syntax Annotation

Brett Reynolds Affiliation: Humber College Email: brett.reynolds@humber.ca Aryaman Arora Nathan Schneider Affiliation: Georgetown University Affiliation: {aa2190, nathan.schneider}@georgetown.edu

Abstract

We introduce the syntactic formalism of the Cambridge Grammar of the English Language (CGEL) to the world of treebanking through the CGELBank project. We discuss some issues in linguistic analysis that arose in adapting the formalism to corpus annotation, followed by quantitative and qualitative comparisons with parallel UD and PTB treebanks. We argue that CGEL provides a good tradeoff between comprehensiveness of analysis and usability for annotation, which motivates expanding the treebank with automatic conversion in the future.

1 Introduction

Parsing hierarchical syntactic structure is a central endeavour in computational linguistics (Jurafsky and Martin, 2021; Kübler et al., 2009). Many syntactic theories and annotation frameworks exist. The venerable Penn Treebank (Marcus et al., 1994, PTB;) has been the leading approach to annotating English constituent structure; its detailed annotation guidelines have been applied to many corpora over the years (Bies et al., 2012; Pradhan et al., 2013, e.g.,). Other theories applied to large-scale annotation for English have included Universal Dependencies (Nivre et al., 2016, UD;), Combinatory Categorial Grammar (Hockenmaier and Steedman, 2007), Role and Reference Grammar (Bladier et al., 2018), Head-Driven Phrase Structure Grammar (Miyao et al., 2004), and so on. Each formalism makes different theoretical claims (e.g., are there transformations?) and computational tradeoffs (e.g. complexity vs. parsing efficiency).

Figure 1: CGELBank-style tree for interrogative clause which Liz bought (as in There are three options; I wonder which Liz bought).

In this paper, we introduce a treebank for English built on the syntactic formalism of the Cambridge Grammar of the English Language (Huddleston and Pullum, 2002, CGEL;). CGEL is the authoritative descriptive grammar for English and analyses many syntactic phenomena in extreme detail with minimal theoretical claims (section 2). Our motivations for introducing yet another treebank are: (a) existing annotation guidelines for other formalisms cannot approach the depth of CGEL; (b) neither of the most common frameworks for descriptive annotation in English (PTB and UD) offers a coherent account of both constituent structure and grammatical functions; and (c) annotating a real-world treebank is a strong test of the expressivity and consistency of the CGEL formalism.

Below, we delve into the linguistic issues surmounted in building the initial version of CGELBank (section 3).¹¹ 1 https://github.com/nert-nlp/cgel/ We then compare the CGEL trees with UD and PTB trees of the same sentences (section 4).

2 What is CGEL?

CGEL is the most comprehensive and up-to-date descriptive grammar of English (Brew, 2003). It uses a morphosyntactic formalism for describing English from first principles (Pullum and Rogers, 2008), wherein the hierarchical structuring of spans as constituents is supplemented with labelling of the dependency-like grammatical functions between them.

CGEL posits a distributionally-defined set of nine lexical categories, from which we developed a part-of-speech tagset with 11 tags: N (noun), N_pro (pronoun), V (verb), V_aux (auxiliary verb), P (preposition), D (determinative), Adj (adjective), Adv (adverb), Sdr (subordinator), Coordinator, and Int (interjection). Pronouns and proper nouns are a subset of nouns, though we have created a distinct tag for pronouns; auxiliary verbs are a subset of verbs. All of these categories except subordinator and coordinator project higher-level phrasal constituents, e.g. N $\leftarrow$ Nom (nominal) $\leftarrow$ NP (noun phrase).

Each CGEL phrase has exactly one Head function along with zero or more dependents. There are also a few non-phrasal dependent constituents in flat structures (e.g. Coordinate). Phrases are typically binary- or unary-branching, but $n$ -ary branches are also possible. CGEL contrasts adjuncts (Mod, Supplement) with complements (Comp and subtypes shown in figure 4 of appendix A). It also analyses many complex syntactic phenomena often ignored in computational work, e.g. gaps (section 3.3) and scope of coordination (section 3.3.4).

Multilingual syntactic formalisms such as Universal Dependencies (Nivre et al., 2016, UD;) lack the expressivity of CGEL for specifically English due to the necessity of cross-lingual consistency.²² 2 E.g., the general principle of content-heads, leading to verbal auxiliaries being treated as dependents of the main verb in English but main-verb copulae (which are not used in some languages) being analysed as dependents of the predicate. CGEL also differs from the Penn Treebank (Marcus et al., 1994, PTB;) in associating grammatical functions to each edge between phrasal categories in a tree, adopting the concept of headedness from dependency grammars. CGEL also analyzes “transformational” phenomena in English with gaps, but in a less overwhelming manner than PTB, limiting their uses to ellipsis and relations that would otherwise be non-projective.

As a whole, we find that CGEL combines the best of both worlds—comprehensive analysis of complex and long-tail linguistic phenomena with the minimal parse complexity needed to express those, as well as both dependency and constituency layers. And it does so in human-readable fashion, precisely defining its terminology and defending its analyses in a nearly 2000-page volume that is widely referenced by linguists. Finally, a companion textbook (Huddleston and Pullum, 2005; Huddleston et al., 2021) introduces language learners to the major points of English syntax. A CGEL-style treebank (CGELBank), potentially with a parser, would therefore be of interest to English learners familiar with the framework.

3 Linguistic decisions

Despite its breadth, depth, and specificity Culicover (2004), there are elements of English grammar that are not fully specified in CGEL. While CGEL does use some corpus data in its analyses and descriptions of English, the grammar is largely based on contrived examples. CGEL also only describes an idealized standard variety of English—internet-sourced text is out-of-distribution. As a treebank forces us to be explicit in our linguistic decisions, here we describe a number of the issues we ran into and the (not necessarily final) decisions we made.

3.1 Categorizing individual lexemes

Designing part-of-speech (POS) tagsets and delineating boundaries between tags has long been a contentious problem in treebanking (Atwell, 2008). CGEL is detailed but non-exhaustive in this regard.

In developing CGELBank, we had to collate the many mentions of lexemes and their categories distributed throughout CGEL,³³ 3 E.g., “One borderline case is else. This is an adverb when following or, as in Hurry up or else you’ll miss the bus, but arguably a preposition when it postmodifies interrogative heads and compound determinatives.” (CGEL, fn. 5, p. 15) along with the careful application of CGEL principles to the categorization of hundredes of lexemes not explicitly mentioned.⁴⁴ 4 Carried out since 2006 in consultation with Huddleston and Pullum and recorded in the Simple English Wiktionary. Examples of words categorized this way include the determinative said (e.g., as in said contract), the coordinator slash (e.g., Dear God slash Allah slash Buddha slash Zeus), and the preposition o’clock (Pullum and Reynolds, 2013).

3.2 Simplifying and un-simplifying

As shown in CGEL’s figure 5b (p. 48; reproduced in the appendix as figure 5), CGEL uses a variety of subtypes of head within clause structure: Nucleus is the head of a clause which is itself a clause, Predicate is a VP that is the head of a clause, and Predicator is the V that is the head of a VP. We dispense with these subtypes, simply using Head in all cases, as shown in figure 1.

In some cases, CGEL simplifies tree structures by removing intermediate unary nodes, such as by removing an intermediate Head:Nom between Head:N and its projected NP. We always include such nodes, as in figure 1. Moreover, we show complements as sisters of N and V but modifiers as sisters of Nom and VP, where CGEL again simplifies at times.

3.3 Gaps

CGEL posits gaps in tree structures when a constituent appears in prenucleus position, as in which Liz bought in figure 1, but is inconsistent in indicating it. In most cases, we have decided to explicitly indicate a gap. We identify the following unclear cases and present our decisions.

3.3.1 Subject gaps

In open interrogatives such as (3.3.1a), “inversion accompanies the placement in prenuclear position of a non-subject interrogative phrase” (CGEL, p. 95), while there is no inversion in those like (3.3.1b).

\ex

. Ẇhat did she tell you? .̱ Who told you that?

This suggests two possible analyses for the structure of (3.3.1b): either they both have a prenucleus and a co-indexed gap, or only (a) does, the who in (b) being a normal subject Maling (2000). Unfortunately, it is not clear to us which position CGEL takes. The discussion on p. 96 suggests that there is no inversion in (3.3.1b) because there is no prenucleus, and thus no gap. However, this is not a consistent rule throughout the text.⁵⁵ 5 CGEL recognizes explicitly that a subject gap may exist in a construction like Who_i [do you think [ _i was responsible]]? (p. 1082), but this involves a subordinate clause. Given the ambiguity, we have taken what we see as the standard position that there is indeed a gap (Maling, 2000; Bies et al., 1995, e.g.) in clauses with subjects that have been questioned or relativized.

3.3.2 Adjunct gaps

The issue here concerns adjuncts such as the PP in after lunch, we left and whether or not they are co-indexed to a VP-final gap. CGEL recognizes that “adjuncts may also be located in prenuclear position” (p. 1372), but “there need be no anaphoric link with a pronoun [or gap] in the nucleus of the clause” (p. 1410), as in As for the concert-hall, the architect excelled herself, where the underlined section is in prenucleus position but cannot appear clause finally. Other examples of items appearing in pre- or postnuclear position without a corresponding gap include the relative clause in an it-cleft like It’s the director who was sacked and markers (e.g., whether it works, p. 956).

Because adjuncts appear in a variety of locations in clause structure, with some not appearing clause finally, we have decided against including a gap. The exception is in relative and open interrogative clauses, where CGEL explicitly marks a gap (e.g., That’s not the reason [why_i [he did it _i]], p. 1086).

3.3.3 Phrasal genitives

CGEL has the genitive suffix ’s attaching to the last word in an NP such as [somebody local’s]. In the case of an NP ending in a gap, we take the ’s as attached to that gap, as in a guy I know ’s house.

3.3.4 Coordination and comparatives

CGEL calls coordinations such as The PM arrived at six and the Queen an hour later Gapped Coordination, and writes examples like this with a “gap” as shown here between the Queen and an hour later (p. 1338). But this is not the same kind of gap we find in long-distance dependencies such as the subject relatives discussed above; rather, this is ellipsis. Consequently, CGELBank does not include a gap in tree structure. Instead, we have a nonce coordinate NP + PP with daughters Subj:NP + PredComp:PP or the like. Similarly, we do not treat the “gaps” in comparatives as gaps in tree structure.

3.4 Branching & tree structure

3.4.1 Extraposition

CGEL posits extraposed subjects and objects, such as the underlined clause in It’s a good thing that we left early,⁶⁶ 6 These are semantic agents, patients, etc., but not syntactic Subj and Obj. saying that its extraposed subject is “in a matrix clause containing be + a short predicative complement” (p. 953). Unfortunately, this leaves the precise structure unclear.

After considering various options, we have decided to attach the extraposed constituent as a second complement in the VP with ternary branching. Despite the seeming difference implied by the labels “extraposed” and “internalized”, we see this as analogous to the position of the internalized complement—the by phrase—in the VP in passive clauses (p. 674).

3.4.2 Coordinates & markers

A coordination is a non-headed construction with coordinates as daughters (CGEL p. 1278). Therefore, coordinates are neither heads nor dependents. Consider, though, the following coordination the guests and indeed his family too (p. 1278), reproduced here as figure 2.

Figure 2: CGEL flat coordination—rejected in CGELBank, where indeed his family too is an NP serving as the Head of the second coordinate.

Unlike coordinations, NPs like and indeed his family too are headed constructions. The NP has two modifiers: indeed and too, which, like all modifiers, are dependents requiring a head sister. But if coordinates are not heads, then this NP is headless.

Markers⁷⁷ 7 Though CGEL uses “marker” both non-technically (e.g., marker of distinctively informal style), and technically as a function term, we discuss only the latter. are sisters of heads when they are subordinators (see (9) on p. 954 and (51) on p. 1187), so a marker is a dependent. This, however, is incompatible with Marker:Coordinators like and in and indeed his family too being the sister to the Coordinate:NP his family (p. 1277).

Given these facts, the NP his family in figure 2 must be a head and not a coordinate. We generalize from this to the principle that a coordinate is only the daughter of coordination, and a marker is always a dependent with a sister head.

3.4.3 Indirect complements

Indirect complements are those such as the underlined phrase in enough time to complete the work which are licensed by a dependent in the phrase, here the Det enough, but CGEL is inexplicit about it attachment. We construct a superordinate phrase of the head type and branch the indirect complement from that, as in [[enough time]_Head:NP [to do the work]_{Comp_rel:Clause}]_NP. When the complement is further delayed, we branch it from the nearest possible parent phrase, as in figure 3.

Figure 3: CGELBank structure of the indirect complement of so.

3.4.4 Verbless clauses

CGEL’s treatment of so-called “verbless clauses” (VlCs; ch. 14§10) is incompatible with its treatment of clauses in general. Though CGEL provides no definition of a clause, a clause is a phrase (p. 23) and a phrase is a constituent with a head and possible dependents (p. 22). A predicate is a VP that functions as the head of a clause (p. 24). Therefore, a clause is a phrase headed by a VP. But VlCs have no verb and no VP, so they must not be clauses in the syntactic sense that CGEL implies. That is not to deny that, like clauses, VlCs express a semantic connection between a predicand and a preciate, but there are other constructions in CGEL that do this without being analyzed as clauses (e.g., it made them happy; p. 217).

Seeing no clear alternative, we treat PPs like with his hands in his pockets as having two complements, analogous to complex transitive verbs (e.g., made him happy) and PPs like while happy as taking predicative complements, similar to as. Finally, we analyze supplement VlCs (e.g., bag in hand, they set out) as headless nonce constructions like those in gapped coordination (see section 3.4.2).

3.4.5 Names

Despite CGEL’s claim that “to a large extent the syntactic structure of proper names conforms to the rules for the structure of ordinary NPs,” (p. 517), it also observes that “there is no convincing evidence for treating one element as head” in personal names (p. 519). Additionally, it notes that “titles [of books, etc.] are much less constrained than other kinds of proper name and are excluded from [our] account” (p. 517). For these reasons, we have chosen to treat proper names such as Pierre Vinken as single lexical items. We similarly treat chemical compounds such as carbon dioxide as single lexical items.

In the dataset itself, multiple tokens in such a construction are analysed using the Flat relation, structured similarly to headless coordination.

3.4.6 Supplements

Supplements are adjuncts which are only loosely attached to a phrase, “set off from the rest in speech by separate intonational phrasing and in writing by punctuation” (CGEL p. 25). A clear example is:

\ex

. The necklace, which her mother gave her , was in the safe

There are, however, other cases where the distinction between supplements and modifiers is quite fuzzy. A fronted if-PP, for instance, meets the prosodic and punctuational criteria of the definition. But we currently have no practical and principled way to say whether the same PP at the end of a VP, for instance, is a modifier or a supplement. We have proceeded on intuition using discussions to try to resolve disagreements.

Another issue is that the structural treatment of supplements in CGEL is sui-generis and inconsistent with typical trees. It consists of an anchor node to which a supplement such as the underlined relative clause in section 3.4.6 is attached by means of a dashed arrow going from the supplement to the anchor. Under this analysis, the anchor and the supplements are not in a head-dependent relationship (p. 1350). For convenience and practicality, we instead attach supplements as dependents to the anchor head, even if that leads to three or more branches from the node, so that the NP in section 3.4.6 would be [the necklace which her mother gave her].

3.4.7 Miscellaneous

•

We treat comma splices as distinct clauses.
•

Clauses set off by punctuation such as commas, dashes, or (semi-)colons are analyzed as supplements.
•

We take particles such as up in pick up the kids to be complements in a ternary branching VP along with the head and the object (CGEL p. 280).
•

Ungrammatical aspects of the source sentence are kept and corrected using a special annotation.

4 Comparison with existing frameworks

Having resolved some of the linguistic issues that arose while annotating a treebank in the CGEL formalism, we look towards comparing CGELBank with two established formalisms for computational syntax: Universal Dependencies (Nivre et al., 2020; de Marneffe et al., 2021) and Penn Treebank (Marcus et al., 1994).

4.1 Corpus

To enable quantitative comparisons, two selections of sentences were annotated as CGELBank trees:

•

EWT, 100 sentences from the English Web Treebank, which were previously annotated with Penn and UD trees (Bies et al., 2012; Silveira et al., 2014)—we annotated CGELBank trees;
•

Ling, 65 sentences observed to be linguistically interesting---we separately annotated CGELBank and UD trees.⁸⁸ 8 UD trees were generated with Stanza v1.4.0 (Qi et al., 2020) and hand-corrected in the UD Annotatrix tool (Tyers et al., 2017)

This resulted in a small set of parallel data. Summary statistics about our datasets are reported in table 2, and a breakdown by CGELBank labels (POS, phrasal category, grammatical function) in table 1. CGELBank trees were originally created in LaTeX using the forest package and later converted into a PENMAN-like textual notation (Mathiessen and Bateman, 1991). Further details are in appendix B.

4.2 POS tags

POS Phrasal Category Gram. Function 691 N 1113 Nom 4444 Head 361 V 899 NP 627 Mod 327 P 807 VP 409 Comp 303 D 619 Clause 403 Obj 265 N_pro 343 PP 299 Det 224 V_aux 302 DP 295 Subj 173 Adj 195 AdjP 209 Coordinate 133 Adv 134 AdvP 205 Marker 104 Sdr 103 Coordination 88 PredComp 101 Coordinator 86 Clause_rel 68 Supplement 4 Int 6 NP+PP 53 Flat 4 IntP 52 Det-Head 3 NP+AdvP 43 Prenucleus 96 GAP 2 NP+Clause 10 Postnucleus

Table 1: Counts in CGELBank POS tags, phrasal categories, and grammatical functions. Special phrasal categories for coordination and some functions are not listed due to low frequency.

The part-of-speech tagset used in CGELBank contains only 11 tags, fewer than the equivalent tagset for aligned tokens in both UD (18) and PTB (37).⁹⁹ 9 CGELBank currently omits punctuation tokens, which account for some missing POS categories in UD and PTB. Of the substantive PTB tags we are missing only WP$ (whose). The multilingual UD tagset is coarse due to the language-specific nature of fine-grained labels. The PTB tagset is fine-grained, reflecting morphological inflections of verbs, nouns, pronouns, adjectives, and adverbs, as well as WH status of pronouns, determiners, and adverbs. Even if the PTB tagset were coarsened, the correspondences across the three tagsets would not be exact.

Quantitive comparison.¹⁰¹⁰ 10 Numbers reported henceforth were computed on a preliminary version of the CCGBank release (except for table 1, which is current as of this writing). The comparison to PTB and UD identified inconsistencies in the CGEL trees that were subsequently revised. We report numbers prior to the revision because the revision was influenced by the comparison.

Applying information theoretic measures to our data, we can estimate the overall uncertainty (i.e. entropy) of the tagset, and whether the UD or PTB POS tag provides greater information about the CGELBank POS tag for a word (i.e. conditional entropy given another distribution).

	$\displaystyle H(X)$	$\displaystyle=-\sum_{x\in\mathcal{X}}{p(x)\log{p(x)}}$		(1)
	$\displaystyle H(X\mid Y)$	$\displaystyle=-\sum_{x\in\mathcal{X},y\in\mathcal{Y}}{p(x,y)\log{\frac{p(x,y)}{p(y)}}}$		(2)

We induced an alignment of tokens between parallel trees (given that our hand-annotated CGELBank trees omitted punctuation, for example) and compared the aligned tokens’ tags. As expected, when it comes to predicting the CGELBank tag, tags from other frameworks are informative, producing a marked reduction in entropy. By this measure, the PTB tagset proves slightly more informative than UD.

	EWT	Ling	Combined
Trees	99	65	164
Tokens	2102	908	3010
% UD–CGELBk matched	86.6%	86.8%	86.6%
$H(\text{CGELBk POS})$	2.85	2.87	2.87
$H(\text{CGELBk POS}\mid\text{UD POS})$	0.38	0.46	0.42
$H(\text{CGELBk POS}\mid\text{PTB POS})$	0.34	0.40	0.38

Table 2: Statistics for our data, including sizes and tag entropy values. Note: some UD tokens, such as punctuation, are omitted from CGELBank trees (unmatched).

Differences.

The major contributors to tag uncertainty are the CGELBank categories of determinatives (D), prepositions (P), and subordinators (Sdr). These categories do not correspond neatly to the POS hierarchies defined by UD and PTB, leading to complex overlapping of the distribution of tags.

As summarized in table 3, CGEL adopts an expansive view of the preposition (P) category, incorporating function words that head traditional prepositional phrases as well as distributionally similar phrases (including intransitive particles, adverbials taking the form of a clause, deictic adverbial words, and so on). According to CGEL, characteristics unifying traditionally disparate categories as prepositions include that they are mostly closed-class and uninflecting; can mostly be modified by right; and can mostly appear adnominally, adverbially, and as complement of be (but not become).

As for determinatives (D), CGEL again takes a broad view of the situation, shown in table 4. The articles a and the are the prototypical determinatives. Beyond those, determinatives are lexemes that are mutually exclusive with the articles in NP structure (*the this book) and/or can occur in the partitive construction (many of them) (p. 539). Unlike UD and PTB, determinatives (such as this) functioning as the sole constituent in the NP are not called pronouns; they are instead analyzed as fused determiner-heads of the NP (section 4.4, item 4).

CGELBank PTB UD Relativizer that Sdr WDT PRON Infinitive marker to Sdr TO PART Complementizer that, whether, if Sdr IN SCONJ Clause-marking before, in, because, if* … P IN SCONJ Adjunct clause–marking when, where … P WRB SCONJ NP-marking of, before, in, to, out (of) … P IN ADP Idiomatic verb particle out, up … P RP ADP Spatial particle out, up … P RB ADV Adverbial outside, here, then … P RB ADV

Table 3: Tagging of prepositions, subordinators, and related items in the three frameworks.
* Non-complementizer (conditional) use of if.

Named entities are another source of tag divergence. CGELBank makes no distinction between proper nouns and common nouns, unlike the other formalisms. PTB goes as far as to label every content word in a proper name as NNP: e.g., Supreme Court is labeled Adj by CGEL but NNP by PTB.

CGELBank PTB UD Article a, an, the D DT DET Demonstrative this, those … D DT DET, PRON either, neither D DT, CC, RB DET, CCONJ, ADV both D DT, PDT, CC, RB DET, CCONJ, ADV Quantifier all D DT, PDT DET Quantifier some, any, every, each D DT DET Quantifier few, little, many, much D JJ ADJ Quantifier no D DT, RB DET, ADV none D NN NOUN Nominal anyone, nothing … D NN PRON Adverbial anywhere, nowhere D RB ADV what Adj, D*, N_pro WP, WDT PRON, DET which D, N_pro WDT PRON, DET one D, N, N_pro CD, NN, PRP NUM, NOUN, PRON Other cardinal number, no suffix D, N CD NUM hundreds, thousands, … N NNS NOUN

Table 4: Selected CGELBank determinatives and related categories in other frameworks.
* CGEL has what as a pronoun, determinative, or adjective (pp. 540, 909–910). The determinative use of what is not attested in CGELBank.

4.3 Gaps

Following the discussion in section 3.3, we sought to characterize CGELBank gaps empirically. We attempted to align them with PTB null/empty elements in the EWT set based on relative location between tokens in the sentence. This reveals that CGELBank’s gaps largely correspond to PTB elements *T* (e.g. trace of WH-movement; 28 of 33 aligned) and *RNR* (Right Node Raising; 2 of 2 aligned). There are 10 unaligned CGELBank gaps in EWT: these indicate noncanonical ordering of elements in a clause, whereas PTB signals this via a special constituent label—or not at all. Conversely, CGELBank does not use gaps for controlled subjects or other omitted subjects of nonfinite clauses (PTB *PRO*), nor for passivization.

4.4 Tree structures

With regard to parse structure, CGELBank offers a systematic and coherent encoding of both phrase structure (as in PTB) and grammatical relations (as in UD). In this sense, CGELBank parses are more informative than either PTB or UD parses.

PTB.

Broadly speaking, CGELBank and Penn Treebank adopt comparable inventories of phrasal categories.¹¹¹¹ 11 The main differences are that PTB uses finer-grained clause labels and assigns distinct categories to WH-phrases. CGEL defines a taxonomy of clause types, but CGELBank only distinguishes relative clauses (Clause_rel) from others (Clause). More clause types may be added in the future. But the trees look very different for the following major reasons:

1.

Heads and Grammatical Functions: PTB does not specify heads within phrases or systematically provide the grammatical function of a phrase with respect to its parent phrase.¹²¹² 12 In PTB, a limited set of functions are labeled on appropriate phrases, notably subjects and certain types of adjuncts. Therefore, heuristic head rules (Collins, 1999) are necessary to convert to dependency structures.
2.

Branching: Constituents in the PTB are notoriously flat, with potentially multiple adjunct modifiers and complements all attaching at the same level. Often they are heuristically binarized for parsing. CGEL constituents are mostly binary already, with outer attachments for adjuncts, which makes the underlying grammar more parsimonious.
3.

Empty Categories: The PTB contains a variety of empty categories reflecting Government and Binding theory (Chomsky, 1981). Our treebank contains a single notion of a gap (section 3.3), limited primarily to unbounded dependency constructions and ellipsis; gaps do not represent control, for example.
4.

Fusion: CGEL’s fused-head construction occurs when the Head function is fused with a dependent function in NP structure (p. 332; more in ch. 5§9). The Prenucleus:NP in figure 1 illustrates one of three such fusions (see figure 4). The fused-head construction results in a directed acyclic graph (Pullum and Rogers, 2008). In these cases, the dependent simultaneously performs its usual function (e.g., Det:DP) and the function of Head in the NP, leading it to have two parent nodes in the graph. We maintained the tree structure in our dataset such that the fused-head relations were still recoverable; only one edge was kept but specially labelled as $x$ -Head or Head- $x$ .

PTB has no counterpart to CGEL’s fusion of functions. All PTB parses are properly trees.

UD.

After aligning tokens between UD and CGELBank representations, we find that only 46.6% of them have the same head¹³¹³ 13 To extract heads of tokens from CGELBank trees, we treat both head and fused-head constituents as heads of their sister constituents. In flat structures (e.g. coordination) we take the first element as the head to agree with UD’s representation. in both formalisms. This is largely due to CGEL’s treatment of many function words as heads, since e.g. a preposition projects a prepositional phrase, not a nominal phrase. Consequently, the proportion of UD dependencies with CGELBank equivalents is low for auxiliaries (aux, 4.1% head agreement), copulas (cop, 0.0%), and standard prepositions (case, 0.0%).

UD’s design decision to favor content heads was driven by valid crosslinguistic considerations but leads to awkward treatment of English prepositions and copular constructions Gerdes et al. (2018).¹⁴¹⁴ 14 For example, consider The problem is that the project will soon be behind schedule. In CGEL, the head of the clause is is, with a that-clause as complement, etc. In UD, the main predicate is schedule (since copulas, other auxiliaries, complementizers, and prepositions are considered modifiers). A second consequence of content heads (combined with the lack of constituent structure) is that some structural ambiguities are not resolved by the parse, as in the case of coordination: clause-level, verb phrase--level, and predicate-level coordination produce identical dependency structures.¹⁵¹⁵ 15 E.g., [Quickly dress] and [cook the chicken] vs. Quickly [[dress] and [cook the chicken]] vs. Quickly [dress and cook] the chicken. For all three interpretations, the UD tree will have dress as the clause root with cook (conj) and quickly (advmod) as its dependents, and no indication dress and cook can simultaneously be transitive with a shared direct object.

Several dependency relations in UD accommodate “extrasyntactic” relationships that appear in corpus sentences, such as repair and headless expressions (e.g., personal names). But when it comes to compositional syntactic relations, CGELBank’s inventory of phrasal categories and functions is finer-grained and generally more informative.

One example is complementation: English UD only distinguishes complements from modifiers if they are non-prepositional dependents within a clause—subjects, objects, and complement clauses in UD are not prepositionally marked and can only be dependents of a clausal predicate. Thus, all clausal complements of nouns (the claim that there is life on Mars), as well as prepositional (oblique) clausal complements of any word (interested in living on Mars), are labeled the same as modifier clauses in UD (Przepiórkowski and Patejuk, 2019).

Sometimes UD basic (surface-level) structures provide more information than CGELBank:

1.

The xcomp relation (borrowed from LFG; Dalrymple, 2001) is used within a clause to signal predicate-licensed control or raising. Control and raising are not indicated in CGELBank.
2.

The UD expl relation marks expletive uses of it and there (It is clear that there is more work to do), signaling that these words are nonreferential and present only for syntactic reasons. In CGELBank, these are treated as pronouns filling ordinary grammatical functions (usually Subj)—though some constructions that trigger expletive pronouns, like extraposition (section 3.4.1), are specially marked elsewhere in the tree.
3.

The subject relations in English UD distinguish passive from non-passive subjects. CGEL analyzes the passive construction as combining ordinary morphosyntactic components at the surface level, so no special label is present in the tree.
4.

English UD has special relations for nominal expressions serving as temporal modifiers. Arguably this is a place where semantics has crept into the syntax.
5.

As UD requires every word to be assigned a dependency relation, treebanks are required to make decisions about the internal structure of expressions like dates, though these are not well standardized at present (Zeman, 2021; Schneider and Zeldes, 2021).

In addition, several UD languages, including English, offer enhanced dependency graphs that augment the surface-syntactic tree with deeper relations (Schuster and Manning, 2016). CGELBank reflects mostly “surface structure”, though its coindexation of gaps can be used to recover some of the UD enhanced dependencies (e.g., for relative constructions).

5 Conclusion

Using the analysis developed in CGEL (Huddleston and Pullum, 2002), we introduced a new expressive and linguistically-informed syntactic formalism to corpus annotation of English. We released a small treebank with parallel annotation in CGELBank, UD, and PTB, and compared features of the three. The unique features of the CGEL formalism, combined with its minimal complexity, lend themselves to computational work on English syntax.

In the future, we plan to expand our treebank with more manually-annotated trees, using that to develop means for automatic conversion from UD and/or PTB to CGELBank. A full annotation manual is under development. We also are interested in incorporating more of CGEL’s analysis, particularly in morphology and fine-grained part-of-speech tags.

Acknowledgments

We thank John Payne, Geoffrey Pullum, and Pairoj Kunanupatham for useful discussion on many linguistic issues that arose in annotation as well as the CGEL formalism in general. We also thank various Twitter users (including Rui P. Chaves and Russell Lee-Goldman) who commented on syntactic analyses posted by Brett Reynolds, as well as anonymous reviewers.

References

Atwell (2008) ES Atwell. 2008. Development of tag sets for part-of-speech tagging. In Corpus Linguistics: An International Handbook, volume 1, pages 501–526. Walter de Gruyter.
Bies et al. (1995) Ann Bies, Mark Ferguson, Karen Katz, Robert MacIntyre, Victoria Tredinnick, Grace Kim, Mary Ann Marcinkiewicz, and Britta Schasberger. 1995. Bracketing guidelines for Treebank II style Penn Treebank project.
Bies et al. (2012) Ann Bies, Justin Mott, Colin Warner, and Seth Kulick. 2012. English Web Treebank. Technical Report LDC2012T13, Linguistic Data Consortium, Philadelphia, PA.
Bladier et al. (2018) Tatiana Bladier, Andreas van Cranenburgh, Kilian Evang, Laura Kallmeyer, Robin Möllemann, and Rainer Osswald. 2018. RRGbank: a Role and Reference Grammar corpus of syntactic structures extracted from the Penn Treebank. In Proceedings of the 17th International Workshop on Treebanks and Linguistic Theories (TLT 2018), December 13–14, 2018, Oslo University, Norway, 155, pages 5–16. Linköping University Electronic Press.
Brew (2003) Chris Brew. 2003. The Cambridge Grammar of the English Language, Rodney Huddleston and Geoffrey K. Pullum. Computational Linguistics, 29(1):144–147.
Chomsky (1981) Noam Chomsky. 1981. Lectures on Government and Binding: the Pisa lectures. Number 9 in Studies in Generative Grammar. Foris Publications, Dordrecht, Holland.
Collins (1999) Michael John Collins. 1999. Head-driven statistical models for natural language parsing. Ph.D. thesis, University of Pennsylvania.
Culicover (2004) Peter W Culicover. 2004. Review article: The Cambridge grammar of the English language. Language, 80(1):127–141.
Dalrymple (2001) Mary Dalrymple. 2001. Lexical functional grammar, volume 34 of Syntax and Semantics. Brill.
de Marneffe et al. (2021) Marie-Catherine de Marneffe, Christopher D. Manning, Joakim Nivre, and Daniel Zeman. 2021. Universal Dependencies. Computational Linguistics, 47(2):255–308.
Gerdes et al. (2018) Kim Gerdes, Bruno Guillaume, Sylvain Kahane, and Guy Perrier. 2018. SUD or Surface-Syntactic Universal Dependencies: An annotation scheme near-isomorphic to UD. In Universal Dependencies Workshop 2018.
Hockenmaier and Steedman (2007) Julia Hockenmaier and Mark Steedman. 2007. CCGbank: A Corpus of CCG Derivations and Dependency Structures Extracted from the Penn Treebank. Computational Linguistics, 33(3):355–396.
Huddleston and Pullum (2002) Rodney Huddleston and Geoffrey K. Pullum. 2002. The Cambridge Grammar of the English Language. Cambridge University Press, Cambridge.
Huddleston and Pullum (2005) Rodney Huddleston and Geoffrey K. Pullum. 2005. A Student’s Introduction to English Grammar, 1st edition. Cambridge University Press.
Huddleston et al. (2021) Rodney Huddleston, Geoffrey K. Pullum, and Brett Reynolds. 2021. A Student’s Introduction to English Grammar, 2nd edition. Cambridge University Press.
Jurafsky and Martin (2021) Dan Jurafsky and James H. Martin. 2021. Speech and Language Processing, 3rd edition. Self-published. Draft.
Kübler et al. (2009) Sandra Kübler, Ryan McDonald, and Joakim Nivre. 2009. Dependency parsing. Synthesis Lectures on Human Language Technologies. Morgan & Claypool Publishers.
Maling (2000) Joan Maling. 2000. A simple argument for subject gaps. In Yosef Grodzinsky, Lewis P Shapiro, and David Swinney, editors, Language and the brain. Academic Press, San Diego.
Marcus et al. (1994) Mitchell Marcus, Grace Kim, Mary Ann Marcinkiewicz, Robert MacIntyre, Ann Bies, Mark Ferguson, Karen Katz, and Britta Schasberger. 1994. The Penn Treebank: Annotating predicate argument structure. In Human Language Technology: Proceedings of a Workshop held at Plainsboro, New Jersey, March 8-11, 1994.
Mathiessen and Bateman (1991) Christian M. I. M. Mathiessen and John A. Bateman. 1991. Text Generation and Systemic-Functional Linguistics. Pinter, London.
Miyao et al. (2004) Yusuke Miyao, Takashi Ninomiya, and Jun’ichi Tsujii. 2004. Corpus-oriented grammar development for acquiring a Head-driven Phrase Structure grammar from the Penn Treebank. In International Conference on Natural Language Processing, pages 684–693. Springer.
Nivre et al. (2016) Joakim Nivre, Marie-Catherine de Marneffe, Filip Ginter, Yoav Goldberg, Jan Hajič, Christopher D. Manning, Ryan McDonald, Slav Petrov, Sampo Pyysalo, Natalia Silveira, Reut Tsarfaty, and Daniel Zeman. 2016. Universal Dependencies v1: A multilingual treebank collection. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC’16), pages 1659–1666, Portorož, Slovenia. European Language Resources Association (ELRA).
Nivre et al. (2020) Joakim Nivre, Marie-Catherine de Marneffe, Filip Ginter, Jan Hajič, Christopher D. Manning, Sampo Pyysalo, Sebastian Schuster, Francis Tyers, and Daniel Zeman. 2020. Universal Dependencies v2: An evergrowing multilingual treebank collection. In Proc. of LREC, pages 4027–4036, Marseille, France.
Pradhan et al. (2013) Sameer Pradhan, Alessandro Moschitti, Nianwen Xue, Hwee Tou Ng, Anders Björkelund, Olga Uryupina, Yuchen Zhang, and Zhi Zhong. 2013. Towards robust linguistic analysis using OntoNotes. In Proc. of CoNLL, pages 143–152, Sofia, Bulgaria.
Przepiórkowski and Patejuk (2019) Adam Przepiórkowski and Agnieszka Patejuk. 2019. Nested coordination in Universal Dependencies. In Proc. of the Third Workshop on Universal Dependencies (UDW, SyntaxFest 2019), pages 58–69, Paris, France.
Pullum and Reynolds (2013) Geoffrey K. Pullum and Brett Reynolds. 2013. New members of ‘closed classes’ in English. Unpublished.
Pullum and Rogers (2008) Geoffrey K. Pullum and James Rogers. 2008. Expressive power of the syntactic theory implicit in the Cambridge Grammar of the English Language. In Annual Meeting of the Linguistics Association of Great Britain. University of Essex.
Qi et al. (2020) Peng Qi, Yuhao Zhang, Yuhui Zhang, Jason Bolton, and Christopher D. Manning. 2020. Stanza: A python natural language processing toolkit for many human languages. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pages 101–108, Online. Association for Computational Linguistics.
Schneider and Zeldes (2021) Nathan Schneider and Amir Zeldes. 2021. Mischievous nominal constructions in Universal Dependencies. In Proc. of the Fifth Workshop on Universal Dependencies (UDW, SyntaxFest 2021), pages 160–172, Sofia, Bulgaria.
Schuster and Manning (2016) Sebastian Schuster and Christopher D. Manning. 2016. Enhanced English Universal Dependencies: An improved representation for natural language understanding tasks. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC’16), pages 2371–2378, Portorož, Slovenia. European Language Resources Association (ELRA).
Silveira et al. (2014) Natalia Silveira, Timothy Dozat, Marie-Catherine de Marneffe, Samuel R. Bowman, Miriam Connor, John Bauer, and Christopher D. Manning. 2014. A gold standard dependency corpus for English. In Proc. of LREC, pages 2897–2904, Reykjavík, Iceland.
Tyers et al. (2017) Francis M. Tyers, Mariya Sheyanova, and Jonathan North Washington. 2017. UD annotatrix: An annotation tool for Universal Dependencies. In Proceedings of the 16th International Workshop on Treebanks and Linguistic Theories, pages 10–17, Prague, Czech Republic.
Zeman (2021) Daniel Zeman. 2021. Date and time in Universal Dependencies. In Proc. of the Fifth Workshop on Universal Dependencies (UDW, SyntaxFest 2021), pages 173–193, Sofia, Bulgaria.

Appendix A Functions

The hierarchy of the full set of functions used in the treebank is shown in Figure 4.

{forest}

Figure 4: The hierarchical relation of all functions in the treebank.

Appendix B Corpus design

EWT.

We randomly sampled 200 trees from the portion of sentences in EWT that have token count between 15 and 30 inclusive. These sentences were long enough to naturally contain complex linguistic phenomena that could test our annotation guidelines, and preliminary attempts at rule-based conversion from UD to CGELBank found difficulties in sentences in this length range. 99 of the trees were manually annotated thus far by one of the authors. Another author performed extensive consistency checks and revisions, including splitting one of the sentences into two sentences, resulting in 100 annotated trees.

Ling.

An author encountered linguistically-interesting and difficult-to-analyze sentences in a variety of contexts (news articles/headlines, social media posts, etc.) over the course of a year and manually annotated them in the CGEL formalism. These were annotated before some of the guidelines in this paper were agreed upon, leading to many necessary revisions.

Appendix C More tables

Table 5 lists lemmata with ambiguous POS tags in CGELBank. Table 6 lists all function words found in CGELBank.

{D, Sdr}: that	{P, V, Adj}: like	{N, V}: call try
{D, N_pro}: one	{P, N_pro}: there	{V, V_aux}: do have
{D, Adv}: more	{P, Adv}: as	{Adj, Adv}: very well
	{P, Sdr}: to for if	{Adj, N_pro}: what
	{P, Coordinator, Int, Adv}: so

Table 5: Ambiguous lemmata occurring at least 5 times

D [excluding numerals]: a, a few, a little, all, another, any, anybody, anyone, anything, anywhere, both, each, enough, every, everybody, everyone, everything, least, many, many a, million, more, no, no one, none, one, several, some, someone, something, sometimes, somewhere, that, the, this, three, two, which
N_pro: he, I, it, its, mine, my, one, she, there, they, we, what, which, who, yesterday, you, your
P: @, a.m., about, above, after, against, along, around, as, at, away, back, because, before, behind, between, by, considering, coupled, down, due, during, for, forward, from, here, if, in, in order, including, into, irrespective, like, now, of, off, on, onboard, out, outside, over, past, per, regarding, since, so, so long as, than, then, there, through, to, up, upon, upstairs, when, while, with, within
V_aux: be, can, could, do, have, may, must, should, will, would
Sdr: for, if, that, to, whether
Coordinator: &, -, /, and, but, etc, or, plus, so

Table 6: Full list of function word lemmata in our data by POS

Figure 5: The interrogative clause what Liz bought as originally parsed in CGEL (p. 48, figure 5b).