Text Chunking using Transformation-Based Learning

Lance A. RamshawMitchell P. Marcus

article1995ACL2,017 citations

Demonstrates that text chunking can be framed and solved as a sequence tagging problem using transformation-based learning, achieving over 92% accuracy in identifying non-recursive base noun phrases.

Listen

Full syntactic analysis of unrestricted text is computationally complex and prone to errors. Text chunking—the process of identifying non-overlapping, low-level phrase groups such as basic noun phrases—serves as an efficient preliminary step to full parsing and enables practical applications like automated index-term extraction. The article evaluates whether transformation-based learning, an automated rule-learning method that iteratively corrects baseline guesses using contextual clues, can accurately identify text chunks by framing chunking as a sequential word-tagging problem.

To conduct this evaluation, the researchers derived training and testing datasets from the Penn Treebank corpus of Wall Street Journal text. They tested two distinct targets: basic noun phrases (non-recursive noun phrases up to their heads) and partitioning chunks (dividing full sentences into adjacent noun-type and verb-type segments). Raw texts were first assigned part-of-speech labels, followed by baseline chunk tags based on those labels. The transformation-based system then searched across 100 contextual rule templates to automatically learn an ordered sequence of error-correcting rules, utilizing optimization methods such as static feature indexing and rule-disabling heuristics to manage computational demands.

Key findings demonstrate that transformation-based learning achieves strong accuracy across both chunking objectives. Trained on a 200,000-word dataset and tested on 50,000 words, the system achieved 92.3% recall and 91.8% precision for basic noun phrases, representing an error reduction of roughly 57% to 62% over baseline heuristics. For the more complex sentence-partitioning task, the system reached 88.5% recall and 87.7% precision, achieving over 70% error reduction. Incorporating specific lexical words into the rule templates provided modest gains for basic noun phrases (about a 1% absolute performance increase) but proved significantly more important for partitioning chunks, where it yielded an approximate 5% boost. Furthermore, the resulting models consist of transparent, human-interpretable rules that explain why specific labeling corrections were made.

These results establish that transformation-based chunking provides an accurate, automated, and explainable foundation for natural language processing systems without requiring manual rule-crafting. The system's performance enables faster downstream text processing pipelines while keeping computational overhead relatively low. Next steps proposed by the article include expanding rule templates to reference internal chunk boundaries, enriching the tagset to capture broader context, and extending the methodology to higher-level relational tasks such as dependency parsing and predicate-argument mapping.

Decision-makers should consider certain limitations noted in the evaluation. The primary sources of error stemmed from ambiguous verb forms functioning as modifiers and complex conjunction structures, both of which frequently require broader semantic context that local rule patterns cannot capture. Additionally, minor noise exists due to automatic test-set extraction heuristics and underlying corpus parse variations. Nonetheless, the high reported accuracy provides strong confidence in adopting transformation-based tagging for intermediate linguistic processing.

Bowdoin College University of Pennsylvania part of speech textual data treebanks text chunking transformation-based learning part-of-speech tagging

arXiv: cmp-lg/9505040

Paper: Induction of Decision Trees, J. R. Quinlan (1986). Introduces foundational inductive rule learning and decision tree techniques that provide direct conceptual background for rule-based machine learning paradigms like transformation-based learning.

Paper: Discriminative Training Methods for Hidden Markov Models: Theory and Experiments with Perceptron Algorithms, Michael Collins (2002). Formulates discriminative sequence tagging algorithms and evaluates them directly on standard base noun phrase chunking tasks.
Paper: Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data, J. Lafferty et al. (2001). Introduces conditional random fields as a globally normalized sequence labeling framework, providing a major alternative paradigm for POS tagging and chunking.
Paper: A unified architecture for natural language processing: deep neural networks with multitask learning, Ronan Collobert et al. (2008). Extends text chunking and shallow parsing into a unified multitask deep learning framework without hand-crafted features.
Paper: Word Representations: A Simple and General Method for Semi-Supervised Learning., Joseph Turian et al. (2010). Demonstrates how unsupervised word representations can be directly integrated into supervised sequence models for noun phrase chunking.
Paper: Bidirectional LSTM-CRF Models for Sequence Tagging, Zhiheng Huang et al. (2015). Applies neural bidirectional LSTM-CRF architectures to standard sequence labeling and text chunking benchmarks.
Paper: Natural Language Processing (almost) from Scratch, Ronan Collobert et al. (2011). Presents an end-to-end neural network architecture that handles phrase chunking and tagging from raw text without hand-engineered rule transformations.

Cover for Text Chunking using Transformation-Based Learning

Abstract

Eric Brill introduced transformation-based learning and showed that it can do part-of-speech tagging with fairly high accuracy. The same method can be applied at a higher level of textual interpretation for locating chunks in the tagged text, including non-recursive ``baseNP'' chunks. For this purpose, it is convenient to view chunking as a tagging problem by encoding the chunk structure in new tags attached to each word. In automatic tests using Treebank-derived data, this technique achieved recall and precision rates of roughly 92% for baseNP chunks and 88% for somewhat more complex chunks that partition the sentence. Some interesting adaptations to the transformation-based learning approach are also suggested by this application.

Topics

part of speech textual data treebanks speech signals online learning text data text document semantic information tag recommendations

Keywords

text chunking transformation-based learning part-of-speech tagging Penn Treebank baseNP chunking lexical rule templates attachment disambiguation

1 Introduction
2 Text Chunking
2.1 Existing Chunk Identification Techniques
2.2 Deriving Chunks from Treebank Parses
3 The Transformation-based Learning Paradigm
4 Transformational Text Chunking
4.1 Encoding Choices
4.2 Baseline System
4.3 Rule Templates
5 Algorithm Design Issues
5.1 Organization of the Computation
5.2 Indexing Static Rule Elements
5.3 Heuristic Disabling of Unlikely Rules
6 Results
6.1 Analysis of Initial Rules
6.2 Contribution of Lexical Templates
6.3 Frequent Error Classes
7 Future Directions
8 Conclusions
References

Knowls

Knowl 1 — IOB Tagging Scheme and Partitioning Tag Scheme for Text Chunking
definition
Text chunking can be formulated as a sequential tagging problem by attaching discrete chunk tags to individual words, eliminating the need to insert and balance explicit parentheses or brackets across words.

BaseNP Tagset ( $IOB$ Encoding) To identify non-recursive, base noun phrases (baseNPs) up to the head noun (including determiners and premodifiers, but excluding postmodifying prepositional phrases and clauses), words are labeled using the tagset $\{I, O, B\}$ :
- $I$ (Inside): The word is inside a baseNP.
- $O$ (Outside): The word is outside any baseNP.
- $B$ (Boundary): The word is the first token of a baseNP that immediately follows another baseNP without intervening non-NP tokens.
Any sequence of $\{I, O, B\}$ tags can be deterministically decoded into valid chunk bracketings with local consistency rules: if a $B$ tag directly follows an $O$ tag, it is decoded locally as an $I$ tag.

Sentence-Partitioning Tagset To partition entire sentences into non-overlapping noun-like ( $N$ -type) and verb-like ( $V$ -type) chunks, words are labeled using the tagset $\{BN, N, BV, V, P\}$ :
- $BN$ : The first word of an $N$ -type chunk (which includes base noun phrases as well as prepositions heading prepositional phrases).
- $N$ : Any subsequent word inside an $N$ -type chunk.
- $BV$ : The first word of a $V$ -type chunk (which includes verbs, auxiliaries, and intervening elements such as adjective phrases).
- $V$ : Any subsequent word inside a $V$ -type chunk.
- $P$ : Punctuation tokens, which do not establish or break chunk boundaries.
Local consistency repair: If a $V$ tag immediately follows an $N$ tag without an intervening $BV$ tag, it is decoded as $BV$ .
Knowl 2 — Baseline Initialization and POS-Conditioned Heuristic for Chunk Tagging
model/method
In transformation-based learning (TBL) for text chunking, an initial baseline assignment of chunk tags is generated before any transformational rules are learned or applied.

Part-of-speech (POS) tags are first assigned to all tokens in the text using an automated tagger. The baseline chunker then assigns each word the chunk tag most frequently associated with its assigned POS tag in the training corpus: $\hat{T}_{\text{baseline}}(w_i) = \arg\max_{T \in \mathcal{T}} \text{Count}(\text{POS}(w_i), T)$ where $\mathcal{T}$ is the chunk tagset (such as $\{I, O, B\}$ or $\{BN, N, BV, V, P\}$ ).

In empirical evaluations on Wall Street Journal text from the Penn Treebank, assigning baseline chunk tags conditioned on part-of-speech tags achieves higher initial accuracy than assigning each word its own most frequent chunk tag. In the $\{I, O, B\}$ scheme, because the $B$ tag is only used when two baseNPs directly abut, determiners and nouns are initially assigned the $I$ tag by the POS baseline.
Knowl 3 — Contextual Rule Template Architecture for Transformation-Based Chunking
model/method
Transformation-based chunking learns an ordered sequence of transformation rules from a supervised corpus. Each rule consists of a contextual pattern matching a specific neighborhood of tokens and an action that replaces a word's current chunk tag with a new chunk tag.

The search space is defined by 100 contextual rule templates generated by taking the Cartesian product of 20 word/POS context patterns and 5 chunk tag context patterns:
1. Word and Part-of-Speech Patterns (20 patterns total):
  - 10 lexical patterns over word identities $W$ : current word $W_0$ ; adjacent words $W_{-1}, W_1$ ; word pairs $(W_{-1}, W_0), (W_0, W_1), (W_{-1}, W_1), (W_{-2}, W_{-1}), (W_1, W_2)$ ; and window disjunctions $W_{-1,-2,-3}$ (matching if any of the 3 preceding words matches) and $W_{1,2,3}$ (matching if any of the 3 following words matches).
  - 10 structural patterns over fixed part-of-speech tags $P$ : $P_0, P_{-1}, P_1, (P_{-1}, P_0), (P_0, P_1), (P_{-1}, P_1), (P_{-2}, P_{-1}), (P_1, P_2), P_{-1,-2,-3}, P_{1,2,3}$ .
2. Chunk Tag Patterns (5 patterns over dynamic chunk tags $T$ ):
  - Current chunk tag: $T_0$ .
  - Adjacent chunk tag pairs: $(T_{-1}, T_0)$ and $(T_0, T_1)$ .
  - Two-tag left and right contexts: $(T_{-2}, T_{-1})$ and $(T_1, T_2)$ .
Candidate rules instantiate these templates at locations where the current chunk tag differs from the target tag. In each training pass, the candidate rule that yields the maximal net score (positive tag corrections minus negative tag corruptions) across the corpus is selected and appended to the learned rule sequence.
Knowl 4 — Optimized Rule Search and Static Pattern Indexing for Small-Tagset TBL
algorithm
Standard transformation-based learning (TBL) algorithms for part-of-speech tagging prune the search for the optimal transformation rule using an $[ \text{old\_tag} \times \text{new\_tag} ]$ confusion matrix. In text chunking, the tagset $\mathcal{T}$ contains only 3 to 5 tags, making confusion-matrix partitioning ineffective. To handle 100 templates efficiently over large corpora, the search is organized via positive-score sorting and static pattern indexing.
Input: Supervised training corpus C with words, POS tags, and target chunk tags; Template set H
Output: Ordered sequence of learned transformation rules R
Initialize corpus C with baseline POS-to-chunk predictions
Initialize StaticIndex mapping static pattern keys (word and POS contexts) to token indices in C
loop
    Identify all error locations in C where current_tag != target_tag
    Generate instantiated candidate rules from templates in H at error locations
    For each candidate rule r, compute its positive score pos(r) (number of errors it corrects)
    Sort candidate rules in descending order of pos(r)
    best_rule = null
    best_net_score = 0
    for each candidate rule r in sorted list do
        if pos(r) <= best_net_score then
            break // No remaining candidate rule can exceed best_net_score
        Retrieve candidate matching locations in C using StaticIndex for r's static elements
        Evaluate negative score neg(r) (number of correct tags r would corrupt)
        net_score = pos(r) - neg(r)
        if net_score > best_net_score then
            best_net_score = net_score
            best_rule = r
    if best_rule is null or best_net_score <= 0 then
        break
    Append best_rule to R
    Apply best_rule to update chunk tags in C
return R
The static index indexes only the invariable components of rule antecedents (word tokens and POS tags), avoiding dynamic memory overhead and eliminating the need to scan the entire corpus when computing negative scores for candidate rules.
Knowl 5 — Heuristic Disabling and Re-enabling of Candidate Rules in TBL
algorithm
To accelerate iterative rule evaluation in transformation-based learning across large corpora, candidate rules with low positive scores are temporarily disabled across iterations.
Input: Candidate rule set CandidateRules; Net score of winning rule in current pass best_net; Number of corpus tag changes made in current pass num_changes; Update factor alpha in (0, 1]; Disabling margin delta > 0
Output: Updated active and disabled candidate rule sets
for each active rule r in CandidateRules do
    if pos(r) < best_net - delta then
        Mark r as disabled
        adjusted_pos(r) = pos(r)
for each disabled rule r do
    adjusted_pos(r) = adjusted_pos(r) + alpha * num_changes
    if adjusted_pos(r) >= best_net - delta then
        Mark r as active
By adding a conservative fraction $\alpha \cdot \text{num\_changes}$ to the tracking scores of disabled rules, the algorithm avoids scoring uncompetitive rules in every training pass while ensuring that rules are restored to consideration when accumulated corpus modifications could make them competitive. This provides roughly an order-of-magnitude training speedup with negligible loss in rule sequence quality.

Knowl 6 — Empirical Performance of Transformation-Based Text Chunking

data/table

Transformation-based learning was evaluated on baseNP identification (non-recursive noun phrases) and sentence partitioning (exhaustive $N$ -type and $V$ -type chunking) using Wall Street Journal data from the Penn Treebank. The baseline system assigned chunk tags based on the most frequent chunk tag for each word's part-of-speech tag (predicted by Brill's tagger). Models were trained on training sets of 50K, 100K, and 200K words, stopping after 500 rules, and evaluated on a separate 50K-word test set. A chunk was counted as correct in recall and precision only if both its start and end boundaries matched the reference chunking exactly.

Task / Training Size	Recall	Error Red.	Precision	Error Red.	Tag Accuracy	Error Red.
BaseNP Chunks
Baseline	81.9%	—	78.2%	—	94.5%	—
50K	90.4%	47.2%	89.8%	53.1%	96.9%	44.4%
100K	91.8%	54.8%	91.3%	60.0%	97.2%	49.6%
200K	92.3%	57.4%	91.8%	62.4%	97.4%	53.4%
Partitioning Chunks
Baseline	60.0%	—	47.8%	—	78.0%	—
50K	86.6%	66.6%	85.8%	72.8%	94.4%	74.4%
100K	88.2%	70.4%	87.4%	75.8%	95.0%	77.3%
200K	88.5%	71.1%	87.7%	76.5%	95.3%	78.5%

BaseNP chunking achieved 92.3% recall and 91.8% precision at 200K training words (57.4% and 62.4% error reduction over baseline). Partitioning chunking showed a lower initial POS baseline (60.0% recall, 47.8% precision) due to particle/preposition and clause ambiguities, but achieved over 70% error reduction, reaching 88.5% recall and 87.7% precision.

Knowl 7 — Impact of Lexical Rule Templates on Text Chunking Accuracy

data/table

To quantify the contribution of lexical rule templates (templates conditioning transformations on specific word identities rather than POS and chunk tags alone), models were trained without lexical templates under identical conditions on the 50K-word Penn Treebank test set.

Task / Training Size	Recall	Error Red.	Precision	Error Red.	Tag Accuracy	Error Red.
BaseNP (No Lexical)
Baseline	81.9%	—	78.2%	—	94.5%	—
50K	89.6%	42.7%	88.9%	49.2%	96.6%	38.8%
100K	90.6%	48.4%	89.9%	53.7%	96.9%	44.4%
200K	90.7%	48.7%	90.5%	56.3%	97.0%	46.0%
Partitioning (No Lexical)
Baseline	60.0%	—	47.8%	—	78.0%	—
50K	81.8%	54.5%	81.4%	64.4%	92.4%	65.4%
100K	82.9%	57.2%	83.0%	67.3%	92.9%	67.9%
200K	83.6%	58.9%	83.5%	68.4%	93.9%	72.2%

Comparing these ablation results to the full model demonstrates that:

For baseNP chunking at 200K words, lexical rules provide an improvement of roughly 1.6% in recall (92.3% vs. 90.7%) and 1.3% in precision (91.8% vs. 90.5%), representing about 5% of the overall error reduction.
For sentence-partitioning chunking at 200K words, lexical rules provide a larger improvement of 4.9% in recall (88.5% vs. 83.6%) and 4.2% in precision (87.7% vs. 83.5%), representing roughly 10% of total error reduction. Lexical features are critical in partitioning chunking to disambiguate specific prepositions, particles, and auxiliary verbs.

Knowl 8 — Analysis of Learned Transformation Rules for Chunk Boundary Disambiguation
empirical result
An analysis of the top transformation rules learned on 200K words of Penn Treebank text illustrates how transformation-based learning resolves syntactic ambiguities:

BaseNP Disambiguation Patterns
- Boundary insertion between abutting baseNPs: Because the baseline tags all determiners as $I$ , adjacent baseNPs merge. The second learned rule transforms a determiner's tag to $B$ when preceded by two $I$ tags ( $T_{-2}=I, T_{-1}=I, P_0=\text{DT} \to B$ ). Similar rules change $I$ to $B$ for wh-determiners ( $P_0=\text{WDT}$ ), pronouns ( $P_0=\text{PRP}$ ), or the specific word "who" ( $W_0=\text{who}$ ) when preceded by $T_{-1}=I$ .
- Conjunction within baseNPs: Baseline heuristics assign $O$ to coordinating conjunctions ( $P_0=\text{CC}$ ). Learned rules change $O$ to $I$ when the conjunction occurs between an $I$ tag and a noun ( $T_{-1}=I, P_0=\text{CC}, P_1=\text{NN} \to I$ or $P_1=\text{NNS} \to I$ ), correctly keeping conjoined head nouns inside a single baseNP chunk.
- Lexical reclassification: The word "about" is retagged from $O$ to $I$ when preceded by $T_{-1}=O$ in contexts where it acts as a numeric quantifier rather than a preposition (e.g., "including about four million shares").
Partitioning Chunk Disambiguation Patterns
- Determiner continuation: Determiners ( $P_0=\text{DT}$ ) assigned baseline $BN$ are changed to $N$ when preceded by $T_{-1}=BN$ .
- Sentence and punctuation boundaries: Tokens tagged $N$ are changed to $BN$ at sentence start ( $T_{-1}=Z, W_{-1}=\text{ZZZ}$ ) or following a comma ( $T_{-1}=P, P_{-1}=\text{',''}$ ).
- Verb vs. Noun chunk transitions: Tokens tagged $BN$ are changed to $BV$ when preceding a verb ( $T_1=V, P_1=\text{VB} \to BV$ ).
Knowl 9 — Error Categories in Transformation-Based Text Chunking
limitation
Error analysis on baseNP chunking indicates that two primary linguistic phenomena account for approximately 50% of all chunking errors:
1. Participle Disambiguation (VBG and VBN): The part-of-speech baseline assigns non-chunk tag $O$ to gerund-participles (VBG) and past participles (VBN). Although participles frequently function as prenominal adjectival modifiers inside baseNPs (requiring tag $I$ ), local $n$ -gram templates over words and POS tags often cannot differentiate adjectival premodifiers from main verbal heads or participial clauses.
2. Conjunction Scope Resolution (CC and Commas): Coordinating conjunctions (CC) such as "and" or punctuation commas (,) are assigned $O$ by the baseline. Treebank annotations group conjoined nouns or premodifiers into a single baseNP, but separate conjoined maximal noun phrases into distinct baseNPs. Differentiating between word-level conjunction within an NP and phrase-level or clause-level conjunction frequently requires wide-scope semantic and syntactic dependencies beyond the reach of local $n$ -gram transformation templates.

Coverage note — All major contributions of the paper—including the IOB and partitioning chunk representations, baseline POS heuristic, 100-template rule architecture, search and indexing optimizations, heuristic rule disabling, empirical evaluation on the Penn Treebank, lexical ablation study, learned rule analysis, and error analysis—have been captured as knowls. Minor commentary comparing previous manual evaluation methodologies on third-party systems from related work was omitted.

References

1.Abney, Steven. 1991. Parsing by chunks. In Berwick, Abney, and Tenny, editors, Principle-Based Parsing. Kluwer Academic Publishers.
2.Bourigault, D. 1992. Surface grammatical analysis for the extraction of terminological noun phrases. In Proceedings of the Fifteenth International Conference on Computational Linguistics, pages 977–981.
3.Brill, Eric. 1993a. Automatic grammar induction and parsing free text: A transformation-based approach. In Proceedings of the DARPA Speech and Natural Language Workshop, 1993, pages 237–242.
4.Brill, Eric. 1993b. A Corpus-Based Approach to Language Learning. Ph.D. thesis, University of Pennsylvania.
5.Brill, Eric. 1993c. Rule based tagger, version 1.14. Available from ftp.cs.jhu.edu in the directory /pub/brill/programs/.
6.Brill, Eric. 1994. Some advances in transformation-based part of speech tagging. In Proceedings of the Twelfth National Conference on Artificial Intelligence, pages 722–727. (cmp-lg/9406010).
7.Brill, Eric and Philip Resnik. 1994. A rule-based approach to prepositional attachment disambiguation. In Proceedings of the Sixteenth International Conference on Computational Linguistics. (cmp-lg/9410026).
8.Church, Kenneth. 1988. A stochastic parts program and noun phrase parser for unrestricted text. In Second Conference on Applied Natural Language Processing. ACL.
9.Ejerhed, Eva I. 1988. Finding clauses in unrestricted text by finitary and stochastic methods. In Second Conference on Applied Natural Language Processing, pages 219–227. ACL.
10.Gee, James Paul and François Grosjean. 1983. Performance structures: A psycholinguistic and linguistic appraisal. Cognitive Psychology, 15:411–458.
11.Kupiec, Julian. 1993. An algorithm for finding noun phrase correspondences in bilingual corpora. In Proceedings of the 31st Annual Meeting of the ACL, pages 17–22.
12.Marcus, Mitchell, Grace Kim, Mary Ann Marcinkiewicz, Robert MacIntyre, Ann Bies, Mark Ferguson, Karen Katz, and Britta Schasberger. 1994. The Penn Treebank: A revised corpus design for extracting predicate argument structure. In Human Language Technology, ARPA March 1994 Workshop. Morgan Kaumann.
13.Ramshaw, Lance A. and Mitchell P. Marcus. 1994. Exploring the statistical derivation of transformational rule sequences for part-of-speech tagging. In Proceedings of the ACL Balancing Act Workshop on Combining Symbolic and Statistical Approaches to Language, pages 86–95. (cmp-lg/9406011).
14.Voutilainen, Atro. 1993. NPTool, a detector of English noun phrases. In Proceedings of the Workshop on Very Large Corpora, pages 48–57. ACL, June. (cmp-lg/9502010).

Citation

MLA

Ramshaw, L. A., and M. P. Marcus. “Text Chunking Using Transformation-Based Learning”. ACL Third Workshop on Very Large Corpora, June 1995, Pp. 82-94, 1995, http://arxiv.org/abs/cmp-lg/9505040v1.

APA

Ramshaw, L. A., & Marcus, M. P. (1995). Text Chunking using Transformation-Based Learning. ACL Third Workshop on Very Large Corpora, June 1995, Pp. 82-94. http://arxiv.org/abs/cmp-lg/9505040v1

Chicago

Ramshaw, L. A., and M. P. Marcus. 1995. “Text Chunking Using Transformation-Based Learning”. ACL Third Workshop on Very Large Corpora, June 1995, Pp. 82-94. http://arxiv.org/abs/cmp-lg/9505040v1.

Harvard

Ramshaw, L.A. and Marcus, M.P. (1995) “Text Chunking using Transformation-Based Learning”, ACL Third Workshop on Very Large Corpora, June 1995, pp. 82-94 [Preprint]. Available at: http://arxiv.org/abs/cmp-lg/9505040v1.

Vancouver

1. Ramshaw LA, Marcus MP (1995) Text Chunking using Transformation-Based Learning. ACL Third Workshop on Very Large Corpora, June 1995, pp. 82-94

BibTeX

@article{ramshaw1995text,
  title = {Text Chunking using Transformation-Based Learning},
  author = {Ramshaw, Lance A. and Marcus, Mitchell P.},
  year = {1995},
  journal = {ACL Third Workshop on Very Large Corpora, June 1995, pp. 82-94},
  url = {http://arxiv.org/abs/cmp-lg/9505040v1},
  eprint = {cmp-lg/9505040}
}

Metadata:arXiv

Access the Paper

This paper is available from its original source. Click below to access the PDF.

Open PDF

License: https://creativecommons.org/licenses/by-nc-sa/4.0/

Text Chunking using Transformation-Based Learning

Abstract

Topics

Keywords

Table of Contents

Knowls

Knowl 1 — IOB Tagging Scheme and Partitioning Tag Scheme for Text Chunking

Knowl 2 — Baseline Initialization and POS-Conditioned Heuristic for Chunk Tagging

Knowl 3 — Contextual Rule Template Architecture for Transformation-Based Chunking

Knowl 4 — Optimized Rule Search and Static Pattern Indexing for Small-Tagset TBL

Knowl 5 — Heuristic Disabling and Re-enabling of Candidate Rules in TBL