Built independently by an author, for readers. Read the story and support ChapterPal
ChapterPal Logo
Log inSign up

keyword

dependency parse depths

Dependency parse depths refer to the hierarchical distances of words from the root node within a dependency parse tree, measured as the number of syntactic steps or edges traversed from the root to each specific word. In computational linguistics and natural language processing, sentences are analyzed as directed graphs where the main governing predicate serves as the root, and grammatical dependents form branched pathways below it. A word positioned at the root is assigned a depth of zero, while modifiers, arguments, and subsequent subordinate elements receive increasing depth values corresponding to their level of syntactic embedding. Quantifying these tree depths provides a metric for assessing syntactic hierarchy and evaluating whether latent representations in neural language models capture the vertical structure of sentence grammar.

1 item

A Structural Probe for Finding Syntax in Word Representations

A Structural Probe for Finding Syntax in Word Representations

John Hewitt, Christopher D. Manning

OrganizationsStanford University

Why you should read this

Proposes a geometric probing framework to demonstrate that contextual language models like BERT and ELMo implicitly embed entire hierarchical syntax trees directly within linear subspaces of their vector representations.

Recent work has improved our ability to detect linguistic knowledge in word representations. However, current methods for detecting syntactic knowledge do not test whether syntax trees are represented in their entirety. In this work, we propose a structural probe, which evaluates whether syntax trees are embedded in a linear transformation of a neural network's word representation space. The probe identifies a linear transformation under which squared L2 distance encodes the distance between words in the parse tree, and one in which squared L2 norm encodes depth in the parse tree. Using our probe, we show that such transformations exist for both ELMo and BERT but not in baselines, providing evidence that entire syntax trees are embedded implicitly in deep models' vector geometry.

Added

2026-09-25