What Does BERT Look at? An Analysis of BERT’s Attention

Kevin ClarkUrvashi KhandelwalOmer LevyChristopher D. Manning

article2019ACL2,120 citationsBest Paper Award

Reveals that individual attention heads in BERT encode distinct grammatical relations and coreference patterns, offering an attention-based analysis framework to interpret linguistic knowledge inside transformer models.

Listen

Large neural language models like BERT achieve state-of-the-art performance across natural language processing tasks, yet their internal mechanics remain largely opaque. While prior research focused on evaluating final outputs or internal vector states, the article investigates the internal attention mechanisms of BERT to understand what specific linguistic properties are learned directly from unannotated text.

The article aims to evaluate the surface-level patterns and linguistic capabilities encoded across the 144 attention heads in standard BERT. Specifically, it tests whether individual attention components or linear combinations of them capture grammatical syntax and coreference relationships without explicit task training.

To conduct this evaluation, the study examined attention weight distributions across 1,000 Wikipedia text segments and evaluated grammatical performance using standard benchmark datasets, including the Penn Treebank for dependency parsing and the CoNLL-2012 dataset for coreference resolution. The approach evaluated individual attention heads as direct classifiers and built lightweight diagnostic probes that combine attention weights with fixed word representations, avoiding modifications to the underlying pre-trained model weights.

The analysis reveals several key findings. First, attention heads frequently exhibit broad structural behaviors, such as focusing on adjacent tokens, spreading attention evenly across the sentence, or directing over 50% of their attention mass to separator tokens, which the authors show act as an inactive default state. Second, individual heads specialize heavily in specific grammatical roles; for example, specific heads identify noun determiners with 94.3% accuracy, direct objects of verbs with 86.8% accuracy, and prepositional objects with 76.3% accuracy, outperforming simple positional baselines by 35 to 46 percentage points. Third, a dedicated attention head successfully tracks entity coreference with 65% accuracy, approaching the 69% accuracy of specialized rule-based systems. Finally, combining attention maps with word embeddings in a simple diagnostic parser yields an overall parsing score of 77, indicating that comprehensive syntactic structure is directly encoded in the attention layers.

These findings demonstrate that language models naturally discover hierarchical syntax and coreference purely as an emergent property of self-supervised pre-training, rather than requiring hand-crafted syntactic rules. The results also show significant functional redundancy, as attention heads within the same layer cluster closely in behavior. This redundancy implies opportunities to streamline model architectures, reduce computational and deployment costs, and improve interpretability by pruning inactive or duplicative attention components.

Organizations developing or deploying language models should leverage attention map analysis as a standard diagnostic tool alongside vector evaluations to audit model capabilities. Model optimization teams should explore pruning redundant heads to reduce runtime latency while preserving specialized linguistic heads. Future research should evaluate whether these attention patterns persist across larger architectures and diverse languages, and explore methods to deliberately train compact models with specialized attention distributions.

Confidence in these findings is high for standard English language benchmarks under the BERT-base architecture. However, limitations remain: the quantitative linguistic evaluations were restricted to English datasets with truncated sequence lengths of 128 tokens, and individual heads showed high proficiency only on specific grammatical relations rather than full-sentence parsing independently. Stakeholders should account for these boundary conditions when generalizing the findings to other languages or longer-context models.

Meta Stanford University attention mechanism language model syntactic information Attention Heads BERT

arXiv: 1906.04341 clarkkev/attention-analysis

Paper: BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding, Jacob Devlin et al. (2019). Introduces the bidirectional Transformer architecture and pre-training objectives of BERT, which serves as the direct subject of analysis in this paper.
Paper: Attention is not Explanation, Sarthak Jain et al. (2019). Establishes foundational methodology and critical skepticism regarding whether attention weights faithfully reflect linguistic and model explanations.
Paper: Head-Driven Statistical Models for Natural Language Parsing, Michael Collins (2003). Provides the foundational syntactic head-modifier parsing theory against which individual BERT attention heads are evaluated.
Paper: Deep contextualized word representations, Matthew E. Peters et al. (2018). Pioneered the analysis and extraction of contextualized linguistic representations across deep neural network layers.

Paper: BERT Rediscovers the Classical NLP Pipeline, Ian Tenney et al. (2019). Extends the linguistic analysis of BERT by investigating how its layered representations systematically encode the classical NLP pipeline.
Paper: DeBERTa: Decoding-enhanced BERT with Disentangled Attention, Pengcheng He et al. (2021). Builds on insights into BERT's attention limitations by introducing disentangled attention mechanisms that separately compute content and relative position.
Paper: SpanBERT: Improving Pre-training by Representing and Predicting Spans, Mandar Joshi et al. (2019). Improves upon BERT's token-level attention and representations by pre-training directly on contiguous text spans.
Paper: RoBERTa: A Robustly Optimized BERT Pretraining Approach, Yinhan Liu et al. (2019). Presents a rigorous optimization study of BERT's pre-training design and hyperparameter space based on early BERT analysis.
Paper: Synthesizer: Rethinking Self-Attention for Transformer Models, Yi Tay et al. (2021). Critically re-evaluates the necessity of dot-product self-attention patterns in Transformers by testing synthetic and alignment-free alternatives.
Paper: Language Models as Knowledge Bases?, Fabio Petroni et al. (2019). Examines whether pre-trained models like BERT store factual and relational knowledge directly in their representations.
Paper: On the Relationship between Self-Attention and Convolutional Layers, Jean-Baptiste Cordonnier et al. (2020). Analyzes the theoretical and empirical behavior of multi-head self-attention mechanisms in relation to convolutional operations.

Cover for What Does BERT Look at? An Analysis of BERT’s Attention

Abstract

Large pre-trained neural networks such as BERT have had great recent success in NLP, motivating a growing body of research investigating what aspects of language they are able to learn from unlabeled data. Most recent analysis has focused on model outputs (e.g., language model surprisal) or internal vector representations (e.g., probing classifiers). Complementary to these works, we propose methods for analyzing the attention mechanisms of pre-trained models and apply them to BERT. BERT's attention heads exhibit patterns such as attending to delimiter tokens, specific positional offsets, or broadly attending over the whole sentence, with heads in the same layer often exhibiting similar behaviors. We further show that certain attention heads correspond well to linguistic notions of syntax and coreference. For example, we find heads that attend to the direct objects of verbs, determiners of nouns, objects of prepositions, and coreferent mentions with remarkably high accuracy. Lastly, we propose an attention-based probing classifier and use it to further demonstrate that substantial syntactic information is captured in BERT's attention.

Topics

attention mechanism language model syntactic information syntactic structure named entity recognition svm

Keywords

Attention Heads BERT Attention Heads Head Analysis Model Word SEP BERT Look

1 Introduction
2 Background: Transformers and BERT
3 Surface-Level Patterns in Attention
3.1 Relative Position
3.2 Attending to Separator Tokens
3.3 Focused vs Broad Attention
4 Probing Individual Attention Heads
4.1 Method
4.2 Dependency Syntax
4.3 Coreference Resolution
5 Probing Attention Head Combinations
6 Clustering Attention Heads
7 Related Work
8 Conclusion
References

Knowls

Knowl 1 — Attention-and-Words Probing Classifier for Dependency Parsing
model/method
To evaluate the syntactic knowledge distributed across multiple attention heads of BERT without modifying the underlying model, a parameterized probing classifier is defined over fixed attention maps and word representations. The classifier treats dependency parsing as directed head selection for each word $j$ across candidate heads $i$ .

The Attention-and-Words probing model conditions head importance on word identity by incorporating GloVe word embeddings $v_i, v_j \in \mathbb{R}^d$ for candidate head $i$ and dependent $j$ . The probability distribution over candidate heads $i$ for a given dependent word $j$ is:

$p(i \mid j) \propto \exp \left( \sum_{k=1}^n W_{k,:} (v_i \oplus v_j) \alpha_{ij}^k + U_{k,:} (v_i \oplus v_j) \alpha_{ji}^k \right)$

where $\alpha_{ij}^k$ is the attention weight from word $i$ to word $j$ in attention head $k$ , $\alpha_{ji}^k$ is the reverse attention weight from word $j$ to word $i$ in head $k$ , $n$ is the total number of attention heads across all layers ( $n = 144$ in BERT-base), $\oplus$ denotes vector concatenation, and $W, U \in \mathbb{R}^{n \times 2d}$ are learned parameter matrices that generate word-conditioned importance weights for each attention head.

A simpler Attention-Only probe assigns scalar weights $w_k, u_k \in \mathbb{R}$ to head $k$ without word embeddings:

$p(i \mid j) \propto \exp \left( \sum_{k=1}^n w_k \alpha_{ij}^k + u_k \alpha_{ji}^k \right)$

During training, BERT weights and GloVe vectors remain frozen, and only parameters $W$ and $U$ (or $w$ and $u$ ) are optimized via supervised learning on labeled treebank data.

Knowl 2 — Dependency Parsing Evaluation of Attention Probing Classifiers

data/table

The overall syntactic information captured across all 144 attention heads of BERT-base is evaluated on the Wall Street Journal section of the Penn Treebank annotated with Stanford Dependencies using Unlabeled Attachment Score (UAS).

*The structural probe score is in Undirected Unlabeled Attachment Score (UUAS) on a single BERT layer's vector representations and is not directly comparable.

The Attention-and-Words probe achieves 77 UAS, substantially outperforming the baseline using GloVe with distance features (58 UAS) and the control baseline using attention maps from a randomly initialized Transformer with pre-trained embeddings (30 UAS). This demonstrates that BERT's attention weights inherently encode substantial directed syntactic dependency structure.

Knowl 3 — Syntactic Dependency Specialization in Individual BERT Attention Heads

data/table

Individual attention heads in BERT-base (formatted as <layer>-<head>) specialize in specific syntactic dependency relations. Each attention head is evaluated as an unsupervised classifier where the predicted syntactic head of a word is chosen as the word receiving the maximum attention weight from that word (or vice versa).

Relation	Head	Accuracy (%)	Baseline (%)
All relations (overall UAS)	Head 7-6	34.5	26.3 (offset 1)
Prepositional modifier (prep)	Head 7-4	66.7	61.8 (offset -1)
Object of preposition (pobj)	Head 9-6	76.3	34.6 (offset -2)
Determiner (det)	Head 8-11	94.3	51.7 (offset 1)
Noun compound modifier (nn)	Head 4-10	70.4	70.2 (offset 1)
Nominal subject (nsubj)	Head 8-2	58.5	45.5 (offset 1)
Adjectival modifier (amod)	Head 4-10	75.6	68.3 (offset 1)
Direct object (dobj)	Head 8-10	86.8	40.0 (offset -2)
Adverbial modifier (advmod)	Head 7-6	48.8	40.2 (offset 1)
Auxiliary (aux)	Head 4-10	81.1	71.5 (offset 1)
Possessive modifier (poss)	Head 7-6	80.5	47.7 (offset 1)
Passive auxiliary (auxpass)	Head 4-10	82.5	40.5 (offset 1)
Clausal complement (ccomp)	Head 8-1 Wandering	48.8	12.4 (offset -2)
Marker (mark)	Head 8-2	50.7	14.5 (offset 2)
Phrasal verb particle (prt)	Head 6-7	99.1	91.4 (offset -1)

Baseline accuracies represent the best fixed-offset prediction (the number in parentheses indicates the token offset). In almost all relations except pobj, the direction of highest accuracy occurs when the syntactic dependent attends to the syntactic head word, reflecting the one-to-many nature of heads to dependents.

Knowl 4 — Subword-to-Word Attention Aggregation Method
model/method
Because BERT uses byte-pair subword tokenization, token-level attention maps must be converted into word-level attention maps to evaluate linguistic phenomena at the word level.

Let $I$ be the set of subword tokens comprising word $u$ , and let $J$ be the set of subword tokens comprising word $v$ . Given token-to-token attention weights $\alpha_{\text{token}}(i, j)$ from token $i \in I$ to token $j \in J$ :
1. Attention from a single token $i$ to a split word $v$ is computed by summing over target subwords: $\alpha(i, J) = \sum_{j \in J} \alpha_{\text{token}}(i, j)$
2. Attention from a split word $u$ to word $v$ is computed by taking the mean across all source subwords: $\alpha_{\text{word}}(u, v) = \frac{1}{|I|} \sum_{i \in I} \alpha(i, J) = \frac{1}{|I|} \sum_{i \in I} \sum_{j \in J} \alpha_{\text{token}}(i, j)$
This aggregation preserves the property that total outgoing attention from any word $u$ to all words in the sentence sums to 1. For word-level head probing, the predicted link for word $u$ is $\arg\max_{v \neq u} \alpha_{\text{word}}(u, v)$ , ignoring special tokens [CLS] and [SEP].

Knowl 5 — Coreference Resolution Performance of BERT Attention Head 5-4

data/table

Individual attention heads in BERT-base can perform coreference antecedent selection without fine-tuning. Evaluated on the CoNLL-2012 coreference dataset, the head word of a coreferent mention selects as its antecedent the mention whose head word receives the maximum attention weight.

Model	All (%)	Pronoun (%)	Proper (%)	Nominal (%)
Nearest mention	27	29	29	19
Head-word match	52	47	67	40
Rule-based sieve system	69	70	77	60
Neural coreference (Wiseman et al., 2015)*	83	–	–	–
Head 5-4	65	64	73	58

*The neural coreference system is trained specifically on coreference on full documents, whereas the attention probing is zero-shot on documents truncated to 128 tokens.

BERT Head 5-4 achieves 65% overall accuracy on antecedent selection, outperforming string-matching baselines by 13 percentage points and performing close to the multi-pass rule-based sieve system, demonstrating particular strength on nominal mentions (58%) through fuzzy semantic matching.

Knowl 6 — Separator Token Attention as a No-Op Mechanism in BERT
empirical result
In BERT-base, over 50% of the total attention mass across heads in layers 6–10 is placed directly on the [SEP] delimiter token, despite [SEP] occupying only a tiny fraction (typically 2 out of 128 tokens) of the input sequence.

Evidence indicates that attending to [SEP] functions as a computational no-op (no-operation) when a head's specific function is inapplicable:
1. When the current token is [SEP], over 90% of its own attention is directed back to itself and the other [SEP] token, rather than aggregating broad sentence context.
2. In specialized heads, tokens lacking the relevant syntactic property default to attending to [SEP]. For instance, in Head 8-10 (which links direct objects to verbs), non-noun tokens place their attention primarily on [SEP].
3. Gradient-based feature importance measurements of the masked language modeling loss $\mathcal{L}$ with respect to attention weights $\alpha$ , given by $\left| \frac{\partial \mathcal{L}}{\partial \alpha} \right|$ , drop close to zero for [SEP] beginning in layer 5, indicating that changes in attention weight to [SEP] have negligible influence on model outputs.
Knowl 7 — Positional and Entropy Patterns Across BERT Attention Layers
empirical result
BERT-base attention heads exhibit distinct layer-dependent positional and entropy behaviors:
1. Positional Offsets: Most heads put negligible attention on the current token itself. Early layers contain heads specialized to immediate positional offsets: 4 heads (in layers 2, 4, 7, and 8) place on average $>50\%$ of attention on the previous token ( $t-1$ ), and 5 heads (in layers 1, 2, 2, 3, and 6) place $>50\%$ on the next token ( $t+1$ ).
2. Distribution Entropy: Lower layers (especially layer 1) contain high-entropy attention heads that distribute attention broadly, allocating at most 10% of attention mass to any single token and functioning as bag-of-vectors sentence representations.
3. [CLS] Token Attention: While attention distributions from [CLS] across most layers match normal token entropy profiles, [CLS] in layer 12 exhibits an average entropy of 3.89 nats, attending broadly across the entire input to aggregate sequence-level representations for the next-sentence prediction objective.
Knowl 8 — Layer-Wise Functional Clustering of Attention Heads via Jensen-Shannon Divergence
empirical result
The functional similarity between any pair of attention heads $H_i$ and $H_j$ is quantified by computing the average Jensen-Shannon (JS) divergence between their output attention distributions over tokens in a dataset:

$\text{Distance}(H_i, H_j) = \sum_{\text{token} \in \text{data}} \text{JS}\big(H_i(\text{token}), H_j(\text{token})\big)$

Two-dimensional visualization via Multidimensional Scaling (MDS) reveals that attention heads form clear functional clusters corresponding to specific behaviors (e.g., delimiter attention, fixed positional offsets, and specific syntactic relations). Furthermore, heads within the same layer cluster close to each other, exhibiting substantial intra-layer behavioral similarity and redundancy, which is hypothesized to arise partly from the use of attention dropout during training.

Coverage note — None was omitted; all key empirical analyses, probing architectures, quantitative results for dependency syntax and coreference, and qualitative attention phenomena reported in the paper are covered.

References

1.Yossi Adi, Einat Kermany, Yonatan Belinkov, Ofer Lavi, and Yoav Goldberg. 2017. Fine-grained analysis of sentence embeddings using auxiliary prediction tasks. In ICLR.
2.Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015. Neural machine translation by jointly learning to align and translate. In ICLR.
3.Yonatan Belinkov, Nadir Durrani, Fahim Dalvi, Hassan Sajjad, and James R. Glass. 2017. What do neural machine translation models learn about morphology? In ACL.
4.Terra Blevins, Omer Levy, and Luke S. Zettlemoyer. 2018. Deep rnns encode soft hierarchical syntax. In ACL.
5.Kaylee Burns, Aida Nematzadeh, Alison Gopnik, and Thomas L. Griffiths. 2018. Exploiting attention to reveal shortcomings in memory models. In BlackboxNLP@EMNLP.
6.Kehai Chen, Rui Wang, Masao Utiyama, Eiichiro Sumita, and Tiejun Zhao. 2018. Syntax-directed attention for neural machine translation. In AAAI.
7.Alexis Conneau, Germán Kruszewski, Guillaume Lample, Loïc Barrault, and Marco Baroni. 2018. What you can cram into a single $&!#* vector: Probing sentence embeddings for linguistic properties. In ACL.
8.Andrew M Dai and Quoc V Le. 2015. Semi-supervised sequence learning. In NIPS.
9.Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In NAACL-HLT.
10.Akiko Eriguchi, Kazuma Hashimoto, and Yoshimasa Tsuruoka. 2016. Tree-to-sequence attentional neural machine translation. In ACL.
11.Mario Giulianelli, Jack Harding, Florian Mohnert, Dieuwke Hupkes, and Willem H. Zuidema. 2018. Under the hood: Using diagnostic classifiers to investigate and improve how language models track agreement information. In BlackboxNLP@EMNLP.
12.Yoav Goldberg. 2019. Assessing BERT’s syntactic abilities. arXiv preprint arXiv:1901.05287.
13.Kristina Gulordava, Piotr Bojanowski, Edouard Grave, Tal Linzen, and Marco Baroni. 2018. Colorless green recurrent networks dream hierarchically. In NAACL-HLT.
14.John Hewitt and Christopher D. Manning. 2019. Finding syntax with structural probes. In NAACL-HLT.
15.Sarthak Jain and Byron C. Wallace. 2019. Attention is not explanation. arXiv preprint arXiv:1902.10186.
16.Urvashi Khandelwal, He He, Peng Qi, and Daniel Jurafsky. 2018. Sharp nearby, fuzzy far away: How neural language models use context. In ACL.
17.Joseph B Kruskal. 1964. Multidimensional scaling by optimizing goodness of fit to a nonmetric hypothesis. Psychometrika, 29(1):1–27.
18.Heeyoung Lee, Yves Peirsman, Angel Chang, Nathanael Chambers, Mihai Surdeanu, and Dan Jurafsky. 2011. Stanford’s multi-pass sieve coreference resolution system at the conll-2011 shared task. In CoNLL.
19.Tal Linzen, Emmanuel Dupoux, and Yoav Goldberg. 2016. Assessing the ability of lstms to learn syntax-sensitive dependencies. TACL.
20.Nelson F. Liu, Matt Gardner, Yonatan Belinkov, M. Peters, and Noah A. Smith. 2019. Linguistic knowledge and transferability of contextual representations. In NAACL-HLT.
21.Mitchell P Marcus, Mary Ann Marcinkiewicz, and Beatrice Santorini. 1993. Building a large annotated corpus of english: The Penn treebank. Computational linguistics, 19(2):313–330.
22.David Marecek and Rudolf Rosa. 2018. Extracting syntactic trees from transformer encoder self-attentions. In BlackboxNLP@EMNLP.
23.Rebecca Marvin and Tal Linzen. 2018. Targeted syntactic evaluation of language models. In EMNLP.
24.Paul Michel, Omer Levy, and Graham Neubig. 2019. Are sixteen heads really better than one? arXiv preprint arXiv:1905.10650.
25.Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. Glove: Global vectors for word representation. In EMNLP.
26.Matthew E Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. In NAACL-HLT.
27.Sameer Pradhan, Alessandro Moschitti, Nianwen Xue, Olga Uryupina, and Yuchen Zhang. 2012. Conll-2012 shared task: Modeling multilingual unrestricted coreference in ontonotes. In Joint Conference on EMNLP and CoNLL-Shared Task.
28.Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. 2018. Improving language understanding by generative pre-training. https://blog.openai.com/language-unsupervised.
29.Alessandro Raganato and Jörg Tiedemann. 2018. An analysis of encoder representations in transformer-based machine translation. In BlackboxNLP@EMNLP.
30.Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. Neural machine translation of rare words with subword units. In ACL.
31.Xing Shi, Inkit Padhi, and Kevin Knight. 2016. Does string-based neural mt learn source syntax? In EMNLP.
32.Emma Strubell, Patrick Verga, Daniel Andor, David I Weiss, and Andrew McCallum. 2018. Linguistically-informed self-attention for semantic role labeling. In EMNLP.
33.Mukund Sundararajan, Ankur Taly, and Qiqi Yan. 2017. Axiomatic attribution for deep networks. In ICML.
34.Ian Tenney, Dipanjan Das, and Ellie Pavlick. 2019. Bert rediscovers the classical nlp pipeline. arXiv preprint arXiv:1905.05950.
35.Ian Tenney, Patrick Xia, Berlin Chen, Alex Wang, Adam Poliak, R Thomas McCoy, Najoung Kim, Benjamin Van Durme, Samuel R Bowman, Dipanjan Das, et al. 2018. What do you learn from context? probing for sentence structure in contextualized word representations. In ICLR.
36.Zhaopeng Tu, Baosong Yang, Michael R. Lyu, and Tong Zhang. 2018. Multi-head attention with disagreement regularization. In EMNLP.
37.Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In NIPS.
38.Jesse Vig. 2019. Visualizing attention in transformer-based language models. arXiv preprint arXiv:1904.02679.
39.Elena Voita, David Talbot, Fedor Moiseev, Rico Sennrich, and Ivan Titov. 2019. Analyzing multi-head self-attention: Specialized heads do the heavy lifting, the rest can be pruned. arXiv preprint arXiv:1905.09418.
40.Sam Joshua Wiseman, Alexander Matthew Rush, Stuart Merrill Shieber, and Jason Weston. 2015. Learning anaphoricity and antecedent ranking features for coreference resolution. In ACL.
41.Kelly W. Zhang and Samuel R. Bowman. 2018. Language modeling teaches you more syntax than translation does: Lessons learned through auxiliary task analysis. In BlackboxNLP@EMNLP.

Citation

MLA

Clark, K., et al. “What Does BERT Look At? An Analysis of BERT's Attention”. arXiv, 2019, http://arxiv.org/abs/1906.04341v1.

APA

Clark, K., Khandelwal, U., Levy, O., & Manning, C. D. (2019). What Does BERT Look At? An Analysis of BERT's Attention. arXiv. http://arxiv.org/abs/1906.04341v1

Chicago

Clark, K., U. Khandelwal, O. Levy, and C. D. Manning. 2019. “What Does BERT Look At? An Analysis of BERT's Attention”. arXiv. http://arxiv.org/abs/1906.04341v1.

Harvard

Clark, K. et al. (2019) “What Does BERT Look At? An Analysis of BERT's Attention”, arXiv [Preprint]. Available at: http://arxiv.org/abs/1906.04341v1.

Vancouver

1. Clark K, Khandelwal U, Levy O, Manning CD (2019) What Does BERT Look At? An Analysis of BERT's Attention. arXiv

BibTeX

@article{clark2019what,
  title = {What Does BERT Look At? An Analysis of BERT's Attention},
  author = {Clark, Kevin and Khandelwal, Urvashi and Levy, Omer and Manning, Christopher D.},
  year = {2019},
  journal = {arXiv},
  url = {http://arxiv.org/abs/1906.04341v1},
  eprint = {1906.04341}
}

Metadata:arXiv

Source Code

This paper has an official code repository available. Click below to access the source code.

View Repository

Access the Paper

This paper is available from its original source. Click below to access the PDF.

Open PDF

License: https://creativecommons.org/licenses/by/4.0/

Model	UAS
Right-branching baseline	26
Distances + GloVe	58
Random Init Attn + GloVe	30
Attn (Attention-Only probe)	61
Attn + GloVe (Attention-and-Words probe)	77
Structural probe (Hewitt and Manning, 2019)	80 UUAS*

What Does BERT Look at? An Analysis of BERT’s Attention

Abstract

Topics

Keywords

Table of Contents

Knowls

Knowl 1 — Attention-and-Words Probing Classifier for Dependency Parsing

Knowl 2 — Dependency Parsing Evaluation of Attention Probing Classifiers

Knowl 3 — Syntactic Dependency Specialization in Individual BERT Attention Heads

Knowl 4 — Subword-to-Word Attention Aggregation Method

Knowl 5 — Coreference Resolution Performance of BERT Attention Head 5-4

Knowl 6 — Separator Token Attention as a No-Op Mechanism in BERT

Knowl 7 — Positional and Entropy Patterns Across BERT Attention Layers

Knowl 8 — Layer-Wise Functional Clustering of Attention Heads via Jensen-Shannon Divergence

References

Citation

BibTeX

Source Code

Access the Paper