Back

A Structured Self-attentive Sentence Embedding

Zhouhan LinMinwei FengCicero Nogueira dos SantosMo YuBing XiangBowen ZhouYoshua Bengio

article2017ICLR2,327 citations

Proposes a structured self-attention mechanism that replaces fixed sentence vectors with 2-D matrix embeddings, enabling models to capture multiple semantic aspects of text while providing interpretable visual attention across natural language processing tasks.

Listen

Representing the meaning of full sentences in automated language processing is a critical challenge, especially as complex sentences convey multiple concepts that traditional single-vector summaries fail to capture effectively. Standard approaches typically compress an entire sentence into one fixed vector through sequential processing or basic pooling, which places an excessive burden on the system's memory and leaves internal mechanisms difficult to interpret. The article set out to introduce and evaluate a structured self-attentive sentence embedding model that uses a two-dimensional matrix to extract multiple semantic components from a single sentence without requiring external context.

The authors evaluated their approach across three diverse language-processing tasks: author profiling using social media posts, sentiment classification using customer reviews, and textual entailment using sentence pairs. The architecture couples a bidirectional sequential network with a self-attention mechanism and introduces a specialized regularization penalty to ensure each extracted vector captures a distinct, non-redundant aspect of the text. To support practical deployment, the investigation also tested weight-pruning methods designed to reduce the model's footprint while preserving performance.

The findings demonstrate that the proposed matrix embedding consistently outperforms standard baselines and achieves competitive or superior accuracy against existing models. Specifically, on the review sentiment dataset, the model achieved an accuracy of 64.21% compared to roughly 62% for conventional baseline models, and on author age profiling, it reached 80.45% compared to baseline performance around 77% to 78%. On textual entailment, the method achieved 84.4% accuracy, which is within 0.2 percentage points of the leading benchmark while surpassing other sentence-encoding baselines by more than 1%. Furthermore, the ablation tests confirmed that the diversity penalty significantly improves focus and accuracy, while varying the number of attention vectors showed that extracting multiple distinct components yields a substantial boost over single-vector representations.

These results indicate that self-attention successfully relieves recurrent networks from long-term memory bottlenecks by directly harvesting contextual cues across the sentence, making the architecture well-suited for processing longer passages. In addition, the attention weights provide direct, visual interpretability, allowing decision-makers to audit which words triggered a particular classification. From a deployment perspective, the weight-pruning strategy enables significant reductions in parameters, lowering storage and computational overhead with only minor trade-offs in accuracy.

Organizations developing automated text analysis tools should consider adopting self-attentive matrix embeddings for classification tasks where both accuracy and explainability are critical. If deployed in resource-constrained environments, engineering teams can implement structured pruning to reduce memory requirements substantially. Moving forward, research should explore extending this architecture to longer documents, such as full articles, and developing effective decoder structures to enable unsupervised pre-training, addressing the primary limitation that the current framework relies entirely on supervised, task-specific training.

CIFARIBMMila – Québec Artificial Intelligence Instituteattention mechanismclassificationregularizationSentenceEmbeddingSentence Embedding
arXiv: 1703.03130
Cover for A Structured Self-attentive Sentence Embedding

Abstract

This paper proposes a new model for extracting an interpretable sentence embedding by introducing self-attention. Instead of using a vector, we use a 2-D matrix to represent the embedding, with each row of the matrix attending on a different part of the sentence. We also propose a self-attention mechanism and a special regularization term for the model. As a side effect, the embedding comes with an easy way of visualizing what specific parts of the sentence are encoded into the embedding. We evaluate our model on 3 different tasks: author profiling, sentiment classification, and textual entailment. Results show that our model yields a significant performance gain compared to other sentence embedding methods in all of the 3 tasks.

Topics

attention mechanismclassificationregularizationsentiment classificationtextual entailment

Keywords

SentenceEmbeddingSentence EmbeddingModelHiddenMatrixAttentionLSTMarXivPENALIZATION TERM

Table of Contents

  • 1 Introduction
  • 2 Approach
  • 2.1 Model
  • 2.2 Penalization term
  • 2.3 Visualization
  • 3 Related work
  • 4 Experimental results
  • 4.1 Author profiling
  • 4.2 Sentiment analysis
  • 4.3 Textual entailment
  • 4.4 Exploratory experiments
  • 4.4.1 Effect of penalization term
  • 4.4.2 Effect of multiple vectors
  • 5 Conclusion and discussion
  • References
  • A Pruned MLP for Structured Matrix Sentence Embedding
  • B Detailed Structure of the Model for SNLI Dataset

Knowls

  1. Knowl 1 — Structured Self-Attentive Sentence Embedding

    model/method

    The structured self-attentive sentence embedding model transforms a variable-length sentence into a fixed-size 2-D representation matrix by computing multiple hops of attention over bidirectional LSTM hidden states.

    Let an input sentence of nn tokens be represented by word embedding vectors S=(w1,w2,…,wn)∈Rn×dS = (w_1, w_2, \dots, w_n) \in \mathbb{R}^{n \times d}, where dd is the word embedding dimension. A bidirectional LSTM with uu hidden units in each direction encodes SS into hidden states H=(h1,h2,…,hn)∈Rn×2uH = (h_1, h_2, \dots, h_n) \in \mathbb{R}^{n \times 2u}, where each ht=[h→t;h←t]h_t = [\overrightarrow{h}_t; \overleftarrow{h}_t].

    A self-attention mechanism maps HH to an annotation weight matrix A∈Rr×nA \in \mathbb{R}^{r \times n} using a 2-layer MLP without bias:

    A=softmax(Ws2tanh⁡(Ws1HT))A = \text{softmax}\left(W_{s2} \tanh\left(W_{s1} H^T\right)\right)

    where Ws1∈Rda×2uW_{s1} \in \mathbb{R}^{d_a \times 2u} is a parameter matrix with hyperparameter hidden size dad_a, Ws2∈Rr×daW_{s2} \in \mathbb{R}^{r \times d_a} is a parameter matrix determining the number of attention hops rr, and the softmax(⋅)\text{softmax}(\cdot) operator is applied along the second dimension (over the nn sequence positions) such that ∑k=1nAik=1\sum_{k=1}^n A_{ik} = 1 for each hop i∈{1,…,r}i \in \{1, \dots, r\}.

    The final sentence embedding matrix M∈Rr×2uM \in \mathbb{R}^{r \times 2u} is formed by computing the weighted sum of LSTM hidden representations for each hop:

    M=AHM = A H

    Each of the rr rows of MM captures a distinct semantic aspect or component of the sentence.

  2. Knowl 2 — Frobenius Norm Attention Diversity Regularization

    model/method

    To prevent multi-hop self-attention from extracting redundant representations where multiple hops attend to the same words, an auxiliary penalization term PP is added to the training loss:

    P=∥AAT−I∥F2P = \left\| A A^T - I \right\|_F^2

    where A∈Rr×nA \in \mathbb{R}^{r \times n} is the attention annotation matrix whose rows aia^i sum to 1, I∈Rr×rI \in \mathbb{R}^{r \times r} is the identity matrix, and ∥⋅∥F\|\cdot\|_F denotes the Frobenius norm.

    For any off-diagonal entry i≠ji \neq j, the product (AAT)ij=∑k=1nAikAjk∈[0,1](A A^T)_{ij} = \sum_{k=1}^n A_{ik} A_{jk} \in [0, 1] corresponds to the inner product of two discrete probability distributions over the sentence positions. Minimizing this term toward 0 penalizes overlap between attention distributions across different hops ii and jj.

    Subtracting the identity matrix forces the diagonal elements (AAT)ii=∑k=1nAik2(A A^T)_{ii} = \sum_{k=1}^n A_{ik}^2 toward 1, which encourages each individual attention vector aia^i to concentrate its probability mass on as few tokens as possible rather than spreading diffusely across the sentence.

  3. Knowl 3 — Multiplicative Gated Interaction for Matrix Sentence Textual Entailment

    model/method

    For sentence-pair classification tasks like textual entailment, hypothesis and premise sentences are encoded into separate matrix representations Mh,Mp∈Rr×2uM_h, M_p \in \mathbb{R}^{r \times 2u} using shared bidirectional LSTM and self-attention parameters. The relation between MhM_h and MpM_p is computed via a 3-way multiplicative interaction inspired by factored gated autoencoders.

    For each matrix, row-wise projections are computed via batched dot products with 3-D parameter tensors Wfh,WfpW_{fh}, W_{fp}:

    Fh=batcheddot(Mh,Wfh)F_h = \text{batcheddot}(M_h, W_{fh})

    Fp=batcheddot(Mp,Wfp)F_p = \text{batcheddot}(M_p, W_{fp})

    The semantic relation factor matrix FrF_r is computed via the Hadamard (element-wise) product:

    Fr=Fh⊙FpF_r = F_h \odot F_p

    FrF_r is subsequently flattened and passed through a multi-layer perceptron with a softmax classification layer to predict the relationship category (e.g., entailment, contradiction, or neutral).

  4. Knowl 4 — Structured Weight Pruning for Matrix Sentence Embeddings

    model/method

    Connecting an r×2ur \times 2u matrix sentence embedding MM to a standard fully connected hidden layer of bb units requires r⋅2u⋅br \cdot 2u \cdot b parameters, which can account for ~90% of total model parameters. Structured weight pruning exploits the 2-D symmetry of MM across both the hop and feature dimensions to reduce parameter count.

    1. Vertical pruning: The bb hidden units are split into rr groups of size pp (b=r⋅pb = r \cdot p). The ii-th group connects exclusively to the ii-th row of MM, producing a structured hidden matrix Mv∈Rr×pM^v \in \mathbb{R}^{r \times p} and eliminating a fraction of (r−1)/r(r - 1)/r connections.
    2. Horizontal pruning: A complementary structured hidden representation Mh∈R2u×qM^h \in \mathbb{R}^{2u \times q} is formed by connecting hidden units of size qq exclusively across the identical LSTM feature index across all rr rows.

    The subsequent classification softmax layer connects directly to the combined elements of MvM^v and MhM^h, maintaining expressive power while reducing fully connected parameter requirements by over an order of magnitude.

  5. Knowl 5 — Classification Performance on Author Profiling and Sentiment Analysis

    data/table

    The structured self-attentive sentence embedding model was evaluated against standard BiLSTM and CNN sentence encoders on two classification tasks: 5-class user age range prediction on the PAN-2016 English Twitter Author Profiling dataset (68,485 training, 4,000 validation, 4,000 test), and 5-star sentiment rating prediction on the Yelp dataset (500,000 training, 2,000 validation, 2,000 test). All models utilized 300 hidden units per LSTM direction and a 2-layer ReLU MLP for classification.

    Models Yelp Accuracy Age Accuracy
    BiLSTM + Max Pooling + MLP 61.99% 77.40%
    CNN + Max Pooling + MLP 62.05% 78.15%
    Structured Self-Attentive Model 64.21% 80.45%

    The structured self-attentive embedding achieves a 2.16% accuracy gain on Yelp and a 2.30% gain on the Age dataset over the strongest baseline, demonstrating the value of multi-aspect attention representations over max pooling.

  6. Knowl 6 — Sentence-Encoding Performance on Stanford Natural Language Inference

    data/table

    The structured self-attentive sentence encoder was evaluated on the Stanford Natural Language Inference (SNLI) corpus (570k sentence pairs; 3 classes: entailment, contradiction, neutral) under the strict sentence-encoding evaluation setting, where hypothesis and premise sentences are encoded into representations independently without cross-sentence attention.

    Model Test Accuracy
    300D LSTM encoders (Bowman et al., 2016) 80.6%
    600D (300+300) BiLSTM encoders (Liu et al., 2016b) 83.3%
    300D Tree-based CNN encoders (Mou et al., 2015a) 82.1%
    300D SPINN-PI encoders (Bowman et al., 2016) 83.2%
    300D NTI-SLSTM-LSTM encoders (Munkhdalai Yu, 2016a) 83.4%
    1024D GRU encoders with SkipThoughts pre-training (Vendrov et al., 2015) 81.4%
    300D NSE encoders (Munkhdalai Yu, 2016b) 84.6%
    Structured Self-Attentive Encoder (Our method) 84.4%

    The structured self-attentive encoder outperforms all standard sentence encoders by ≥1.0%\ge 1.0\% accuracy, trailing only the Neural Semantic Encoder (NSE) by 0.2%.

  7. Knowl 7 — Impact of Frobenius Attention Diversity Regularization

    data/table

    Ablation experiments evaluating the Frobenius norm penalization term P=∥AAT−I∥F2P = \|AA^T - I\|_F^2 demonstrate that penalizing attention redundancy directly improves classification performance on Yelp and Twitter Age datasets.

    Penalization coefficient Yelp Accuracy Age Accuracy
    1.0 (with penalty) 64.21% 80.45%
    0.0 (without penalty) 61.74% 79.27%

    Qualitative attention visualizations reveal that models trained without the penalization term collapse to redundant attention across all rr hops, frequently focusing on uninformative stop words (such as 'it'). Models trained with the penalization term exhibit distinct, diverse attention across different hops and attend specifically to key task-relevant phrase segments.

  8. Knowl 8 — Sensitivity of Sentence Representations to the Number of Attention Hops

    empirical result

    Varying the number of attention rows rr from 1 to 30 on the Age and SNLI datasets demonstrates the quantitative value of 2-D matrix representations compared to 1-D vector sentence embeddings.

    When r=1r = 1, the representation reduces to a standard single-vector attentive sentence embedding, yielding performance comparable to ordinary pooling and baseline vector encoders. Increasing rr from 1 to 10 produces a steep gain in test accuracy. For values of rr between 10 and 30, test set accuracy curves remain high and largely invariant, confirming that extracting multiple semantic aspects is essential while making the architecture robust across a broad hyperparameter range for rr.

  9. Knowl 9 — Parameter Reduction and Accuracy Trade-Off in Structured Pruning

    data/table

    Structured weight pruning of the fully connected layer attached to the matrix embedding was evaluated across Yelp, Age, and SNLI datasets. In each case, MvM^v and MhM^h pruned layers were compared against the unpruned baseline with bb hidden units.

    Configuration Hidden Layer Softmax Other Total Accuracy
    Yelp, Original (b=3000b=3000) 54M 15K 1.3M 55.3M 64.21%
    Yelp, Pruned (p=150,q=10p=150, q=10) 2.7M 52.5K 1.3M 4.1M 63.86%
    Age, Original (b=4000b=4000) 72M 20K 1.3M 73.2M 80.45%
    Age, Pruned (p=25,q=20p=25, q=20) 822K 63.75K 1.3M 2.1M 77.32%
    SNLI, Original (b=4000b=4000) 72M 12K 22.9M 95.0M 84.43%
    SNLI, Pruned (p=300,q=10p=300, q=10) 5.6M 45K 22.9M 28.6M 83.16%

    Pruning reduces the hidden layer parameter count by over 90% (e.g., from 54M to 2.7M on Yelp and 72M to 822K on Age) while incurring only modest drops in accuracy (e.g., 0.35% on Yelp and 1.27% on SNLI).

  10. Knowl 10 — Obstacle to Unsupervised Training of Matrix Sentence Embeddings

    limitation

    The structured self-attentive sentence embedding model relies on supervised downstream task objectives and cannot be trained out-of-the-box in an unsupervised reconstruction framework (e.g., autoencoding).

    Because the matrix embedding extracts semantic components across rr rows without a predefined ordering or explicit alignment prior, a sequential decoder does not know how the latent aspects stored in the rows should be partitioned and sequenced to reconstruct the original token stream. Searching over all potential row assignments using standard neural decoders introduces a high risk of overfitting.

Coverage note — None was omitted; all key contributions including the self-attention mechanism, the Frobenius diversity regularizer, visualization techniques, structured weight pruning, gated entailment modeling, benchmark results, and ablations are covered.

References

  1. 1.Yoshua Bengio, Réjean Ducharme, and Pascal Vincent. A neural probabilistic language model. In Advances in Neural Information Processing Systems, pp. 932–938, 2001.
  2. 2.Samuel R Bowman, Gabor Angeli, Christopher Potts, and Christopher D Manning. A large annotated corpus for learning natural language inference. arXiv preprint arXiv:1508.05326, 2015.
  3. 3.Samuel R Bowman, Jon Gauthier, Abhinav Rastogi, Raghav Gupta, Christopher D Manning, and Christopher Potts. A fast unified model for parsing and sentence understanding. arXiv preprint arXiv:1603.06021, 2016.
  4. 4.Jianpeng Cheng, Li Dong, and Mirella Lapata. Long short-term memory-networks for machine reading. In Conference on Empirical Methods in Natural Language Processing (EMNLP). Association for Computational Linguistics, 2016.
  5. 5.Junyoung Chung, Caglar Gulcehre, KyungHyun Cho, and Yoshua Bengio. Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv:1412.3555, 2014.
  6. 6.Cicero dos Santos and Maira Gatti. Deep convolutional neural networks for sentiment analysis of short texts. In Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers, pp. 69–78, 2014.
  7. 7.Cicero dos Santos, Ming Tan, Bing Xiang, and Bowen Zhou. Attentive pooling networks. arXiv preprint arXiv:1602.03609, 2016.
  8. 8.Minwei Feng, Bing Xiang, Michael R. Glass, Lidan Wang, and Bowen Zhou. Applying deep learning to answer selection: a study and an open task. In 2015 IEEE Workshop on Automatic Speech Recognition and Understanding, ASRU 2015, Scottsdale, AZ, USA, December 13-17, 2015, pp. 813–820, 2015.
  9. 9.Felix Hill, Kyunghyun Cho, and Anna Korhonen. Learning distributed representations of sentences from unlabelled data. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 1367–1377, San Diego, California, June 2016. Association for Computational Linguistics. URL http://www.aclweb.org/anthology/N16-1162.
  10. 10.Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 9(8): 1735–1780, 1997.
  11. 11.Nal Kalchbrenner, Edward Grefenstette, and Phil Blunsom. A convolutional neural network for modelling sentences. arXiv preprint arXiv:1404.2188, 2014.
  12. 12.Yoon Kim. Convolutional neural networks for sentence classification. arXiv preprint arXiv:1408.5882, 2014.
  13. 13.Ryan Kiros, Yukun Zhu, Ruslan R Salakhutdinov, Richard Zemel, Raquel Urtasun, Antonio Torralba, and Sanja Fidler. Skip-thought vectors. In Advances in neural information processing systems, pp. 3294–3302, 2015.
  14. 14.Quoc V Le and Tomas Mikolov. Distributed representations of sentences and documents. In ICML, volume 14, pp. 1188–1196, 2014.
  15. 15.Ji Young Lee and Franck Dernoncourt. Sequential short-text classification with recurrent and convolutional neural networks. arXiv preprint arXiv:1603.03827, 2016.
  16. 16.Peng Li, Wei Li, Zhengyan He, Xuguang Wang, Ying Cao, Jie Zhou, and Wei Xu. Dataset and neural recurrent sequence labeling model for open-domain factoid question answering. arXiv preprint arXiv:1607.06275, 2016.
  17. 17.Wang Ling, Lin Chu-Cheng, Yulia Tsvetkov, and Silvio Amir. Not all contexts are created equal: Better word representations with variable attention. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pp. 1367–1372, Lisbon, Portugal, September 2015. Association for Computational Linguistics.
  18. 18.Yang Liu, Chengjie Sun, Lei Lin, and Xiaolong Wang. Learning natural language inference using bidirectional LSTM model and inner-attention. CoRR, abs/1605.09090, 2016a.
  19. 19.Yang Liu, Chengjie Sun, Lei Lin, and Xiaolong Wang. Learning natural language inference using bidirectional lstm model and inner-attention. arXiv preprint arXiv:1605.09090, 2016b.
  20. 20.Mingbo Ma, Liang Huang, Bing Xiang, and Bowen Zhou. Dependency-based convolutional neural networks for sentence embedding. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, volume 2, pp. 174–179, 2015.
  21. 21.Horia Margarit and Raghav Subramaniam. A batch-normalized recurrent network for sentiment classification. In Advances in Neural Information Processing Systems, 2016.
  22. 22.Roland Memisevic. Learning to relate images. IEEE transactions on pattern analysis and machine intelligence, 35(8):1829–1846, 2013.
  23. 23.Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781, 2013.
  24. 24.Lili Mou, Rui Men, Ge Li, Yan Xu, Lu Zhang, Rui Yan, and Zhi Jin. Natural language inference by tree-based convolution and heuristic matching. arXiv preprint arXiv:1512.08422, 2015a.
  25. 25.Lili Mou, Hao Peng, Ge Li, Yan Xu, Lu Zhang, and Zhi Jin. Discriminative neural sentence modeling by tree-based convolution. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pp. 2315–2325, Lisbon, Portugal, September 2015b. Association for Computational Linguistics. URL http://aclweb.org/anthology/D15-1279.
  26. 26.Tsendsuren Munkhdalai and Hong Yu. Neural tree indexers for text understanding. arXiv preprint arXiv:1607.04492, 2016a.
  27. 27.Tsendsuren Munkhdalai and Hong Yu. Neural semantic encoders. arXiv preprint arXiv:1607.04315, 2016b.
  28. 28.Hamid Palangi, Li Deng, Yelong Shen, Jianfeng Gao, Xiaodong He, Jianshu Chen, Xinying Song, and Rabab Ward. Deep sentence embedding using long short-term memory networks: Analysis and application to information retrieval. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 24(4):694–707, 2016.
  29. 29.Ankur P. Parikh, Oscar Tackstrom, Dipanjan Das, and Jakob Uszkoreit. A decomposable attention model for natural language inference. In Proceedings of EMNLP, 2016.
  30. 30.Jeffrey Pennington, Richard Socher, and Christopher D Manning. Glove: Global vectors for word representation. In EMNLP, volume 14, pp. 1532–43, 2014.
  31. 31.Richard Socher, Jeffrey Pennington, Eric H. Huang, Andrew Y. Ng, and Christopher D. Manning. Semi-supervised recursive autoencoders for predicting sentiment distributions. In Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing, pp. 151–161, Edinburgh, Scotland, UK., July 2011. Association for Computational Linguistics. URL http://www.aclweb.org/anthology/D11-1014.
  32. 32.Richard Socher, Alex Perelygin, Jean Y Wu, Jason Chuang, Christopher D Manning, Andrew Y Ng, and Christopher Potts. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the conference on empirical methods in natural language processing (EMNLP), volume 1631, pp. 1642. Citeseer, 2013.
  33. 33.Kai Sheng Tai, Richard Socher, and Christopher D. Manning. Improved semantic representations from tree-structured long short-term memory networks. In Proceedings of ACL, pp. 1556–1566, 2015.
  34. 34.Ming Tan, Cicero dos Santos, Bing Xiang, and Bowen Zhou. Improved representation learning for question answer matching. In Proceedings of ACL, pp. 464–473, Berlin, Germany, August 2016. Association for Computational Linguistics. URL http://www.aclweb.org/anthology/P16-1044.
  35. 35.Theano Development Team. Theano: A {Python} framework for fast computation of mathematical expressions. arXiv e-prints, abs/1605.0, 2016. URL http://arxiv.org/abs/1605.02688.
  36. 36.Ivan Vendrov, Ryan Kiros, Sanja Fidler, and Raquel Urtasun. Order-embeddings of images and language. arXiv preprint arXiv:1511.06361, 2015.
  37. 37.Wenpeng Yin and Hinrich Schütze. Convolutional neural network for paraphrase identification. In Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 901–911, 2015.

Citation

MLA
Lin, Z., et al. “A Structured Self-attentive Sentence Embedding”. arXiv, 2017, http://arxiv.org/abs/1703.03130v1.
APA
Lin, Z., Feng, M., Santos, C. N. dos ., Yu, M., Xiang, B., Zhou, B., & Bengio, Y. (2017). A Structured Self-attentive Sentence Embedding. arXiv. http://arxiv.org/abs/1703.03130v1
Chicago
Lin, Z., M. Feng, C. N. dos . Santos, et al. 2017. “A Structured Self-attentive Sentence Embedding”. arXiv. http://arxiv.org/abs/1703.03130v1.
Harvard
Lin, Z. et al. (2017) “A Structured Self-attentive Sentence Embedding”, arXiv [Preprint]. Available at: http://arxiv.org/abs/1703.03130v1.
Vancouver
1. Lin Z, Feng M, Santos CN dos, Yu M, Xiang B, Zhou B, Bengio Y (2017) A Structured Self-attentive Sentence Embedding. arXiv

BibTeX

@article{lin2017structured,
  title = {A Structured Self-attentive Sentence Embedding},
  author = {Lin, Zhouhan and Feng, Minwei and Santos, Cicero Nogueira dos and Yu, Mo and Xiang, Bing and Zhou, Bowen and Bengio, Yoshua},
  year = {2017},
  journal = {arXiv},
  url = {http://arxiv.org/abs/1703.03130v1},
  eprint = {1703.03130}
}
Metadata:arXiv

Access the Paper

This paper is available from its original source. Click below to access the PDF.

Open PDF

License: Published with permission