End-To-End Memory Networks

Sainbayar SukhbaatarArthur SzlamJason WestonRob Fergus

article2015NeurIPS2,755 citations

Proposes an end-to-end differentiable memory network architecture that uses multi-hop recurrent attention over external memory, enabling effective training on question answering and language modeling without step-by-step supervision.

Listen

The article addresses the challenge of building artificial intelligence systems that perform multiple reasoning steps over stored information and capture long-term dependencies in sequential data such as text. These capabilities are essential for practical question-answering and language-generation applications, yet many existing neural models struggle because they lack explicit, accessible memory or require extensive manual supervision during training.

The article set out to develop and test a neural network architecture that maintains an external memory, performs multiple attention-based reads or hops over that memory, and produces an output after several internal steps, all trained end-to-end from input-output pairs alone.

The approach was evaluated through controlled experiments on twenty synthetic question-answering tasks that require different forms of deduction and on standard language-modeling benchmarks. The model was compared against recurrent baselines such as LSTMs and against earlier memory networks that needed strong supervision of supporting facts. Key design choices tested included sentence representations, the number of memory hops, and training schedules that temporarily removed intermediate nonlinearities.

The main findings are that the end-to-end model achieves error rates within a few percentage points of strongly supervised memory networks while using far less supervision, that increasing the number of memory hops consistently lowers error on reasoning tasks, and that the same architecture slightly outperforms tuned recurrent networks on language modeling, reaching 111 perplexity on Penn Treebank and 147 on Text8. Position-aware sentence encoding and a two-phase linear-start training procedure each contributed measurable gains, especially on tasks sensitive to word order.

These results indicate that explicit memory with recurrent attention can be trained scalably without per-step labels, lowering the cost and effort needed to apply such models to new domains. The performance edge from multiple hops suggests that allowing several internal reasoning steps before producing an answer improves accuracy on problems that require chaining facts or maintaining context over long sequences.

Further work should focus on scaling the memory access mechanism to larger stores, for example through multiscale attention or hashing, and on closing the remaining gap to strongly supervised systems on the most difficult synthetic tasks. Additional experiments with real-world data and larger vocabularies would also help establish whether the observed advantages hold outside controlled settings.

The reported gains rest on relatively small training sets and repeated random restarts to mitigate variance; results on the largest language-modeling corpus reflect only a single training run. Readers should therefore treat the exact numerical margins as indicative rather than definitive until confirmed on broader data.

Meta New York University language model question answering random access memory End-to-End Memory Networks recurrent neural networks attention mechanism

arXiv: 1503.08895 facebook/MemNN

Paper: Neural Turing Machines, Alex Graves et al. (2014). Introduces the foundational architecture for coupling neural network controllers with differentiable external memory addressable via soft attention mechanisms.
Paper: Neural Machine Translation by Jointly Learning to Align and Translate, Dzmitry Bahdanau et al. (2015). Pioneers the soft attention mechanism over sequential representations that forms the mathematical basis for reading from memory hops in End-To-End Memory Networks.
Paper: Long Short-Term Memory, Sepp Hochreiter et al. (1997). Presents the foundational recurrent gating architecture that the source adopts as standard baselines and seeks to enhance with explicit external memory.
Paper: Sequence to Sequence Learning with Neural Networks, Ilya Sutskever et al. (2014). Establishes the sequence-to-sequence neural framework used as a comparative benchmark and baseline for capturing multi-step dependencies.
Paper: Natural Language Processing (almost) from Scratch, Ronan Collobert et al. (2011). Develops early foundational techniques for learning end-to-end continuous text representations and embeddings without handcrafted linguistic features.

Paper: Relational recurrent neural networks, Adam Santoro et al. (2018). Extends memory-augmented recurrent networks by incorporating self-attention across memory slots to model relational reasoning over multi-step tasks.
Paper: Matching Networks for One Shot Learning, Oriol Vinyals et al. (2016). Builds upon attention-based memory retrieval across external support sets to enable rapid one-shot learning.
Paper: Titans: Learning to Memorize at Test Time, Ali Behrouz et al. (2024). Advances external neural memory architectures into modern sequence models through dynamic test-time memorization.
Paper: Pointer Sentinel Mixture Models, Stephen Merity et al. (2016). Applies memory-attention mechanisms over past hidden states to let language models selectively copy words directly from recent context.
Paper: Test-time regression: a unifying framework for designing sequence models with associative memory, Ke Alexander Wang et al. (2025). Provides a unified theoretical framework formalizing how neural sequence models perform associative recall and memory updates at test time.
Paper: MeMo: Memory as a Model, Ryan Wei Heng Quek et al. (2026). Extends the principle of decoupled, explicit memory storage by formulating dedicated parametric memory models for modular question answering.
Paper: Training Large Language Models to Reason in a Continuous Latent Space, Shibo Hao et al. (2024). Generalizes multi-hop continuous reasoning by enabling autoregressive language models to reason iteratively within continuous latent space rather than explicit tokens.

Abstract

We introduce a neural network with a recurrent attention model over a possibly large external memory. The architecture is a form of Memory Network (Weston et al., 2015) but unlike the model in that work, it is trained end-to-end, and hence requires significantly less supervision during training, making it more generally applicable in realistic settings. It can also be seen as an extension of RNNsearch to the case where multiple computational steps (hops) are performed per output symbol. The flexibility of the model allows us to apply it to tasks as diverse as (synthetic) question answering and to language modeling. For the former our approach is competitive with Memory Networks, but with less supervision. For the latter, on the Penn TreeBank and Text8 datasets our approach demonstrates comparable performance to RNNs and LSTMs. In both cases we show that the key concept of multiple computational hops yields improved results.

Topics

language model question answering random access memory treebanks network architecture recurrent neural networks

Keywords

End-to-End Memory Networks recurrent neural networks attention mechanism external memory computational hops Penn Treebank language models sentence representation machine translation

1 Introduction
2 Approach
2.1 Single Layer
2.2 Multiple Layers
3 Related Work
4 Synthetic Question and Answering Experiments
4.1 Model Details
4.2 Training Details
4.3 Baselines
4.4 Results
5 Language Modeling Experiments
5.1 Training Details
5.2 Results
6 Conclusions and Future Work
References
A Results on 10k QA dataset
B Visualization of attention weights in QA problems

Knowls

Knowl 1 — Single-Layer Continuous Memory Network Architecture
model/method
The single-layer End-to-End Memory Network takes a discrete sequence of inputs $x_1, \dots, x_n$ (such as sentences or words) to store in memory and a query $q$ , where each item contains tokens from a dictionary of size $V$ , and outputs a discrete prediction $\hat{a}$ . The architecture consists of three core steps:
1. Input Memory Representation: Each input item $x_i$ is mapped to a continuous memory vector $m_i \in \mathbb{R}^d$ via an embedding matrix $A \in \mathbb{R}^{d \times V}$ . The query $q$ is mapped to an internal query state vector $u \in \mathbb{R}^d$ via an embedding matrix $B \in \mathbb{R}^{d \times V}$ . When items are represented as bags of words with one-hot word vectors $x_{ij}$ and $q_j$ , $m_i = \sum_j A x_{ij}$ and $u = \sum_j B q_j$ . The match between query state $u$ and memory $m_i$ is computed using an inner product followed by a softmax: $p_i = \text{Softmax}(u^T m_i) = \frac{\exp(u^T m_i)}{\sum_{j=1}^n \exp(u^T m_j)}$ where $p = [p_1, \dots, p_n]^T$ is an attention probability vector over the inputs.
2. Output Memory Representation: Each input $x_i$ is assigned a continuous output vector $c_i \in \mathbb{R}^d$ via an embedding matrix $C \in \mathbb{R}^{d \times V}$ (e.g., $c_i = \sum_j C x_{ij}$ ). The retrieved memory response vector $o \in \mathbb{R}^d$ is the attention-weighted sum: $o = \sum_{i=1}^n p_i c_i$
3. Final Prediction: The sum of the memory output $o$ and query vector $u$ is passed through a weight matrix $W \in \mathbb{R}^{V \times d}$ and a softmax function to generate the predicted token distribution: $\hat{a} = \text{Softmax}(W(o + u))$
All parameters ( $A, B, C, W$ ) are continuous and differentiable, allowing end-to-end training via standard cross-entropy loss and backpropagation.
Knowl 2 — Multi-Hop Memory Network Stacking
model/method
To perform $K$ sequential computational hops over memory, $K$ continuous memory layers are stacked vertically. For each hop $k \in \{1, \dots, K\}$ :
1. Query Update: The input state to the first layer is the embedded query $u^1 = B q \in \mathbb{R}^d$ . For subsequent layers $k+1$ , the query state is updated by adding the memory output $o^k \in \mathbb{R}^d$ to the input state $u^k \in \mathbb{R}^d$ : $u^{k+1} = u^k + o^k$
2. Layer-Wise Memory Access: At layer $k$ , input memory vectors $\{m_i^k\}_{i=1}^n$ and output memory vectors $\{c_i^k\}_{i=1}^n$ are obtained using layer-specific embedding matrices $A^k$ and $C^k$ . Attention probabilities and memory response vectors are computed as: $p_i^k = \text{Softmax}\left((u^k)^T m_i^k\right) = \frac{\exp\left((u^k)^T m_i^k\right)}{\sum_{j=1}^n \exp\left((u^k)^T m_j^k\right)}$ $o^k = \sum_{i=1}^n p_i^k c_i^k$
3. Final Prediction: At the top of the network, the output distribution over vocabulary size $V$ is computed from the combined top state: $\hat{a} = \text{Softmax}(W u^{K+1}) = \text{Softmax}(W(o^K + u^K))$ where $W \in \mathbb{R}^{V \times d}$ is the output classification matrix.
Because the operations at every layer are smooth, the error signal backpropagates through all $K$ memory hops directly to the input embeddings without needing intermediate hop supervision.
Knowl 3 — Weight Tying Schemes in Multi-Hop Memory Networks
model/method
To ease training and reduce parameter count across $K$ stacked memory layers in an End-to-End Memory Network, two weight tying strategies are used:
1. Adjacent Weight Tying: The output embedding matrix of layer $k$ is tied to the input embedding matrix of layer $k+1$ : $A^{k+1} = C^k \quad \text{for } k \in \{1, \dots, K-1\}$ In addition, the query embedding matrix is tied to the first layer input embedding matrix ( $B = A^1$ ), and the prediction matrix is tied to the top layer output embedding matrix ( $W^T = C^K$ ).
2. Layer-wise (RNN-like) Weight Tying: The input and output embedding matrices are shared across all hops: $A^1 = A^2 = \dots = A^K = A$ $C^1 = C^2 = \dots = C^K = C$ Under this scheme, a learnable linear transformation matrix $H \in \mathbb{R}^{d \times d}$ is added to the state update between hops: $u^{k+1} = H u^k + o^k$
Layer-wise weight tying casts the multi-hop memory network as a recurrent neural network where recurrence runs over computational memory hops (producing internal attention outputs) rather than over token steps in time.
Knowl 4 — Position Encoding for Sentence Representations
model/method
To preserve word order within sentences without recurrent processing, Position Encoding (PE) weights the embedding of each word according to its position in the sentence.

For a sentence $x_i = \{x_{i1}, x_{i2}, \dots, x_{iJ}\}$ consisting of $J$ words, where each word $x_{ij}$ is a one-hot column vector of vocabulary size $V$ , the input memory vector $m_i \in \mathbb{R}^d$ is computed as: $m_i = \sum_{j=1}^J l_j \odot (A x_{ij})$ where $\odot$ represents element-wise (Hadamard) multiplication, $A \in \mathbb{R}^{d \times V}$ is the word embedding matrix, and $l_j \in \mathbb{R}^d$ is a column vector whose $k$ -th element (for dimension index $k \in \{1, \dots, d\}$ and word position $j \in \{1, \dots, J\}$ ) is: $l_{kj} = \left(1 - \frac{j}{J}\right) - \left(\frac{k}{d}\right)\left(1 - \frac{2j}{J}\right)$

The same position encoding vector $l_j$ is used to construct output memory vectors $c_i = \sum_{j=1}^J l_j \odot (C x_{ij})$ and query vectors $u = \sum_{j=1}^J l_j \odot (B q_j)$ .
Knowl 5 — Temporal Memory Encoding and Random Noise Regularization
model/method
To capture the chronological order of sentences in narrative reasoning tasks, End-to-End Memory Networks incorporate learned temporal context vectors and time-jitter regularization:
1. Temporal Encoding: Sentences are indexed in reverse chronological order relative to the query such that $x_1$ is the sentence immediately preceding the query. The input representation $m_i$ and output representation $c_i$ for sentence $x_i$ are modified by adding learned temporal embedding row vectors $T_A(i) \in \mathbb{R}^d$ and $T_C(i) \in \mathbb{R}^d$ from temporal matrices $T_A$ and $T_C$ : $m_i = \sum_j A x_{ij} + T_A(i)$ $c_i = \sum_j C x_{ij} + T_C(i)$ $T_A$ and $T_C$ are learned during training and follow the same weight tying constraints as $A$ and $C$ .
2. Random Noise (RN) Regularization: To prevent overfitting to specific temporal slot indices and regularize $T_A$ , empty "dummy" memories are randomly injected into the story sequence at training time with a probability of 10%. This jitters the relative temporal distance of sentences and improves generalization on small training sets.
Knowl 6 — Linear Start Training Strategy for Memory Networks
model/method
End-to-End Memory Networks with multi-hop softmax attention can suffer from early gradient saturation and get trapped in local minima on deduction and induction tasks. The Linear Start (LS) strategy mitigates this through two-phase training:
1. Phase 1 (Linear Start): The softmax functions in each intermediate memory layer are removed, setting the attention weighting directly to dot products: $p_i^k = (u^k)^T m_i^k$ . The network remains linear throughout its memory hops, retaining only the final output softmax over vocabulary tokens. Training begins with an initial learning rate $\eta = 0.005$ .
2. Phase 2 (Nonlinear Resumption): When the validation loss plateaus in Phase 1, the softmax functions are re-inserted into all intermediate memory layers, and standard end-to-end training resumes until convergence.
Linear Start prevents premature attention focus during initial parameter updates, dramatically reducing error on tasks such as basic induction (reducing error on bAbI Task 16 from 52.1% to 1.6%).
Knowl 7 — End-to-End Memory Network Architecture for Language Modeling
model/method
The End-to-End Memory Network can be adapted for word-level language modeling to predict the next word token given a context buffer of the preceding $N$ words:
1. Memory Storage: Each of the previous $N$ words is stored in a separate memory slot. Since each memory slot contains a single word, position within the sentence is not needed, and memory vectors are formed by adding word embeddings to temporal position embeddings: $m_i = A x_i + T_A(i)$ and $c_i = C x_i + T_C(i)$ for $i \in \{1, \dots, N\}$ .
2. Fixed Query State: Since language modeling has no external question, the initial query state $u^1 \in \mathbb{R}^d$ is fixed to a constant vector with all components equal to $0.1$ ( $u^1 = \mathbf{0.1}$ ).
3. Recurrent Hops and Non-linearity: Layer-wise weight tying is used ( $A^1 = \dots = A^K = A$ and $C^1 = \dots = C^K = C$ ). To facilitate deep gradient propagation, ReLU activations are applied to half of the hidden units during state updates: $u^{k+1} = \text{ReLU}\left(H u^k + o^k\right)$ where $H \in \mathbb{R}^{d \times d}$ is a learned linear transformation.
4. Prediction: The probability distribution over vocabulary size $V$ for the next token is given by $\hat{a} = \text{Softmax}(W u^{K+1})$ .

Knowl 8 — bAbI Synthetic QA Benchmark Error Rates

data/table

The table compares test error rates (%) across the 20 bAbI synthetic question answering tasks for strongly supervised Memory Networks (MemNN), weakly supervised baselines (LSTM, MemNN-WSH), and End-to-End Memory Network (MemN2N) variants using 1k training examples per task, along with overall mean errors on 1k and 10k datasets.

	Baselines			MemN2N (1k training examples)
Task	MemNN (Supervised)	LSTM	MemNN WSH	BoW	PE	PE LS	PE LS RN	1 hop PE LS joint	2 hops PE LS joint	3 hops PE LS joint	PE LS RN joint	PE LS LW joint
1: 1 supporting fact	0.0	50.0	0.1	0.6	0.1	0.2	0.0	0.8	0.0	0.1	0.0	0.1
2: 2 supporting facts	0.0	80.0	42.8	17.6	21.6	12.8	8.3	62.0	15.6	14.0	11.4	18.8
3: 3 supporting facts	0.0	80.0	76.4	71.0	64.2	58.8	40.3	76.9	31.6	33.1	21.9	31.7
4: 2 argument relations	0.0	39.0	40.3	32.0	3.8	11.6	2.8	22.8	2.2	5.7	13.4	17.5
5: 3 argument relations	2.0	30.0	16.3	18.3	14.1	15.7	13.1	11.0	13.4	14.8	14.4	12.9
6: yes/no questions	0.0	52.0	51.0	8.7	7.9	8.7	7.6	7.2	2.3	3.3	2.8	2.0
7: counting	15.0	51.0	36.1	23.5	21.6	20.3	17.3	15.9	25.4	17.9	18.3	10.1
8: lists/sets	9.0	55.0	37.8	11.4	12.6	12.7	10.0	13.2	11.7	10.1	9.3	6.1
9: simple negation	0.0	36.0	35.9	21.1	23.3	17.0	13.2	5.1	2.0	3.1	1.9	1.5
10: indefinite knowledge	2.0	56.0	68.7	22.8	17.4	18.6	15.1	10.6	5.0	6.6	6.5	2.6
11: basic coreference	0.0	38.0	30.0	4.1	4.3	0.0	0.9	8.4	1.2	0.9	0.3	3.3
12: conjunction	0.0	26.0	10.1	0.3	0.3	0.1	0.2	0.4	0.0	0.3	0.1	0.0
13: compound coreference	0.0	6.0	19.7	10.5	9.9	0.3	0.4	6.3	0.2	1.4	0.2	0.5
14: time reasoning	1.0	73.0	18.3	1.3	1.8	2.0	1.7	36.9	8.1	8.2	6.9	2.0
15: basic deduction	0.0	79.0	64.8	24.3	0.0	0.0	0.0	46.4	0.5	0.0	0.0	1.8
16: basic induction	0.0	77.0	50.5	52.0	52.1	1.6	1.3	47.4	51.3	3.5	2.7	51.0
17: positional reasoning	35.0	49.0	50.9	45.4	50.1	49.0	51.0	44.4	41.2	44.5	40.4	42.6
18: size reasoning	5.0	48.0	51.3	48.1	13.6	10.1	11.1	9.6	10.3	9.2	9.4	9.2
19: path finding	64.0	92.0	100.0	89.7	87.4	85.6	82.8	90.7	89.9	90.2	88.0	90.6
20: agent's motivation	0.0	9.0	3.6	0.1	0.0	0.0	0.0	0.0	0.1	0.0	0.0	0.2
Mean error (%) (1k)	6.7	51.3	40.2	25.1	20.3	16.3	13.9	25.8	15.6	13.3	12.4	15.2
Failed tasks (>5% err, 1k)	4	20	18	15	13	12	11	17	11	11	11	10
Mean error (%) (10k)	3.2	36.4	39.2	15.4	9.4	7.2	6.6	24.5	10.9	7.9	7.5	11.0
Failed tasks (>5% err, 10k)	2	16	17	9	6	4	4	16	7	6	6	6

Key results:

MemN2N trained weakly without supporting fact labels achieves 12.4% mean error on 1k problems, approaching strongly supervised MemNN (6.7%) and outperforming weak baselines (LSTM: 51.3%, MemNN-WSH: 40.2%).
Increasing memory hops from 1 hop (25.8%) to 3 hops (13.3%) reduces error significantly.
On the 10k training set with non-linearity and $d=100$ , MemN2N achieves 4.2% mean error with only 3 failed tasks.

Knowl 9 — Language Modeling Perplexity Comparison

data/table

Word-level language modeling performance evaluated on the Penn Treebank (10k vocabulary) and Text8 (44k vocabulary) corpora demonstrates the effect of scaling memory hops and memory buffer size.

	Penn Treebank				Text8
Model	# hidden	# hops ( $K$ )	Memory ( $N$ )	Test perp.	# hidden	# hops ( $K$ )	Memory ( $N$ )	Test perp.
RNN	300	-	-	129	500	-	-	184
LSTM	100	-	-	115	500	-	-	154
SCRN	100	-	-	115	500	-	-	161
MemN2N	150	2	100	121	500	2	100	187
MemN2N	150	3	100	122	500	3	100	178
MemN2N	150	4	100	120	500	4	100	162
MemN2N	150	5	100	118	500	5	100	154
MemN2N	150	6	100	115	500	6	100	155
MemN2N	150	7	100	114	500	7	100	147
MemN2N	150	6	25	118	500	6	25	163
MemN2N	150	6	50	114	500	6	50	166
MemN2N	150	6	75	114	500	6	75	158
MemN2N	150	6	100	115	500	6	100	155
MemN2N	150	6	125	112	500	6	125	157
MemN2N	150	6	150	114	500	6	150	154
MemN2N	150	7	200	111	-	-	-	-

Key observations:

Adding memory hops consistently improves perplexity: increasing hops from 2 to 7 reduces test perplexity from 121 to 114 on Penn Treebank and from 187 to 147 on Text8 (with memory size 100).
MemN2N outperforms RNN, LSTM, and Structurally Constrained Recurrent Net (SCRN) baselines, reaching 111 perplexity on Penn Treebank (with 7 hops, memory size 200) and 147 on Text8.
MemN2N achieves these results using approximately $1.5\times$ the parameters of an RNN with identical hidden units, compared to LSTM which uses approximately $4\times$ the parameters.

Knowl 10 — Alternating Local n-gram and Global Cache Attention in Memory Networks
empirical result
Analysis of average attention weights $p_i^k$ across memory positions during 6-hop language modeling shows structured functional differentiation across hops:
1. Functional Division: Certain hops place sharp attention exclusively on recent words (small memory offsets), functioning as a smoothed local $n$ -gram model. Other hops distribute attention broadly across the entire memory window, functioning as an un-decayed context cache.
2. Alternating Hops: The model tends to alternate between local attention hops and broad cache lookup hops throughout the multi-hop computation.
3. Uniform Cache Retention: Unlike standard RNN recurrent states that experience exponential decay over time, the cache attention in End-to-End Memory Networks maintains approximately uniform average activation across all memory positions regardless of temporal distance.
Knowl 11 — Computational Scalability Limitations of Continuous Memory Lookups
limitation
While continuous soft attention enables end-to-end training via gradient descent, it imposes scalability and optimization constraints:
1. Linear Lookup Cost: Computing inner-product attention distributions and weighted sums across all memory vectors scales linearly ( $O(N)$ ) with memory buffer size $N$ , making full-memory attention computationally expensive for extremely large document sets or knowledge bases.
2. Supervision Gap on Complex Deduction: Without supporting-fact supervision at each hop, continuous memory networks on small training sets (1k examples) can fail to converge on complex relational tasks (such as path finding and positional reasoning) compared to strongly supervised discrete memory models.

Coverage note — No substantial contributed material was omitted; all core architectural components, training techniques, experimental results across QA and language modeling tasks, and stated limitations are covered.

References

1.C. G. Atkeson and S. Schaal. Memory-based neural networks for robot learning. Neurocomputing, 9:243–269, 1995.
2.D. Bahdanau, K. Cho, and Y. Bengio. Neural machine translation by jointly learning to align and translate. In International Conference on Learning Representations (ICLR), 2015.
3.Y. Bengio, R. Ducharme, P. Vincent, and C. Janvin. A neural probabilistic language model. J. Mach. Learn. Res., 3:1137–1155, Mar. 2003.
4.J. Chung, C¸ . Gulc¸ehre, K. Cho, and Y. Bengio. Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint: 1412.3555, 2014.
5.S. Das, C. L. Giles, and G.-Z. Sun. Learning context-free grammars: Capabilities and limitations of a recurrent neural network with an external stack memory. In In Proceedings of The Fourteenth Annual Conference of Cognitive Science Society, 1992.
6.J. Goodman. A bit of progress in language modeling. CoRR, cs.CL/0108005, 2001.
7.A. Graves. Generating sequences with recurrent neural networks. arXiv preprint: 1308.0850, 2013.
8.A. Graves, G. Wayne, and I. Danihelka. Neural turing machines. arXiv preprint: 1410.5401, 2014.
9.K. Gregor, I. Danihelka, A. Graves, and D. Wierstra. DRAW: A recurrent neural network for image generation. CoRR, abs/1502.04623, 2015.
10.S. Hochreiter and J. Schmidhuber. Long short-term memory. Neural computation, 9(8):1735–1780, 1997.
11.A. Joulin and T. Mikolov. Inferring algorithmic patterns with stack-augmented recurrent nets. NIPS, 2015.
12.J. Koutn´ık, K. Greff, F. J. Gomez, and J. Schmidhuber. A clockwork RNN. In ICML, 2014.
13.M. P. Marcus, M. A. Marcinkiewicz, and B. Santorini. Building a large annotated corpus of english: The Penn Treebank. Comput. Linguist., 19(2):313–330, June 1993.
14.T. Mikolov. Statistical language models based on neural networks. Ph. D. thesis, Brno University of Technology, 2012.
15.T. Mikolov, A. Joulin, S. Chopra, M. Mathieu, and M. Ranzato. Learning longer memory in recurrent neural networks. arXiv preprint: 1412.7753, 2014.
16.M. C. Mozer and S. Das. A connectionist symbol manipulator that discovers the structure of context-free languages. NIPS, pages 863–863, 1993.
17.B. Peng, Z. Lu, H. Li, and K. Wong. Towards Neural Network-based Reasoning. ArXiv preprint: 1508.05508, 2015.
18.J. Pollack. The induction of dynamical recognizers. Machine Learning, 7(2-3):227–252, 1991.
19.K. Steinbuch and U. Piske. Learning matrices and their applications. IEEE Transactions on Electronic Computers, 12:846–862, 1963.
20.M. Sundermeyer, R. Schluter, and H. Ney. LSTM neural networks for language modeling. In ¨ Interspeech, pages 194–197, 2012.
21.W. K. Taylor. Pattern recognition by means of automatic analogue apparatus. Proceedings of The Institution of Electrical Engineers, 106:198–209, 1959.
22.J. Weston, A. Bordes, S. Chopra, and T. Mikolov. Towards AI-complete question answering: A set of prerequisite toy tasks. arXiv preprint: 1502.05698, 2015.
23.J. Weston, S. Chopra, and A. Bordes. Memory networks. In International Conference on Learning Representations (ICLR), 2015.
24.K. Xu, J. Ba, R. Kiros, K. Cho, A. Courville, R. Salakhutdinov, R. Zemel, and Y. Bengio. Show, Attend and Tell: Neural Image Caption Generation with Visual Attention. ArXiv preprint: 1502.03044, 2015.
25.W. Zaremba, I. Sutskever, and O. Vinyals. Recurrent neural network regularization. arXiv preprint arXiv:1409.2329, 2014.

Citation

MLA

Sukhbaatar, S., et al. “End-To-End Memory Networks”. arXiv, 2015, https://doi.org/10.48550/arxiv.1503.08895.

APA

Sukhbaatar, S., Szlam, A., Weston, J., & Fergus, R. (2015). End-To-End Memory Networks. arXiv. https://doi.org/10.48550/arxiv.1503.08895

Chicago

Sukhbaatar, S., A. Szlam, J. Weston, and R. Fergus. 2015. “End-To-End Memory Networks”. Preprint, ArXiv. https://doi.org/10.48550/arxiv.1503.08895.

Harvard

Sukhbaatar, S. et al. (2015) “End-To-End Memory Networks”. arXiv. Available at: https://doi.org/10.48550/arxiv.1503.08895.

Vancouver

1. Sukhbaatar S, Szlam A, Weston J, Fergus R (2015) End-To-End Memory Networks. https://doi.org/10.48550/arxiv.1503.08895

BibTeX

@misc{https://doi.org/10.48550/arxiv.1503.08895,
  doi = {10.48550/ARXIV.1503.08895},
  url = {https://arxiv.org/abs/1503.08895},
  author = {Sukhbaatar, Sainbayar and Szlam, Arthur and Weston, Jason and Fergus, Rob},
  keywords = {Neural and Evolutionary Computing (cs.NE), Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
  title = {End-To-End Memory Networks},
  publisher = {arXiv},
  year = {2015},
  copyright = {arXiv.org perpetual, non-exclusive license}
}

Metadata:DOI registry

Source Code

This paper has an official code repository available. Click below to access the source code.

View Repository

Access the Paper

This paper is available from its original source. Click below to access the PDF.

Open PDF

License: Authors

End-To-End Memory Networks

Abstract

Topics

Keywords

Table of Contents

Knowls

Knowl 1 — Single-Layer Continuous Memory Network Architecture

Knowl 2 — Multi-Hop Memory Network Stacking

Knowl 3 — Weight Tying Schemes in Multi-Hop Memory Networks

Knowl 4 — Position Encoding for Sentence Representations

Knowl 5 — Temporal Memory Encoding and Random Noise Regularization

Knowl 6 — Linear Start Training Strategy for Memory Networks

Knowl 7 — End-to-End Memory Network Architecture for Language Modeling