Back

Topics

meta-learningrandom access memorynetwork architecturerecurrent neural networks

Keywords

MemoryMeta-LearningNeural NetworksMemory-Augmented Neural NetworksLearningClassesEpisodeNetworkClassi CationMANN

Table of Contents

  • 1. Introduction
  • 2. Meta-Learning Task Methodology
  • 3. Memory-Augmented Model
  • 3.1. Neural Turing Machines
  • 3.2. Least Recently Used Access
  • 4. Experimental Results
  • 4.1. Data
  • 4.2. Omniglot Classification
  • 4.2.1. PERSISTENT MEMORY INTERFERENCE
  • 4.2.2. CURRICULUM TRAINING
  • 4.3. Regression
  • 5. Discussion & Future Work
  • 6. Acknowledgements
  • References

Knowls

  1. Knowl 1 — Episodic Meta-Learning Formulation with Time-Offset Labels

    model/method

    In the episodic meta-learning framework, model parameters θ\theta are trained to minimize expected loss across a distribution of datasets p(D)p(\mathcal{D}):

    θ∗=arg⁡min⁡θED∼p(D)[L(D;θ)]\theta^* = \arg\min_\theta \mathbb{E}_{\mathcal{D} \sim p(\mathcal{D})}[\mathcal{L}(\mathcal{D}; \theta)]

    Each episode presents a dataset D={(xt,yt)}t=1T\mathcal{D} = \{(x_t, y_t)\}_{t=1}^T, where xtx_t is an input sample (such as an image or vector) and yty_t is its ground-truth target (such as a class label or continuous function value).

    To prevent the model from learning static sample-to-class mappings directly in its weights, inputs and labels are presented in a temporally offset sequence:

    (x1,null),(x2,y1),(x3,y2),…,(xT,yT−1)(x_1, \text{null}), (x_2, y_1), (x_3, y_2), \dots, (x_T, y_{T-1})

    At time step tt, the network receives the current input xtx_t together with the previous target yt−1y_{t-1}, and is evaluated on predicting the conditional distribution p(yt∣xt,D1:t−1;θ)p(y_t \mid x_t, \mathcal{D}_{1:t-1}; \theta). Class labels are randomly shuffled across episodes so that correct identification on the first instance of a class requires guessing, but subsequent instances within the episode can achieve high accuracy if the model rapidly binds representations (xt,yt)(x_t, y_t) in memory.

  2. Knowl 2 — Least Recently Used Access Memory Writing Mechanism

    algorithm

    The Least Recently Used Access (LRUA) module is a content-based memory writing mechanism designed for meta-learning. It writes new information to either the least recently used memory locations or the most recently used memory locations, avoiding sequence-dependent location addressing.

    Memory usage is tracked by a usage weight vector wtu∈RNw_t^u \in \mathbb{R}^N across NN memory slots, decayed by parameter γ∈[0,1]\gamma \in [0, 1] and incremented by read weights wtrw_t^r and write weights wtww_t^w:

    wtu=γwt−1u+wtr+wtww_t^u = \gamma w_{t-1}^u + w_t^r + w_t^w

    The least-used slots are identified by a binary indicator vector wtlu∈{0,1}Nw_t^{lu} \in \{0, 1\}^N:

    wtlu(i)={0if wtu(i)>m(wtu,n)1if wtu(i)≤m(wtu,n)w_t^{lu}(i) = \begin{cases} 0 & \text{if } w_t^u(i) > m(w_t^u, n) \\ 1 & \text{if } w_t^u(i) \le m(w_t^u, n) \end{cases}

    where m(wtu,n)m(w_t^u, n) represents the nn-th smallest element of wtuw_t^u, and nn equals the number of read heads.

    Write weights wtww_t^w are computed as a convex combination of previous read weights wt−1rw_{t-1}^r and previous least-used weights wt−1luw_{t-1}^{lu} gated by a learnable scalar α\alpha:

    wtw=σ(α)wt−1r+(1−σ(α))wt−1luw_t^w = \sigma(\alpha) w_{t-1}^r + (1 - \sigma(\alpha)) w_{t-1}^{lu}

    where σ(α)=11+e−α\sigma(\alpha) = \frac{1}{1 + e^{-\alpha}}.

    Input: Prior memory matrix Mt−1M_{t-1}, key vector ktk_t, previous read weights wt−1rw_{t-1}^r, previous usage weights wt−1uw_{t-1}^u, gate parameter α\alpha, decay parameter γ\gamma, read count nn
    Output: Updated memory matrix MtM_t, write weights wtww_t^w, usage weights wtuw_t^u
    Compute least-used indicator wt−1luw_{t-1}^{lu} from wt−1uw_{t-1}^u using threshold m(wt−1u,n)m(w_{t-1}^u, n)
    Compute write weights wtw=σ(α)wt−1r+(1−σ(α))wt−1luw_t^w = \sigma(\alpha) w_{t-1}^r + (1 - \sigma(\alpha)) w_{t-1}^{lu}
    Zero out rows ii of Mt−1M_{t-1} where wt−1lu(i)==1w_{t-1}^{lu}(i) == 1
    for each memory row ii from 11 to NN do
        Mt(i)=Mt−1(i)+wtw(i)ktM_t(i) = M_{t-1}(i) + w_t^w(i) k_t
    end for
    Compute read weights wtrw_t^r from cosine similarity of ktk_t with rows of MtM_t
    Update usage weights wtu=γwt−1u+wtr+wtww_t^u = \gamma w_{t-1}^u + w_t^r + w_t^w
    return MtM_t, wtww_t^w, wtuw_t^u

  3. Knowl 3 — Content-Based Memory Retrieval in Memory-Augmented Neural Networks

    model/method

    In a Memory-Augmented Neural Network (MANN), an external memory matrix Mt∈RN×MM_t \in \mathbb{R}^{N \times M} contains NN row vectors of dimension MM at time step tt. A controller network (e.g., an LSTM or feedforward network) processes the current input and emits a query key kt∈RMk_t \in \mathbb{R}^M.

    Addressing is performed purely by cosine similarity between the key ktk_t and each memory row Mt(i)M_t(i):

    K(kt,Mt(i))=kt⋅Mt(i)∥kt∥∥Mt(i)∥K(k_t, M_t(i)) = \frac{k_t \cdot M_t(i)}{\|k_t\| \|M_t(i)\|}

    A softmax over all NN memory rows yields normalized read weights wtr∈RNw_t^r \in \mathbb{R}^N:

    wtr(i)=exp⁡(K(kt,Mt(i)))∑j=1Nexp⁡(K(kt,Mt(j)))w_t^r(i) = \frac{\exp(K(k_t, M_t(i)))}{\sum_{j=1}^N \exp(K(k_t, M_t(j)))}

    The retrieved memory vector rt∈RMr_t \in \mathbb{R}^M is formed as the linear combination:

    rt=∑i=1Nwtr(i)Mt(i)r_t = \sum_{i=1}^N w_t^r(i) M_t(i)

    Vector rtr_t is supplied as input to the classification layer and forwarded as an additional input to the controller at the next step.

  4. Knowl 4 — Test-Set Accuracies on 5-Class Omniglot Classification Across Successive Instance Presentations

    data/table

    In 5-class Omniglot classification episodes using one-hot encoded labels, models and human participants are evaluated on accuracy as the number of instances presented per class increases from the 1st to the 10th presentation within an episode.

    Model 1st (%) 2nd (%) 3rd (%) 4th (%) 5th (%) 10th (%)
    Human 34.5 57.3 70.1 71.8 81.4 92.4
    Feedforward 24.4 19.6 21.1 19.9 22.8 19.5
    LSTM 24.4 49.5 55.3 61.0 63.6 62.5
    MANN 36.4 82.8 91.0 92.6 94.9 98.1

    The MANN equipped with Least Recently Used Access outperforms humans and non-augmented baselines at every instance. On the 2nd instance (one-shot classification), MANN achieves 82.8%82.8\%, exceeding human performance (57.3%57.3\%) and standard LSTM (49.5%49.5\%). On the 1st instance, MANN scores 36.4%36.4\% (above the 20%20\% chance level), reflecting an educated guessing strategy where it discounts labels already bound in memory to other classes.

  5. Knowl 5 — Test-Set Accuracies on String-Labeled Omniglot Classification Comparing Memory Mechanisms

    data/table

    Models trained for 100,000 episodes on Omniglot with 5-character string labels (55=31255^5 = 3125 possible labels) were evaluated on test episodes containing either 5 or 15 distinct classes across 10 instance presentations per class.

    Model Controller # Classes 1st (%) 2nd (%) 3rd (%) 4th (%) 5th (%) 10th (%)
    kNN (raw pixels) – 5 4.0 36.7 41.9 45.7 48.1 57.0
    kNN (deep features) – 5 4.0 51.9 61.0 66.3 69.3 77.5
    Feedforward – 5 0.0 0.2 0.0 0.2 0.0 0.0
    LSTM – 5 0.0 9.0 14.2 16.9 21.8 25.5
    MANN Feedforward 5 0.0 8.0 16.2 25.2 30.9 46.8
    MANN LSTM 5 0.0 69.5 80.4 87.9 88.4 93.1
    kNN (raw pixels) – 15 0.5 18.7 23.3 26.5 29.1 37.0
    kNN (deep features) – 15 0.4 32.7 41.2 47.1 50.6 60.0
    Feedforward – 15 0.0 0.1 0.0 0.0 0.0 0.0
    LSTM – 15 0.0 2.2 2.9 4.3 5.6 12.7
    MANN (LRUA) Feedforward 15 0.1 12.8 22.3 28.8 32.2 43.4
    MANN (LRUA) LSTM 15 0.1 62.6 79.3 86.6 88.7 95.3
    MANN (NTM) LSTM 15 0.0 35.4 61.2 71.7 77.7 88.4

    The MANN with an LSTM controller and Least Recently Used Access (LRUA) achieves 62.6%62.6\% on the 2nd instance and 95.3%95.3\% on the 10th instance in the 15-class setting. It significantly outperforms the original Neural Turing Machine (NTM) addressing mechanism (35.4%35.4\% on the 2nd instance and 88.4%88.4\% on the 10th instance), demonstrating the efficacy of pure content-based LRUA for episodic binding over location-based sequence addressing.

  6. Knowl 6 — Omniglot Meta-Learning Dataset Setup and Combinatorial Labeling Scheme

    experimental setup

    The Omniglot dataset contains over 1600 handwritten character classes with 20 examples per class, downscaled to 20×2020 \times 20 pixels. Data augmentation includes random translations, random continuous rotations, and discrete rotations (90∘,180∘,270∘90^\circ, 180^\circ, 270^\circ). 1200 original classes (plus augmentations) form the training set, while the remaining 423 classes (plus augmentations) form the test set.

    Two labeling setups are used:

    1. One-Hot Vectors: Classes are assigned 5-dimensional one-hot vectors for 5-class episodes.
    2. Combinatorial Character Strings: For episodes scaling to 15 or more classes, labels are formed as length-5 strings from the character set {′a′,′b′,′c′,′d′,′e′}\{'a', 'b', 'c', 'd', 'e'\}, encoded as concatenated one-hot vectors of length 25 (5×55 \times 5). This produces 55=31255^5 = 3125 possible labels, preventing label reuse across episodes and forcing memory-based binding without expanding standard output layer dimensions.

  7. Knowl 7 — Effects of Persistent External Memory and Proactive Interference on MANN Learning

    empirical result

    When the external memory matrix of a MANN is not cleared between consecutive episodes, residual information from previous episodes creates proactive interference.

    In 5-class Omniglot classification with persistent memory, learning progresses substantially slower than in the clean-memory condition, and the network fails to develop the rapid spike in accuracy seen when memory is reset between episodes. However, when episode length is extended (e.g., 10 unique classes over episodes of length 75), the network learns to manage slot overwriting and reaches classification accuracies comparable to the memory-wiped baseline.

  8. Knowl 8 — Curriculum-Based Class Capacity Scaling for Memory-Augmented Neural Networks

    empirical result

    A curriculum learning strategy scales MANN classification capacity across large numbers of classes. Training begins with episodes of 15 classes, and every 10,000 training episodes the maximum number of classes is increased by 1 (with episode lengths set to 10 times the maximum class count), reaching 25 classes at 100,000 episodes.

    When evaluated without additional weight updates on test episodes containing up to 50 and 100 classes, the model maintains high accuracy, exhibiting gradual performance decay as the number of classes approaches 100.

  9. Knowl 9 — Few-Shot Function Regression and Proximal Interpolation with Memory-Augmented Neural Networks

    empirical result

    MANNs applied to regression meta-learn to estimate continuous functions from small sample sets. In each episode, target functions are sampled from a Gaussian Process (GP) prior with fixed hyperparameters. The network receives input sequences (xt,f(xt−1))(x_t, f(x_{t-1})) with inputs xt∈Rdx_t \in \mathbb{R}^d for d∈{1,2,3}d \in \{1, 2, 3\}.

    Because input xx-values are sampled continuously and presented only once per episode, exact retrieval is impossible; the model must interpolate by performing content-based reads over proximal stored points. The MANN's negative log-likelihood predictions closely track the true Gaussian Process predictions (which have direct access to ground-truth hyperparameters), and predictive variance appropriately increases for query inputs distant from previously observed data.

Coverage note — None was omitted; all primary architectural components, algorithmic mechanics, and empirical results across classification and regression are captured.

References

  1. 1.Braun, Daniel A, Aertsen, Ad, Wolpert, Daniel M, and Mehring, Carsten. Motor task variation induces structural learning. Current Biology, 19(4):352–357, 2009.
  2. 2.Brazdil, Pavel B, Soares, Carlos, and Da Costa, Joaquim Pinto. Ranking learning algorithms: Using ibl and meta-learning on accuracy and time results. Machine Learning, 50(3):251–277, 2003.
  3. 3.Caruana, Rich. Multitask learning. Machine learning, 28 (1):41–75, 1997.
  4. 4.Cowan, Nelson. The magical mystery four how is working memory capacity limited, and why? Current Directions in Psychological Science, 19(1):51–57, 2010.
  5. 5.Giraud-Carrier, Christophe, Vilalta, Ricardo, and Brazdil, Pavel. Introduction to the special issue on meta-learning. Machine learning, 54(3):187–193, 2004.
  6. 6.Graves, Alex, Wayne, Greg, and Danihelka, Ivo. Neural turing machines. arXiv preprint arXiv:1410.5401, 2014.
  7. 7.He, Kaiming, Zhang, Xiangyu, Ren, Shaoqing, and Sun, Jian. Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. arXiv preprint arXiv:1502.01852, 2015.
  8. 8.Hochreiter, Sepp and Schmidhuber, Jürgen. Long short-term memory. Neural computation, 9(8):1735–1780, 1997.
  9. 9.Hochreiter, Sepp, Younger, A Steven, and Conwell, Peter R. Learning to learn using gradient descent. In Artificial Neural NetworksICANN 2001, pp. 87–94. Springer, 2001.
  10. 10.Jankowski, Norbert, Duch, Włodzisław, and Grabczewski, Krzysztof. Meta-learning in computational intelligence, volume 358. Springer Science & Business Media, 2011.
  11. 11.Lake, Brenden M, Salakhutdinov, Ruslan, and Tenenbaum, Joshua B. Human-level concept learning through probabilistic program induction. Science, 350(6266):1332–1338, 2015.
  12. 12.Mnih, Volodymyr, Kavukcuoglu, Koray, Silver, David, Rusu, Andrei A, Veness, Joel, Bellemare, Marc G, Graves, Alex, Riedmiller, Martin, Fidjeland, Andreas K, Ostrovski, Georg, et al. Human-level control through deep reinforcement learning. Nature, 518(7540):529–533, 2015.
  13. 13.Rendell, Larry A, Sheshu, Raj, and Tcheng, David K. Layered concept-learning and dynamically variable bias management. In IJCAI, pp. 308–314. Citeseer, 1987.
  14. 14.Schmidhuber, Jürgen, Zhao, Jieyu, and Wiering, Marco. Shifting inductive bias with success-story algorithm, adaptive levin search, and incremental self-improvement. Machine Learning, 28(1):105–130, 1997.
  15. 15.Schweighofer, Nicolas and Doya, Kenji. Meta-learning in reinforcement learning. Neural Networks, 16(1):5–9, 2003.
  16. 16.Silver, David, Huang, Aja, Maddison, Chris J, Guez, Arthur, Sifre, Laurent, van den Driessche, George, Schrittwieser, Julian, Antonoglou, Ioannis, Panneershelvam, Veda, Lanctot, Marc, et al. Mastering the game of go with deep neural networks and tree search. Nature, 529(7587):484–489, 2016.
  17. 17.Thrun, Sebastian. Lifelong learning algorithms. In Learning to learn, pp. 181–209. Springer, 1998.
  18. 18.Underwood, Benton J. Interference and forgetting. Psychological review, 64(1):49, 1957.
  19. 19.Vilalta, Ricardo and Drissi, Youssef. A perspective view and survey of meta-learning. Artificial Intelligence Review, 18(2):77–95, 2002.
  20. 20.Weston, Jason, Chopra, Sumit, and Bordes, Antoine. Memory networks. arXiv preprint arXiv:1410.3916, 2014.
  21. 21.Yu, Dong and Deng, Li. Automatic Speech Recognition. Springer, 2012.

Citation

MLA
Santoro, A., et al. “Meta-learning with Memory-augmented Neural Networks”. International Conference on Machine Learning, 2016, pp. 1842–50, http://proceedings.mlr.press/v48/santoro16.pdf.
APA
Santoro, A., Bartunov, S., Botvinick, M., Wierstra, D., & Lillicrap, T. (2016). Meta-learning with memory-augmented neural networks. International Conference on Machine Learning, 1842–1850. http://proceedings.mlr.press/v48/santoro16.pdf
Chicago
Santoro, A., S. Bartunov, M. Botvinick, D. Wierstra, and T. Lillicrap. 2016. “Meta-learning with Memory-augmented Neural Networks”. International Conference on Machine Learning, 1842–50. http://proceedings.mlr.press/v48/santoro16.pdf.
Harvard
Santoro, A. et al. (2016) “Meta-learning with memory-augmented neural networks”, International Conference on Machine Learning, pp. 1842–1850. Available at: http://proceedings.mlr.press/v48/santoro16.pdf.
Vancouver
1. Santoro A, Bartunov S, Botvinick M, Wierstra D, Lillicrap T (2016) Meta-learning with memory-augmented neural networks. International Conference on Machine Learning 1842–1850

BibTeX

@article{santoro2016meta,
  title = {Meta-learning with memory-augmented neural networks},
  author = {Santoro, Adam and Bartunov, Sergey and Botvinick, Matthew and Wierstra, Daan and Lillicrap, Timothy},
  year = {2016},
  journal = {International Conference on Machine Learning},
  pages = {1842-1850},
  url = {http://proceedings.mlr.press/v48/santoro16.pdf}
}
Metadata:DOI registry

Access the Paper

This paper is available from its original source. Click below to access the PDF.

Open PDF

License: Published with permission