keyword
meta-learning methods
Meta-learning methods, commonly described as learning to learn techniques, are machine learning algorithms designed to train models across a diverse set of tasks so they can rapidly generalize and adapt to new, unseen tasks using only a small amount of data. Unlike traditional machine learning approaches that optimize a model for a single dedicated task, meta-learning methods focus on optimizing the learning process itself by extracting shared representations, optimal initial parameter initializations, or adaptive optimization rules from multiple previous learning experiences. These methods encompass optimization-based algorithms that enable fast task adaptation through few gradient updates, metric-based architectures that learn generalizable similarity metrics for pattern matching, and model-based or in-context learning frameworks that process examples sequentially to infer task context. As a result, meta-learning methods are widely utilized to address few-shot classification and regression problems, automate hyperparameter optimization and architecture search, correct for noisy training signals, and facilitate rapid personalization in distributed environments.
3 items
Noisy Correspondence Learning with Meta Similarity Correction
Haochen Han, Kaiyao Miao, Qinghua Zheng, Minnan Luo
Why you should read this
Proposes a meta-learning framework that trains a correction network on clean and mismatched meta-data to rectify similarity scores and filter out mismatched cross-modal pairs during retrieval training.
Despite the success of multimodal learning in cross-modal retrieval task, the remarkable progress relies on the correct correspondence among multimedia data. However, collecting such ideal data is expensive and time-consuming. In practice, most widely used datasets are harvested from the Internet and inevitably contain mismatched pairs. Training on such noisy correspondence datasets causes performance degradation because the cross-modal retrieval methods can wrongly enforce the mismatched data to be similar. To tackle this problem, we propose a Meta Similarity Correction Network (MSCN) to provide reliable similarity scores. We view a binary classification task as the meta-process that encourages the MSCN to learn discrimination from positive and negative meta-data. To further alleviate the influence of noise, we design an effective data purification strategy using meta-data as prior knowledge to remove the noisy samples. Extensive experiments are conducted to demonstrate the strengths of our method in both synthetic and real-world noises, including Flickr30K, MS-COCO, and Conceptual Captions. Our code is publicly available.
Added
2026-09-26
Meta-learning via Language Model In-context Tuning
Yanda Chen, Ruiqi Zhong, Sheng Zha, George Karypis, He He
Why you should read this
Proposes in-context tuning, a meta-learning method that fine-tunes language models directly on concatenated task prompts and demonstration examples to outperform gradient-based meta-learning while substantially decreasing sensitivity to example selection and ordering.
The goal of meta-learning is to learn to adapt to a new task with only a few labeled examples. Inspired by the recent progress in large language models, we propose in-context tuning (ICT), which recasts task adaptation and prediction as a simple sequence prediction problem: to form the input sequence, we concatenate the task instruction, labeled in-context examples, and the target input to predict; to meta-train the model to learn from in-context examples, we fine-tune a pre-trained language model (LM) to predict the target label given the input sequence on a collection of tasks. We benchmark our method on two collections of text classification tasks: LAMA and BinaryClfs. Compared to MAML which adapts the model through gradient descent, our method leverages the inductive bias of pre-trained LMs to perform pattern matching, and outperforms MAML by an absolute 6% average AUC-ROC score on BinaryClfs, gaining more advantage with increasing model size. Compared to non-fine-tuned in-context learning (i.e. prompting a raw LM), in-context tuning meta-trains the model to learn from in-context examples. On BinaryClfs, ICT improves the average AUC-ROC score by an absolute 10%, and reduces the variance due to example ordering by 6x and example choices by 2x.
Added
2026-09-26
LEAF: A Benchmark for Federated Settings
Sebastian Caldas, Peter Wu, Tian Li, Jakub Konecný, H. B. McMahan, Virginia Smith, Ameet Talwalkar
Why you should read this
Proposes LEAF, an open-source benchmarking framework providing realistic datasets, standardized evaluation metrics, and reference implementations to enable reproducible research across federated, multi-task, and meta-learning environments.
Modern federated networks, such as those comprised of wearable devices, mobile phones, or autonomous vehicles, generate massive amounts of data each day. This wealth of data can help to learn models that can improve the user experience on each device. However, the scale and heterogeneity of federated data presents new challenges in research areas such as federated learning, meta-learning, and multi-task learning. As the machine learning community begins to tackle these challenges, we are at a critical time to ensure that developments made in these areas are grounded with realistic benchmarks. To this end, we propose LEAF, a modular benchmarking framework for learning in federated settings. LEAF includes a suite of open-source federated datasets, a rigorous evaluation framework, and a set of reference implementations, all geared towards capturing the obstacles and intricacies of practical federated environments.
Added
2026-09-18