keyword
language similarity
Language similarity is the degree of linguistic resemblance between two or more natural languages across structural, lexical, syntactic, morphological, and phonological dimensions. It encompasses both genealogical relatedness, such as shared ancestry within a language family, and typological alignment, including common word orders, grammatical features, and vocabulary overlap. In computational linguistics and natural language processing, language similarity is typically quantified using typological databases, phylogenetic trees, or token-level lexical overlap. These measurements serve as primary indicators for evaluating and predicting the effectiveness of cross-lingual transfer, zero-shot generalization, and multilingual model performance across diverse source and target languages.
2 items
Litmus (Re)Agent: A Benchmark and Agentic System for Predictive Evaluation of Multilingual Models
Avni Mittal, Shanu Kumar, Sandipan Dandapat, Monojit Choudhury
Why you should read this
Introduces a 1,500-question benchmark and a DAG-orchestrated agentic system to accurately predict multilingual model performance across target languages and tasks when direct evaluation data is missing from published literature.
We study predictive multilingual evaluation: estimating how well a model will perform on a task in a target language when direct benchmark results are missing. This problem is common in multilingual deployment, where evaluation coverage is sparse and published evidence is uneven across languages, tasks, and model families. We introduce a controlled benchmark of 1,500 questions spanning six tasks and five evidence scenarios. The benchmark separates accessible evidence from ground truth, enabling evaluation of systems that must infer missing results from incomplete literature evidence. We also present Litmus (Re)Agent, a DAG-orchestrated agentic system that decomposes queries into hypotheses, retrieves evidence, and synthesises predictions through feature-aware aggregation. Across six systems, Litmus (Re)Agent achieves the best overall performance, with the largest gains in transfer-heavy scenarios where direct evidence is weak or absent. These results show that structured agentic reasoning is a promising approach to multilingual performance estimation under incomplete evidence.
Added
2026-09-30
Towards a Common Understanding of Contributing Factors for Cross-Lingual Transfer in Multilingual Language Models: A Review
Fred Philippy, Siwen Guo, Shohreh Haddadan
Why you should read this
Categorizes and synthesizes empirical findings across five key drivers of cross-lingual transfer in multilingual language models to reconcile conflicting literature and guide more effective zero-shot cross-lingual adaptation.
In recent years, pre-trained Multilingual Language Models (MLLMs) have shown a strong ability to transfer knowledge across different languages. However, given that the aspiration for such an ability has not been explicitly incorporated in the design of the majority of MLLMs, it is challenging to obtain a unique and straightforward explanation for its emergence. In this review paper, we survey literature that investigates different factors contributing to the capacity of MLLMs to perform zero-shot cross-lingual transfer and subsequently outline and discuss these factors in detail. To enhance the structure of this review and to facilitate consolidation with future studies, we identify five categories of such factors. In addition to providing a summary of empirical evidence from past studies, we identify consensuses among studies with consistent findings and resolve conflicts among contradictory ones. Our work contextualizes and unifies existing research streams which aim at explaining the cross-lingual potential of MLLMs. This review provides, first, an aligned reference point for future research and, second, guidance for a better-informed and more efficient way of leveraging the cross-lingual capacity of MLLMs.
Added
2026-09-26