OpenNMT: Open-Source Toolkit for Neural Machine Translation

Guillaume KleinYoon KimYuntian DengJean SenellartAlexander M. Rush

article2017ACL1,970 citationsBest Demonstration Award

Presents OpenNMT, an efficient and modular open-source framework that allows researchers to easily implement, train, and evaluate novel neural machine translation architectures and multimodal models.

Listen

Neural machine translation has emerged as the leading approach for automated language translation, substantially outperforming older phrase-based and rule-based systems in human evaluations. However, adoption across research and industry has been hindered by a divide in software tooling: proprietary industrial systems remain closed to the public, while existing academic implementations often function as specialized research code lacking the performance, modularity, and operational readiness required for production deployments.

The article introduces OpenNMT, an open-source framework designed to provide an efficient, modular, and extensible platform for training and deploying neural machine translation models across academic and industrial environments.

To establish credibility and verify performance, the authors implemented the core translation system in Lua/Torch alongside a lightweight Python version, incorporating aggressive internal memory sharing, multi-graphics processing unit (GPU) parallelization, and specialized deployment targets including a C++ runtime. The authors benchmarked OpenNMT against established systems, specifically comparing it to Nematus on standard English-to-German translation tasks and evaluating its performance on a large-scale multilingual corpus covering five Romance languages across twenty language pairs.

The evaluation revealed several key findings regarding efficiency, translation quality, and extensibility. First, OpenNMT achieves superior speed and accuracy compared to the established Nematus baseline; on English-to-German translation benchmarks, it achieved higher translation accuracy (19.34 BLEU versus 18.25) and delivered a 63% increase in training throughput and an 81% increase in inference speed under sub-word tokenization. Second, the system's memory-sharing optimization reduces GPU memory usage by 70%, enabling larger batch sizes, while parallel processing across eight GPUs provides a 3.5-fold overall speed-up to reach equivalent training loss. Third, training a single unified model across twenty language pairs systematically improved translation quality across all pairs compared to independent training models, yielding gains of up to 10.4 BLEU points. Finally, case studies confirmed that the modular codebase effectively generalizes to non-standard tasks, including image-to-text optical character recognition, speech recognition, sentence summarization, and dialogue response generation.

These findings indicate that organizations no longer need to compromise between research flexibility and production efficiency. The significant reductions in memory footprint and training duration translate directly into lower hardware infrastructure costs and faster development cycles. Furthermore, the compact codebase—consisting of approximately 4,000 lines compared to legacy systems requiring over 100,000 lines—lowers the technical barrier to maintaining and auditing translation infrastructure.

Decision-makers and engineering teams should consider adopting OpenNMT for both sequence-to-sequence research and enterprise translation deployments, utilizing the specialized C++ runtime where efficient CPU deployment is required. Organizations with multi-language workflows should leverage unified multilingual models to maximize performance across low-resource language pairs while minimizing model maintenance overhead. Ongoing development should focus on expanding community contributions and tracking future model architecture advances.

While the reported benchmarks provide high confidence in the system's computational efficiency and translation fidelity across the tested English and Romance language benchmarks, potential adopters should exercise appropriate caution. System performance and memory behavior were primarily evaluated on specific hardware configurations and language sets; organizations deploying models to non-Romance languages or resource-constrained edge environments should conduct targeted pilots before full-scale deployment.

Harvard University SYSTRAN machine translations statistical machine translation open source software neural machine translation OpenNMT system open-source toolkit

arXiv: 1709.03815 OpenNMT/OpenNMT

Paper: Effective Approaches to Attention-based Neural Machine Translation, Minh-Thang Luong et al. (2015). Introduces the global and local attention mechanisms for sequence-to-sequence neural machine translation that form the architectural foundation for OpenNMT's default models.
Paper: Neural Machine Translation by Jointly Learning to Align and Translate, Dzmitry Bahdanau et al. (2015). Establishes the core neural machine translation framework with joint alignment and translation that OpenNMT standardizes and implements.
Paper: Neural Machine Translation of Rare Words with Subword Units, Rico Sennrich et al. (2016). Presents byte pair encoding for subword segmentation, a necessary preprocessing technique used in OpenNMT pipelines to handle open-vocabulary translation.
Paper: Sequence to Sequence Learning with Neural Networks, Ilya Sutskever et al. (2014). Introduces the foundational sequence-to-sequence recurrent neural network architecture used for translation tasks that OpenNMT was built to train and serve.
Paper: Moses: Open Source Toolkit for Statistical Machine Translation, Philipp Koehn et al. (2007). Provides the prior open-source paradigm and standard toolkit for statistical machine translation that OpenNMT was developed to succeed for neural architectures.
Paper: Improving Neural Machine Translation Models with Monolingual Data, Rico Sennrich et al. (2016). Details methods for integrating monolingual corpora via back-translation into NMT training, an essential operational pipeline supported by NMT frameworks.
Paper: Sequence-Level Knowledge Distillation, Yoon Kim et al. (2016). Formulates sequence-level knowledge distillation techniques frequently deployed in OpenNMT workflows to compress neural translation models for production.
Paper: Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation, Yonghui Wu et al. (2016). Describes practical, production-scale NMT engineering designs—including length normalization and coverage penalties—reproduced and modularized within OpenNMT.

Paper: fairseq: A Fast, Extensible Toolkit for Sequence Modeling, Myle Ott et al. (2019). Advances open-source neural sequence modeling and translation toolkits beyond OpenNMT by introducing multi-GPU scaling, mixed-precision FP16 training, and native Transformer optimization in PyTorch.
Paper: SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing, Taku Kudo et al. (2018). Develops a language-independent, raw-text subword tokenizer that streamlines and modernizes the preprocessing pipelines utilized by translation toolkits like OpenNMT.
Paper: Transformers: State-of-the-Art Natural Language Processing, Thomas Wolf et al. (2019). Generalizes open-source sequence toolkit design from modular NMT libraries into a unified ecosystem and model hub across broader NLP architectures and pretrained models.
Paper: The Annotated Transformer, Sasha Rush et al. (2022). Provides a direct, line-by-line PyTorch reference implementation of the attention-based Transformer architectures that later superseded recurrent NMT toolkits.
Paper: Multilingual Denoising Pre-training for Neural Machine Translation, Yinhan Liu et al. (2020). Extends neural sequence-to-sequence translation pipelines by incorporating large-scale multilingual denoising pre-training before fine-tuning translation tasks.

Cover for OpenNMT: Open-Source Toolkit for Neural Machine Translation

Abstract

We introduce an open-source toolkit for neural machine translation (NMT) to support research into model architectures, feature representations, and source modalities, while maintaining competitive performance, modularity and reasonable training requirements.

Topics

machine translations statistical machine translation open source software

Keywords

neural machine translation OpenNMT system open-source toolkit recurrent neural networks hierarchical attention word embeddings asynchronous training sentence summarization language modeling

1 Introduction
2 Description
3 Ecosystem
4 Community
5 Conclusion

Knowls

Knowl 1 — OpenNMT Architecture and Modular Framework Design
model/method
OpenNMT is an open-source framework for neural machine translation (NMT) and general sequence-to-sequence modeling implemented in Lua/Torch (with a matching API implementation in Python/PyTorch). The architecture models conditional translation probability $p(w_{1:T} \mid x_{1:S}) = \prod_{t=1}^T p(w_t \mid w_{1:t-1}, x_{1:S}; \theta)$ using an attention-based encoder-decoder neural network.

The framework provides native support for:
- Encoder-Decoder Recurrent Units: Multi-layer stacked Long Short-Term Memory (LSTM) or Gated Recurrent Unit (GRU) networks (2 to 16 vertical layers).
- Input Feeding: Feeding the attention vector from step $t-1$ back into the decoder input alongside the word embedding at step $t$ .
- Modular Attention Layers: Standard global attention, local attention, sparsemax attention, hierarchical attention, and structured attention based on graphical model inference.
- Factored Representations: Joint generation and conditioning on words and auxiliary linguistic features (e.g., casing, morphological tags) by abstracting feature inputs and output generators from the core translation graph.
- Decoding: Standalone beam search with hypothesis tracking.
- Standalone Preprocessing: Language-independent reversible tokenization tools supporting subword Byte Pair Encoding (BPE).
Knowl 2 — Time-Series GPU Memory Sharing in Sequence-to-Sequence Models
model/method
In recurrent neural network training on GPUs, intermediate activation allocations often constrain maximum minibatch size. OpenNMT reduces memory footprint by leveraging the deterministic time-series execution flow of sequence-to-sequence architectures.

Prior to starting model training, OpenNMT performs dynamic computation graph exploration to detect intermediate tensor buffers whose lifecycles do not overlap across consecutive recurrent time steps. It then shares and reuses these pre-allocated internal memory buffers across unrolled clones rather than maintaining independent memory allocations. For default NMT configurations, this dynamic buffer reuse achieves a 70% reduction in GPU memory utilization without sacrificing execution speed, allowing larger batch sizes on a single GPU.

Knowl 3 — OpenNMT English-to-German Benchmark Comparison against Nematus

data/table

The performance of OpenNMT was benchmarked against the Nematus NMT toolkit on English-to-German (EN $\to$ DE) translation using the WMT 2015 dataset, evaluated on newstest2014. Both systems used a 2-layer stacked RNN with 500 hidden units per layer, word embedding dimension of 300, batch size of 64, beam search width of 5, and were trained for 13 epochs on an Intel Core i7-5930K CPU with a single NVIDIA GeForce GTX 1080 GPU under CUDA 8.0 and cuDNN 5005.

Vocabulary Setting	System	Training Speed	Translation Speed	BLEU
		(tokens/sec)	(tokens/sec)
$V = 50\text{k}$	Nematus	3393	284	17.28
$V = 50\text{k}$	OpenNMT	4185	380	17.60
$V = 32\text{k}$ (BPE)	Nematus	3221	252	18.25
$V = 32\text{k}$ (BPE)	OpenNMT	5254	457	19.34

OpenNMT demonstrates higher training throughput (up to $1.63\times$ faster on 32k BPE) and translation throughput (up to $1.81\times$ faster on 32k BPE) while achieving comparable or higher BLEU scores under identical model architectures and hyperparameter configurations.

Knowl 4 — Inference Throughput Across OpenNMT Runtime Deployments

data/table

OpenNMT provides specialized deployment runtimes tailored for different serving environments: a batched Torch GPU/CPU runtime and a lightweight standalone C++ CPU runtime using the Eigen linear algebra library for production CPU servers.

The table reports decoding speed in processed source tokens per second on an Intel Core i7 / NVIDIA GeForce GTX 1080 setup across different batch sizes and beam search widths:

Batch Size	Beam Size	Torch GPU (tok/s)	Torch CPU (tok/s)	C++/Eigen CPU (tok/s)
1	5	209.0	24.1	62.2
1	1	166.9	23.3	84.9
30	5	646.8	104.0	116.2
30	1	535.1	128.5	392.7

Batching on GPU provides up to a $3.1\times$ throughput increase (from 209.0 to 646.8 tokens/sec with beam 5). For CPU-only deployment environments, the dedicated C++/Eigen implementation achieves between $1.1\times$ and $3.6\times$ higher throughput than the Torch CPU implementation.

Knowl 5 — Multilingual Neural Translation Across Five Romance Languages

data/table

A single multilingual OpenNMT model was trained across 20 translation directions involving five Romance languages: Spanish (ES), French (FR), Italian (IT), Portuguese (PT), and Romanian (RO). The dataset consisted of 4 million sentences from Europarl, GlobalVoices, and TED parallel corpora, where every sentence is parallel across all five languages (multi-source, multi-target), preprocessed with a 32,000-token shared Byte Pair Encoding (BPE) vocabulary.

The table presents translation BLEU scores for all 20 language pairs, where the value in parentheses ( $\Delta$ ) denotes the BLEU improvement compared to a model trained exclusively on that single language pair:

Source $\downarrow$ / Target $\to$	ES	FR	IT	PT	RO
ES	—	32.7 (+5.4)	28.0 (+4.6)	34.4 (+6.1)	28.7 (+6.4)
FR	32.9 (+3.3)	—	26.3 (+4.3)	30.9 (+5.2)	26.0 (+6.6)
IT	31.6 (+5.3)	31.0 (+5.8)	—	28.0 (+5.0)	24.3 (+5.9)
PT	35.3 (+10.4)	34.1 (+4.7)	28.1 (+5.6)	—	28.7 (+5.0)
RO	35.0 (+5.4)	31.9 (+9.0)	26.4 (+6.3)	31.6 (+7.3)	—

Joint multilingual training consistently outperforms individual bilingual training across all 20 directions, with BLEU gains ranging from $+3.3$ (FR $\to$ ES) to $+10.4$ (PT $\to$ ES), demonstrating positive cross-lingual transfer.

Knowl 6 — Synchronous and Asynchronous Multi-GPU Data Parallelism in OpenNMT
model/method
OpenNMT implements multi-GPU data-parallel distributed training supporting two synchronization strategies:
1. Synchronous Data Parallelism: Each GPU hosts an identical replica of master model parameters and executes forward and backward computations on a distinct data minibatch. Gradients from all GPUs are aggregated synchronously to update the master parameters, followed by parameter resynchronization across all GPUs before the next minibatch.
2. Asynchronous Data Parallelism: GPUs process minibatches independently and asynchronously accumulate their local gradients directly into the central parameter server.
On an 8-GPU cluster, asynchronous training yields a $6\times$ per-epoch throughput speedup compared to single-GPU execution. Accounting for slight gradient inefficiency during asynchronous updates, the overall wall-clock speedup to converge to equivalent validation loss is $3.5\times$ .
Knowl 7 — Multi-Modal Sequence-to-Sequence Extensions in OpenNMT
model/method
The modular encoder abstraction in OpenNMT allows adaptation to non-textual input modalities by substituting the standard recurrent encoder:
- Image-to-Text (Visual Markup Decompiler / OCR): The source recurrent network is replaced with a deep Convolutional Neural Network (CNN) feature extractor operating directly on input images. The decoder attends over the visual feature map via spatial attention to generate LaTeX markup, requiring fewer than 500 lines of adaptation code (im2text).
- Speech-to-Text: The text encoder is replaced with a Pyramidal Recurrent Neural Network encoder to downsample high-resolution audio spectrogram features across successive recurrent layers prior to attention decoding.
Knowl 8 — OpenNMT Non-Translation Sequence-to-Sequence Task Replication
empirical result
OpenNMT generalizes beyond machine translation to broader sequence transduction tasks:
- Sentence Summarization: On the Gigaword abstractive summarization benchmark, OpenNMT achieved a ROUGE-1 score of 33.13, reproducing the attentive recurrent neural network results of Chopra et al. (2016).
- Dialogue Response Generation: Trained on 14 million sentence pairs from the OpenSubtitles dataset, OpenNMT matched the language modeling perplexity benchmarks reported by Vinyals and Le (2015).

Coverage note — None was omitted; all core architectural, efficiency, modularity, and empirical benchmarking contributions were included.

References

1.Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2014. Neural Machine Translation By Jointly Learning To Align and Translate. In ICLR. pages 1–15. https://doi.org/10.1146/annurev.neuro.26.041002.131047.
2.Samuel R. Bowman, Luke Vilnis, Oriol Vinyals, Andrew M. Dai, Rafal Jozefowicz, and Samy Bengio. 2016.´ Generating sentences from a continuous space. In Proceedings of the 20th SIGNLL Conference on Computational Natural Language Learning, CoNLL 2016, Berlin, Germany, August 11-12, 2016. pages 10–21. http://aclweb.org/anthology/K/K16/K16-1002.pdf.
3.William Chan, Navdeep Jaitly, Quoc V. Le, and Oriol Vinyals. 2015. Listen, attend and spell. CoRR abs/1508.01211. http://arxiv.org/abs/1508.01211.
4.Kyunghyun Cho, Bart van Merrienboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014. Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation. In Proc of EMNLP.
5.Sumit Chopra, Michael Auli, Alexander M Rush, and SEAS Harvard. 2016. Abstractive sentence summarization with attentive recurrent neural networks. Proceedings of NAACL-HLT16 pages 93–98.
6.Junyoung Chung, Caglar Gulcehre, KyungHyun Cho, and Yoshua Bengio. 2014. Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv:1412.3555 .
7.Josep Crego, Jungi Kim, and Jean Senellart. 2016. Systran’s pure neural machine translation system. arXiv preprint arXiv:1602.06023 .
8.Jeffrey Dean, Greg Corrado, Rajat Monga, Kai Chen, Matthieu Devin, Mark Mao, Andrew Senior, Paul Tucker, Ke Yang, Quoc V Le, et al. 2012. Large scale distributed deep networks. In Advances in neural information processing systems. pages 1223–1231.
9.Yuntian Deng, Anssi Kanervisto, and Alexander M. Rush. 2016. What you get is what you see: A visual markup decompiler. CoRR abs/1609.04938. http://arxiv.org/abs/1609.04938.
10.Chris Dyer, Jonathan Weese, Hendra Setiawan, Adam Lopez, Ferhan Ture, Vladimir Eidelman, Juri Ganitkevitch, Phil Blunsom, and Philip Resnik. 2010. cdec: A decoder, alignment, and learning framework for finite-state and context-free translation models. In Proc ACL. Association for Computational Linguistics, pages 7–12.
11.Sepp Hochreiter and Jurgen Schmidhuber. 1997. Long short-¨ term memory. Neural computation 9(8):1735–1780.
12.Mike Schuster Quoc V. Le Maxim Krikun Yonghui Wu Zhifeng Chen Nikhil Thorat Fernanda Vigas Martin Wattenberg Greg Corrado Macduff Hughes Jeffrey Dean Johnson. 2016. Google’s multilingual neural machine translation system: Enabling zero-shot translation .
13.Philipp Koehn, Hieu Hoang, Alexandra Birch, Chris Callison-Burch, Marcello Federico, Nicola Bertoldi, Brooke Cowan, Wade Shen, Christine Moran, Richard Zens, et al. 2007. Moses: Open source toolkit for statistical machine translation. In Proc ACL. Association for Computational Linguistics, pages 177–180.
14.Minh-Thang Luong, Hieu Pham, and Christopher D. Manning. 2015. Effective Approaches to Attention-based Neural Machine Translation. In Proc of EMNLP.
15.Andre FT Martins and Ram´ on Fernandez Astudillo. 2016.´ From softmax to sparsemax: A sparse model of attention and multi-label classification. arXiv preprint arXiv:1602.02068 .
16.G. Neubig. 2017. Neural Machine Translation and Sequence-to-sequence Models: A Tutorial. ArXiv e-prints .
17.Graham Neubig. 2013. Travatar: A forest-to-string machine translation engine based on tree transducers. In Proc ACL. Sofia, Bulgaria.
18.Rico Sennrich and Barry Haddow. 2016. Linguistic input features improve neural machine translation. arXiv preprint arXiv:1602.02892 .
19.Rico Sennrich, Barry Haddow, and Alexandra Birch. 2015. Neural machine translation of rare words with subword units. CoRR abs/1508.07909. http://arxiv.org/abs/1508.07909.
20.Ilya Sutskever, Oriol Vinyals, and Quoc V. Le. 2014. Sequence to Sequence Learning with Neural Networks. In NIPS. page 9. http://arxiv.org/abs/1409.3215.
21.Oriol Vinyals and Quoc Le. 2015. A neural conversational model. arXiv preprint arXiv:1506.05869 .
22.Jason Weston, Sumit Chopra, and Antoine Bordes. 2014. Memory networks. CoRR abs/1410.3916. http://arxiv.org/abs/1410.3916.
23.Yonghui Wu, Mike Schuster, Zhifeng Chen, Quoc V Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey, et al. 2016. Google’s neural machine translation system: Bridging the gap between human and machine translation. arXiv preprint arXiv:1609.08144 .
24.Kelvin Xu, Jimmy Ba, Ryan Kiros, Kyunghyun Cho, Aaron C. Courville, Ruslan Salakhutdinov, Richard S. Zemel, and Yoshua Bengio. 2015. Show, attend and tell: Neural image caption generation with visual attention. CoRR abs/1502.03044. http://arxiv.org/abs/1502.03044.
25.Zichao Yang, Diyi Yang, Chris Dyer, Xiaodong He, Alex Smola, and Eduard Hovy. 2016. Hierarchical attention networks for document classification. In Proc ACL.

Citation

MLA

Klein, G., et al. “OpenNMT: Open-source Toolkit for Neural Machine Translation”. arXiv, 2017, http://arxiv.org/abs/1709.03815v1.

APA

Klein, G., Kim, Y., Deng, Y., Crego, J., Senellart, J., & Rush, A. M. (2017). OpenNMT: Open-source Toolkit for Neural Machine Translation. arXiv. http://arxiv.org/abs/1709.03815v1

Chicago

Klein, G., Y. Kim, Y. Deng, J. Crego, J. Senellart, and A. M. Rush. 2017. “OpenNMT: Open-source Toolkit for Neural Machine Translation”. arXiv. http://arxiv.org/abs/1709.03815v1.

Harvard

Klein, G. et al. (2017) “OpenNMT: Open-source Toolkit for Neural Machine Translation”, arXiv [Preprint]. Available at: http://arxiv.org/abs/1709.03815v1.

Vancouver

1. Klein G, Kim Y, Deng Y, Crego J, Senellart J, Rush AM (2017) OpenNMT: Open-source Toolkit for Neural Machine Translation. arXiv

BibTeX

@article{klein2017opennmt,
  title = {OpenNMT: Open-source Toolkit for Neural Machine Translation},
  author = {Klein, Guillaume and Kim, Yoon and Deng, Yuntian and Crego, Josep and Senellart, Jean and Rush, Alexander M.},
  year = {2017},
  journal = {arXiv},
  url = {http://arxiv.org/abs/1709.03815v1},
  eprint = {1709.03815}
}

Metadata:arXiv

Source Code

This paper has an official code repository available. Click below to access the source code.

View Repository

Access the Paper

This paper is available from its original source. Click below to access the PDF.

Open PDF

License: https://creativecommons.org/licenses/by/4.0/

OpenNMT: Open-Source Toolkit for Neural Machine Translation

Abstract

Topics

Keywords

Table of Contents

Knowls

Knowl 1 — OpenNMT Architecture and Modular Framework Design

Knowl 2 — Time-Series GPU Memory Sharing in Sequence-to-Sequence Models

Knowl 3 — OpenNMT English-to-German Benchmark Comparison against Nematus

Knowl 4 — Inference Throughput Across OpenNMT Runtime Deployments

Knowl 5 — Multilingual Neural Translation Across Five Romance Languages

Knowl 6 — Synchronous and Asynchronous Multi-GPU Data Parallelism in OpenNMT

Knowl 7 — Multi-Modal Sequence-to-Sequence Extensions in OpenNMT

Knowl 8 — OpenNMT Non-Translation Sequence-to-Sequence Task Replication

References

Citation

BibTeX

Source Code

Access the Paper