CodaLab Competitions: An Open Source Platform to Organize Scientific Challenges Adrien Pavao, Isabelle Guyon, Anne-Catherine Letournel, Dinh-Tuan Tran, Xavier Baró, Hugo Jair Escalante, Sergio Escalera, Tyler Thomas, Zhen Xu
4Paradigm Inc. Centre de Visió per Computador ChaLearn CNRS Instituto Nacional de Astrofísica, Óptica y Electrónica LISN Tier0 Software Development LLC Universitat de Barcelona Universitat Oberta de Catalunya Université Paris-Saclay Why you should read this Presents an open-source framework that enables researchers to design, host, and scale reproducible machine learning competitions using custom Docker environments, multi-phase evaluations, and external compute resources.
CodaLab Competitions is an open source web platform designed to help data scientists and research teams to crowd-source the resolution of machine learning problems through the organization of competitions, also called challenges or contests. CodaLab Competitions provides useful features such as multiple phases, results and code submissions, multi-score leaderboards, and jobs running inside Docker containers. The platform is very flexible and can handle large scale experiments, by allowing organizers to upload large datasets and provide their own CPU or GPU compute workers.
