Towards Reproducible Machine Learning Research in Natural Language Processing

Half-day tutorial at ACL Conference 2022.

Overview

While recent progress in the field of ML has been significant, the reproducibility of these cutting-edge results is often lacking, with many submissions lacking the necessary information in order to ensure subsequent reproducibility. Despite proposals such as the Reproducibility Checklist and reproducibility criteria at several major conferences, the reflex for carrying out research with reproducibility in mind is lacking in the broader ML community. We propose this tutorial as a gentle introduction to ensuring reproducible research in ML, with a specific emphasis on computational linguistics and NLP.

Materials

Organizers

Ana Lucic
Ana Lucic

University of Amsterdam

Maurits Bleeker
Maurits Bleeker

University of Amsterdam

Samarth Bhargav
Samarth Bhargav

University of Amsterdam

Jessica Zosa Forde
Jessica Zosa Forde

Brown University

Koustuv Sinha
Koustuv Sinha

McGill University

Jesse Dodge
Jesse Dodge

Allen Institute for AI

Sasha Luccioni
Sasha Luccioni

HuggingFace

Robert Stojnic
Robert Stojnic

Facebook AI Research