Publications

Tom Sherborne, Naomi Saphra, Pradeep Dasigi, Hao Peng (2024). TRAM: Bridging Trust Regions and Sharpness Aware Minimization. International Conference on Learning Representations (ICLR).

Cite URL

Angelica Chen, Ravid Shwartz-Ziv, Kyunghyun Cho, Matthew L. Leavitt, Naomi Saphra (2024). Sudden Drops in the Loss: Syntax Acquisition, Phase Transitions, and Simplicity Bias in MLMs. International Conference on Learning Representations (ICLR).

Cite URL

Zachary Ankner, Naomi Saphra, Davis Blalock, Jonathan Frankle, Matthew L. Leavitt (2024). Dynamic Masking Rate Schedules for MLM Pretraining. European Association for Computational Linguistics (EACL).

Cite URL

Naomi Saphra, Eve Fleisig, Kyunghyun Cho, Adam Lopez (2023). First Tragedy, then Parse: History Repeats Itself in the New Era of Large Language Models. arXiv preprint.

Cite URL

Ian Berlot-Attwell, A. Michael Carrell, Kumar Krishna Agrawal, Yash Sharma, Naomi Saphra (2023). Attribute Diversity Determines the Systematicity Gap in VQA. arXiv preprint.

Cite URL

Dieuwke Hupkes, Mario Giulianelli, Verna Dankers, Mikel Artetxe, Yanai Elazar, Tiago Pimentel, Christos Christodoulopoulos, Karim Lasri, Naomi Saphra, Arabella Sinclair, Dennis Ulmer, Florian Schottmann, Khuyagbaatar Batsuren, Kaiser Sun, Koustuv Sinha, Leila Khalatbari, Maria Ryskina, Rita Frieske, Ryan Cotterell, Zhijing Jin (2023). State-of-the-art generalisation research in NLP: a taxonomy and review. Nature Machine Intelligence.

Cite DOI URL

Michael Hu, Angelica Chen, Naomi Saphra, Kyunghyun Cho (2023). Latent State Transitions in Training Dynamics. Transactions of Machine Learning Research (TMLR).

Cite URL

Jeevesh Juneja, Rachit Bansal, Kyunghyun Cho, João Sedoc, Naomi Saphra (2023). Linear Connectivity Reveals Generalization Strategies. ICLR.

URL

Naomi Saphra (2023). Interpretability Creationism. The Gradient.

Cite URL

Bingchen Zhao, Yuling Gu, Jessica Zosa Forde, Naomi Saphra (2022). One Venue, Two Conferences: The Separation of Chinese and American Citation Networks. NeurIPS Workshop on Cultures in AI.

PDF Cite

Josef Valvoda, Naomi Saphra, Jonathan Rawski, Adina Williams, Ryan Cotterell (2022). Benchmarking Compositionality with Formal Languages. COLING.

PDF

Naomi Saphra (2021). Training dynamics of neural language models. The University of Edinburgh.

Cite

Thibault Sellam, Steve Yadlowsky, Jason Wei, Naomi Saphra, Alexander D'Amour, Tal Linzen, Jasmijn Bastings, Iulia Turc, Jacob Eisenstein, Dipanjan Das, Ian Tenney, Ellie Pavlick (2021). The MultiBERTs: BERT Reproductions for Robustness Analysis. ICLR.

PDF

Jennifer C White, Tiago Pimentel, Naomi Saphra, Ryan Cotterell (2021). A Non-Linear Structural Probe. NAACL.

PDF

Mohammad Tahaei, Kami Vaniea, Naomi Saphra (2020). Understanding Privacy-Related Questions on Stack Overflow. CHI.

PDF

Tiago Pimentel, Naomi Saphra, Adina Williams, Ryan Cotterell (2020). Pareto Probing: Trading Off Accuracy for Complexity. EMNLP.

PDF

Naomi Saphra, Adam Lopez (2020). LSTMs Compose (and Learn) Bottom-Up. Findings of EMNLP.

PDF

Naomi Saphra, Adam Lopez (2019). Understanding Learning Dynamics Of Language Models with SVCCA. NAACL.

PDF

Naomi Saphra, Adam Lopez (2019). Sparsity Emerges Naturally in Neural Language Models. ICML Workshop on Identifying and Understanding Deep Learning Phenomena.

PDF

Naomi Saphra, Adam Lopez (2016). Evaluating Informal-Domain Word Representations With UrbanDictionary. The First Workshop on Evaluating Vector Space Representations for NLP.

PDF

Naomi Saphra, Adam Lopez (2015). AMRICA: an AMR Inspector for Cross-language Alignments. Proc. of NAACL demo.

PDF

A. Vedaldi, S. Mahendran, S. Tsogkas, S. Maji, B. Girshick, J. Kannala, E. Rahtu, I. Kokkinos, M. B. Blaschko, D. Weiss, B. Taskar, K. Simonyan, Naomi Saphra, S. Mohamed (2014). Understanding Objects in Detail with Fine-grained Attributes. CVPR.

PDF

Ryan Cotterell, Adithya Renduchintala, Naomi Saphra, Chris Callison-Burch (2014). An Algerian Arabic-French Code-Switched Corpus. LREC Workshop on Free/Open-Source Arabic Corpora and Corpora Processing Tools.

PDF

Nathan Schneider, Brendan O'Connor, Naomi Saphra, David Bamman, Manaal Faruqui, Noah A. Smith, Chris Dyer, Jason Baldridge (2014). A framework for (under) specifying dependency syntax without overloading annotators. ACL Linguistic Annotation Workshop.

PDF

Matthew Blaschko, Ross B. Girshick, Juho Kannala, Iasonas Kokkinos, Siddarth Mahendran, Subhransu Maji, Sammy Mohamed, Esa Rahtu, Naomi Saphra, Karen Simonyan (2012). Towards a detailed understanding of objects and scenes in natural images.

Cite URL