Publications
2021
The GEM Benchmark: Natural Language Generation, its Evaluation and Metrics
Sebastian Gehrmann , Tosin Adewumi, Karmanya Aggarwal, Pawan Sasanka Ammanamanchi, Aremu Anuoluwapo, Antoine Bosselut, Khyathi Raghavi Chandu, Miruna Clinciu, Dipanjan Das, Kaustubh D Dhole, Wanyu Du, Esin Durmus, Ondřej Dušek, Chris Emezue, Varun Gangal, Cristina Garbacea, Tatsunori Hashimoto, Yufang Hou, Yacine Jernite, Harsh Jhamtani, Yangfeng Ji, Shailza Jolly, Dhruv Kumar, Faisal Ladhak, Aman Madaan, Mounica Maddela, Khyati Mahajan, Saad Mahamood, Bodhisattwa Prasad Majumder, Pedro Henrique Martins, Angelina McMillan-Major, Simon Mille, Emiel van Miltenburg, Moin Nadeem, Shashi Narayan, Vitaly Nikolaev, Rubungo Andre Niyongabo, Salomey Osei, Ankur Parikh, Laura Perez-Beltrachini, Niranjan Ramesh Rao, Vikas Raunak, Juan Diego Rodriguez, Sashank Santhanam, João Sedoc, Thibault Sellam, Samira Shaikh, Anastasia Shimorina, Marco Antonio Sobrevilla Cabezudo, Hendrik Strobelt, Nishant Subramani, Wei Xu, Diyi Yang, Akhila Yerukola, Jiawei Zhou
[ Website ]
[ Paper ]
Accelerating Antimicrobial Discovery with Controllable Deep Generative Models and Molecular Dynamics
Payel Das, Tom Sercu, Kahini Wadhawan*, Inkit Padhi*; Sebastian Gehrmann* , Flaviu Cipcigan, Vijil Chenthamarakshan, Hendrik Strobelt, Cicero dos Santos, Pin-Yu Chen, Yi Yan Yang, Jeremy Tan, James Hedrick, Jason Crain, Aleksandra Mojsilovic
Nature Biomedical Engineering
[ Paper ]
2020
Proceedings of the 1st Workshop on Evaluating NLG Evaluation
Shubham Agarwal, Ondřej Dušek, Sebastian Gehrmann, Dimitra Gkatzia, Ioannis Konstas, Emiel Van Miltenburg, Sashank Santhanam
Workshop at INLG 2020
[ Workshop Website ]
[ Proceedings ]
Learning to Evaluate Translation Beyond English: BLEURT Submissions to the WMT Metrics 2020 Shared Task
Thibault Sellam, Amy Pu, Hyung Won Chung, Sebastian Gehrmann, Qijun Tan, Markus Freitag, Dipanjan Das, Ankur P Parikh
WMT 2020
[ Paper ]
The Language Interpretability Tool: Extensible, Interactive Visualizations and Analysis for NLP Models
Ian Tenney*, James Wexler*, Jasmijn Bastings, Tolga Bolukbasi, Andy Coenen, Sebastian Gehrmann* , Ellen Jiang, Mahima Pushkarna, Carey Radebaugh, Emily Reif, Ann Yuan
EMNLP Demo 2020
[ Paper ]
[ Website ]
[ Tutorial ]
Interpretability and analysis in neural NLP
Sebastian Gehrmann* , Yonatan Belinkov*, Ellie Pavlick*
Tutorial at ACL 2020
[ Slides ]
ToTTo: A controlled table-to-text generation dataset
Ankur P Parikh, Xuezhi Wang, Sebastian Gehrmann* , Manaal Faruqui, Bhuwan Dhingra, Diyi Yang, Dipanjan Das
EMNLP 2020
[ Paper ]
Investigating gender bias in language models using causal mediation analysis
Jesse Vig*, Sebastian Gehrmann* , Yonatan Belinkov*, Sharon Qian, Daniel Nevo, Yaron Singer, Stuart Shieber
NeurIPS 2020
[ Extended Preprint ]
[ Paper ]
Parent perspectives in shared decision-making for children with medical complexity
Jody Lin, Catherine Clark, Bonnie Halpern-Felsher, Paul Bennett, Shiri Assis-Hassid, Ofra Amir, Yadira Nunez, Nancy Cleary, Sebastian Gehrmann , Barbara Grosz, Lee Sanders
Academic Pediatrics
[ Paper (Paywall) ]
A Corpus for Detecting High-Context Clinical Indications in Intensive Care Patient Notes Focusing on Frequently Readmitted Patients
Edward Moseley, Joy Wu, Jonathan Welt, John Foote Jr., Patrick Tyler, David Grant, Eric Carlson, Sebastian Gehrmann , Franck Dernoncourt, Leo Celi
Proceedings of The 12th Language Resources and Evaluation Conference.
[ Paper ]
exBERT: A Visual Analysis Tool to Explore Learned Representations in Transformer Models
Benjamin Hoover, Hendrik Strobelt, Sebastian Gehrmann
ACL Demos 2020
[ Paper ]
2019
Memory-Augmented Recurrent Neural Networks Can Learn Generalized Dyck Languages
Mirac Suzgun, Sebastian Gehrmann , Yonatan Belinkov, Stuart Shieber
[ Preprint ]
Encoder-Agnostic Adaptation for Conditional Language Generation
Zachary M Ziegler, Luke Melas-Kyriazi, Sebastian Gehrmann , Alexander M Rush
[ Preprint ]
Visual Interaction with Deep Learning Models through Collaborative Semantic Inference
Sebastian Gehrmann *, Hendrik Strobelt*, Robert Krüger, Hanspeter Pfister, Alexander M Rush
IEEE Transactions on Visualization and Computer Graphics (VAST 2019)
[ Paper ]
GLTR: Statistical Detection and Visualization of Generated Text
Sebastian Gehrmann , Hendrik Strobelt, Alexander M Rush
Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: System Demonstrations
Nominated for Best Demo
[ Paper ]
[ Demo ]
[ Code ]
LSTM Networks Can Perform Dynamic Counting
Mirac Suzgun, Sebastian Gehrmann , Yonatan Belinkov, Stuart Shieber
Deep Learning and Formal Languages Workshop at ACL 19
[ Paper ]
Evaluating an Automated Mediator for Joint Narratives in a Conflict Situation
Massimo Zancanaro, Oliviero Stock, Gianluca Schiavo, Alessandro Cappelletti, Sebastian Gehrmann , Daphna Canetti, Ohad Shaked, Shani Fachter, Rachel Yifat, Ravit Mimran
Behaviour & Information Technology
[ Paper ]
Improving Human Text Comprehension through Semi-Markov CRF-based Neural Section Title Generation
Sebastian Gehrmann , Steven Layne, Franck Dernoncourt
Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics
[ Paper ]
Interactive Visual Exploration of Latent Space (IVELS) for Peptide Auto-Encoder Model Selection
Tom Sercu, Sebastian Gehrmann , Hendrik Strobelt, Payel Das, Inkit Padhi, Cicero Dos Santos, Kahini Wadhawan, Vijil Chenthamarakshan
ICLR 2019 Workshop on Deep Generative Models for Highly Structured Data
[ Paper ]
Generating Abstractive Summaries with Finetuned Language Models
Sebastian Gehrmann *, Zachary Ziegler*, Alexander Rush
Proceedings of the 12th International Conference on Natural Language Generation (Challenge Track)
[ Paper ]
Margin Call: an Accessible Web-based Text Viewer with Generated Paragraph Summaries in the Margin
Nabah Rizvi, Sebastian Gehrmann , Franck Dernoncourt
Proceedings of the 12th International Conference on Natural Language Generation (Demo Track)
[ Paper ]
2018
Bottom-Up Abstractive Summarization
Sebastian Gehrmann , Yuntian Deng, Alexander Rush
EMNLP 2018
[ Paper ]
[ Code ]
Debugging Sequence-to-Sequence Models with Seq2Seq-Vis
Hendrik Strobelt*, Sebastian Gehrmann*, Michael Behrisch, Adam Perer, Hanspeter Pfister, Alexander Rush
BlackboxNLP workshop at EMNLP 2018
[ Paper ]
[ Demo ]
End-to-End Content and Plan Selection for Natural Language Generation
Sebastian Gehrmann , Falcon Dai, Henry Elder, Alexander Rush
Proceedings of the 11th International Conference on Natural Language Generation
Best ROUGE/METEOR/CIDEr
[ Paper ]
[ Code ]
[ Challenge Website ]
Towards Controllable Generation of Diverse Natural Language
Henry Elder, Sebastian Gehrmann , Alexander O’Connor, Qun Liu
Proceedings of the 11th International Conference on Natural Language Generation (E2E NLG challenge track)
[ Paper ]
[ Challenge Website ]
Seq2Seq-Vis: A Visual Debugging Tool for Sequence-to-Sequence Models
Hendrik Strobelt*, Sebastian Gehrmann *, Michael Behrisch, Adam Perer, Hanspeter Pfister, Alexander Rush
IEEE Transactions on Visualization and Computer Graphics (VAST 2018)
Honorable Mention
[ Paper ]
[ Demo ]
[ Code ]
Comparing deep learning and concept extraction based methods for patient phenotyping from clinical narratives
Sebastian Gehrmann , Franck Dernoncourt, Yeran Li, Eric Carlson, Joy Wu, Jonathan Welt, John Foote Jr., Edward Moseley, David Grant, Patrick Tyler, Leo Celi
PLoS One
[ Paper ]
[ Code ]
[ Data ]
2017
LSTMVis: A Tool for Visual Analysis of Hidden State Dynamics in Recurrent Neural Networks
Hendrik Strobelt*, Sebastian Gehrmann *, Hanspeter Pfister, Alexander Rush
IEEE Transactions on Visualization and Computer Graphics (InfoVis 2017)
[ Paper ]
[ Demo ]
[ Code ]
Behind the Scenes: A Medical Natural Language Processing Project
Joy Wu, Franck Dernoncourt, Sebastian Gehrmann , Patrick Tyler, Edward Moseley, Eric Carlson, David Grant, Yeran Li, Jonathan Welt, Leo Celi
International Journal of Medical Informatics
[ Paper ]
2015
Deploying AI Methods to Support Collaborative Writing: a Preliminary Investigation
Sebastian Gehrmann , Lauren Urke, Ofra Amir, Barbara Grosz
Proceedings of the 33rd Annual ACM Conference, Extended Abstracts on Human Factors in Computing Systems
[ Paper ]