UCL NLP

Tweets byucl_nlp

News

11/11/2020:
Don’t Read Too Much into It: Adaptive Computation for Open-Domain Question Answering will appear at EMNLP 2020. We propose an adaptive computation method that significantly reduces the computational cost of ODQA systems while retraining a similar performance.
09/11/2020:
Beat the AI: Investigating Adversarial Human Annotation for Reading Comprehension has been accepted at TACL 2020, and we’ll be presenting it at EMNLP 2020. We’ve also publicly released the dataset and you can try to beat our best model and submit to our online leaderboard through Dynabench.
28/09/2020:
Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks has been accepted at NeurIPS 2020. We’ve also released a blog post and released the code as part of the HuggingFace ecosystem. Check out a demo of RAG here
04/09/2020:
KILT: a Benchmark for Knowledge Intensive Language Tasks is now available on ArXiv! KILT is set of tools and data to accelerate research progress on open domain and knowledge intensive NLP, including open domain QA, fact checking, relation extraction and entity linking. KILT will make your work easier, more comparable and reproducible, and allow researchers to share components more easily.
check out the code, and leaderboard, and HuggingFace integrations.
07/08/2020:
Question and Answer Test-Train Overlap in Open-Domain Question Answering Datasets is now on ArXiv! Do you use NaturalQuestions, TriviaQA, or WebQuestions? It turns out 60% of test set answers are also in the train set. More surprising, 30% of test questions have a close paraphrase in the train set. We look at what this means for models. Annotations and code available here:
06/07/2020:
R4C: A Benchmark for Evaluating RC Systems to Get the Right Answer for the Right Reason was presented at ACL 2020. It previously won the “Best Linguistic Resource” award at the 26th annual meeting of the Japanese Association for Natural Language Processing.
24/06/2020:
“How Context Affects Language Models' Factual Predictions” has been been awarded Best Paper at AKBC 2020!

News Archive →

People

Sebastian Riedel

Professor

Sebastian Riedel

Sebastian works in NLP and Machine Learning. He is particularly interested in helping machines to read more accurately by leveraging...

Pontus Stenetorp

Lecturer

Pontus Stenetorp

Pontus works somewhere in the intersection between Natural Language Processing and Machine Learning. He is particularly interested in...

Tim Rocktäschel

Affiliated Faculty (Lecturer)

Tim Rocktäschel

Reinforcement Learning – Deep Learning – Natural Language Processing

Tim’s work focuses on developing agents that are...

Ed Grefenstette

Honorary Reader

Ed Grefenstette

Ed is interesting in teaching machines to understand and communicate using language (formal and natural), and in both neural and symbolic...

Pasquale Minervini

Senior Research Fellow, Principal Investigator for H2020 CLARIFY

Pasquale Minervini

Pasquale is interested in Machine Learning and Reasoning. He is the Principal Investigator of the CLARIFY H2020 project on the UCL end.

Linqing Liu

PhD Student

Linqing Liu

Linqing is a first year PhD student with broad interests in NLP and Machine Learning. She is currently working on question answering.

Matko Bošnjak

PhD Student

Matko Bošnjak

Matko’s interests include both natural and unnatural language processing, and their interplay. Specifically, he’s enjoying...

Max Bartolo

PhD Student

Max Bartolo

Max’s current research focuses on natural language processing, with particular interest in question answering and machine reasoning.

Maximilian Mozes

PhD Student

Maximilian Mozes

Maximilian’s research focuses on the intersection of adversarial machine learning and natural language processing.

Mikayel Samvelyan

PhD Student

Mikayel Samvelyan

Mikayel’s interests lie in reinforcement learning, natural language processing and multi-agent systems.

Minqi Jiang

PhD student

Minqi Jiang

Minqi studies how decision-making agents can make use of compositional structures, like language, to learn more generalizable behaviors.

Patrick Lewis

PhD Student

Patrick Lewis

Patrick is a first year PhD student, interested in Transfer Learning, Machine Reading and leveraging world knowledge to improve predictions...

Tom Crossland

PhD Student

Tom Crossland

Tom is an astrophysicist working with the MR group and the Mullard Space Science Laboratory, interested in Machine Learning applications to...

Yao Lu

PhD Student

Yihong Chen

PhD Student

Yihong Chen

Yihong is interested in almost everything, currently working on methods that empower efficient learning of symbols. Welcome any messages...

Yuxiang Wu

PhD Student

Yuxiang Wu

Yuxiang is a third-year PhD student, interested in Question Answering, Knowledge Base, and other knowledge related tasks.

Zhengyao Jiang

PhD Student

Zhengyao Jiang

Zhengyao is interested in topics about reinforcement leanring, relational inductive bias, and bridging symbolic reasoning and deep learning....

Alastair Roberts

Visiting Researcher

Alastair Roberts

Alastair’s interests lie in natural language processing & machine learning. His current work is focused on question answering.

Alumni

Andreas Vlachos

Now a senior lecturer at University of Cambridge

Johannes Welbl

Now a Research Scientist at DeepMind

Luke Hewitt

Now a PhD student at MIT

Gerasimos Lampouras

Now a research associate at University of Sheffield

Saku Sugawara

Now back to being a Ph.D. student at the University of Tokyo.

Saku Sugawara

Former Visiting PhD Student

Sonse Shimaoka

Now a master student at Tohoku University

Zhao Zhang

Now back to being a PhD student at the Chinese Academy of Sciences.

Zhao Zhang

Former Visiting PhD Student

Guillaume Bouchard

Now a Research Manager at Facebook

Thomas Demeester

Now a post-doc at University of Ghent

Jason Naradowsky

Now a research scientist at Preferred Networks (PFN)

Théo Trouillon

Now back to being a PhD student at Xerox Research Centre Europe

Marzieh Saeidi

Now a Research Scientist at Facebook

Isabelle Augenstein

Now an associate professor at University of Copenhagen

Naoya Inoue

Now an assistant professor at Tohoku University

Tim Dettmers

Now a PhD student at University of Washington

V. Ivan Sanchez

Now an NLP researcher at Lenovo

Andres Campero

Now back to being a PhD student at MIT

Andres Campero

Former Visiting PhD Student

Takuma Yoneda

Now a student at Toyota Technological Institute at Chicago

Georgios Spithourakis

Now a ML engineer at PolyAI

Publications

Don't Read Too Much into It: Adaptive Computation for Open-Domain Question Answering

Yuxiang Wu, Sebastian Riedel, Pasquale Minervini, Pontus Stenetorp. In EMNLP, 2020.

Beat the AI: Investigating Adversarial Human Annotation for Reading Comprehension

Max Bartolo, Alastair Roberts, Johannes Welbl, Sebastian Riedel, Pontus Stenetorp. In TACL, 2020.

Answering Complex Open-Domain Questions with Multi-Hop Dense Retrieval

Wenhan Xiong, Xiang Lorraine Li, Srini Iyer, Jingfei Du, Patrick Lewis, William Yang Wang, Yashar Mehdad, Wen-tau Yih, Sebastian Riedel, Douwe Kiela, Barlas Oğuz. ArXiv 2020.

KILT: a Benchmark for Knowledge Intensive Language Tasks

Fabio Petroni, Aleksandra Piktus, Angela Fan, Patrick Lewis, Majid Yazdani, Nicola De Cao, James Thorne, Yacine Jernite, Vassilis Plachouras, Tim Rocktäschel, Sebastian Riedel. ArXiv 2020.

MLQA: Evaluating Cross-lingual Extractive Question Answering

Patrick Lewis, Barlas Oguz, Ruty Rinott, Sebastian Riedel, Holger Schwenk. In ACL 2020.

Question and Answer Test-Train Overlap in Open-Domain Question Answering Datasets

Patrick Lewis, Pontus Stenetorp, Sebastian Riedel. ArXiv 2020.

View All →

Datasets

AdversarialQA (from Beat the AI)

In this work we investigate this annotation methodology and apply it in three different settings, collecting a total of 36,000 samples with progressively stronger models in the annotation loop. This allows us to explore questions such as the reproducibility of the adversarial effect, transfer from data collected with varying model-in-the-loop strengths, and generalisation to data collected without a model. We find that training on adversarially collected samples leads to strong generalisation to non-adversarially collected datasets, yet with progressive performance deterioration with increasingly stronger models-in-the-loop. Furthermore, we find that stronger models can still learn from datasets collected with substantially weaker models-in-the-loop. When trained on data collected with a BiDAF model in the loop, RoBERTa achieves 39.9F1 on questions that it cannot answer when trained on SQuAD - only marginally lower than when trained on data collected using RoBERTa itself (41.0F1).

Max Bartolo, Alastair Roberts, Johannes Welbl, Sebastian Riedel, Pontus Stenetorp.

KILT: a Benchmark for Knowledge Intensive Language Tasks

KILT is a resource for training, evaluating and analyzing NLP models on Knowledge Intensive Language Tasks. KILT has been built from 11 datasets representing 5 tasks. All these datasets have been grounded in a single pre-processed wikipedia dump, allowing for fairer and more consistent evaluation and enabling new task setups such as multitask and transfer learning with minimal effort. KILT also provides tools to analyze and understand the predictions made by models, as well as the evidence they provide for their predictions.

Fabio Petroni, Aleksandra Piktus, Angela Fan, Patrick Lewis, Majid Yazdani, Nicola De Cao, James Thorne, Yacine Jernite, Vassilis Plachouras, Tim Rocktäschel, Sebastian Riedel.