UCL NLP

Tweets byucl_nlp

News

23/04/2021:
Our paper, Question and Answer Test-Train Overlap in Open-Domain Question Answering, has won a Best Paper award at EACL 2021! Congrats to authors Patrick, Pontus and Sebastian.
01/04/2021:
Our paper Complex Query Answering with Neural Link Predictors, a paper based on Erik’s MSc thesis supervised by Pasquale, won an Outstanding Paper Award at ICLR 2021!
15/02/2021:
PAQ: 65 Million Probably-Asked Questions and What You Can Do With Them has been released on ArXiv! Two times faster and more accurate QA models by using a huge collection of 65M automatically-generated QA-pairs. This leads to a flexible QA system that can be optimised for memory, speed or accuracy. Check out the PAQ data here, code and models coming soon!
12/01/2021:
Answering Complex Open-Domain Questions with Multi-Hop Dense Retrieva has been accepted at ICLR 2021. This simple recursive retrieval gets state of the art results without requiring additional resources such as hyperlink networks.
11/01/2021:
Stereotype and Skew: Quantifying Gender Bias in Pre-trained and Fine-tuned Language Models has been accepted at EACL 2021. We propose two intuitive metrics, skew and stereotype, that quantify and analyse the gender bias present in contextual language models when tackling the WinoBias pronoun resolution task.
11/01/2021:
Question and Answer Test-Train Overlap in Open-Domain Question Answering Datasets has been accepted at EACL 2021. If you’re doing open domain QA, evaluate your model on our test sets to see whether it generalizes, or is just memorizing the training set. Get the data here
07/01/2021:
Complex Query Answering with Neural Link Predictors, our state-of-the-art approach for answering complex queries on large and incomplete Knowledge Graphs, will appear at ICLR 2021 as an Oral – top 2% of all publications!

News Archive →

People

Sebastian Riedel

Professor

Sebastian Riedel

Sebastian works in NLP and Machine Learning. He is particularly interested in helping machines to read more accurately by leveraging...

Pontus Stenetorp

Lecturer

Pontus Stenetorp

Pontus works somewhere in the intersection between Natural Language Processing and Machine Learning. He is particularly interested in...

Tim Rocktäschel

Affiliated Faculty (Lecturer)

Tim Rocktäschel

Reinforcement Learning – Deep Learning – Natural Language Processing

Tim’s work focuses on developing agents that are...

Ed Grefenstette

Honorary Reader

Ed Grefenstette

Ed is interesting in teaching machines to understand and communicate using language (formal and natural), and in both neural and symbolic...

Pasquale Minervini

Senior Research Fellow, Principal Investigator for H2020 CLARIFY

Pasquale Minervini

Pasquale is interested in Machine Learning and Reasoning. He is the Principal Investigator of the CLARIFY H2020 project on the UCL end.

Linqing Liu

PhD Student

Linqing Liu

Linqing is a first year PhD student with broad interests in NLP and Machine Learning. She is currently working on question answering.

Matko Bošnjak

PhD Student

Matko Bošnjak

Matko’s interests include both natural and unnatural language processing, and their interplay. Specifically, he’s enjoying...

Max Bartolo

PhD Student

Max Bartolo

Max’s current research focuses on natural language processing, with particular interest in question answering and machine reasoning.

Maximilian Mozes

PhD Student

Maximilian Mozes

Maximilian’s research focuses on the intersection of adversarial machine learning and natural language processing.

Mikayel Samvelyan

PhD Student

Mikayel Samvelyan

Mikayel’s interests lie in reinforcement learning, natural language processing and multi-agent systems.

Minqi Jiang

PhD student

Minqi Jiang

Minqi studies how decision-making agents can make use of compositional structures, like language, to learn more generalizable behaviors.

Patrick Lewis

PhD Student

Patrick Lewis

Patrick is a first year PhD student, interested in Transfer Learning, Machine Reading and leveraging world knowledge to improve predictions...

Tom Crossland

PhD Student

Tom Crossland

Tom is an astrophysicist working with the MR group and the Mullard Space Science Laboratory, interested in Machine Learning applications to...

Yao Lu

PhD Student

Yihong Chen

PhD Student

Yihong Chen

Yihong is interested in almost everything, currently working on methods that empower efficient learning of symbols. Welcome any messages...

Yuxiang Wu

PhD Student

Yuxiang Wu

Yuxiang is a third-year PhD student, interested in Question Answering, Knowledge Base, and other knowledge related tasks.

Zhengyao Jiang

PhD Student

Zhengyao Jiang

Zhengyao is interested in topics about reinforcement leanring, relational inductive bias, and bridging symbolic reasoning and deep learning....

Alastair Roberts

Visiting Researcher

Alastair Roberts

Alastair’s interests lie in natural language processing & machine learning. His current work is focused on question answering.

Alumni

Andreas Vlachos

Now a senior lecturer at University of Cambridge

Johannes Welbl

Now a Research Scientist at DeepMind

Luke Hewitt

Now a PhD student at MIT

Gerasimos Lampouras

Now a research associate at University of Sheffield

Saku Sugawara

Now back to being a Ph.D. student at the University of Tokyo.

Saku Sugawara

Former Visiting PhD Student

Sonse Shimaoka

Now a master student at Tohoku University

Zhao Zhang

Now back to being a PhD student at the Chinese Academy of Sciences.

Zhao Zhang

Former Visiting PhD Student

Guillaume Bouchard

Now a Research Manager at Facebook

Thomas Demeester

Now a post-doc at University of Ghent

Jason Naradowsky

Now a research scientist at Preferred Networks (PFN)

Théo Trouillon

Now back to being a PhD student at Xerox Research Centre Europe

Marzieh Saeidi

Now a Research Scientist at Facebook

Isabelle Augenstein

Now an associate professor at University of Copenhagen

Naoya Inoue

Now an assistant professor at Tohoku University

Tim Dettmers

Now a PhD student at University of Washington

V. Ivan Sanchez

Now an NLP researcher at Lenovo

Andres Campero

Now back to being a PhD student at MIT

Andres Campero

Former Visiting PhD Student

Takuma Yoneda

Now a student at Toyota Technological Institute at Chicago

Georgios Spithourakis

Now a ML engineer at PolyAI

Publications

PAQ: 65 Million Probably-Asked Questions and What You Can Do With Them

Patrick Lewis, Yuxiang Wu, Linqing Liu, Pasquale Minervini, Heinrich Küttler, Aleksandra Piktus, Pontus Stenetorp, Sebastian Riedel. ArXiv, 2021.

Complex Query Answering with Neural Link Predictors

Erik Arakelyan, Daniel Daza, Pasquale Minervini, Michael Cochez. ICLR (Outstanding Paper Award), 2021.

Pretrained Language Models for Biomedical and Clinical Tasks: Understanding and Extending the State-of-the-Art

Patrick Lewis, Myle Ott, Jingfei Du, Veselin Stoyanov. In ClinicalNLP2020 @ EMNLP2020.

Don't Read Too Much into It: Adaptive Computation for Open-Domain Question Answering

Yuxiang Wu, Sebastian Riedel, Pasquale Minervini, Pontus Stenetorp. In EMNLP, 2020.

Beat the AI: Investigating Adversarial Human Annotation for Reading Comprehension

Max Bartolo, Alastair Roberts, Johannes Welbl, Sebastian Riedel, Pontus Stenetorp. In TACL, 2020.

Answering Complex Open-Domain Questions with Multi-Hop Dense Retrieval

Wenhan Xiong, Xiang Lorraine Li, Srini Iyer, Jingfei Du, Patrick Lewis, William Yang Wang, Yashar Mehdad, Wen-tau Yih, Sebastian Riedel, Douwe Kiela, Barlas Oğuz. ICLR 2021, 2020.

View All →

Datasets

PAQ: 65 Million Probably-Asked Questions and What You Can Do With Them

Patrick Lewis, Yuxiang Wu, Linqing Liu, Pasquale Minervini, Heinrich Küttler, Aleksandra Piktus, Pontus Stenetorp, Sebastian Riedel.

AdversarialQA (from Beat the AI)

In this work we investigate this annotation methodology and apply it in three different settings, collecting a total of 36,000 samples with progressively stronger models in the annotation loop. This allows us to explore questions such as the reproducibility of the adversarial effect, transfer from data collected with varying model-in-the-loop strengths, and generalisation to data collected without a model. We find that training on adversarially collected samples leads to strong generalisation to non-adversarially collected datasets, yet with progressive performance deterioration with increasingly stronger models-in-the-loop. Furthermore, we find that stronger models can still learn from datasets collected with substantially weaker models-in-the-loop. When trained on data collected with a BiDAF model in the loop, RoBERTa achieves 39.9F1 on questions that it cannot answer when trained on SQuAD - only marginally lower than when trained on data collected using RoBERTa itself (41.0F1).

Max Bartolo, Alastair Roberts, Johannes Welbl, Sebastian Riedel, Pontus Stenetorp.

KILT: a Benchmark for Knowledge Intensive Language Tasks

KILT is a resource for training, evaluating and analyzing NLP models on Knowledge Intensive Language Tasks. KILT has been built from 11 datasets representing 5 tasks. All these datasets have been grounded in a single pre-processed wikipedia dump, allowing for fairer and more consistent evaluation and enabling new task setups such as multitask and transfer learning with minimal effort. KILT also provides tools to analyze and understand the predictions made by models, as well as the evidence they provide for their predictions.

Fabio Petroni, Aleksandra Piktus, Angela Fan, Patrick Lewis, Majid Yazdani, Nicola De Cao, James Thorne, Yacine Jernite, Vassilis Plachouras, Tim Rocktäschel, Sebastian Riedel.