2026
- 08/10/2026:
Watch this space! We’re busy catching up on everything the group has been up to since 2023.
2023
- 10/07/2023:
Our paper, What the DAAM: Interpreting Stable Diffusion Using Cross Attention, has won a Best Paper Award at ACL 2023! Congrats to authors Linqing and Pontus!
2022
- 18/05/2022:
Our work Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity has been selected as an outstanding paper at ACL 2022!
- 26/03/2022:
The call for participation for the Shared Task at the DADC Workshop co-located with NAACL ‘22 in Seattle is now live! We have three fantastic tracks for you to participate in. Sign up here!
- 19/03/2022:
Additional resources from our work on Improving Question Answering Model Robustness with Synthetic Adversarial Data Generation at EMNLP 2021 are now available! We are releasing a collection of synthetically-generated adversarial QA pairs and related resources as well as the models used to generate the questions.
- 28/02/2022:
Our AAAI 2022 tutorial, On Explainable AI: From Theory to Motivation, Industrial Applications, XAI Coding & Engineering Practices, was an outstanding success, with more than 600 attendees – check it out! Congratulations Pasquale and collaborators!
- 20/01/2022:
AdversarialQA is currently the 3rd most downloaded QA dataset on Huggingface 🤗 Datasets right after the benchmark SQuADv1.1 and SQuADv2!
- 04/01/2022:
Our proposal for the First Workshop on Dynamic Adversarial Data Collection has been accepted! See you at NAACL ‘22 in Seattle!
2021
- 01/12/2021:
Pasquale is joining the School of Informatics at the University of Edinburgh as a faculty member, and is currently recruiting PhD students! If you would like to work with an ICLR 2021 Outstanding Paper Award winner in an amazing NLP department, make sure to get in touch with him!
- 23/11/2021:
Relation Prediction as an Auxiliary Training Objective for Improving Multi-Relational Graph Representations is now SoTA on two OGB link property prediction datasets ogbl-biokg and ogbl-wikikgv2. Check out how to reproduce these results here!
- 04/10/2021:
Our paper Relation Prediction as an Auxiliary Training Objective for Improving Multi-Relational Graph Representations will be presented at AKBC 2021! We propose to incorporate relation prediction into the 1vsAll objective for training better knowledge graph embeddings. Check out our code here.
- 15/09/2021:
Maximilian presented his research on adversarial examples in NLP at the UCL AI Centre seminar series. The recording is available here.
- 01/08/2021:
Our paper Implicit MLE: Backpropagating Through Discrete Exponential Family Distributions will be presented at NeurIPS 2021! We propose a way to back-propagate through algorithms – check out our video, as well as Yannic Kilcher’s video on our paper! Congrats Pasquale and collaborators!
- 23/04/2021:
Our paper, Question and Answer Test-Train Overlap in Open-Domain Question Answering, has won a Best Paper award at EACL 2021! Congrats to authors Patrick, Pontus and Sebastian.
- 01/04/2021:
Our paper Complex Query Answering with Neural Link Predictors, based on Erik’s MSc thesis supervised by Pasquale, won an Outstanding Paper Award at ICLR 2021!
- 15/02/2021:
PAQ: 65 Million Probably-Asked Questions and What You Can Do With Them has been released on ArXiv! Two times faster and more accurate QA models by using a huge collection of 65M automatically-generated QA-pairs. This leads to a flexible QA system that can be optimised for memory, speed or accuracy. Check out the PAQ data here, code and models coming soon!
- 12/01/2021:
Answering Complex Open-Domain Questions with Multi-Hop Dense Retrieval has been accepted at ICLR 2021. This simple recursive retrieval gets state-of-the-art results without requiring additional resources such as hyperlink networks.
- 11/01/2021:
Question and Answer Test-Train Overlap in Open-Domain Question Answering Datasets has been accepted at EACL 2021. If you’re doing open domain QA, evaluate your model on our test sets to see whether it generalizes, or is just memorizing the training set. Get the data here
- 11/01/2021:
Stereotype and Skew: Quantifying Gender Bias in Pre-trained and Fine-tuned Language Models has been accepted at EACL 2021. We propose two intuitive metrics, skew and stereotype, that quantify and analyse the gender bias present in contextual language models when tackling the WinoBias pronoun resolution task.
- 07/01/2021:
Complex Query Answering with Neural Link Predictors, our state-of-the-art approach for answering complex queries on large and incomplete Knowledge Graphs, will appear at ICLR 2021 as an Oral – top 2% of all publications!
2020
- 11/11/2020:
Don’t Read Too Much into It: Adaptive Computation for Open-Domain Question Answering will appear at EMNLP 2020. We propose an adaptive computation method that significantly reduces the computational cost of ODQA systems while retraining a similar performance.
- 09/11/2020:
Beat the AI: Investigating Adversarial Human Annotation for Reading Comprehension has been accepted at TACL 2020, and we’ll be presenting it at EMNLP 2020. We’ve also publicly released the dataset and you can try to beat our best model and submit to our online leaderboard through Dynabench.
- 28/09/2020:
Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks has been accepted at NeurIPS 2020. We’ve also released a blog post and released the code as part of the HuggingFace ecosystem. Check out a demo of RAG here
- 04/09/2020:
KILT: a Benchmark for Knowledge Intensive Language Tasks is now available on ArXiv! KILT is set of tools and data to accelerate research progress on open domain and knowledge intensive NLP, including open domain QA, fact checking, relation extraction and entity linking. KILT will make your work easier, more comparable and reproducible, and allow researchers to share components more easily.
check out the code, and leaderboard, and HuggingFace integrations. - 07/08/2020:
Question and Answer Test-Train Overlap in Open-Domain Question Answering Datasets is now on ArXiv! Do you use NaturalQuestions, TriviaQA, or WebQuestions? It turns out 60% of test set answers are also in the train set. More surprising, 30% of test questions have a close paraphrase in the train set. We look at what this means for models. Annotations and code available here
- 06/07/2020:
R4C: A Benchmark for Evaluating RC Systems to Get the Right Answer for the Right Reason was presented at ACL 2020. It previously won the “Best Linguistic Resource” award at the 26th annual meeting of the Japanese Association for Natural Language Processing.
- 24/06/2020:
“How Context Affects Language Models’ Factual Predictions” has been been awarded Best Paper at AKBC 2020!
- 09/06/2020:
Pasquale’s paper Learning Reasoning Strategies in End-to-End Differentiable Proving will appear at ICML 2020! We propose a neuro-symbolic reasoning model that can learn to dynamically select and generate rules conditioned on the goal during their reasoning process via gradient-based optimisation.
- 01/06/2020:
UCL NLP members authored two chapters in Knowledge Graphs for eXplainable Artificial Intelligence: Foundations, Applications and Challenges, edited by IOS Press!
- 22/05/2020:
Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks is now up on ArXiv. State-of-the-art open domain Question Answering, and more factual and specific generation, by combining the power of pre-trained dense retrieval and seq2seq models.
- 01/05/2020:
Our paper on how to improve unsupervised factual predictions using retrieved context,“How Context Affects Language Models’ Factual Predictions” has been accepted at AKBC2020!
- 05/04/2020:
Make Up Your Mind! Adversarial Generation of Inconsistent Natural Language Explanations to be presented at ACL 2020! In this collaboration with Oxford and DeepMind researchers we show that you can invert NLP models and make them produce mutually inconsistent explanations.
- 04/04/2020:
MLQA: Evaluating Cross-lingual Extractive Question Answering has been accepted at ACL 2020! Lets make sure question answering works in all languages. Get started here
- 01/03/2020:
Towards machine-assisted meta-studies: the Hubble constant appeared in the Monthly Notices of the Royal Astronomical Society.
- 25/02/2020:
Our new work in collaboration with NYU and FAIR on Unsupervised Question Decomposition for multi-hop QA is now up on ArXiv. Break down complex questions into a list of simple questions without any labelled data. Strong, robust results on HotPotQA, and don’t need any supporting fact annotation.
- 07/02/2020:
Assessing the Benchmarking Capacity of Machine Reading Comprehension Datasets was presented at AAAI 2020.
- 01/02/2020:
Differentiable Reasoning on Large Knowledge Bases and Natural Language will be presented at AAAI 2020 as an oral (4.5% acceptance rate)! We show how neuro-symbolic reasoning can be scaled to very large knowledge bases and text corpora.
2019
- 16/10/2019:
We’re releasing a new cross-lingual Question Answering dataset, MLQA: Evaluating Cross-lingual Extractive Question Answering Check out the dataset here
- 03/09/2019:
Language Models as Knowledgebases? accepted at EMNLP 2019! Code for the paper’s experiments, and the LAMA probe are available here
- 13/06/2019:
NLProlog: Reasoning with Weak Unification for Question Answering in Natural Language accepted at ACL 2019! Code and synthetic training data is available here
- 13/06/2019:
Unsupervised Question Answering by Cloze Translation accepted at ACL 2019! Code and synthetic training data will be available here
2018
- 28/09/2018:
Interpretation of Natural Language Rules in Conversational Machine Reading accepted at EMNLP 2018 - CodaLab challenge are now LIVE at sharc-data.github.io!
- 26/09/2018:
Wronging a Right: Generating Better Errors to Improve Grammatical Error Detection accepted at EMNLP 2018 - code available at this link!
- 22/08/2018:
Cape achieves new SoTA on the TriviaQA Wiki dataset codalab Leaderboard. More details can be found here. Cape is super easy to use, extend and integrate into all kinds of software!
- 19/08/2018:
Adversarially Regularising Neural NLI Models to Integrate Logical Background Knowledge accepted at CoNLL 2018 - code available at this link!
- 11/08/2018:
We are proud of our 2nd place in the EMNLP 2018 FEVER shared task (fever.ai) thanks to the amazing work of @takuma_ynd!
- 06/08/2018:
We will be giving a tutorial on Machine Reading at UAI 2018!
- 27/05/2018:
Jack the Reader – A Machine Reading Framework accepted at ACL, System Demonstrations track!
- 21/05/2018:
Numeracy for Language Models: Evaluating and Improving their Ability to Predict Numbers accepted at ACL!
- 11/05/2018:
Behavior Analysis of NLI Models: Uncovering the Influence of Three Factors on Robustness accepted at HLT-NAACL!
- 07/01/2018:
Convolutional 2D Knowledge Graph Embeddings accepted at AAAI!
2017
- 02/10/2017:
The 6th Workshop on Automated Knowledge Base Construction (AKBC 2017) returns to NIPS: submit your papers by October 21st!
- 04/09/2017:
End-to-end Differentiable Proving accepted at NIPS!
- 26/07/2017:
Adversarial Sets for Regularising Neural Link Predictors accepted at UAI!
- 23/07/2017:
Programming with a Differentiable Forth Interpreter accepted at ICML!
- 13/06/2017:
A Supervised Approach to Extractive Summarisation of Scientific Papers, a paper based on Ed Collins’ MEng thesis, accepted at CoNLL!
- 24/04/2017:
Frustratingly Short Attention Spans in Neural Language Modeling, a paper based on Michal Daniluk’s MSc Machine Learning project, accepted to ICLR! Michal also received the MSc Machine Learning Programme Director’s Award (2015-2016) for Outstanding Project Report (Second Place)
- 11/04/2017:
SemEval 2017 Science task description paper preprint now available online.
- 10/04/2017:
Tim Rocktäschel is awarded a Google Ph.D. Fellowship in Natural Language Processing
- 07/04/2017:
Multi-Task Learning of Keyphrase Boundary Classification accepted at ACL!
- 03/04/2017:
Neural Architectures for Fine-grained Entity Type Classification wins outstanding paper award at EACL!
- 03/04/2017:
2 papers by our group accepted at EACL!
- 24/02/2017:
We co-organised a Poetry AI workshop at UCL, at which humans acted as a neural network to generate poetry. Slides of the event are available here.
- 21/02/2017:
Wired article discussing our project on machine reading of scientific publications and how it could aid peer review
- 09/02/2017:
SemEval 2017 Science results are announced. Congratulations to the winning teams, s2_end2end, MayoNLP and MIT!
- 28/01/2017:
We are co-organising the first workshop for women and underrepresented minorities in NLP (WiNLP) at ACL 2017. Consider participating!
- 12/01/2017:
We are co-organizing the SemEval 2017 Task 10: Extracting Keyphrases and Relations from Scientific Publications (ScienceIE). Consider participating!
2016
- 01/11/2016:
emoji2vec: Learning Emoji Representations from their Description, a paper based on Ben Eisner’s UCLMR internship, won the best paper award at SocialNLP 2016!
- 12/08/2016:
Defining Words with Words: Beyond the Distributional Hypothesis was awarded the best proposal award at RepEval!
- 09/07/2016:
We are co-organizing a workshop on Neural Abstract Machines & Program Induction (NAMPI) at NIPS 2016. Consider submitting a paper!