Stereotype and Skew: Quantifying Gender Bias in Pre-trained and Fine-tuned Language Models has been accepted at EACL 2021. We propose two intuitive metrics, skew and stereotype, that quantify and analyse the gender bias present in contextual language models when tackling the WinoBias pronoun resolution task.