Title: Simplicity bias leads to amplified performance disparities. Abstract: The simple idea that not all things are equally difficult has surprising implications when applied in a fairness context.
In this work we explore how "difficulty" is model-specific, such that different models find different parts of a dataset challenging.
When difficulty correlates with group information, we term this difficulty disparity.
Drawing a connection with recent work exploring the inductive bias towards simplicity of SGD-trained models, we show that when such a disparity exists, it is further amplified by commonly-used models.
We quantify this amplification factor across a range of settings aiming towards a fuller understanding of the role of model bias. We also present a challenge to the simplifying assumption that ``fixing'' a dataset is sufficient to ensure unbiased performance.