they behave reasonably as expected. But, even in those cases, understanding how the model works can sometimes be of great help. Back in 2015, Google was embarrassed by its Photos service labeling pictures of black people as “gorillas” [@simonite2018when]. Why was the image recognition software doing so? Certainly not because it is racist and deliberately meant to offend people, but rather because it was making the wrong correlations between the features extracted from the pictures. Google’s answer to the problem was to ban the words “gorilla”, “chimp”, “chimpanzee”, and “monkey” from the Photos lexicon altogether, avoiding people (and monkeys) being assigned such labels, and this workaround remains in place, almost three years on. Although there may be many different reasons for Google’s approach towards the issue, it becomes clear that the complexity of the machine learning algorithm employed for image recognition may have prevented a proper quick fix, that is, it wasn’t possible to quickly interpret the model, identify, and adjust its malfunctioning parts in a timely manner.

However, misclassifications can cause much more than annoyance. The greatest importance of model interpretability rests on decision-making systems, whose outputs can have a material impact on the lives of individuals. Artificial intelligence 