Research and practice in clinical neurosciences often involve cognitive assessment. However, this has traditionally used a nomothetic approach, comparing the performance of patients to normative samples. This method of defining abnormality places the average test performance of neurologically healthy individuals at its center. However, evidence suggests that neurological ‘abnormalities’ are very common, as is the diversity of cognitive abilities. The veneration of central tendency in cognitive assessment, i.e., equating typicality with healthy or ideal, is, I argue, misguided on neurodiversity, bio-evolutionary, and cognitive neuroscientific grounds. Furthermore, the use of average performance as an anchor point for normal performance is unreliable in practice and frequently leads to the mischaracterization of cognitive impairments. Examples are explored of how individuals who are already vulnerable for socioeconomic reasons can easily be over-pathologized. At a practical level, by valuing diversity rather than typicality, cognitive assessments can become more idiographic and focused on change at the level of the individual. The use of existing methods that approach cognitive assessment ideographically is briefly discussed, including premorbid estimation methods and informant reports. Moving the focus away from averageness to valuing diversity for both clinical cognitive assessments and inclusion of diverse groups in research is, I argue, a more just and effective way forward for clinical neurosciences.