**Monographs in the Psychology of Education** *Series Editor:* Daniel H. Robinson

# Hoben Thomas

# Sex Differences in Reading and Math Test Scores of Children

A Heterodoxical Model

# **Monographs in the Psychology of Education**

**Series Editor**

Daniel H. Robinson , The University of Texas at Arlington, Arlington, TX, USA

The purpose of the series, Monographs in the Psychology of Education, is to examine behavioral, cognitive, and academic assessments and interventions – with emphasis on the feasibility and reproducibility of research methods and conclusions – to determine their real-world effectiveness for students in classrooms worldwide. Series volumes provide critical reviews of evidence-based methods and may be authored or edited (multiple contributors). The books in this series serve as a critical resource for researchers, professors, and graduate students as well as clinicians, practitioners, and policy in school and clinical child psychology, special and general education, social work and all interrelated disciplines.

Hoben Thomas

# Sex Differences in Reading and Math Test Scores of Children

A Heterodoxical Model

Hoben Thomas Department of Psychology Pennsylvania State University University Park, PA, USA

ISSN 2662-7574 ISSN 2662-7582 (electronic) Monographs in the Psychology of Education ISBN 978-3-031-41271-4 ISBN 978-3-031-41272-1 (eBook) https://doi.org/10.1007/978-3-031-41272-1

© The Editor(s) (if applicable) and The Author(s) 2024. This book is an open access publication.

**Open Access** This book is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this book are included in the book's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the book's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use.

The publisher, the authors, and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, expressed or implied, with respect to the material contained herein or for any errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This Springer imprint is published by the registered company Springer Nature Switzerland AG The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland

Paper in this product is recyclable.

*To the love of my life, The Reverend Patricia Menné Thomas, whom I lost at the beginning of the pandemic to dementia.*

# **Preface**

Conventional wisdom is that the gender-gap or mean difference in math test scores favoring boys is negligible, vanishing, and on a path toward closure given appropriate societal adjustments. Furthermore, stereotypes and bias are the most frequently cited reasons for boys slightly larger math mean test scores. In fact, the mean gap favoring boys in math has existed since the earliest onset of achievement testing in the USA. It was evident in Stone's 1908 doctoral dissertation data and in New York City's massive math testing program in schools starting around 1910. The gap exists worldwide in most developed countries on the PISA test, as well as on the "Nation's Report Card" or NAEP tests in the USA. Of far larger size, but of nearly negligible interest, is the mean gap dramatically favoring girls in reading. In fact, girls are so dominant in reading it is difficult to find *any* studies reporting boys have larger test score means, at least in developed countries. That reading has favored girls in test score mean has been known at least since Gray's dissertation on reading tests in 1917. The convolution or finite mixture model in this book explains test score sex differences for both tasks. Interestingly, the mean differences, for both reading and math, are the *smallest* of test score sex differences. The test score variance differences, with larger variance for boys on both tasks, simply dwarf the mean differences for both tasks. The focus for more than 40 years has been on the *smaller* test score sex differences, the mean differences—arguably the wrong differences on which to focus. The theory, which is marginally more complicated than the probability model for the flip of two coins, explains these mean and variance test score differences for both tasks. The theory does so by modeling a biological mechanism which has been for decades out of favor, is nearly universally ignored, and is often disparaged. The monograph's spirit is well captured by the title of mathematician Keith Devlin's book, *The Math Gene*.

University Park, PA, USA Hoben Thomas June 2023

# **Acknowledgments**

Thomas P. Hettmansperger, a long-time friend, colleague, and often coauthor is to be thanked for his help. His fingerprints are everywhere on the analytical developments. Wesley Jamison provided helpful guidance on overall organization, and especially on Chap. 1. Derek S. Young provided helpful comments as well. David R. Hunter read an early version of this work. His thoughtful comments were the basis for some rewriting and additional analysis. Finally, sincere thanks goes to Tuomas Pekkarinen, of the Helsinki School of Economics who provided means and standard deviations for the 2003 PISA scores, thereby allowing substantial additional analyses using international data. Sadly, he died in November 2022.

# **Contents**

# **Chapter 1 Focus on Math and Reading Test Score Inequalities**

Worldwide there has been recognition that boys' and girls' reading and math test score distributions are different. What has not been recognized is precisely how these distributions differ.

Large-scale achievement testing in the U.S. started around 1897 by J. M. Rice [1]. By 1902, the results of Rice's large-scale U.S. elementary school arithmetic reasoning tests for children were available [2]. By 1908, following C. W. Stone's efforts [3], it was clear that sixth grade boys and girls have different achievement test score distributions on each of two math tests Stone constructed. Stone's data likely provide the first evidence of sex differences in math test score distributions in the U.S.A. and perhaps globally as well. His data are typical of many other more recent examples, with small mean differences favoring boys on both tests. Figure 1.1 reveals, for the first time, data from Stone's fundamental test which shows the smaller mean difference of his two tests. Stone was surely unaware of this mean difference however and Fig. 1.1 which is a kernel density estimation plot, a graph simply unimaginable for Stone. His tools are paper and pencil. Consequently, Stone computed no quantities and no graph from these data. He likely could only stare at the tabled data on 250 boys and 250 girls he provided for each of two tests. He mused at the apparent variabilities he saw. "Doubtless the most noteworthy feature of these tables is the wide variability of achievements [3, p. 32]."

Stone's data, and Fig. 1.1 which shows that the boys' distribution is slightly right tilted relative to the girls' distribution, make clear that today's emphasis on the "gender gap," the small mean difference favoring boys in math, is nothing new. It was evident at the dawn of achievement testing, well more than 100 years ago. This fact also suggests the gap is not going away anytime soon. Early investigators may have had an inkling of such distributional differences but were unable to explicitly quantify their beliefs, which was certainly true for Stone. That distributional differences were evident at the outset of testing clearly weakens any contemporary claims that sex differences in math are the consequence of policy decisions or evolving social or environmental forces.

© The Author(s) 2024

H. Thomas, *Sex Differences in Reading and Math Test Scores of Children*, Monographs in the Psychology of Education, https://doi.org/10.1007/978-3-031-41272-1\_1

**Fig. 1.1** Kernel density estimation plot of the sixth grade test scores of 250 boys and 250 girls on Stone's (1908) fundamental math test. The boys' mean is 3.193, and the girls' mean is 3.026, with corresponding standard deviations of 1.048 and 0.996

Commentary and other table entries Stone provides indicate he was well aware of sex differences in test score variability, albeit assessed with methods now obsolete and evident in small samples of his data. Also, evident in Stone's early data is a pattern of statistical features evident today in math testing studies large and small, in the U.S.A. and worldwide, patterns unrecognized by Stone—with some patterns unrecognized by investigators of sex differences in math even today. There will be more on this matter momentarily.

The situation with reading tests is similar. By 1917, Gray [4] recognized there were reading test score distributional differences between boys and girls, evident in the reading tests he constructed. But like Stone, Gray could not have appreciated the manner by which these test score distributions differed, although he did report girls' sample mean surpassed boys in oral reading in all grades one to eight of the elementary school years. Today, that girls surpass boys in reading mean is a nearly invariant universal global test result, at least in developed countries. And like the pattern of differences evident in Stone's math testing, a similar unrecognized pattern exists in reading testing data as well.

Let *x*¯*b* and *x*¯*g* denote test score sample means of boys and girls, respectively, with *s*2 *b* and *s*2 *g* their corresponding sample variances. The way sex differences are defined today is by effect size *d* with

$$d = \frac{\bar{x}\_b - \bar{x}\_g}{\sqrt{(s\_b^2 + s\_g^2)/2}} \cdot$$

While *d* can be viewed as an empirical method of gauging sex differences, most of the writers treat it as an inferential quantity, in which case the appropriateness of *d*'s use rests on the probability model for *d* or *δ,*

$$
\delta = \frac{\mu\_b - \mu\_\mathcal{g}}{\sigma\_c},
$$

and *μb* and *μg* are the population means for boys and girls, respectively. *σc* is the assumed common population standard deviation. So *d* presumably estimates *δ.*

However, *d* has led nowhere. It has provided no assistance in understanding the origins of sex differences in reading or math or why they persist. Worse, outcomes of *d* meta-analyses are often puzzling. A common math finding is that the mean *d,* that is, *d,*¯ is positive but small and favors boys. For example, in Lakin [7], the overall math *d*¯ = 0*.*077*.* Why these small (scaled) mean differences appear has befuddled investigators [8–10].

Meanwhile, it has long been recognized that typically

$$\mathbf{s}\_b \rhd \mathbf{s}\_{\mathfrak{g}} \text{ or } \mathbf{s}\_{\mathfrak{b}}^2 / \mathbf{s}\_{\mathfrak{g}}^2 > 1$$

holds for both tasks, reading and math. These inequalities have also long perplexed researchers [9, 11–14].

Writers and those with a penchant for *d*-based meta-analyses, as well as many researchers, have slavishly, seemingly lemminglike, taken *d* as the ultimate sex differences scalar index. It has even been claimed *d* is useful for *any* sex differences variable [15, p. 177]. *δ,* the model for *d,* is conceptually inappropriate as the need to report *s*2 *b /s*2 *g* implicitly makes clear. More importantly, *d* is a misleading way of viewing sex differences, at least for math and reading test score data. That is because *d* has directed attention toward mean differences and misdirected attention away from the *far larger* variance differences in data. Once these differences are properly recognized, one is forced to recognize the inadequacies of *d.* More centrally however, it forces a need to rethink how differences between the sexes should be viewed, again at least for reading and math test score data.

The key for understanding is not to focus on scaled sample mean differences, as *d* requires. Rather, focus instead on summary data statistical *inequalities.* In particular, focus on pairs of inequalities. These are the data patterns that must be explained by any creditable framework which is claimed to account for sex differences in math and reading.

Define the following two sets, the inequalities of which may appear familiar, first for math testing and then reading testing, respectively:

$$mo = \{ \bar{x}\_b > \bar{x}\_g \text{ & } s\_b > s\_g \} \text{ and } $$

$$ro = \{ \bar{x}\_g > \bar{x}\_b \text{ & } s\_b > s\_g \}.$$

The inequalities of the following two sets have apparently never before been recognized. Again, the first set is for math testing the second set is for reading

testing:

$$\begin{aligned} mdo &= \{ s\_b^2 - s\_\varrho^2 > \bar{x}\_b - \bar{x}\_\varrho, \text{ given } mo \} \text{ and} \\\\ rdo &= \{ s\_b^2 - s\_\varrho^2 > \bar{x}\_\varrho - \bar{x}\_b, \text{ given } ro \}. \end{aligned}$$

Call these four sets collectively,

$$\mathcal{S} = \{mo, ro, mdo, rdo\}.$$

Read *mo* as "math order," *ro* as "reading order," *mdo* as "math difference order," and *rdo* as "reading difference order." The primary task of this monograph can be easily stated: model the data structures of S.

One's intuition with numbers can often lead one astray, regardless of how technically skilled one might be [16], and there may be repeated instances in the following pages when this may be the case. So, consider

$$
\bar{x}\_b = 152, \bar{x}\_\mathfrak{g} = 149, \mathfrak{s}\_b = 34 + 3, \mathfrak{s}\_\mathfrak{g} = 34.
$$

Relative sizes are easy numerical comparisons, so it is easily seen in this math test example that *mo* is satisfied. However, *mdo* may seem not only a strange comparison but intuitively more challenging as well. One easily observes the difference *sb* − *sg* = 3 which might seem small. But the required difference to consider is *s*2 *b* − *s*2 *g* = 213, which perhaps seems larger than intuitively expected and far larger than *x*¯*b* − ¯*xg*. Small differences in standard deviations can balloon into large differences in variance. There is not a unique way of thinking about the matter, but the numerical values of *sb* and *sg* above were written in the form *w* + *b* and *w*, respectively, a difference independent of *w.* The variance difference *(w* + *b)*2 − *w*2 = *b*2 + 2*bw* increases quadratically in *b* or linearly in *w,* so the example provides perhaps an "intuitive correction" going forward. The example is from Table 2.3, Chap. 2, "The Nation's Report Card" 12th grade 2019 math testing result.

Although it was claimed decades ago that the sex mean differences on cognitive tests were disappearing [17], or at least in more recent decades that the mean sex differences in math have decreased [18], or that the test gaps in math mean are ". . . close to zero in developed countries...[19, p. 1219]," keeping alive perhaps the hope that the small mean sex difference favoring boys in math test scores will soon vanish with suitable societal changes. This hope is simply wishful thinking. The empirical fact remains that, from the perspective of more than a century, the inequalities of S have widely held, and they continue to do so worldwide, in developed countries, with an occasional exception, but only for math. Furthermore, the mean gap is not even the largest sex differences gap as S makes clear. Moreover, at least in the U.S.A., the small math mean difference disadvantaging girls has garnered nearly all the attention. By contrast, girls' mean advantage in reading is

*multiples larger* than boys' mean advantage in math. Yet by comparison, the interest in sex differences in reading has been little more than a footnote.

The inequalities of S are so ubiquitous in the literature, as will be demonstrated in Chap. 2, they may be said to *characterize* the structure of the literature for children's reading and math test score summary statistics. The data for Stone's fundamental test displayed in Fig. 1.1 satisfies *mo.* His reasoning test satisfies *mdo*.  Importantly, they tightly constrain the class of models that accounts for them in a coherent way. Happily, as it turns out, accounting for these inequalities also provides the basis for explaining other widely recognized empirical facts long unaccounted for in the reading and math observational test score sex differences literature.

This monograph has four main goals:

Indeed, boys are far more overrepresented in the bottom of the reading test distribution than they are overrepresented in the top part of the math test distribution, as will be seen later. The goal is to address these puzzles in a coherent way within a suitable model.

4. The final goal is to provide model parameter estimates, thus illuminating, with numerous associated graphics, the model's plausibility.

While the focus here is narrow and confined, namely the reading and math *observational* test score literature, that literature is substantial. It includes reported test standardization data, large sample surveys, and observational studies of children's test scores obtained in classroom settings. That girls achieve higher grades than boys in the classroom, a success that does not seem to translate to other settings is a wellrecognized and important fact [20]. However, it is an example of a matter that is outside the neighborhood of focus here.

Efforts to address sex differences, especially in math, nearly always cast a far larger net. For Ceci and Williams, 2010 [5], test score differences are only a small part of their far larger agenda. Their primary focus is why there are relatively few women in engineering, computer science, and other math-oriented occupations. These issues are certainly compelling but are not addressed here. However, some

findings from the present analysis seem to have relevance for understanding sex differences observed in larger social contexts. When this is recognized, the possible implications will be noted.

The path taken here is different and likely to be unfamiliar.  No conventional statistical approaches are featured here. This announcement should not surprise. That is because no conventional approach has been able to coherently explain the inequalities of concern nor the many perplexing empirical facts of data. The evidence is clear: after decades of attention, there has been no compelling explanation, using a myriad of conventional statistical approaches, why the most fixated feature of attention, which has essentially defined interest in sex differences in math, the mean gender gap, exists or why it persists. Furthermore, the task is more complex than explaining the mean difference, the focus of nearly all research. The much larger variance differences must be addressed as well. And this goal must be achieved not just for math sex differences but also for reading sex differences.

Consequently, a new framework must be provided. It is developed below. That new framework is a probability model called model Y. It is developed in Chap. 4 and is the framework within which all sex differences in test score distributions are viewed. The model forms the only basis for interpretation of sex differences throughout. In addition, of course, Y requires a new parameter estimation or statistical procedure which is provided. One feature of the approach is that there is no appeal to covariates as is commonly the case. Covariates may be important, but they play no role in the analyses that follow.

In addition, the path forward is likely to be, for some readers at least, unpopular. It would certainly be preferable if this were not the case. However, to explain test score sex differences coherently, there appears no choice: a genetical theory of X-linkage appears to be the only viable perspective available, and when suitably modeled, it is remarkably parsimonious. It requires just three parameters, the same number as *δ,* so it might be hoped readers can warm to the idea. Such an idea was decades ago mocked or ignored, an attitude that persists today, at least in some quarters. However, a model so based accounts remarkably well for the sex differences observed, as well as the inequalities. To anticipate matters, the empirical inequalities of S are just *expectations* under the model proposed.

Genetical models have never been widely popular, at least in most psychological circles, and their role in explaining behaviors is often grudgingly acknowledged if acknowledged at all. However, it is important to recognize that genetical models have never been proposed to explain behaviors when other coherent explanations were available. As Ceci and Williams 2010 note, advances in understanding ". . . comes from free and open debate in which all sides present their best evidence and no one is excoriated for arguing the unpopular side [5, p. 219]."

This plea for openness was expressed more than a decade ago. Is the current psychological science environment open to unpopular perspectives? Staddon [21] certainly does not think so. He castigates individuals, universities, funding institutions, and more for their lack of openness in science to ideas and intellectual

diversity. Staddon compares the current situation, especially with respect to understanding group cognitive differences, with Lysenkoism [21, p. 167]. In fact, the spirit of Lysenkoism is now seen as threatening *all of science* [22].

Can one, nonetheless, expect a spirit of openness to ideas and conceptualizations regarding unexplained sex differences in reading and math test scores? One certainly hopes so especially for what follows here. But the overall outlook appears cloudy at best if the path forward is to report only the most austere empirical facts.  Yet it is written in a remarkably cautious style.  They strive to avoid any scintilla that would suggest mechanisms producing such differences by stressing ". . . we assiduously avoid discussing particular causal explanations of gender/sex differences and similarities or indicating our personal preferences for any theories of causation [23, p. 1340])." And some readers may view their discussion as seeming to apologize for even pointing out that there *are* sex differences, perhaps fearing they might antagonize some readers (in particular, journal editors). They write near their close ". . . our insights that sex/gender differences can be simultaneously large and small might appear to some readers to threaten gender equality [23, p. 1354]." Should this last quote be understood to mean simply recognizing that to "cook meat on the grill" is a more masculine than feminine task is likely to pose a threat to gender equality? One can guess at the ramifications for the study of sex differences should this writing style become the template for the future. It would surely curtail a core goal of the scientific process, causal explanations, or as Judea Pearl would put it: Why? [24].

Doubtlessly, anyone who has studied the sex differences literature has observed the inequality pairs in *mo* and *ro.* They are simply too dominant in the literature to avoid notice. The surprise is that the *mdo* and *rdo* inequalities widely hold. In fact, the size of the sample variance differences usually *far* exceeds the sample mean differences. It is difficult to imagine any other fact of empirical data that would signal a more robust rejection of conventional perspectives on how sex differences are viewed, namely that mean differences capture the essence of distributional differences, at least for math and reading sex differences test scores. These empirical facts, to be surveyed in Chap. 2, would seem to provide overwhelming conceptual problems for those who claim the sex differences in math are vanishing, ignorable, or may not exist at all [25].

Small sex differences, in sample means, especially in math, have enjoyed the attention while variance differences have largely been ignored, or at least from a substantive perspective, discounted. The reasons for this emphasis on means and not variances are not difficult to discern and involve both technical and psychological considerations. One psychological reason is that it is easy to *think* in terms of group differences as simply realizations of visual images likely to be familiar: two identically shaped typically normal-like or "bell-shaped" distributions shifted slightly apart, consequently differing only in mean. This is an example of what are

termed *location-shift* models; the intuition is compelling. *δ* is a scaled version of a location-shift model. Where variances and their differences are concerned, these intuitive advantages seem to disappear.

The dominance of effect size *d* as a sex difference defining assessment tool has certainly reinforced this mean difference perspective. Of course, the widely used expression "gender gap" reinforces this cognitive perspective. Sometimes variance parameters are called *nuisance parameters* [26, p. 3] signaling the perceived unimportance of variances. The most commonly used statistical models often assume equal population variances, with a corresponding central attention on population means. Analytically unequal population variances have been historically troublesome for statistical theory and consequently applied researchers as well. Some of these issues seem practically unimportant now with computing replacing analysis, as with the bootstrap [27].

Still, thinking intuitively about the mechanisms that might produce variance differences in behavior seems cognitively difficult. Yet sample variance differences are by far the dominate feature of reading and math test score sex difference summary statistics. This is the reality, and this reality has been unchanged for decades. To repeat, any sex differences theory must account for variance differences and the mean differences as well. This fact was recognized by Feingold in 1992 [28] who argued, correctly, that to understand cognitive sex differences both means and variances must *jointly* be considered. But efforts to do so have not subsequently appeared.

Now, fully 30 years later appears an article with title "Joint Consideration of Means and Variances Might Change the Understanding of Etiology" [29]. The article is a welcome addition. But it focuses largely on adoption or twin studies within the context of conventional behavioral genetics approaches and does not consider its larger conceptual importance. No new framework is proposed. The place to start is at the model level and perhaps with discrete probability models. That is because in all popular ones, the binomial, negative binomial, Poisson, and geometric, the means and variances are linked through their shared common parameters. Instead, the authors focus at the data analysis level.

The literature review will consider only readily accessible reading and math observational test score literature and focuses on whether the inequalities of S are satisfied or not. Thus, mostly reported are studies which report the key quantities of focus, the sample means, and sample variances (or standard deviations) for both boys and girls. These statistics will be denoted by *V* defined by

$$V = \{\bar{x}\_b, \bar{x}\_g, s\_b, s\_g\}.$$

If sample sizes are also available, *V* will appear with six elements

$$V = \{\bar{\mathfrak{x}}\_b, \bar{\mathfrak{x}}\_{\mathfrak{g}}, \mathfrak{s}\_b, \mathfrak{s}\_{\mathfrak{g}}, n\_b, n\_{\mathfrak{g}}\},$$

where *nb* and *ng* are boys' and girls' sample sizes, respectively.

In all that follows, *V* are the only data of focus. No new data are reported. Unless *V* is explicitly reported, *mdo* and *rdo* cannot be evaluated. If *mdo* holds or *rdo* holds, then *mo* or *ro* must hold as well. However, *mo* or *ro* can hold while the more constraining inequalities of differences, *mdo* or *rdo,* can fail. They can fail simply because of the noise of real data, and probabilistically, they are more likely to do so. Sometimes reported are *d* and *s*2 *b /s*2 *g*, which enables *mo* or *ro* to be evaluated, but without being able to explicitly define the elements of *V ,* in which case *mdo* or *rdo* cannot be evaluated.

Thus, the thrust of the literature review is to illustrate the dominance of the inequalities in several domains. This perspective departs sharply from the current practice which defaults to *d* as the sex difference index, which is often coupled with the belief that small *d*, however consistently signed, or that ratios of *s*2 *b /s*2 *g* however consistently greater than one, are apparently ignorable.

In fact, the consistency of *mo* and *ro* in the observational math and reading test score literature must stand as one of the most remarkably consistent and easily observable empirical findings in the child literature. And the evidence has been but a mouse click away. To anticipate, simply glance at the "Nation's Report Card" data [30] reproduced in Table 2.1 through Table 2.4 in Chap. 2. That *mo* or *ro* widely hold is immediately evident. Less obvious is that the inequalities of *rdo* and *mdo* nearly always hold as well. These results are based on millions of children assessed over decades using evolving but remarkably sophisticated and representative research protocols.

There has long been a dismissive attitude toward small differences in sample values, especially sample means. For example, Miller and Halpern write "Sex differences in average mathematics test performance also decreased during the 1970s to 1980s and have since remained small to *negligible* [18, p. 38, italics added]." In fact, they go further explicitly *defining* cognitive sex differences as mean differences, so variance differences are apparently irrelevant. One would expect *s*2 *b /s*2 *g* to fluctuate about one, if *δ,* the model for *d*, were appropriate. Additionally, one would expect *d* to fluctuate about zero if boys and girls followed the same test score distribution. In fact, neither is the case. Miller and Halpern [18] reference Lakin [7] to support their claim. Lakin reported data implying 28 *V.* Of these, 26 satisfied *mo*. Thus, all but two implied *V* satisfied both *x*¯*b > x*¯*g* and *sb > sg.* Furthermore, Lakin employed CogAT test standardization data with huge sample sizes and data spanning decades.

A dramatic example of *s*2 *b /s*2 *g >* 1 consistency was dispatched as unimportant. It appears in *Science*, where a green colored headline proclaims: "Standardized tests in the U.S. indicate girls now score just as well as boys in math [9, p. 494]." The data source is state math test data from 10 U.S. states. In Table S1 [31] are reported *d*, variance ratios, and sample sizes, data equivalent to 66 *V* . The sample sizes are typically in the tens to hundreds of thousands. Of the 66 variance ratios, 65 satisfy *s*2 *b /s*2 *g >* 1. This fact was noted but dismissed with the observation: "However, none are very large and. . . the male variance is not markedly greater than female variance [31, p. 2]." Such remarkable consistencies need an explanation, not dismissal.

Much more variables were the 66 *d* which range from −0*.*13 to 0.10, with *d*¯ = −*.*007 a rare negative value computed from data in their Table S1. However, how the tests vary from state to state and their corresponding content appears unknown. It is only on math tests with a reasoning component that boys display a typically small mean advantage. This will be clear in Chap. 2. These data also speak to the consistency with which the variance differences, *s*2 *b* −*s*2 *g* are often more consistently positive (or equivalently, *s*2 *b /s*2 *g >* 1), while the mean differences *x*¯*b* − ¯*xg* often seem more apt to fluctuate in sign—but only for math: for reading, *x*¯*b*− ¯*xg >* 0 essentially never appears.

The perspective here is that it is the *consistency* of the outcomes over realizations of *V* that is far more important than the sizes of the mean differences within *V* which, only for math, and not reading, are often small. This attitude contrasts sharply with some other perspectives and attitudes toward regularities in empirical data. An example concerns *d* or *d,*¯ which, as noted earlier, has persistently been observed in math meta-analyses to be positive but small, favoring boys. This empirical fact apparently has been an irritation. A "solution" has been proposed: simply *define* the problem away! Hyde and colleagues have long defined *d* ≤ 0*.*10 as *trivial* [9, 15, 25, 32] (The definition probably intended is |*d*| ≤ 0*.*10.) or alternatively as ". . . an effect so small as to be considered no gender difference [33, p. 8801]." A conceptual explanation of why small *d >* 0 persistently appears in math studies and corresponding *d >*¯ 0 in meta-analyses will be provided later in Chap. 6.

Kagan in 2012 viewed psychological research as in crisis and suggested several reforms researchers should take to fix the matter. Kagan writes: "The most important reform urges a search for patterns in the body of evidence...[34, p. 249]." The thrust here is very much in this spirit. A goal of the literature review is to establish the consistency of the patterns of inequalities of S over several domains, including children's ages, countries, tests, and time frames.

Following the literature review in Chap. 2 and a chapter on alternative perspectives in Chap. 3, the theory will be developed in Chap. 4. As noted, it will be revealed that the inequalities of S are simply expectations under the theory proposed. Consequently, the data inequalities are nothing surprising. Subsequently, in Chap. 5, many math and reading *V* examples will be analyzed, their parameter estimates given, with many solutions graphically portrayed. Chapter 6 summarizes matters, extends the analysis in some ways, and applies the model framework to selected settings.

Transparency and Openness (TOP): all analytical results, estimation procedures, and computer code are contained herein or are in Appendix A. All *V* which have been identified, which are publicly available and readily accessible, form the basis for the review in Chap. 2. It would appear that the research reported here satisfies TOP level three criteria for all eight standards, where applicable [35].

**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/ ), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

# **Chapter 2 Literature Review with Focus on Inequalities**

Evidence of the inequalities of S come from several sources. They are widely evident in large scale U.S. national and international PISA studies, which provide compelling evidence. Numerous other studies including the earliest U.S. achievement tests also provide similar evidence.

### **2.1 U.S. National and International Studies**

#### *2.1.1 NAEP: "The Nation's Report Card"*

U.S. reading and math data are available from the largest most representative and congressionally mandated U.S. assessment, the National Assessment of Educational Progress (NAEP), known as the "Nation's Report Card" [30]. From NAEP's 1969 start when in the early days John Tukey was technical advisor, it has evolved both technically and legislatively. Millions of children have been assessed over the years, with from 10,000 to 20,000 children involved in nationwide grade level assessment [36]. There can be little doubt that NAEP tests are by far the best estimates of children's math and reading achievements for the U.S.A. and thus correspondingly information on sex differences as well. The NAEP tests have been called the "gold standard" for monitoring children's academic progress [37, p. vii]. However, the estimates provided are not conventional simple random sample estimates nor do they possess random sample estimate properties.

The NAEP sampling procedure is multistage, clustered, and stratified. First, the U.S.A. is partitioned into primary sampling units (PSUs) which involve one or more counties, and then there is sampling of schools within the PSU and then students within schools. About 30 children are tested within sampled schools [38] with each tested child receiving a *random sample* of test items, in one of many different test booklets [26, Chapter 2]. Thus, no child receives more than a very small sample of items, and estimates of individual children are not possible, but groups of children can be compared. Coupled with the NAEP complex sampling procedure is the estimation procedure involving Item Response Theory, latent variables, multiple imputation, numerical integration, student weights, and more [26, 37, 39]. A further consequence is that sample sizes are not meaningfully associated with estimated quantities.

The most elementary generally recognized necessary requirement of any statistical estimate is that it is *consistent.* That is, one "does better" as sample size increases, so the sample estimates approach their parameter values as sample sizes increase. The consistency of the estimates in NAEP-like settings is not assured [26, p. 29]. This fact is made intuitively plausible at the individual subject level because each child is given only a few of the possible test items available. Consequently, the consistency of an estimate of a child's ability may not be achieved, a fact that can have implications in subsequent procedures. For the NAEP estimated means and standard deviations, standard errors are not reported. Their calculation, usually involving *jackknifing*, is not straightforward given the complexity of the sampling and estimate procedures [39].

These details concerning the NAEP sampling and estimation procedures make it abundantly clear that conventional textbook statistical tools useful for random samples are simply inappropriate. However, some other investigators have, nevertheless, forged ahead with standard statistical procedures [40].

These same general procedures employed in the NAEP surveys are employed in the international surveys, such as the PISA tests, noted below, which were modeled after the NAEP procedures. And these facts have further implications for estimates *based* on *V* estimates from large sample survey procedures. It means, as is noted subsequently, that there is no guidance for the construction of standard errors of parameter estimates based on *V* from large sample surveys.

Tables 2.1 and 2.2 display NAEP math and reading means and standard deviations at Grades 4 and 8. Tables 2.3 and 2.4 display math and reading at Grade 12 [30]. The columns of these tables are labeled *x*¯*b, x*¯*g, sb,* and *sg.* This notation is for consistency only; they are not conventional sample means and standard deviations for reasons just addressed. In these tables, those *V* which fail to satisfy either *mo* or *ro* appear in bold font. Otherwise, all *V* satisfy the stronger order relation, *mdo* or *rdo*. In Table 2.1, of 26 *V* for math, 23 satisfy *mdo,* while 23 of 27 reading *V* satisfy *rdo* in Table 2.2. Similarly, Tables 2.3 and 2.4 display twelfth grade math and reading data. For twelfth grade reading, seven of eight *V* satisfy *rdo.* All five math *V* satisfy *mdo.* Collectively, of the 66 NAEP *V ,* in these tables 58 or all but 8 satisfy *mdo* or *rdo.* Seven of the eight failures are ties, likely the result of rounding.

The mean differences in math are always small. The boys' mean never exceeds the mean of the girls by more than three points, but it is the consistency, spanning decades, which is striking. While the mean differences in math are always small, the mean differences in reading favoring girls are substantially larger, and girls *always*  have larger means. The girls' mean reading advantage ranges from 6 to 15 points, and girls never show less than a 9-point mean advantage at the eighth and twelfth

**Table 2.1** NAEP Grades 4 and 8 Math

Note: Bold font denotes *mo* failure

Note: Bold font denotes *ro* failure. NA = not available

Note: All *V* satisfy *mdo*

**Table 2.2** NAEP Grades 4 and 8 Reading

Note: Bold font denotes *ro* failure

grade levels. Of those *V* satisfying *mdo*, the median ratio of *(s*2 *b* − *s*2 *g)/(x*¯*b* − ¯*xg)* is 72, so the variance differences dwarf the mean differences. The median of the corresponding *rdo* reading ratio *(s*2 *b* − *s*2 *g)/(x*¯*g* − ¯*xb)* is 13. In summary, *mdo* and *rdo* widely hold in the NAEP reading and math data. The story is little changed in other settings.

#### *2.1.2 PISA Tests*

The 2015 Programme for International Student Assessment (PISA) tests [41, 42], which involve 5000 students from each country, reveal that for those 15 to 16 year olds, girls' reading average exceeded boys' average in all 44 countries, averaging 27 points. Among 42 countries with math data, boys' average exceeded the girls' average in 35 countries by 8 points. Variances are not easily accessible for the 2015 data.

Tables 2.5 and 2.6 display the PISA means and standard deviations for math and reading for 41 countries in the 2003 testing cycle [43, 44]. While these published reports do not contain means and standard deviations for each country, they were graciously provided by Tuomas Pekkarinen, of the Helsinki School of Economics. Again, these means and standard deviations are not conventional sample estimates, although the column labels use familiar notation. In Table 2.5 *mdo* holds for 36 of 41 countries; for the five countries shown in bold font, *mo* fails. In Table 2.6, all 41 reading *V* satisfy *rdo.*

As with the NAEP data, the sizes of the boys math mean exceeding the girls math mean are far smaller than for the corresponding reading differences favoring girls. The 39 positive *x*¯*b* − ¯*xg* math differences ranged from 1.21 to 28.84 with mean difference 11.34. For reading, all *x*¯*g* − ¯*xb* were positive, with differences ranging from 13.27 to 57.76 and mean difference 33.63. Clearly, girls dominate boys in every OECD country in reading at least in 2015 and 2003.

Note: *mo* fails for countries in bold font. 36 countries satisfy *mdo*

Note: All 41 countries satisfy *rdo*

#### *2.1.3 IEA PIRLS and TIMSS Tests*

The International Association for the Evaluation of Educational Achievement (IEA) sponsors the Progress in International Reading Literacy Study or PIRLS test which assesses reading among fourth graders and the Trends in International Mathematics and Science Study or TIMSS math tests which assess fourth and eighth graders as well as a more advanced TIMSS-advanced math test administered in the last year of secondary school. Among 79 *V ,* mostly different countries in the 2001 and 2006 PIRLS reading assessments [14], 62 satisfied *rdo.* Never did the boys' mean exceed the girls' mean in reading. Among 55 countries and 55 *V* in 2015 TIMSS fourth graders math test, 32 of 55 satisfied *mdo*; for the 2015 TIMSS eighth graders, among 46 *V* 19 satisfied *mdo*, while among 10 counties in the 2015 TIMMSadvanced math, 7 satisfied *mdo* [45]. The TIMSS fourth and eighth grade tests but perhaps not on the advanced test reveal outcome patterns relatively different from those for NAEP or PISA tests, and the reason seems to be the content of the tests. It is generally understood that the TIMSS items assess more basic skills than do the PISA and NAEP math tests [46] and thus are correspondingly less comparable to them. As will be noted shortly, it has been known for a century from large sample New York City school data, girls exceed boys on some arithmetic tests in grade school and high school.

#### **2.2 Publicly Accessible Individual Studies Reporting** *V*

Six thousand entering students in 47 California junior colleges received the math portion of the Iowa High School test during the 1929–1930 class year. *mdo* holds [47]. Of 21 *V* 19 satisfy *ro*, and among these 19, 17 also satisfy *rdo* for seven grades and three reading tests [48]. Among a combined third grade sample, *rdo* holds [49]. For 10-year old children from eight schools, *rdo* holds [50]. Among four *V* for math-precocious kindergarten and preschoolers, *mo* always holds and *mdo* holds for three *V* [51]. Among eight *V* for reading and four *V* for math, *mdo* and *rdo* always hold [52, 53]. Among elementary Grades 2 through 6, five *V , mdo* always holds [54]. Among SAT assessments, *mdo* holds for all eight *V* [12]. In Johns Hopkins University talent searches employing the SAT math scores, *mdo* holds for all 10 *V* [55, 56]. Among twins assessed at ages 7, 9, and 10 and for three different math skills and one reading test at each age, *mo* and *ro* hold for all 12 *V* [57]. Lakin was noted above [7]. She reported standardization data implying 28 *V* with results spanning 27 years, for ages 9 to 17, 26 of 28 *V* satisfy *mo*, and *mdo* could not be evaluated. Among four *V* summarizing Taiwanese children's math test performance, four satisfied *mo*, and three satisfied *mdo* [58]. In an Australian sample for children of fifth year schooling for math *mdo* holds, while for reading *sb > sg* fails [59]. Cascella [60] reports two large sample *V* based on fifth and tenth grade Italian children, both satisfy *mdo*. A massive 2013 Italian math study involved

". . . the entire population of Italian children in school years 2, 5, 6, 8 and 10, [61, p. 4]." Reporting implied all years satisfy *mo* except year 5 where *sb* = *sg. mdo* could not be evaluated. Two *V* for sixth grade Kenyan children, one *V* for private schools, the other for public schools, both satisfy *mdo* [62]. Maccoby and Jacklin [53, p. 90] report two *V* for Project Talent where *mdo* holds for both and one ACT test *V* where *mo* holds but *mdo* fails. They report two other studies for which *mo* fails.

In the preceding paragraph, there is reference to 88 math *V.* All but nine appear independent of one another, and all but five *V* satisfy *mo* or *mdo.* For reading, reference is made to 35 *V.* All but three *V* appear independent of one another, and all but three satisfy *ro* or *rdo.*

#### **2.3 Early Twentieth Century U.S. Reading and Math Tests**

The earliest empirical findings on achievement test score sex differences provide a useful backdrop for viewing contemporary hypotheses regarding sex differences in math and reading achievement testing. There are, however, difficulties in assessing this early literature. While *V* are desired, they are infrequently available with the elements of *V* being replaced by other easier to compute quantities. However, it was evident early on that boys and girls differed both in their math and in their reading achievement test distributions as Gray and Stone reported in their 1917 and 1908 doctoral dissertations, respectively [3, 4]. These differences largely hold independent of age or grade level or type of test. The main exception would seem to be that girls exceed boys in tests of elementary arithmetic.

Remarkably, the early investigators tested large sample sizes which they could not hope to fully analyze given the tools available to them. The earliest U.S. achievement testing appears to have been initiated by J. M. Rice, a New York City educated physician who ended his practice in 1888, then studied psychology at universities in Leipzig and Jena, before turning his interest to achievement testing. He constructed achievement tests for both spelling and math. He reported in 1897 results of testing 33,000 children on his spelling test [1]. He then turned to math achievement testing. Rice was said by Stone to have developed the earliest math achievement test in the U.S.A. [3, p. 95]. Rice constructed eight arithmetic word problems for each grade, fourth through eighth, then ultimately tested 5,903 children in 18 schools in seven cities [2]. Rice never addressed issues of sex differences in test performance.

Around 1900, only paper and pencil calculation of any desired quantities was likely possible; not until 1902 was a printing calculator with addition and subtraction operations introduced. Only decades later would four operation machines be available [63]. Graphs were done manually as well and of course so was test scoring and evaluation. One result was that only a portion of the data collected were numerically summarized in some fashion. Because the sample mean and sample standard deviation required algebraic calculations, they were often replaced by

easier to compute approximations. Thorndike wrote a book addressing the problem [64]. A key quantity for Thorndike was the median, which requires essentially only ordering data and counting. The median could replace the mean, and the median formed the basis for other approximations. The Average Deviation or *AD* = - *x* |*x* − *M*|*/n*, where *x* denotes a test score, *n* is sample size, and *M* denoted the mean, or the median if the mean was not available. AD approximated the standard deviation. The coefficient of variation, *s/x,*¯ the sample standard deviation divided by the sample mean, could be replaced by *AD/M.* For Pearson's *r*, two approximations were suggested which, in examples Thorndike provided, were much larger than *r*, sometimes by 20% [64, p. 31].

One silver lining of these early computational difficulties may have been the tendency to report tabulated raw frequency data, sometimes with little comment other than what could be gleaned by a visual glance. Constructing tables of score frequencies would seem far easier than computing summary quantities by hand.

Most early U.S. sex difference studies are summarized by Lincoln (1927) [65] in numerous tables. Table and page references in this section below are all to Lincoln, unless noted otherwise. Lincoln's book was his Harvard education doctorate, and according to him, it was the first attempt to gather together sex difference research findings on a myriad of variables. Two earlier reviews actually appeared [66, 67] which Lincoln does not reference. This is understandable, however, because the sources report no useful numerical data.

There is clear evidence of *mdo* holding in the early literature. While *V* are reported for some math results, no *V* appear to have been reported for early reading results for reasons just noted. Already noted were Stone's [3] efforts constructing two tests, he called fundamental and reasoning, intended to assess sixth grade children. Others used his tests more widely. Based on the analysis of data, he reports [3, p. 30–32], and as noted earlier, one test satisfies *mo*, and the other *mdo.* Stone's test data will be analyzed in Chap. 5.

In his 1916 University of Chicago dissertation, Gray produced a series of oral and silent reading tests which were widely used early on (versions of which are used today). He was probably the first one to report sex differences in reading achievement. He provides no useful data for analysis, but his Diagram XIII [4, p. 127] is reproduced here as Fig. 2.1. "The diagram shows that in all grades girls do better than boys in oral reading [4, p. 126]."

Gray's tests were the basis for providing likely the first large scale assessment of reading achievement, in the 1918 Survey of the St. Louis Public Schools [68, 69] a massive work of which a portion was devoted to reading, writing, arithmetic, and music assessments; only for the reading tests are there data on sex differences. The volumes were intended as supporting documents for a proposed school bond offering.

Both the oral and silent reading tests developed by Gray were based on short paragraphs each child read, during which a child's time in seconds to read each paragraph was recorded, along with several types of reading errors. An oral reading score depended on the seconds taken to read each of twelve paragraphs together with the number of errors made. For silent reading, the length of time required to

read three short paragraphs determined a child's silent reading rate with answers provided to questions about the content read. The results are summarized in two tables in Lincoln which appeared earlier [69]. Table 35, based on 5,118 children, shows girls exceed boys in oral reading for Grades 1 to 8 at seven of eight grade levels. The silent reading test scores, Table 36, were based on 4,463 children with two performance indices, quality scores, and rate scores. For five grades, two through eight, boys exceed girls in all grades on quality, which presumably reflects the ". . . ability to master the thought of what is read [69, p. 170]." Using the reading rate score, or reading speed, girls exceed boys in four of the seven grade levels and are tied at one grade. The number of boys and girls at each grade in these tests was typically above 300.

Lincoln reports reading scores by other investigators and other tests in four additional tables. In all cases, the performance variable favors girls, sometimes by wide margins. Table 37 reports a before and after experiment comparing reading rates. For six grade levels at the beginning, girls' averages exceed boys' averages in five grades. At end, girls exceed boys in silent reading for four of six grades. Table 38 displays both rate and comprehension scores for six grades in rural Iowa. For both indices, girls exceed boys in five of six grade level comparisons. Table 39 reports quality scores at nine ages, 7 to 15; girls exceed boys in median scores at all ages.

And when speed was the performance variable, girls are vastly superior, at eight of nine ages, reported in Table 40. Concerning sex differences in reading variability, in three of four tables [65, pp. 149-154] with sex comparisons spanning ages 8 to 15 years, boys' reading test standard deviations exceed the standard deviation for girls in both quality and reading speed in five or more of seven comparisons in each table. Lincoln concludes "The weight of the evidence seems to indicate that girls are somewhat superior to boys in reading [65, p. 72]."

S. A. Courtis [70] constructed eight arithmetic tests of varying content that were used in a New York city school testing program around 1910. These tests will be considered in more detail later. Test 8, in Lincoln's Table 106, a reasoning test of eight word problems, was administered to 13,629 boys and 13,542 girls (probably in Grades 4 to 8); based on an analysis of these data and discussed further below, *mdo* holds. Brooks [71] used Stone's reasoning test. *mo* holds for 5 of 7 ages, 9 to 15. The results are in Table 34. Thorndike [72] constructed four arithmetic reasoning word problems. With substantial Massachusetts school personnel help, 4,640 children were tested. Reported were the percentage of boys exceeding the girls' median. In 22 of 24 comparisons, in an unnumbered table [65, p. 61] for grades 6 to 9, boys' percentage exceeded 50%, which might be a proxy for *x*¯*b > x*¯*g*. No index of variability is given. Interestingly, in a footnote, Lincoln reports that around 1917 Cyril Burt ". . . found the same difference in London schools [65, p. 60]." In yet an additional math reasoning test, spanning five grade levels, fourth through sixth, all 10 comparisons favored boys. The index was the percentage of boys exceeding the girls' median [65, p. 59].

From 1100 to 1200 New York city students of each sex were tested in each of 18 grade levels from forth to twelfth (e.g., 4A, 4B to 12A, 12B); results are given for two tests selected by Courtis which he thought revealed the greatest sex difference. Seventeen of the eighteen test medians, Table 27, favor girls on Test 3, a 120 item multiplication test of single-digit numbers (e.g., 3 × 4). Lincoln writes "There appears a very clear superiority in favor of girls [65, p. 57]." Turn the page for Table 28: for Courtis Test 6, a 16-item reasoning test, all word problems, Lincoln comments: "Here we find the differences largely in favor of boys...[65, p. 58];" 14 of 18 grades favored boys. These tables and associated commentary provide clear evidence, in large sample data, that early on it was recognized that girls exceeded boys on measures of central tendency for certain arithmetic tests.

In summary, the early literature does provide evidence of *mdo* and *mo* holding for arithmetic reasoning tests. It also demonstrates that very early in achievement testing's history, there was good evidence that girls exceed boys on certain arithmetic tests. Should *V* for reading have been reported, it appears that many studies would have satisfied *ro* and likely *rdo* as well.

As a historical note, it seems important to recognize E. L. Thorndike's impact on the early development of both math and reading achievement tests. From 1899 forward, Thorndike spent his entire career at Teacher's College, Columbia University. He was Stone's statistical and conceptual advisor on math achievement testing. Gray, a pioneer in reading tests, while earning his BA in 1913 and doctorate in 1916 from the University of Chicago, earned a master's at Columbia University

in 1914, where he was influenced by Thorndike. Lincoln footnotes other Thorndike students, for example, Brooks [71]. For a very different perspective on Thorndike and his early influence, see Shields [73].

**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/ ), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

# **Chapter 3 Varying Viewpoints on Sex Differences**

Conceptual deficiencies, the surprising claim of no math test score sex differences, and other efforts to understand or explain math and other sex differences in task performance are of concern here. Those readers primarily interested in methods, models, and subsequent results may skip to Chap. 4 with little loss in continuity.

#### **3.1 Similarities and Hellinger Distances**

When Hyde in 2005 introduced her similarities hypothesis which ". . . holds that males and females are similar on most but not all psychological variables [74, p. 581]," she could have given the idea of similarities, an old psychological concept, a rigorous definition, and at the same time, proposed another more appropriate index of sex differences. But this did not happen. *d* although widely used then as now, was well-recognized, at that time, to be a conceptually inadequate index of sex differences at least for some tasks. The need to report *s*2 *b /s*2 *g* in addition to *d* clearly implies that recognition, at least implicitly. If the population variances of the two sexes were plausibly regarded as the same, there would be no need to report variance ratios. Furthermore, much earlier Feingold in 1992 [28] presciently argued, as noted earlier, that to understand sex differences, both means and variances must be addressed. Feingold seemed to be calling for a new conceptual model for sex differences. Just how this goal could be achieved was apparently not obvious, as there appears to have been no proposal to do so in the subsequent decades.

However, simply exchanging *d* and its model *δ* for an index that relaxes the population variance equality constraint would have been an important conceptual improvement. It might also have changed the psychology of how sex differences are viewed, away from the simple and misleading notion that sex differences are primarily, if not just exclusively, mean differences.

Instead, the notion of similarities and effect size *d* were tightly tied. In addition, there was no indication of just what variable attributes the similarities hypothesis were intended to capture other than effect size differences. Yet in the language of psychology, the notion of "psychological variable similarities" certainly extends well beyond the narrow effect size mean differences index [75]. The similarities hypothesis seems to have remained conceptually untethered to anything other than effect size and those often observed small values of |*d*| in various tasks.

A more appropriate scalar index of sex differences that is easy to compute, assuming the variables of both sexes follow normality, is the Hellinger distance denoted as *H* [76]. *H* overcomes obvious deficiencies of *d* and unlike *δ, H* is a metric, that is, *H* is a measure of distance between a pair of probability distributions. It does require that the distributions of concern be specified, which is not required for *δ*. But distributional normality is implicitly assumed seemingly always, in settings where *d* is of focus, so this added assumption seems benign. *H* does not require equal population variance for both sexes, an Achilles heal for *δ* and consequently *d*. Under *δ* and thus *d,* two distributions are conceptualized as different by locationshift, obviously conceptually wrong at least for sex differences in reading and math test score distributions. *H* does not depend on how the distributions of boys and girls test scores are conceptualized as different. *H* is bounded in the interval zero to one, with zero indicating the distributions are identical and one if the distributions are disjoint.

*H* and a function of *H* for use in what follows will be defined below in Chap. 4. As a scalar, estimates of *H* make it easy to compare the differences, in for example, boys' and girls' PISA math test score distributions for different countries, assuming *V* are available. Thus *H* can put the issue of U.S. test score sex differences into a broader global perspective. *H* could provide a unifying common sex differences metric that reflects variance differences even if variances were otherwise ignored. Any alternative index proposed is unlikely to replace *d* because *d* is too well ingrained in the fabric of research. But perhaps *H* can be reported along with *d. H* has a clear conceptual interpretation as a distance. *d* has no clear interpretation, at least for sex differences in reading and math test scores.

One might observe that what Hyde seemed to have wanted, with her similarities hypothesis, was a *metric* or distance index on the separation of probability distributions of boys and girls on whatever the current variable of interest happened to be. She writes "Crucial to meta-analysis is the concept of effect size, which measures the *magnitude* of an effect—in this case the *magnitude* of gender difference [74, p. 582, italics added]." The word "magnitude," which Hyde used repeatedly, usually carries with it in science at least the notion of a distance or metric, a property that the index she was advocating lacked. To write "measures the magnitude" clearly suggests Hyde wanted a "yard stick" which she did not have.

#### **3.2 The False Claim of Parity in Math Testing**

In 2019, Hyde and others [15] cite a large-scale meta-analysis [9] and conclude ". . . girls had reached parity with boys in mathematics [15, p. 177]." They report a second meta-analysis [10] which ". . . accumulated data from 242 studies, representing the testing of more than 1.2 million people. Overall, *d* = 0*.*05*,* again indicating no gender difference [15, p. 177]."

While these statements sound impressive and may seem initially compelling, sex differences in math achievement test score distributions have been recognized and well documented for more than 100 years [3, 70, 71]. Furthermore, data demonstrating these differences have long been available, as detailed in Chap. 2. Consequently, the claim that there are no gender differences in math test scores seems preposterous.

However, statements that boys and girls are "equal" in math performance [25, p. 377] show "no gender difference," the differences are "trivial or nonexistent," or that "parity" has been achieved [15, p. 177] or similar characterizations have appeared in publications spanning well more than a decade [9, 10, 15, 25, 32, 33]. Consequently, it is interesting to understand the basis for this remarkable "no gender difference" or "parity" in math testing claim, made by varying subsets of a set of 13 different researchers authoring the just referenced publications.

First, consider what a "no difference" claim actually means. The precise intended meaning of the claim seems unclear because it appears to have has never been explicitly defined. However, their argument rests on average *d* values from largescale meta-analyses. Thus the claim is properly seen as a *conditional* claim because *d* addresses only mean differences in distributions, ignoring variance differences. But no conditions are placed on their summary statements. For example, "Our analysis shows that, for grades 2 to 11, the general population no longer shows a gender difference in math skills, consistent with the gender similarities hypothesis [9, p. 495]." This statement makes clear their claim is unconditional.

The only plausible approach to understanding is that the "parity" or "no gender differences" claims are to be taken as *equivalent* to the hypothesis that boys and girls share the same math achievement test score distributions. With this equivalency, the claims are easily dispatched without statistical concerns, using data they conveniently provide. Under the identical distribution hypothesis, as already noted, *s*2 *b /s*2 *g*, over studies, should fluctuate about one, while *d* should, over studies, fluctuate about zero. As discussed earlier in Chap. 1 in Table S1 [31] are listed 66 variance ratios; 65 are greater than one. Under the parity hypothesis, 33 would be expected to be so. In Table 7 [10, p. 1132] which reports NAEP data (and is similar to Table 2.1 here) are listed 36 variance ratios, all of which are greater than one; only 18 would be expected. In the same table are listed 36 *d* values; 34 are positive. Only 18 positive would be expected.

There is no surprise that this analysis shows the no difference hypothesis is false. While below their Table 7, the authors do state that "Overall, we conclude that a small gender difference favoring boys in complex problem solving is still present in high school [10, p. 1132]." That is not the message the authors wish to convey. In their abstract, they state "Overall, *d* = 0*.*05 indicating no gender difference ...[10, p. 1132]."

The puzzle is how these authors justify a no sex differences in math test scores claim when they must be aware of where such an elementary and obvious analysis leads. The answer appears easily given. The first and most important fact is that they ignore data consistencies in their reasoning. The foremost feature of data to recognize in addressing psychology's crisis, according to Kagan [34], is patterns in the data. The remarkably consistent outcome patterns of *d >* 0 and *s*2 *b /s*2 *g >* 1 are clearly the most striking feature of their Table 7 [10, p. 1l32] and immediately noticeable by anyone. Furthermore, this consistency is arguably the most striking feature of their article. This being so, it apparently does not matter how many small positive *d* there are or how many variance ratios greater than one there are, apparently for some readers and some of these authors, the no difference or parity claim is unshakeable.

The second fact is that often no standard errors or other indices of uncertainty are given for their summary quantities, typically values of *d*¯. They simply report the statistic of interest and assert their belief. It needs to be acknowledged, however, that the construction of standard errors for meta-analysis may not be as straightforward as commonly believed. What are claimed as appropriate standard errors are often wrongly constructed. Shuster's research on these matters is convincing [77].

Set aside the state test data-based claim that "The weighted mean is 0.0065, consistent with no gender difference [9, p. 495];[31, p. 8]." The fact that 65 of 66 *s*2 *b /s*2 *g* exceed one, in huge sample sizes, decisively falsifies any parity claim for these state math test data. This outcome consistency seems remarkable when it is recognized that how the different tests differ in math content is unknown, that the grade levels ranged from grade 2 to 11, and that likely there were wide ranges of differences in test administration, scoring, and recording procedures employed among the different states. Such consistency simply rarely occurs in the psychological literature. But given the fact that *mdo* widely holds, it is perhaps not surprising that a rejection of the parity idea for the state test data comes from variances, not means. It seems reasonable to suspect that the near-zero weighted *d*¯ is because of the likely variability of the content of the state tests, keeping in mind that boys' advantage on math tests seems related to a test's reasoning component.

Similar subjectively judged decisions appear elsewhere. Referring to an earlier posting of NAEP data, similar to Table 2.3, Hyde writes "For these items, at grade 12, the average effect size was *d* = 0*.*07*,* indicating that girls had reached parity with boys even for complex problem solving at the high school level [25, p. 381]." And earlier in the same source appears ". . . girls' math performance is equal to that of boys. . . [25, p. 377]." Hyde reminds the reader she defined *d* ≤ 0*.*10 as trivial [25, p. 379].

It is interesting to note that for the 31 math *V ,* in Tables 2.1 and 2.3, all but one *d* are, by Hyde's definition, trivial. What is one expected to conclude? The answer apparently desired is that the remarkably consistent sex differences in the NAEP data gathered over decades, revealing small mean differences favoring boys,

and at three grade levels, using the most sophisticated protocol in U.S. history, and involving millions of children, are to be dismissed as substantively trivial.

In summary, U.S. large sample representative data concerning sex differences in math testing show that for math tests with a reasoning component, the mean for boys is nearly always modestly larger than the mean for girls, and the variances for boys are nearly always far larger than the variance for girls. There appears no evidence this conclusion is inappropriate for nearly all international settings as well, at least for developed countries, as Table 2.5 reveals.

#### **3.3 Sex Differences and Searching for Answers**

Other researchers clearly recognize there exist, and have existed, sex differences in math testing, as well as reading. A key question is why such differences occur. Efforts to find definitive answers in test score data have largely failed. The most recent efforts have focused on math testing. As an example, Italian children's PISA math gap, which is the mean sex difference favoring boys, has been among the largest in Europe. To understand why, the 2013 Italian national assessment examination, the INVALSI, for school years 2, 5, 6, 8, and 10 was examined [61]. The "sample" was the entire population of school children's scores, more than 125,859 tests. Employing what they term as dynamic regression models, the effort yielded no definitive answers, but they conclude ". . . girls systematically underperform boys, even after controlling for an array of individual and family background characteristics, and that the average gap increases with children's age [61, p. 1]." In a rewrite, they suggest stereotypes play a major role in both reading and math test score sex differences. Although understanding the PISA math mean sex difference was the motivation for their study, they write in closing: "The analysis of the reasons why the gender gap in math exists and how it can be reduced is beyond the scope of our contribution [78, p. 39]."

Stereotypes regarding girls and women have long been viewed as important variables for degrading girls' and women's test performance and thus explaining sex differences in math performance, as the previous [78] study illustrates. This perspective is a recurring theme in the math sex differences literature [79]. Stereotype threat has been well researched and has been shown, in some studies, to deleteriously impact on girls' math test scores [80]. Boys' reading scores can be similarly influenced [81]. However, after assessing 15 years of evidence, it was concluded stereotype threat is unlikely to importantly influence girls' math performance [82]. Even if the results had turned out differently and stereotype threat were shown to be a viable and important variable, because variance differences were ignored, the results would be unable to address the core concern here: how the inequalities noted at the outset, in S, originate and why do they persist.

Six variables are said to mediate sex differences in math [83]: 1-Stereotype threat, 2-ostracism and gender identification, 3-self-sufficiency, 4-teacher and parent factors, 5-math-related experiences, and 6-math test anxiety. Evidence is cited for each. Collectively, it is argued "It has simply never been established that there is any meaningful and substantial sex difference in mathematics ability that is not massively confounded with factors related to individual experience [83, p. 42]." If true, these facts would seem to only magnify the importance of understanding the inequalities of focus here because they have managed to "poke through" the massive confounding claimed and nearly universally appear in the summary sample statistics. It is difficult, however, to see the relevance of such research on sex differences of primary concern here. That is because, again, of the failure to consider what are, after all, the largest sex differences, the variance differences.

A multiyear Herculean effort reviewing more than four hundred publications culminated in Ceci, Williams, and Barnett [84, 85] and a corresponding lowerkey book [5]. Their explicit, well-defined, and important main target goal was to explain the observed frequency participation differences among men and women in scientific and math-related occupations. This is not an explicit concern addressed here, but many issues they address overlap with issues of focus here. Yet at the end, it was not possible to find any explicit statement rendered by these authors regarding why such frequency differences between the sexes exist, and persist, and at different rates in different countries often with differences in languages and in different cultures.

There is a literature, to be noted below, which specifically addresses similar frequency differences in success rates observed in boys' and girls' performances on certain cognitive tasks. Such findings, if they had been considered, could have been suggestive of corresponding sex differences in adults. Consequently, a different conclusion might have been provided than the one expressed: ". . . we believe that the evidence. . . points to nonbiological/ability factors as the *major* causes of the underrepresentation of women in mathematically intensive careers [5, p. xii; italics in original]."

The strategy here, as noted earlier, is to explicitly focus on the specific set of empirical conditions of S*,* as the target for theoretical explanation. This narrow strategy may ultimately lead to greater overall understanding of sex differences than framing a far larger more encompassing set of goals.

#### **3.4 Genetics: The Oldest Recognized Sex Differences Influence**

Genetical X-linked influences have been recognized since the Talmud [86]. Furthermore, their trait influence is thought to be stable, with change occurring very slowly over long periods of time [87]. It is notable that interest in sex differences has probably always been triggered by frequency disparities in task performance, often most easily recognized in observational settings.

O'Connor in 1943 [88] appears to have been the first to propose that X-linked gene influences could explain behavioral sex differences in performance unexplainable within alternative frameworks. Boys outperformed girls on his "wiggly blocks" task. In brief, given an X-linked gene in two alleles, if the recessive form has relative frequency *q* and is performance enhancing, then the proportion of *q* boys should exceed the proportion of *q*2 girls. Or alternatively, the mean task performance for the boys should exceed the mean task performance for girls.

However, not until the 1960 to 1980 interval were there efforts to address O'Connor's suggestion. The earliest approaches involved familial correlations. While a correlational approach cannot address the inequalities of focus here, this early history appears to have continued to shape perspectives today and in unhelpful ways.

Under a bivariate discrete binary outcome X-linked Mendelian model (each marginal distribution is Bernoulli, with 0 or 1 outcome), the expected correlations among pairs of related individuals, for example, mother-daughter, or sister-brother, can be specified, given only the gene frequency. However, measurements on variables possibly mediated by X-linked influences are typically observations on outcome variables such as achievement test scores typically regarded as continuously distributed. It was widely assumed, wrongly, that continuous bivariate test score correlations could serve as a proxy for estimating the desired discrete Mendelian correlations. The general failure of the correlations computed on continuous data to yield what was hoped to be estimates of the Mendelian correlations, provoked strong rebukes concerning even the possibility of X-linked effects. For example, "the validity of the hypothesis is unfounded [89]," or "Sex differences. . . Not an X-linked effect [90]."

What went unrecognized was a conceptual problem: there was no basis for assuming a discrete bivariate Mendelian correlation could be estimated from correlations obtained from continuous bivariate test score model. Indeed, it was shown that attempting to construct a conceptual basis for doing so was essentially analytically impossible or at least intractable [91].

The consequence of this result implied these early correlational approaches were misguided and conceptually flawed. Nevertheless, the general failure of a correlation approach, along with the strongly worded, largely condemnatory spirit of the attacks appears to have carried over to present times. Indeed, the very idea that one might believe any trait might be attributable to a single-gene effect signals to some writers, even today, that such persons are beyond redemption and also are apparently stupid: "An education in psychology is not sufficient to overcome the appeal of single genes [92, p. 495]." It may well be these authors wish to reconsider their belief. A *single gene* TKTL1 may well be responsible for distinguishing cognitive abilities of modern humans from Neanderthals [93].

Besides the fact that genetical arguments are often, understandably, psychologically unappealing, there was another issue. Many X-linked recessive traits are deleterious, and some lethal, and suggesting a recessive gene might enhance performance seemed implausible, especially given the role of dominance in the theory of natural selection [94, 95]. This perspective has been revised, and today the role of recessive genes as having beneficial effects is an active area of research [96].

The acknowledgment of X-linkage as a possible factor explaining sex differences remains largely absent in the contemporary psychological sex differences literature concerning reading and math, although not the wider literature addressing sex differences more generally. The possibility of other forms of genetical influence, namely polygene effects playing a role in observed sex differences, seems more widely acknowledged, in the psychological literature, although rarely explicitly embraced, at least in the recent psychological sex differences literature.

An important notable earlier exception to this recent trend are the perspectives of Eleanor Maccoby and Carol Jacklin [53].  They argued, far more broadly than is being argued here, that the origins of psychological sex differences were determined by ". . . certain sexlinked biological predispositions, but to say so is not to deny the importance of social learning. [53, p. 275]." Their entire book is framed within a biological perspective. Consider their book-ending section title: "Is Biology Destiny? [53, p. 373]." They clearly embraced the likelihood of biological factors playing a role by setting boundary conditions: "A variety of social institutions are viable within the framework set by biology [53, p. 374]." And they addressed explicitly, and in some detail, the possibility of recessive X-linkage as a possible sex differences factor in spatial task settings [53, pp. 121–122; 361].

Part of the reluctance to embrace genetical influences is understandable: one reason is there is often missing any explicit convincing mechanistic connection between general statements of genetical influences on sex differences, and the observed sex differences that triggered the interest in the first place, such as the observed relative frequency differences arising on certain task performances, or the obvious lopsided participation rates among men and women in certain mathoriented occupations. A coherent conceptual framework is needed which accounts for these observational differences and which also accounts for the summary statistical inequalities.

If math talent is facilitated by an X-linked recessive gene, all sons of math talented women, twelve Israeli university mathematicians, should be math talented. Math talent was defined as an SAT math score greater than 700. Among ten women who agreed to participate, all six boys were talented. Of ten girls, one was expected to be talented, but none were. Hypothesizing that each sex is equally likely to be talented is easily rejected *p <* 0*.*0004. While small, the study [97] is remarkable.

More recently, there has been a wide recognition of the plausibility that X-linked genetical factors can contribute to phenotypical behavioral differences between boys and girls [87, 105–108]. In addition, the documented importance of X-linkage for understanding trait distribution continues to grow [109].

**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/ ), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

# **Chapter 4 Genetical and** Y **Models for Math and Reading**

This chapter details first the genetical model and then the probability model Y for math and reading. While the formality of the approach is distinctive, crucial to the endeavor is just how sex differences are construed.  This is the key to conceptual understanding.

It is this focus on within sex differences, not between sex differences, which appears to set the approach apart from all other efforts to explain sex differences. Viewing matters from an effect size perspective is simply putting on a conceptual blindfold.

Only the model specifications and results of the analyses appear below. The arguments appear in Appendix A. The most important analytical results state that the inequality orderings on the parameters of Y correspond with the empirical inequalities of *mo* and *mdo* in the case of math and *ro* and *rdo* in the case of reading. Consequently, the elements of S are simply *expectations* under Y.

Two "toy" examples to hopefully aid intuition and illustrate the model's usefulness in both generating data and estimating parameters appear at the start of Chap. 5. The moment estimation procedure is detailed in Appendix A.3, and an R code implementation [110] is given in Appendix A.4.

#### **4.1 The Genetical Model**

"X-linked genes, especially escape genes. . . contribute to sex differences [105, p. 241]." X-linkage, also referred to as sex-linkage, is applied here to math and reading test score sex differences in boys and girls.  Girls have two X chromosomes, one from their father and one from their mother. For girls,

H. Thomas, *Sex Differences in Reading and Math Test Scores of Children*, Monographs in the Psychology of Education, https://doi.org/10.1007/978-3-031-41272-1\_4

the genes of one of these X chromosomes are mostly inactivated, but some genes escape this inactivation. Considering biallelic genes that escape inactivation, girls have genotypes *AA, Aa, aA*, and *aa*, while boys with a single X have genotypes *A* or *a*, with capital *A* denoting the dominant and lower case *a* the recessive allele. Assume for girls *AA, Aa*, and *aA* all lead to the same phenotypic test performance for girls as does *A* for boys. Assume *a* boys and *aa* girls display identical test score performance. Assume *A* and *a* have relative frequencies Pr*(A)* = *p* ∈ *(*0*,* 1*),* Pr*(a)* = *q* ∈ *(*0*,* 1*)*, and *p* + *q* = 1*.* Assume girls' four genotypes *AA, aA, Aa*, and *aa* follow a binomial with *(p* + *q)*2 = *p*2 + 2*pq* + *q*2 with Pr*(aa)* = *q*2*.* The pairs of probabilities of focus: for girls *q*2 and 1 − *q*2 and for boys *q* and 1 − *q*.

In math, *a* and *aa* with probabilities *q* and *q*2, respectively, for boys and girls, are assumed to facilitate high math performance. They become coefficients of latent probability distributions in the probability model to explain sex differences in high math test score performance. For reading, a gene with allele frequencies 1 − *q* and 1 − *q*2 is assumed to facilitate high reading performance for boys and girls, respectively, and these too become coefficients of latent reading distributions. Of course, different genes are assumed to be involved for each task.

# **4.2 Model** Y **for Math**

The fundamental idea is that there are, for boys' and girls' math and reading test scores, two latent unobserved subpopulations within each sex. The shapes of these two subpopulations define the shape of each sex's population, that is, each sex's test score distribution. One subpopulation is composed of high scoring individuals, and the other subpopulation is composed of low scoring individuals. It is assumed all individuals are members of one, and only one, of these subpopulations. Assume for the moment that all high scoring individuals, both boys and girls, have outcomes *μ*2, while all low scoring individuals have outcome *μ*1 with *μ*1 *< μ*2. *μ*1 and *μ*2 are outcomes of a binary two-outcome "Bernoulli-like" random variable *B*: *Bb* for boys and *Bg* for girls. While *Bb* and *Bg* have identical outcomes, their outcome probabilities are different.

Now focus on math. The probability of *Bb* = *μ*2 is *q* or *P (Bb* = *μ*2*)* = *q* and *P (Bb* = *μ*1*)* = 1 − *q.* For girls, *P (Bg* = *μ*2*)* = *q*2 and *P (Bg* = *μ*1*)* = 1 − *q*2*.* So the probability of the *B* outcomes reflects the gene frequencies just given above, with the recessive gene facilitating higher test scores for math.

Of course, the outcomes of test scores are not discrete binary outcomes *μ*1 or *μ*2. Rather they are usually regarded as continuous. Thus, another random variable is required, *N*: it is assumed to be independent of *B*, and it has mean zero and variance *σ*2 *>* 0, and like *Bb* and *Bg*, there is a pair, *Nb* for boys and *Ng* for girls. *Nb* and *Ng* have identical but unspecified distributions and reflect other sources of variance influencing test scores.

So far, all the above theory is latent and unobserved. What is observed is the additive composition of *N* and *B.* The result of this addition is that *μ*1 and *μ*2 become the means of two latent probability distributions which represent the two subpopulations.

Define Y by *Y* = *B* + *N* where a realization of *Y,* that is, *y*, is a math test score and where

$$Y = B + N = \begin{cases} Y\_b = B\_b + N\_b & \text{math for boys,} \\\\ Y\_\mathcal{g} = B\_\mathcal{g} + N\_\mathcal{g} & \text{math for girls.} \end{cases} \tag{4.1}$$

With E(·*)* denoting expectation and var*(*·*)* denoting variance,

 $\mathrm{E}(Y\_b) = \mu\_b = (1 - q)\mu\_1 + q\mu\_2$  and  $\mathrm{E}(Y\_\mathcal{g}) = \mu\_\mathcal{g} = (1 - q^2)\mu\_1 + q^2\mu\_2$ .  $\mathrm{var}(Y\_b) = \sigma\_b^2 = q(1 - q)(\mu\_2 - \mu\_1)^2 + \sigma^2$ ;  $\mathrm{var}(Y\_\mathcal{g}) = \sigma\_\mathcal{g}^2 = q^2(1 - q^2)(\mu\_2 - \mu\_1)^2 + \sigma^2$ .

Only realizations of *Yb* and *Yg* are observed. Importantly, the *only* sex differences are the discrete outcome probabilities for *Bb* and *Bg* given above. The distributions of *Yb* and *Yg*, along with three inequalities, may be given.

The *Yb* and *Yg* distributions are

$$f\_b(\mathbf{y}) = (1 - q)f\_1(\mathbf{y}) + qf\_2(\mathbf{y}) \text{ the distribution for boys}$$

and

$$f\_{\mathcal{S}}(\mathbf{y}) = (1 - q^2)f\_1(\mathbf{y}) + q^2 f\_2(\mathbf{y}) \text{ the distribution for girls.}$$

*fk(y)* = *f (y*;*μk, σ*2*), k* = 1*,* 2, and *f* is the distribution of *Nb* and *Ng*; *f* can be either continuous or discrete and is otherwise unspecified. Please see Appendix A.1 for the argument.

The distribution of *Yb* is not the same as the distribution of *Yg* plus a constant, provided 0 *<q<* 1*.* Thus, Y is not a location-shift model. The right sides of *fb(y)* and *fg(y)* are latent and unobserved. But their parameters *q,μ*1*, μ*2, and *σ*2 can be estimated, as will be illustrated in the next chapter.

The corresponding lower tail distribution functions *Fb(y)* for boys and *Fg(y)* for girls are (if *y* is continuous)

$$F\_b(\mathbf{y}) = (1 - q) \int\_{-\infty}^{\mathbf{y}} f\_1(w) dw + q \int\_{-\infty}^{\mathbf{y}} f\_2(w) dw.$$

$$F\_{\mathcal{S}}(\mathbf{y}) = (1 - q^2) \int\_{-\infty}^{\mathbf{y}} f\_1(w) dw + q^2 \int\_{-\infty}^{\mathbf{y}} f\_2(w) dw.$$

Again, *fk(y)* = *f (y*;*μk, σ*2*), k* = 1*,* 2, and *f* is the distribution of *Nb* and *Ng*.

The population analogs of the empirical inequalities *mo* and *mdo* are given in the following three inequalities:

$$\operatorname{E}(Y\_b) = \mu\_b \succ \mu\_\mathfrak{g} = \operatorname{E}(Y\_\mathfrak{g}) \tag{4.2}$$

$$\text{var}(Y\_b) = \sigma\_b^2 > \sigma\_\varrho^2 = \text{var}(Y\_\varrho) \text{ and } \sigma\_b > \sigma\_\varrho \text{ if } 0 < q < .618 \tag{4.3}$$

$$
\sigma\_b^2 - \sigma\_g^2 > \mu\_b - \mu\_g \text{ given that (4.2), (4.3) and } L \text{ hold.}\tag{4.4}
$$

*L* = {*μd* = *μ*2 − *μ*1 *>* 1&0 *<q<(*[5 − 4*/μd* ] 1*/*2 − 1*)/*2 *< .*618}. *L* is a weak condition, requiring only that *μ*2 − *μ*1 *>* 1 and like (4.3) fixes an upper bound on the size of *q.* Please see Appendix A.2 for the arguments.

Inequalities (4.2), (4.3), and (4.4) reveal Y is easily falsifiable. In (4.4) that (4.2), (4.3) and *L* hold. Thus, failures in data of *mo* or *mdo* to hold might be taken as falsifying Y, although sampling variability must be considered. Clearly, by inspection, E*(Yb)* and var*(Yb)* share common parameters and are functionally related and similarly for girls.

The key point in the above development is that Y generates the same analytical inequality orderings, in the model, inequalities (4.2), (4.3), and (4.4), as are observed in data, namely *mo* and *mdo*. Thus, under Y, *mo* and *mdo* are expected outcomes.

# **4.3 Model** Y **for Reading**

The reading model is analytically similar to the model for math. For math, the recessive gene is assumed to code for higher scores. For reading, the recessive gene with frequency *q* is assumed to be deleterious and thus codes for lower reading test scores. From a model perspective, the values of *q* and 1 − *q* for boys and *q*2 and 1 − *q*2 for girls are interchanged wherever they appear: thus, *q*2 is interchanged with 1 − *q*2 for girls, while 1 − *q* is interchanged with *q* for boys. Two inequalities change: *μg > μb* and *σ*2 *b* − *σ*2 *g > μg* − *μb.* The distributions *fb(y)* and *fg(y)* for reading simply interchange their right-side coefficients of *fk(y), k* = 1*,* 2*.* As with Y for math, the observed inequalities *ro* and *rdo* are simply expectations under Y for reading.

Different genes are assumed to underpin performance in math than underpin performance in reading. *q* is used to denote the gene frequencies in either case because, in almost all cases, no confusion should result in whether the *q* refers to reading or math. Occasionally, *q* may refer to either or both tasks. When confusion seems possible and the distinction is important, *qm* and *qr,* respectively, for math and reading *q* will distinguish the settings. Hats denote estimates for all parameters, e.g., *q*ˆ for the *q* estimate and *σ*ˆ for the *σ* estimate.

#### **4.4 Hellinger Distance** *H*

The Hellinger distance [76] will be used to measure the distance between the probability distributions of boys and girls in some illustrative examples in Chap. 5. *H* is a metric, that is, a distance, unlike *δ* the model for *d. δ* assumes equal population variances while *H* does not. For continuous data, the square of the distance between the boys' and girls' probability distributions is

$$H^2 = 1 - \int \sqrt{f\_b(\mathbf{y}) f\_\mathbf{g}(\mathbf{y})} d\mathbf{y}.$$

*H* is zero if the two probability distributions are identical, and *H* is one if the two probability distributions do not share the same support; that is, their distributions over the horizontal axis are disjoint. For some continuous distributions, including the normal, there are algebraic expressions requiring only the parameters and which do not require integration [76].

Boys' and girls' test score distributions are usually quite similar and often are nearly identical. They share the same support, that is, boys and girls share the same test score outcomes. The result is that the estimated *H* values are often very small. Consequently, reported below are estimates of *Hd* = 100*H. Hd* remains a distance, scaled between zero and one hundred.

**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/ ), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

# **Chapter 5 Model Estimation and Illustration of Test Score Distributions**

Parameter estimates under Y are provided for several reading and math test *V* and their estimated test score distributions from a variety of settings around the world, along with their graphic portrayals.

While model estimates can be provided for nearly all *V* which satisfy *mo* or *ro*, the only samples to be considered below are those that appear to be representative of the larger population. *V* for SAT tests, although as noted in Chap. 2 well satisfy *mdo*, will not be considered. That is because people who take the SAT choose to do so, and so they certainly do not reflect a more general population.

All data explored below are either contained herein or are easily accessible online. As noted earlier, the estimation algorithm is given in Appendix A.3, while an R code implementation [110] appears in Appendix A.4. Those *V* chosen as examples below illustrate similarities and contrasts for difference settings.

#### **5.1 Math Examples**

To assist understanding of Y, defined as Equation 4.1 and discussed in Chap. 4, are two toy examples. Both masquerade as math test score settings. And both illustrate the ease of generating data under Y and the corresponding estimation of parameters. Implementation using coins and poker chips can cause some tedium.

#### *5.1.1 Example 1: Coin Tosses*

On one side of each of two pennies, mark 0, and on the other side, mark 10. On one side of a nickel, mark 5, and on the other side, mark −5. To simulate a virtual boy's test score, flip one penny and the nickel. The penny plays the role of the random variable *Bb* with outcomes 0 and 10, each with probability *q* = 1*/*2. The nickel plays the role of the random variable *Nb* with outcomes −5 and 5, each with probability one-half. Record the sum of the observed outcomes of both coins. This sum is a realized value *y* of *Yb* from the model *Yb* = *Bb* + *Nb*. Now repeat 100 times, thus obtaining "test scores" for 100 virtual boys.

Perhaps the following may further aid intuition: suppose the penny tossed for boys lands 0. Then following the nickel's toss, the outcome of the summed values is either −5 or 5. Should the penny have landed 10, the corresponding summed outcome following the nickel's toss would be 5 or 15. Thus, the two penny outcomes, 0 or 10, are each the centers of two distributions each with variance 25, the nickel's variance, resulting in a two-component mixture distribution with component means 0 and 10 and component variance 25. Each component's weight coefficient is one-half the penny's outcome probabilities. This paragraph completely specifies a discrete two-component mixture distribution.

For girls, flip all three coins. If both pennies show 10, take the value shown on the nickel, add 10, and record. If the outcome for the pennies is otherwise, simply record the outcome shown on the nickel. The two pennies play the role of *Bg* with outcome 10 with probability *q*2 = 1*/*4 and zero with probability 1−*q*2 = 3*/*4*.* The nickel plays for girls the role of *Ng*. The result is a realized value *y* of *Yg* = *Bg*+*Ng*. Repeat the procedure 100 times, recording the data for 100 virtual girls. The possible outcomes for both sexes are −5*,* 5, and 15. One *V* realization is

$$V = \langle \bar{\mathbf{x}}\_b = \mathbf{5}, \bar{\mathbf{x}}\_{\mathcal{g}} = 2.400, \mathbf{s}\_b = 7.247, \mathbf{s}\_{\mathcal{g}} = 6.454, n\_b = 100, n\_{\mathcal{g}} = 100 \rangle,$$

which satisfies *mdo.* Below are the estimates, with hats, and the parameter values in parentheses:

$$\begin{array}{l}\hat{q} = 0.396(q = 0.5),\\\hat{q}^2 = 0.157(q^2 = 0.25),\\\hat{\mu}\_1 = 0.694(\mu\_1 = 0),\\\hat{\mu}\_2 = 11.562(\mu\_2 = 10), \text{ and}\\\hat{\sigma}^2 = 25.142(\sigma^2 = 25).\end{array}$$

#### *5.1.2 Example 2: Poker Chips in Three Urns*

Three urns are labeled *Bb* and *Bg* the boys' urn and girls' urn, respectively, and *N. Bb* contains 40 chips labeled 4 and 60 labeled 2; *Bg* contains 16 chips labeled 4 and 84 chips labeled 2. *N* contains 50 chips labeled −2 and 50 labeled 2. For boys sample with replacement, one chip from urn *Bb* and one from urn *N* (which may be thought of as a value of *Nb*) add the numbers, save the result, and repeat 100 times. Repeat the process for girls except urn *Bg* is used. Thus, there are 100 virtual "observations" for each sex. The possible outcomes are 0*,* 2*,* 4, and 6. A *V*

realization is

$$V = \{ \bar{\mathbf{x}}\_b = 2.780, \bar{\mathbf{x}}\_\mathbf{g} = 1.980, \mathbf{s}\_b = 2.272, \mathbf{s}\_\mathbf{g} = 1.980, n\_b = 100, n\_\mathbf{g} = 100 \}.$$

The elements of *V* satisfy *mdo.* The parameter estimates have hats with parameter values in parentheses:

$$\begin{array}{l}\hat{q} = 0.35\mathbf{\hat{S}}(q = 0.400),\\\hat{q}^2 = 0.126(q^2 = 0.160),\\\hat{\mu}\_1 = 1.540, (\mu\_1 = 2),\\\hat{\mu}\_2 = 5.033(\mu\_2 = 4) \text{ and}\\\hat{\sigma}^2 = 2.471(\sigma^2 = 4).\end{array}$$

Y for each sex is a mixture model with two latent component distributions *f*1*(y)* and *f*2*(y)*. The two most important decisions usually encountered in constructing mixture models are first, the specification of the number of components. Here there are always two for each sex. The second concern is the specification of the latent unobserved component distributions. In most mixture models, the latent components are assumed to be normal distributions. However, typically there is no theory or empirical basis for justifying their specification. A wrong specification of the component distributions can lead to misunderstandings and inappropriate conclusions. There is usually no easy way to assess the consequences of wrongly specifying the mixture components, and thus criticisms or concerns that the components might be wrongly specified are difficult to counter. This problem is perhaps the central concern of many critics of the latent variables mixture model approach to understanding individual differences.

This concern is not relevant here because under Y, as noted earlier, *f*1*(y)* and *f*2*(y)* the component distributions do not require specification for their parameters to be estimated. They can be any probability distribution, discrete or continuous. Consequently, Y consists of a pair of weak semiparametric mixture models. However, to construct graphs of the corresponding solutions provided below, the components must be specified. Thus *post* estimation, when component distributions graphically appear, they are constructed assuming the component distributions *f*1*(y)* and *f*2*(y)* are normal distributions.

An additional unusual advantage of the current estimation procedure, very unlike most solutions of finite mixtures, is that the solutions are closed form. That is, the solution output will always be the same with the same *V* input. In conventional mixture solutions, unlike the special situation here, and where the components are assumed to be normal distributions, iterative procedures often with randomly specified starting conditions are required. Consequently, the same mixture solution, given identical inputs with each solution, cannot be guaranteed.

#### *5.1.3 Example 3: Italian Math Test*

Considerably, more detail will accompany this example which may serve as a guide for interpreting the remaining examples for both math and reading. Cascella in 2020 [60] reported the large sample testing of Italian children, Grades 5 and 10, using the Italian INVALSI math achievement test. For both grades, *mdo* is satisfied. Grade 5 children, aged 10 years, are considered here.

The starting point is the summary statistics data vector *V* the only information. *V* = {¯*xb* = 198*.*913*, x*¯*g* = 191*.*881*, sb* = 43*.*058*, sg* = 39*.*845*, nb* = 15453*, ng* = 15415}*.*

$$\tilde{V} = \{198.913, 191.985, 43.065, 39.837\}.$$

*V* is given here for easy comparison with *V* and will be explained momentarily below. Below are the parameter estimates with standard errors in parentheses:

$$\begin{array}{l}\hat{q} = 0.178(0.023),\\ \hat{q}^2 = 0.032(0.010),\\ \hat{\mu}\_1 = 190.362(0.572),\\ \hat{\mu}\_2 = 238.492(4.34), \text{and}\\ \hat{\sigma}^2 = 1516.197(21.59). \end{array}$$

Define X-linked heritability for boys as

$$h\_{b\chi}^2 = \widehat{\text{var}}(B\_b) / \widehat{\text{var}}(Y\_b).$$

and similarly for girls. *h*2 *bx* = 0*.*182*(*0*.*015*)* and for girls *h*2 *gx* = 0*.*045*(*0*.*007*).*

Once the parameter estimates are obtained, the predicted values are reported in *V .* - These values may be compared with the observed values in *V.* The differences between *V* and *V* may be taken as an index of how well the model accounts for the data. The elements of *V* are the estimated values of the sample means and sample standard deviations, given the model estimates, and are computed from the parameter estimates where the estimates replace the model parameter values. Thus, these are approximately the expected values given the model estimates. The expressions to do so are given above in Chap. 4.

For example, the boys' predicted value of *y*¯, which is 198.913 in this example, is the estimated expected value

$$
\widehat{\mathcal{E}}(Y\_b) = (1 - \widehat{q})\widehat{\mu}\_1 + \widehat{q}\widehat{\mu}\_2.
$$

The girls' predicted standard deviation *sg* is var *(Yg)* with

$$\widehat{\text{var}}(Y\_{\mathfrak{g}}) = (\widehat{\mu}\_2 - \widehat{\mu}\_1)^2 \widehat{q}^2 (1 - \widehat{q}^2) + \widehat{\sigma}^2.$$

**Fig. 5.1** Example 3. Thick lines are *f*ˆ *b(y)* for boys and *f*ˆ *g(y)* girls, and thin lines are their two components. The ordinates are at *μ*ˆ 1 and *μ*ˆ2

In this example, it is 39*.*837*.* As the above comparison of *V* and *V* reveals, the model estimates are in excellent agreement with the data.

Figure 5.1 provides a graphical solution with the post estimation assumption that *f*1*(y)* and *f*2*(y)*, the latent components, are normally distributed. The estimated means of these two latent component distributions are shown by the vertical ordinates at the locations of *μ*ˆ 1 and *μ*ˆ 2, the estimated component means. The component distributions for both boys and girls share the same standard deviation, *σ*ˆ = 38*.*94*.* The two bold lines, dashed and solid, are the estimates of *fb(y)* and *fg(y)*, namely the estimated population models for boys and girls, respectively. If the test scores of the 30,868 children were available, these bold lines should resemble the histograms for boys and girls separately if the latent normal component distributions are reasonable specifications. The bold lines are determined by the thin lines of the unobserved components. There is a pair of thin lines for each sex. The two much smaller, thinner lines in the upper tails represent the distributions for boys (dashed lines) and girls (solid lines) for higher scoring latent component distributions. The shapes of these component distributions for boys and girls are identical; the girls' component is smaller because the weight *q*ˆ2 associated with this component is smaller for girls than the *q*ˆ weight for boys.

In particular, these two upper tail components with their estimated coefficient weights are *q*ˆ2*f*ˆ 2*(y)* = 0*.*032*f*ˆ 2*(y)* for girls, and for boys it is *q*ˆ*f*ˆ 2*(y)* = 0*.*178*f*ˆ 2*(y)*. *f*ˆ 1*(y)* and *f*ˆ 2*(y)* denote the probability distributions with estimates replacing their parameters. The other pair of thin lines, in the lower tails, are those associated with the boys' and girls' latent lower score component, *f*1*(y).* The girls' lower scoring component thin solid line is graphically indistinguishable in some regions of the graph. Precisely, these are *(*1− ˆ*q*2*)f*ˆ 1*(y)* = 0*.*968*f*ˆ 1*(y)* for girls and *(*1− ˆ*q)f*ˆ 1*(y)* = 0*.*822*f*ˆ 1*(y)* for boys. The girls' lower scoring weighted component line tracks the girls' *f*ˆ *g(y)* closely because the lower component weight 0*.*968 is nearly one. The boys' lower scoring weighted component tracks *f*ˆ *b(y)* less closely in most of the regions of the graph because the lower score component weight for the boys departs more sharply from one, namely 0*.*822*.*

The upper tail of *f*ˆ *b(y)* for boys clearly shows more probability mass than the corresponding upper upper tail region of *f*ˆ *g(y)* for the girls. It is commonly observed, e.g., [11], that upper tail region for boys' distribution "is fatter" than the distribution for girls. This figure, Fig. 5.1, illustrates that Y can explain this fact. An analytically based explanation will be provided later in Chap. 6.

In Fig. 5.1, the lines associated with the lower scoring weighted latent probability distributions, for both boys and girls, extend well above 250. Neither the model, Y nor the data imply, as mistakenly might be thought, that higher scoring boys or girls only come from the higher scoring component. While probabilities depend on the test score taken as the reference point (the lower limit of an integral), in fact, in this example, more higher scoring boys and girls can come from the lower than from the higher scoring components. To make this important point precise and using the equations defined in Chap. 4, the total estimated proportion of girls above *μ*ˆ 2 ≈ 238 is 0*.*123 = 1 − *F*ˆ *g(*238*)*, while *(*1 − ˆ*q*2*)* ∞ 238 *f*ˆ 1*(y)dy* = 0*.*107 the area under the girl's weighted lower scoring component but above 238 and *q*ˆ2 ∞ 238 *f*ˆ 2*(y)dy* = 0*.*016 the area under the girl's weighted higher scoring component and above 238. Thus, by far the largest proportion of higher scoring girls, those with scores above 238, comes from the lower scoring, not the higher scoring component, and in this example, similarly for boys as well.

Keep in mind these probabilities are estimated under the assumption the latent components have normal distributions. The reality may be somewhat different. To repeat, this example illustrates the important fact that higher scoring children come from both mixture components.

One can estimate the number of girls and boys in these upper components by computing *q*ˆ2*ng* ≈ 486 girls while *qn*ˆ *b* ≈ 2745 boys. The X-linked heritability estimate for girls is very small, 0*.*045, a tiny proportion of total variance for girls: var *(Bg)* + var *(Ng)* = 1587*.*010 with var *(Bg)* = 70*.*812 and var *(Ng)* = 1516*.*197*.* For boys, the X-linked heritability is much larger 0*.*182, but still relatively small. Thus, the proportion of X-linked variance accounted for in Y is a small fraction of the total variance for both sexes, a finding entirely expected.

The probability that a given girl's test score *y* "comes from" the higher scoring second component can also be estimated:

$$
\widehat{P}(\text{component 2} \mid \text{y}) = \frac{\widehat{q}^2 \widehat{f}\_2(\text{y})}{\widehat{f}\_8(\text{y})}.
$$

However, to do so requires *f*1*(y)* and *f*2*(y)* to be specified, perhaps as normal distributions.

#### **<IP_ADDRESS> Standard Errors**

Standard errors associated with the parameter estimates are reported above in this example because the elements of *V* seem to have been the result of random sampling assumptions, at least approximately. These standard errors were obtained using the *parametric bootstrap* [27, p. 53], assuming that *post* estimation, the component distributions *f*1*(y)* and *f*2*(y)* are normal in distribution. Whenever standard errors appear, they were obtained by the parametric bootstrap, the components are assumed to be normal, and the elements of *V* are assumed to be obtained by random sampling, at least approximately.

Standard errors for parameter estimates obtained from large sample survey *V* , for NAEP or PISA, are not given. For one, there are no sample sizes associated with the estimates in *V* , and for another, estimates in *V* are far removed from random sample estimates. These facts were noted in Chap. 2. There is simply no guidance on how one might proceed to construct a standard error for, say, *q*ˆ estimated from NAEP data, and thus no standard errors appear.

#### *5.1.4 Example 4: CogAT or Cognitive Abilities Test*

The CogAT is a multiple choice test, with varying types of number puzzles, analogies, and series. This example employs an earlier version of the CogAT from 1987 [111] and involved eighth grade children. The observed *V* which satisfies *mdo* and predicted *V* are

> *V* = {101*,* 100*,* 16*,* 14*,* 5085*,* 5148}*, V* -= {101*,* 100*,* 16*.*091*,* 13*.*896}*.*

Below are parameter estimates with standard errors in parentheses:

$$\begin{array}{l} \hat{q} = 0.015(0.008), \\ \hat{q}^2 = 0.0002(0.0003), \\ \hat{\mu}\_1 = 99.98(0.19), \\ \hat{\mu}\_2 = 166.84(38), \\ \hat{\sigma}^2 = 192.08(5.6), \\ h\_{\delta\chi}^2 = 0.258(0.037), \text{and} \\ h\_{g\chi}^2 = 0.005(0.003). \end{array}$$

The X-linked heritability for girls is vanishing small, but for boys much larger.

Figure 5.2 graphically illustrates this solution, which is very unlike Fig. 5.1. It reveals no clear visual separation between *f*ˆ *b(y)* and *f*ˆ *g(y)* nor among the latent component distributions either. That is because *q*ˆ = 0*.*015 is very small, so the higher component probability weights for both sexes are near zero, driving these higher scoring components toward zero for all *y*. Specifically, for girls *q*ˆ2*f*ˆ 2*(y)* = 0*.*0002*f*ˆ 2*(y)*, and for boys *q*ˆ*f*ˆ 2*(y)* = 0*.*015*f*ˆ 2*(y).* Thus, the weighted first component distributions for both sexes and *f*ˆ *b(y)* and *f*ˆ *g(y)* all graphically essentially coincide. Some evidence of the second component boys' distribution is just noticeable in the right tail of Fig. 5.2.

**Fig. 5.2** Example 4. Bold lines are *f*ˆ *b(y)* for boys and *f*ˆ *g(y)* girls. The second higher component distribution for boys is just graphically distinguishable. The lower scoring component lines merge with the *f*ˆ *b(y)* and *f*ˆ *g(y)* distributions and are not graphically distinguishable

Compare the bootstrap standard error associated with *μ*ˆ 1 of 0*.*19 with the standard error associated with *μ*ˆ 2 of 38. The uncertainty regarding the location of the second component mean *μ*2 is large. That is because there are few observations expected to be in the extreme right tails of the distributions on which to base the location of *μ*2 and so the uncertainty with respect to the location of *μ*2 is large. One only expects about 1 ≈ 0*.*0002 × 5*,*148 girl and about 77 ≈ 0*.*015 × 5*,*085 boys in the right higher scoring components.

Even though a glance at Fig. 5.2 suggests there are no sex differences in math, the inequalities in *V* require an explanation. While *x*¯*g* and *x*¯*b* differ by only one, they are more than three standard errors apart. *V* is close to *V* , so Y provides a coherent explanation, although the sex differences are not evident graphically.

#### *5.1.5 Example 5: Stone's 1908 Math Tests*

As noted at the outset, Stone [3] constructed two tests: a fundamental test the results of which were displayed in Fig. 1.1 and the other one an arithmetical reasoning test. Both tests were intended for use with sixth grade children. There are 14 numerical fundamental items which assessed addition, subtraction, multiplication, and division. The first of which was the addition of the following numbers (arranged in a column): 2375, 4052, 6345, 260, 5041, and 1543. Problem 14: Multiply 96879 by 896. There were 12 reasoning problems. The first is "If you buy 2 tablets at 7 cents each, and a book for 65 cents, how much change should you receive from a two-dollar bill?" The items of both tests were presented in an increasing order of difficulty, with the sequence determined at least in part, by earlier pretesting. Children were given 12 minutes for the fundamentals and 15 minutes for the reasoning test. The number and ages of children tested were not specified, but comments suggest about 3000 sixth graders received both tests in 26 different school systems.

The scoring for both tests employed item weighting which was based on Thorndike's belief that "arithmetic is but an abstract name for a number of partially independent abilities [3, p. 20]." Weights were designed to reflect the difficulty of the item. The precise way by which weighting was determined seems unclear. In the end, scores for the 12 reasoning items ranged from 0 to 15.2, while for the 14 fundamental items, scores ranged from 0.3 to 6.3.

Stone provided data from 250 boys and 250 girls, ". . . 500 pupils chosen at random from four representative public school systems [3, p. 30]" in his Table X and Table XI. These data, aggregated over the four school systems from his two tables, are given in Table 5.1 for the fundamentals test and in Table 5.2 for the reasoning test. The data from Table 5.1 provided the basis for Fig. 1.1.

Stone's effort was impressive for the time. He had to score, weight items of individual tests, apparently all by hand, with medians replacing means, and Thorndike's AD or average deviation replacing the standard deviation. He credits

**Table 5.1** Stone's (1908) Fundamental Data

**Table 5.2** Stone's (1908) Reasoning Data

his wife for doing much of the statistical analysis. Much of thrust of his work was to compare test performances of different school systems and their relative performance on the two tests. For example, he compares the ratio of boys to girls in their variabilities, using ratios of AD/median which he called a coefficient of variability. Ratios of these quantities defined his variability index. These ratios are reported for smaller numbers of boys and girls for several different school systems. His fundamental test results satisfy *mo*:

$$V = \{3.193, 3.026, 1.048, 0.996, 250, 250\}$$

with

$$\dot{V} = \{3.193, 3.026, 1.059, 0.984\}$$

close to *V* . The estimates are

$$\begin{array}{l}\hat{q} = 0.175, \\ \hat{q}^2 = 0.031, \\ \hat{\mu}\_1 = 2.991, \\ \hat{\mu}\_2 = 4.147, \\ \hat{\sigma}^2 = 0.929, \\ h\_{bx}^2 = 0.172, \text{and} \\ h\_{g\chi}^2 = 0.041. \end{array}$$

Figure 5.3 displays the model estimates for Stone's fundamentals test. The higher scoring latent components for both boys and girls are visible in the figure, which resembles Fig. 5.1.

Stone's reasoning test results satisfy *mdo*:

$$V = \{6.486, 5.440, 2.532, 2.316, 250, 250\},$$

while

$$
\hat{V} = \{6.486, \textbf{5.440}, \textbf{2.537}, \textbf{2.312}\}.
$$

The estimates are

$$\begin{array}{l}\hat{q} = 0.500, \\ \hat{q}^2 = 0.250, \end{array}$$

**Fig. 5.3** Stone's Fundamentals Test, Example 5, which may be compared with Fig. 1.1

$$\begin{array}{l} \bar{\mu}\_1 = 4.393, \\ \hat{\mu}\_2 = 8.579, \\ \hat{\sigma}^2 = 2.057, \\ h\_{bx}^2 = 0.680, \text{and} \\ h\_{\mathcal{g}^\chi}^2 = 0.615. \end{array}$$

The reasoning test estimate of *q* is far larger than the fundamental test estimate or estimates from the previous two examples. The estimates reflect, of course, weights Stone assigned to each child's test score answer.

Stone stated his primary goal was ". . . to extract knowledge of the relation between distinctive educational procedures and the resulting products [3, p. 7]." Since his primary interest was in educational procedures, why would he have chosen to provide, as the only raw data tabled, data on boys and girls separately, when numerous other tabulations more central to his stated goal were possible? A speculation is that he observed sex differences in test performance among the 3000 boys and girls he observed in the 26 school systems, in the six chiefly north eastern states he visited; perhaps it was this information he wished to preserve, even though he could not do the analysis he might have wished to do.

#### *5.1.6 Example 6: The Courtis Arithmetic Tests*

In 1909, the Russell Sage Foundation of New York City issued a damning report concerning the city's school system ". . . which indicated that retardation was costing the city millions of dollars annually [112, p. 53]." By any measure, the consequence was extraordinary. A huge investigation was mounted which considered numerous aspects of the educational operation, covering janitorial services, school layout, salaries, teachers' ethnic backgrounds, and more. The concern here is with the only assessment of school achievement, the eight arithmetic tests developed by Stuart A. Courtis. At the time, he was in charge of arithmetic at the Detroit Home and Day School, later the Liggett School, which continues today as the University Liggett School. Later, Courtis joined the University of Michigan as an education professor.

The background and motivation for his tests are set out in five articles spanning the years 1909 to 1911 [113–117]. Subsequent references in this section are all to yet another publication, covering the interval 1911 to 1912 which is the interval Courtis provides for in his report [70]. Courtis constructed all the tests [70, pp. 402-414], supervised the testing and data analysis, and wrote the portion of the final report on testing. It contains more than 150 detailed pages with 50 hand-drawn figures or graphs and 55 tables some spanning multiple pages, with seemingly countless computations, mostly averages. It is a remarkable effort given the limited technology of the period. Interestingly, Courtis declined compensation and received only his expenses [112, p. 58].

Testing involved 33,350 children. The target grades were fourth to eighth, with the addition of two high schools, so data are available through Grade 12. Testing occurred in 903 classes in 52 schools in all five New York city boroughs.

The tests covered basic operations and arithmetic reasoning. Collectively, they were difficult tests. Tests 1 to 4 assessed addition, subtraction, multiplication, and division, respectively, each having 120 items. Test 5 required copying 240 numbers; Tests 6 with 16 items and Test 8 with 8 items were word reasoning tests. Test 7 called fundamentals presented a mixture of arithmetic items requiring all four operations, some very difficult. One item required adding eight five-digit numbers. Children were allowed *one minute* on tests 1 to 6, certainly speed tests. The test forms were labeled as speed tests.

Discussion will focus on those tests which, to varying degrees, clearly address sex differences in performance. These are Tests 1, 3, 6, 7, and 8. *V* can be constructed from data provided by Courtis for one grade level for Test 6 and an aggregate of more than 26,000 children for Test 8. The reasons Tests 6 and 8 can be used for analysis, and not the others, is because they had limited numbers of items, for which exact success frequencies were given. Data for the other tests required grouping the scores into intervals, and in addition, for some intervals data were not available. Testing involved 18 grade levels, 4A and 4B to 12A and 12B, and at each grade level from 1100 to 1200 boys, and a similar number of girls were tested. However, grade level data for both sexes are given only sparingly. One can trust the averages reported. That is because Courtis writes that the averages are aided by ". . . mechanical tabulation made the average. . . easily computed [70, p. 432]." AD, average deviations, approximations which replaced standard deviations, followed Thorndike [64].

#### **<IP_ADDRESS> Courtis Tests 1 and 3: Addition and Multiplication**