So the question becomes: How much higher must one of the averages be be-
fore we conclude that the difference is not just a chance fluctuation but infact represents a real difference between the two styles of waitressing? This is
the question of statistical significance.
Statistical Significance
How can researchers tell if different group averages represent real effects or
just chance fluctuations? Fortunately, statisticians have developed formulasthat allow us to estimate the likelihood that the difference between the
averages could have occurred by chance alone. There are many types of sta-
tistical tests, each appropriate for different types of data and different re-
search designs. Some of the more common tests are an analysis of variance,
achi-square test, and a correlation coefficient.
Returning to the waitress example, if the two averages differed by an
amount so small that it could have been caused by a chance fluctuation, we
say the difference has not reached statistical significance . Conversely, if the dif-
ference is so large that in all likelihood it was not caused by chance but reflects
a true difference between the two waitressing styles, we say the difference is
statistically significant. In the latter case, the conclusion would be that one style
of waitressing does seem to result in better tips than the other style.
However, statistical tests do not really provide a yes or no answer to our
question. All they tell us is the statistical probability that the difference be-
tween the groups was caused by chance. But this observation raises anotherquestion. Suppose you apply a statistical test to the waitress ’data and find
that a difference this large would occur by chance one out of every three
times you conducted the study. What would you conclude? That the different
averages represent a real effect? It would be difficult to have much confidence
in such a statement. The difference might be real, but there is a high probabil-
ity that the finding is just a fluke. So when can we say we have a real differ-
ence? Traditionally, psychologists us a significance level of .05 to answer this
question. This means that if the difference between the scores is so large that
it would occur less than 5% of the time by chance, the difference is probably
genuine.
But keep in mind that statistically significant findings are not necessarily
“significant ”in all ways. When researchers use a large number of partici-
pants, even small differences can be statistically significant. Whether the dif-
ference is large enough to be important is another question. In response tothis concern, investigators often examine and report the size of the differencethrough statistical values known as effect size indicators.
Correlation Coefficients
The correlation coefficient is a favorite statistic among personality research-
ers, and one that will pop up from time to time in this book. The correlation
coefficient is the appropriate statistical test when we want to understand therelationship between two measures. For example, we might be interested in
the relationship between loneliness and depression. We could ask a largeStatistical Analysis of Data 31