Wednesday, July 20, 2011

Atlanta Addendum: Duncan and Geography Tests

I wasn't planning on doing this, but two things came out today that demanded it of me.

Arne Duncan's Latest
First, there's Arne Duncan's article at in which he states with pride that the federal government is planning to spend $350 million (!!) "toward developing a new generation of assessments."  Wow. $350 million for new tests.

How many times have new tests been developed, implemented, and hated by the masses, scrapped, re-written based on new and improved standards, implemented, and you know the rest?  How many times?

The reason this keeps happening is revealed in my last post on Atlanta's cheating scandal.  Because testing will never solve our educational problems, but people at the top like Duncan clearly think it will.  He is convinced these tests will provide useful information on students and teachers, if we just have good enough tests.  As I stated in that post, you will never be able to write a test that properly evaluates students and teachers to the levels Duncan expects them to. 

A test is incapable of evaluating a teacher's entire skill set.  A test is incapable of evaluating whether a student should graduate or not.  A single test cannot do this, and never will, and we will keep spending gobs and gobs of money on the fruitless search for the "perfect test" that must surely be out there somewhere until we finally figure this out. 

Of course, we the teachers have already figured this out.  It's the people at the top who need the enlightenment. 

I mean, I thought we were in debt.  Aren't they talking about raising taxes and the debt limit, and cutting like four trillion in spending?  Aren't we nearing economic Armageddon, and two branches of the government are falling over their own ineptitude trying to fix it?  And yet they've got enough to waste $350 million on yet another set of new tests? 

If you want to know why the tea party thrives, this is it.  I personally disagree with much of their rhetoric, but when they want to abolish the Department of Education, I can sympathize in light of this ridiculous waste of money.  If this is what the DOE is going to blow its wad on, maybe we are better off without them.

Geography Tests -- Yet Another Example
And then we have this beauty in the newspaper today.  (  This kind of thing should reveal to people like Duncan why testing will never provide what he wants from it.

Only 20% of students can pass geography tests in 4th, 8th, and 12th grade, it reports.

We get goofy articles like this regularly.  Sometimes they're about history, or math, or science.  The conclusion is always the same: Americans are getting stupider, so let's see what jokes Leno and Colbert come up with this time. 

But this, once again, is yet another classic case of Testing Gone Bad.  Let's look at the specifics, right out of the article:

1. Tests were given to national samples totaling about 25,000 students.  How many students are there in this nation?  Millions.  Tens of millions, in fact.  Now, I know about statistics, and that small samples can represent large ones.  But this becomes a problem with education.  Read on to see why.

2. This so-called geography test gives questions such as this: "Explain the effect of a monsoon in India." This is for eighth graders.

Now, put both of those points together.  What if one class, not tested in the sample, does a whole unit on India, and all the students there know so much about monsoons that they're sick of them though they've never even seen one.  But what if another class, that was tested in the sample, learned about the effects of blizzards in Siberia and the effects of tornadoes in Missouri and the effects of earthquakes in Chili.  But oh, they didn't learn about monsoons, and in fact most of the students have never heard of the word. 

Further, these students know little about India, because at their school, India gets addressed in ninth grade World History. 

So now, you have tested students who actually do know about the substance of this monsoon question, but do not know the specific instance given here.  Therefore many of them will get it wrong or leave it blank.

Do you see why this is a problem?  Tests such as this will never be able to avoid this problem!  Teachers teach what they teach, and unless you're going to mandate that every single eighth grade social studies teacher (oops, what about the schools that don't teach that in eighth grade?) teaches about monsoons in India, then questions like this will continue to make us look bad.

The truth is, you don't know what these students know about geography based on this question.  All it tells you is they don't know about monsoons in India.  That's it.  They might understand earthquakes in Chili, blizzards in Siberia, sandstorms in Libya, drought in Australia, and deforestation in the Amazon, but because the test writers decided to ask about India, the student was stumped. 

Monsoon itself is a very unusual word.  This is not a word the average eighth grader is going to encounter on the street.  Especially if English is their second language and they aren't from southeast Asia. 

Now maybe you're thinking, "So ask questions about all those other things too."  Except that no one is going to learn all those things.  No one can possibly learn about every possible culture and landform.  If you ask about all of them, you are almost guaranteeing that most students will miss a lot of the questions. 

3. Finally, you notice something else not in the article.  In fact, you almost never hear this information in any literature about tests like this.  And that is: What is passing?

Who decides what makes a passing score on these tests?  Why is that number passing?  On what "data" is this based?  You see, someone has to decide all these things.  So, we read that 20% are considered competent in geography.  What if you lowered the passing score by a couple points?  Sometimes you have a large swath of students really close to passing.  This has happened in our school a few times on our state tests.  Does that matter?  Well, it matters if it's your kid.  And it matters if the test has been forced down our throats.

Taking my earlier comment, if you throw in questions about 25 different cultures and places and weather patterns and industries, you better have a lower passing score requirement, because no student will be able to get all or most of those questions right.  Do we know the test writers have taken this into account?  Have they ever taught?  When was the last time they were in a classroom?

What's the Point?
The point of all this is simple: Tests such as this are basically impossible to implement on a national level.  You cannot accurately assess student knowledge in a subject like geography on so large a scale.  The only way is if they knew they were going to be tested beforehand, and had a chance to learn the material that would be on the test.

This is what AP tests presume to do.  You have the whole year to learn the content, and then the test at the end evaluates how well you learned it.  But, assuming you have a competent teacher, there shouldn't be any huge surprises on this test. 

I taught AP Chemistry for the first time this year.  It was very difficult, and the test is one of the hardest.  But my students told me they recognized everything on the test.  Even if they didn't know it well enough to get it right, they recognized it.  I learned something about this, and if I understand it well enough, I will be able to do it right.

That means I did my job.  That's my evaluation.  I delivered the content of the course the students would be evaluated on if they chose to take the AP test. 

But this test serves a different purpose.  The AP test is not determining if a student passes.  They are determining if the student knows enough to get credit for the college course.  They are not evaluating the high school course, but the college one.  So we expect that many of the students will not pass these tests, even though they have learned a lot, been challenged, and hopefully passed the high school one.

But if you want a test that every student must be able to pass in order to graduate, you must--you are morally, ethically, and professionally obligated--to make the passing score attainable for all students.  Do we know the testing industry understands this?  Or the politicians like Duncan? 

And the AP test is specific to my class.  But geography is not so condensed a subject.  As I've shown, there is literally no end to the different cultures, landforms, weather, social development, farming and industrial advancements that can be studied, especially if you include history. 

And no one--no one--could ever argue or expect students to know all of them.  You don't.  Neither do I.  In fact, I would bet most geography professors in colleges and universities would say they don't know them all.

So why are we wasting all this time giving stupid tests to eighth graders then?  Don't we have better things to be doing with our class time? 

Like, I don't know, learning?

No comments: