Worried About Your Certification Exam? Look to Heavens

— Milton Packer wonders what you need to prove you have a brain


In the "Wizard of Oz," the Scarecrow exhibits a profound inferiority complex in his mistaken belief that he lacks a brain. Although his extraordinary cognitive talents solve most of challenges that Dorothy faces during her travels, he remains downtrodden until the Wizard resolves his self-doubt at the end. All he needs to prove that he has a brain is -- a diploma (which, of course, the Wizard provides).

In the past, it was possible to show people that you were worthy by simply doing worthy things. If you were an outstanding plumber, word spread quickly around town, and you were incredibly busy. If you were an inspiring teacher, students would beat a path to your door. And if you knew a great deal about cardiology, colleagues would refer their challenging patients with heart disease to you for your insights and advice.

Not any more. These days, you can claim expertise in any human endeavor only if you have a certificate, a degree, a diploma, or some other formal designation that your knowledge and skills have been recognized by some authoritative body.

You cannot be recognized as speech pathologist without having completed your paperwork. You cannot teach in many public schools unless you have a degree in education. And you may not be able to practice medicine in your area of expertise unless you are certified.

It is hard to argue against the principle that those who seek to practice medicine should be able to meet certain minimum standards. The question is: how can people prove that they are good enough? What criteria will be used to judge competency? And which authoritative body will make those judgments?

Certification and recertification have become extremely emotional topics in medicine, as is evidenced by countless essays and opinion pieces on medical websites. Physicians are justifiably annoyed when some organization claims the right to oversee their expertise. And they become livid when that same organization not only requires them to demonstrate their worth at regular intervals but also charges them a hefty fee for granting them the opportunity to take an examination and acquire a stamp of approval.

The problems with certification and recertification are too numerous to count. How are questions on an examination supposed to validly test a physician's expertise and experience? Does the certification process demonstrate a practitioner's ability to listen or to express compassion? Or to make decisions that are in the patient's interest?

Even if one were to reluctantly concede that a multiple-choice format is a reasonable way of gauging someone's knowledge, what constitutes an appropriate question? And more importantly, what is a correct answer?

All too often, a physician's strategy during the certification examination is not necessarily to provide the right answer, but instead, to give the "expected" answer. My post last week about guidelines led many readers to write that they could not ignore the guidelines -- even if they believed that they were wrong -- because the guidelines represented the "expected" answers to questions asked during the recertification process. If a clinician had independently read and analyzed the literature and had reached a conclusion that differed from that in a guideline, they would still need to provide the guideline-directed answer to the question during the formal examination.

For many physicians, the more you know, the more likely it is that you will get the question wrong.

Many years ago, a friend was studying for her certification examination; the table before her was strewn with booklets filled with condensed tidbits of information along with scores of exemplar questions that were supposed to resemble those in a real exam. I picked up her study guide for cardiology, and for my own amusement, I decided to take the exam at the end without having read any of the preparatory materials.

When I finished, I was distressed to find that I had answered a few questions incorrectly. To understand why, I went to the section in the back of the guide, which not only provided the answers, but also provided citations from the medical literature that supported the choice of the "correct" answer.

One question that I answered "incorrectly" was about heart failure, my own area of expertise and research. The "correct" answer provided in the guide was clearly wrong. But amazingly, the guide cited one of my own papers in the New England Journal of Medicine to support the "wrong" answer!

In the current climate, most physicians have learned that -- to have their expertise formally acknowledged by certification -- they must provide the "expected" answer, not necessarily the correct one.

Theoretically, it is possible for certification bodies to invent an entire body of "correct answers" that are based purely on opinions and misinterpretations of the medical literature. The lack of validity would not matter. All they would need to do is to declare the document as the authoritative source of the "truth". Once that is done, they would be allowed to test whether practitioners have dutifully memorized the right answers.

Do you think that such an extreme idea would never happen?

For nearly 600 years, the Inquisition represented a group of institutions within the Roman Catholic Church that were tasked with combating public heresy. Created to suppress religious dissent, the Inquisitors asked questions that had predetermined correct answers. Those providing the "wrong" answers met an inglorious fate.

Last week, the New York Times ran a story about the certification process in the discipline of -- astrology! Like it or not, astrology has played a major role in intellectual history and human decision-making for centuries. A thousand or so years ago, if you were reputed to be a great astrologer, royalty and military leaders would routinely seek your advice about important decisions they needed to make.

But those halcyon days are gone. According to the Times, to be recognized as an expert astrologer, you need to be certified, and that depends on your ability to pass a number of very difficult examinations. The examinations test your detailed knowledge of the teachings, principles, and equations enshrined in several authoritative books. The sacred volumes contain highly complex mathematical analyses of the heavens, replete with explanations of the importance of the precise position of various planetary bodies at the time of someone's birth or the timing of some momentous decision. The process is profoundly serious -- even though everyone (including all those who take the exam) knows that it is all fake.

Remember this story the next time you learn something with the intent of being able to provide the "expected" answer rather than the correct one. Certification tests your ability to give back a series of expected answers; it does not evaluate your skills, experience and effectiveness as a practitioner or a healer.

Is my heavenly example far-fetched? Just think. In 2006, in a highly controversial move, the International Astronomical Union arbitrarily decided to change its guidelines for the definition of a planet. Until that time, for nearly 100 years, Pluto was widely regarded as a planet -- until it apparently failed its recertification exam.