I just got off the phone with an older relative who is convinced that she should stop the thyroid medicine that she’s been taking for decades.
She claims it made her arm hurt.

She definitely needs the medicine because she had her thyroid gland removed years ago to treat a cancerous tumor. And her doctor has closely monitored her thyroid level for years.

I asked her how she can be sure that this medicine, which she has taken for over 30 years without problems, caused her arm to hurt?

She replied that she stopped the medicine for five days and now her arm feels better, so the medicine must have caused the pain. Right?

That’s what scientists call anecdotal evidence. It’s one story, or anecdote. But it’s not really proof that stopping the thyroid medicine made her arm stop hurting.

A lot of things can make arm pain disappear; perhaps the most likely explanation is that a minor sprain or strain went away with the passage of time.
To really determine whether a treatment works—whether it’s adding a medicine, stopping a medicine, performing an operation or something else—it’s necessary to study the treatment scientifically. One highly effective method is called a randomized controlled trial (RCT).

In that type of experiment, patients are randomly assigned to the treatments that are being compared. For example, one third of the patients might receive a low dose of a study medicine. Another third would take a higher dose of the study medication. And the last third might take a placebo, or might take a known effective treatment for comparison.

Whenever possible the study is “blinded.” In a double blind RCT, neither the treating doctor nor the patient knows what treatment the patient is receiving. In a single blind RCT, the doctor knows what treatment the patient is receiving, but the patient does not.

The blinding reduces bias.

For example, patients who know that they are receiving active medication might expect to feel better and report more improvement than the patients who know they are receiving placebo. Or the active medicine group may expect, and therefore report, more side effects.

But anecdotes are very appealing, and are often very persuasive, especially for people who haven’t been trained to think via the scientific method.
Anecdotes are stories, and the human brain evolved to learn from stories. We didn’t evolve to learn things by performing statistical analysis; it’s a learned art.

One reason anecdotes are so appealing is that we often identify with the person reporting the story. If your best friend swears that a particular cream prevented a cut from scarring, it’s hard not to believe it. You like your friend. You trust your friend. She wouldn’t lie to you. If you don’t believe her, the emotional part of your brain feels bad, as if you are accusing your friend of lying.

Even when the logical part of your brain tells you that your friend really can’t know whether her success was due to (1) the expected disease course (the initial cut and the patient’s skin type would have healed without scarring regardless of treatment) or (2) the placebo effect (your friend expected the cream to work and so overlooked minor scarring.)

Human beings are designed to be emotional creatures, and anecdotes, often in the form of testimonials, are often effective marketing ploys precisely because they appeal to our emotions.

There’s curiosity: what happened and how did it work? There’s vanity: it worked for me, and it might work for you, too, because you are special. There’s hope: the doctors say nothing can be done but someone tells the tale of something that worked for them. And fear: if I don’t try this, will I miss the one thing that might have worked?

Anecdotes are not bad. The problem arises when people fail to recognize that they aren’t sufficient proof of what works or doesn’t work. Anecdotes can be a valuable starting point for scientific inquiry. If 100 people all claim that a particular treatment successfully cured a medical problem, then maybe it’s worth scientific investigation.

Back to my relative with the arm pain. If she really wants to know whether the thyroid medicine made her arm hurt, she could restart it. If the pain comes back, she could stop the medicine and see if it goes away. And repeat this several times to see if it is a reproducible effect. But I wouldn’t recommend it in this case. Thyroid hormone is required for normal metabolic function, and a person with no thyroid simply cannot get it from taking vitamins or changing her diet. And there’s no plausible reason to expect thyroid hormone to start causing arm pain after 30 years.

Scientific studies aren’t perfect. That’s why new treatments seldom become the accepted standard after one study shows a favorable outcome. Most doctors and scientists prefer to wait until several different groups investigate new treatments and replicate favorable findings.