Research suggests depression assessment questionnaires can’t reliably compare people with differing intelligence

By Simon Makin edited by Sarah Lewin Frasier

Psychologists have found that two common questionnaires for assessing depression don’t work for comparing people of differing intelligence—and the problem may extend to other conditions and traits.

For a recent study in the journal Intelligence, Stanisław Czerwiński of the University of Gdańsk in Poland and his colleagues investigated how intelligence correlates with mental health. They hypothesized that the association between intelligence and better mental health starts out positive as it approaches the high end of the IQ scale, then turns negative.

The researchers analyzed data from two U.S. surveys that tracked thousands of people over decades. To estimate IQ, these surveys used an aptitude test that measures math and language abilities. Each used a different well-established mental health scale containing questions about things such as mood, sleep and appetite.

On supporting science journalism

If you're enjoying this article, consider supporting our award-winning journalism by subscribing. By purchasing a subscription you are helping to ensure the future of impactful stories about the discoveries and ideas shaping our world today.

The data revealed the curved relation the researchers were expecting: the highest intelligence levels seemed to be associated with declines in mental health. But then the scientists found a problem. To make sure their results were valid, they ran statistics tests to determine whether the mental health measures work the same for people at different intelligence levels, in part by calculating whether responses to individual questions reflect depression to the same extent for everybody. Both scales failed this test, meaning they can’t be used to compare people with differing intelligence—and conclusions like this study’s can’t be trusted.

The finding casts doubt on previous studies that used these tools without accounting for intelligence and suggests depression screening in doctor’s offices may be flawed. “Imagine we’re measuring height, but our ruler is made of Silly Putty, so the length changes,” says Nicole Beaulieu Perez, a psychiatric nurse at New York University not involved in the research, who studies disparities in mental health care and depression assessment. “How can we know how tall these people are?”

The study doesn’t show what causes this effect, but in hindsight, Czerwiński isn’t surprised. The questionnaires require a lot of interpretation of questions and answers, he says: “Very intelligent people may think about mental health differently and maybe experience symptoms differently.”

To better assess depression in groups of people with varying intelligence or to compare groups that might differ in intelligence, researchers will need better tools. Researchers say new approaches could include digitally tracking sleep and other activities or “experience sampling,” in which study participants are asked how they’re feeling at random intervals rather than offering interpretations after the fact.

In a separate study, Perez recently found that evidence showing that depression scales hold steady across other features, such as gender and culture, is inadequate. “Depression is one of the most measured constructs in science, but we have this measurement problem,” she says.

The study analyzed only two mental health scales, but Czerwiński says the problem is probably widespread wherever depression scales are used. The researchers are currently testing the evaluation of other psychological variables as well—they’ve seen similar results for loneliness, Czerwiński says, and they’re exploring personality measures.