5 Questions on measuring mental illness for psychology professor Eiko Fried

Fried on the challenges of measuring mental health and what psychedelic researchers need to consider in designing studies.

Eiko Fried isn’t particularly interested in psychedelics.  As an associate professor of clinical psychology at Leiden University in the Netherlands, Fried’s primary focus is understanding and measuring mental health issues. But he’s become a close observer of psychedelic science because of the intersections with mental health, particularly depression. 

Fried has studied just how hard it can be to even measure depression, let alone quantify whether a specific treatment can meaningfully alleviate its effects. He’s been in the field long enough to see clinicians get excited about the potential of all kinds of “breakthrough” depression treatments like transmagnetic stimulation and electroconvulsive therapy, only for that excitement to fizzle after new treatments didn’t prove to be miracle cures after all. Psychedelics might be another passing fad, Fried says. He supports further research, but he’s adamant that the field needs more rigorous science. The Microdose spoke with Fried about the challenges of measuring mental health and what psychedelic researchers need to consider in designing studies.

Why is it difficult to quantify depression?

If you want to measure what depression is, you need to tell us what depression is. Otherwise, how can I evaluate whether you succeeded in measuring what you want to measure? It's a real issue in our field. People have the Diagnostic and Statistical Manual of Mental Disorders (DSM), which defines depression symptoms, but that is not the same as depression. 

My stance is that a person's mental health state is a momentary state. It’s a bit like your current state, which comes out of a complex system of things that all interact: your disposition, how you slept last night, did somebody fill your coffee mug with decaf this morning without your awareness? All these things play a role. Mental health states are similar: they come out of biological, psychological, and social factors, and your current stress, your personality, life events, adverse experiences, brains, genes, hormones. 

We try to measure depression with questionnaires, which is really tricky. My take is — which is widely shared among other researchers these days — is that scales or diagnostic manuals superimpose a simplified categorical system on this vast, complex landscape of mental health. And that is fine, we need simplification for health care. Otherwise, how would you compare prevalence rates? It’s a practical thing.

What do you see as the limitations to our current methods of measuring depression?

When we try to figure out if people get better from depression, we give them a questionnaire that has many symptoms, and we measure how many symptoms get better. This is the state of the art. But just because people’s symptoms improve does not mean they can go to work again, or that they can have a relationship again, that they're interested in sex, that they have high well-being, that they have recovered their impairment of functioning. People like me have been calling for more meaningful outcome measures in clinical trials. That's not just for psychedelics; it's also for psychotherapy and other areas where you’re trying to measure people. The symptoms are an important first step, but if you really want to understand where a person is, I would really like to know if they can go to work again, if they can do sports again. How is their quality of life? What's their well-being? 

In most of these ketamine studies, the median follow up time was seven days. That's just ridiculous. We should not do that. It should not be funded, and it’s not interesting, because for a severe chronic mental illness like major depressive disorder, it takes more than seven days to recover impairment of functioning, well-being, and quality of life.

As you said, even the most popular depression symptom scales have limitations. You’ve mentioned several different scales and tools clinicians use; how do scientists decide which one to use? Are the popular ones clearly the best, or are there biases at play here, too?

I can tell you a story: in 1972, there was a famous paper by [John] Feighner where he measured mental illness by making lists of symptoms for common mental illnesses, including major depression. If you look at the paper, he took his symptoms for depression from a 1957 paper by [William] Cassidy, with some very minor changes. Then, in 1980, the DSM-III came out. The DSM-IV came out in 1994, and we’re still using the DSM-V today, which came out in 2013. All of the symptoms Cassidy proposed in 1957 are in the DSM-V except for constipation — that was removed.  In 1980, a journalist asked Cassidy, why these symptoms? And Cassidy said, “It sounded about right.” 

And that's fine. What else would he have said? It's not like we had biomarkers. Perhaps Feighner could have taken his depression criteria from somebody else besides Cassady; there were alternative frameworks, and they all had equal empirical support. But he didn't. 

It’s fine as long as we don't misunderstand these categories as true things in the world – they’re more like summaries. They try to approximate. The same holds for many of the other scales; some are more famous because of just past attendance and historical artifacts. 

For something like the DSM, it’s also a matter of science policy. You don't want to change the DSM all the time, so there's this huge force calling for adherence to precedents. And there are political and social events surrounding this, and pharma lobbying to lower thresholds for what constitutes mental illness. All this shapes what the DSM is today. That doesn't mean it's not also based on data, but the categorization you have is to some degree a little arbitrary. 

Psychedelics researchers are discussing elements of study design that can affect outcomes, like whether participants know they’ve received a psychedelic, or what affect participants’ expectations might have. Are there study design issues you wish more researchers were considering?

People say it’s going to be clear to patients whether they’ve been given MDMA or are in the control group, so we might as well not have a control group. But no, that's not how it works. You want to have a treatment as your control group, something like cognitive behavioral therapy that is known to work very well. If you can show me that the treatment group with MDMA- or psilocybin-assisted psychotherapy has better outcomes after a year than your treatment as usual group, after controlling for factors like clinician contact rate — we know that the more you see people, the better they are on average — then sign me up. 

But currently there's no evidence at all for the second bit. In the paper that everybody has cited lately with a 12 month follow up, there is no control group. Many of the people who are better after 12 months got psychotherapy. But depression is an episodic disorder; even if you give people nothing, people could be better 12 months later. That's how the nature of the illness works. There's no evidence that if you replace MDMA or psilocybin with eating small carrots, it would not also help 80% of people — because it's not the [psychedelic] treatment that helps people in this case, it’s a year of having social support and getting psychotherapy. 

I also want us to think about the blinding of the clinicians, because we do this here in Leiden University. I have colleagues sitting next to people holding their hand for 8 hours. You can be the most amazing neutral observer, but you are still rating this person's progress, knowing their treatment group. It's a massive confounder. 

In the research world, scientists are under pressure to produce splashy work, and quickly. But it often takes longer to do rigorous work — and there’s no guarantee that additional effort will yield valuable new insights. What do you think could incentivize scientists to pursue more rigorous study designs and analyses?

I want to add that I really appreciate the work people are doing. I know it's hard, and it takes years to find participants and write papers. But you really have a minimum of ethical responsibility for the people that you say you're helping to communicate your science appropriately.

One thing that comes to mind immediately is the role of scientific journals. They bear a minimal responsibility at least to weed out questionable stuff, no matter how famous the people trying to publish the paper. Then, number two, funders. They could say, we will only fund clinical trials if they're sufficiently powered. Congress could also step in with some mandates. The best thing that could happen to the science of psychedelic research is if the NIH gave out $500 million to one independent group of people who do not sell patents on ketamine, which many of the people in this field somehow do. They could do one or two large studies, rather than studies with just six participants. That would be better for the field, and it would be better for the patients — there would be fewer questions afterward. 

This interview has been edited and condensed for clarity and length.