New research: Voice and speech patterns reveal psychiatric symptoms, but possibly not diagnoses

Can your tone, sentence structure and the number of pauses reveal whether you have depression or suffer from schizophrenia? Yes, according to previous studies. But a new study from Aarhus University now shows that the reality behind our speech patterns is much more complex than previously thought. Lasse Hansen, a PhD student at the Department of Clinical Medicine at Aarhus University is behind the study, and he advises that doctors should keep their eye on the ball if they use tools such as artificial intelligence to listen for mental illnesses in our speech patterns.

“In the study, we analysed speech from people with multiple mental disorders. We found that the differences in how you speak and what you say are not necessarily specific to an individual diagnosis, as hitherto assumed. In other words, you can’t necessarily be confident that previous markers of depression, for example, actually indicate depression. Instead, they may point to a more general condition, or indicate symptoms that may be present in several different disorders.”

The study analysed 3,000 audio recordings of 420 people, some of whom had diagnoses such as autism, depression or schizophrenia, while others were control subjects without a mental diagnosis. 

Participants had to describe what they saw on eight to ten videos, after which the researchers trained an artificial intelligence model with the more than 3,000 recordings to recognise disease-specific ways of speaking. However, the AI failed to hit the target for all diagnoses, says Lasse Hansen:

“If it has to distinguish between a depressed person and a person with no psychiatric symptoms, it performs well, but in more realistic situations, when it has to distinguish between people with several different complex diagnoses, accuracy drops by about 30 per cent.”

Artificial intelligence should be used to find symptoms, not diagnoses

Several companies and researchers are already working on developing artificial intelligence to make diagnoses based on speech patterns. But it is important that they do not take it for granted that artificial intelligence can be trained to provide accurate diagnoses, says Riccardo Fusaroli, associate professor of cognitive science at Aarhus University, who is the last author on the study.

“The potential of voice and speech markers is probably more as a support tool in the diagnosis process, for example to identify potential symptoms and cognitive impairment, than a tool for direct diagnosis.”

This is why Roberta Rocca, assistant professor and co-leader of the project, calls for both industry and researchers to look at the methods again.

“We call for the field to step back a little and start focusing more on clinical relevance, for example by looking at several diagnostic groups at the same time and shifting focus from predicting broad and complex diagnostic groups, such as schizophrenia and depression, to more specific symptoms.”

The research results - more information

• Type of study: machine learning/artificial intelligence on both text (transcribed recordings) and raw audio files. Cohort study
• The project partners are all from AU
• Partially funded by seed funding from the Interacting Minds Centre (“Clinical Voices”).
• Riccardo Fusaroli has been paid for consulting work on related topics at F. Hoffman-La Roche. Lasse Hansen has been on an internship at F. Hoffman-La Roche on a related topic.
• The article has been peer-reviewed and published in a scientific journal
• Read more in the scientific paper: https://rdcu.be/dqHDo

Contact

PhD student, Lasse Hansen
Department of Depression and Anxiety, Department of Clinical Medicine, Health, Aarhus University
Email: lasse.hansen@clin.au.dk
Telephone: +45 87169027