Proceedings of the National Academy of Sciences<\/em>.<\/p>\n\n\n\nThe soft, flexible, postage-stamp-sized device comfortably adheres to the upper chest to sense the subtle vibrations associated with talking and singing. From there, the captured data are instantaneously streamed via Bluetooth to the users\u2019 smartphone or tablet so they can monitor their vocal activities in real time throughout the day and measure cumulative total vocal usage. Custom machine-learning algorithms distinguish the difference between speaking and singing, enabling singers to track each activity separately.<\/p>\n\n\n\n
With the app, users can set their personalised vocal thresholds. When they near that threshold, their smartphone, smartwatch or an accompanying device on the wrist provides real-time haptic feedback as an alert. Then, they can rest their voices before pushing it too far.<\/p>\n\n\n\n
\u201cThe device precisely measures the amplitude and frequency for speaking and singing,\u201d said Northwestern\u2019s John Rogers, a bioelectronics pioneer who led the device\u2019s development. \u201cThose two parameters are most important in determining the overall load that\u2019s occurring on the vocal folds. Being aware of those parameters, both at a given instant and cumulatively over time, is essential for managing healthy patterns of vocalisation.\u201d<\/p>\n\n\n\n
Northwestern\u2019s Theresa Brancaccio, a voice expert who co-led the study, added: \u201cIt\u2019s easy for people to forget how much they use their voice. Seasoned classical singers tend to be more aware of their vocal usage because they have lived and learned. But some people \u2013 especially singers with less training or people, like teachers, politicians and sports coaches, who must speak a lot for their jobs \u2013 often don\u2019t realise how much they are pushing it. We want to give them greater awareness to help prevent injury.\u201d<\/p>\n\n\n\n
For the millions of people who make their livings by speaking or singing, vocal fatigue is a constant, looming threat. The common condition occurs when overused vocal folds swell, making the voice sound raspy and lose endurance. Vocal fatigue negatively affects singers, in particular, altering their abilities to sing clearly or hit the same notes as their healthy voice can. At best, one short period of vocal fatigue can briefly interrupt a singer\u2019s plans. At worst, it can lead to enough damage to derail a career.<\/p>\n\n\n\n
Lack of awareness is the underlying problem. People rarely make the connection between vocal activities and how those activities affect their voices. Most people don\u2019t notice they are overusing their voices until hoarseness has already set in.<\/p>\n\n\n\n
\u201cWhat leads people into trouble is when events stack up,\u201d Brancaccio said. \u201cThey might have rehearsals, teach lessons, talk during class discussions and then go to a loud party, where they have to yell over the background noise. Then, throw a cold or illness into the mix. People have no idea how much they are coughing or clearing their throats. When these events stack up for days, that can put major stress on the voice.\u201d<\/p>\n\n\n\n
Separately, Rogers, in collaboration with researchers at the Shirley Ryan AbilityLab, had developed a wireless wearable device to track swallowing and speech in stroke patients. The bandage-like sensor measures swallowing abilities and speech patterns to monitor stroke patients\u2019 recovery processes. In the early weeks of the Covid-19 pandemic, Rogers\u2019 team modified the technology to monitor coughing, as a key symptom of the illness.<\/p>\n\n\n\n
\u201cI wanted to gather more data and make our tracking system more precise and more accurate,\u201d Brancaccio said. \u201cSo, I reached out to John to see if his sensors could help us gather more information.\u201d<\/p>\n\n\n\n
Rogers said: \u201cI thought it was a great opportunity for us to extend our technologies beyond our very important, but narrowly targeted, uses in health care to something that might capture a broader population of users. Anyone who uses their voice extensively could benefit.\u201d<\/p>\n\n\n\n
The pair also partnered with speech pathologist and voice expert Aaron Johnson to explore how the devices could be used to evaluate and monitor treatment for patients with vocal disorders. Johnson, who co-directs the NYU Langone\u2019s Voice Center, said the small, wireless device could help track patients\u2019 voices in the real world, outside of a clinical setting.<\/p>\n\n\n\n
\u201cA key part of voice therapy is helping people change how and how much they use their voice,\u201d said Johnson, study co-author and associate professor in the department of otolaryngology at NYU Grossman School of Medicine. \u201cThis device will enable patients and their clinicians to understand voice use patterns and make adjustments in vocal demand to reduce vocal fatigue and speed recovery from voice disorders. Generalising vocal techniques and exercises from therapy sessions into daily life is one of the most challenging aspects of voice therapy, and this device could greatly enhance that process.\u201d<\/p>\n\n\n\n
The team modified Rogers\u2019 existing devices to measure vocal load precisely over time. That includes frequency, volume, amplitude, duration and time of day. Like Rogers\u2019 previous devices for Covid-19 and stroke patients, the new device also senses vibrations rather than recording audio. This lets the device detect vocal activity precisely from the user, rather than the ambient noise surrounding them.<\/p>\n\n\n\n
The biggest challenge was to develop algorithms capable of distinguishing speaking from singing. To overcome this, Brancaccio recruited voice and opera students to undertake singing exercises to train the machine-learning algorithms. A team of classical singers with vocal ranges from bass to soprano wore the devices while humming, singing staccato scales and songs, reading and more. Each singer generated 2500 one-second-long windows of singing and 2500 one-second-long windows of speaking.<\/p>\n\n\n
\n