🎶 Gloves to "hear" through touch

Listening to music with your hands might seem like science fiction, but a team at Université Laval is working on some very special gloves. They can transmit sounds in the form of vibrations directly onto the fingers using small speakers. "You could put the piano on the thumb and the flute on the little finger," illustrates Andréanne Sharp, a professor at the Faculty of Medicine and researcher at the CERVO Research Centre, who leads the project.

The technology, developed jointly with Professor Jérémie Voix from the École de technologie supérieure (ETS), relies on activating the brain areas linked to each finger to add a "tactile cue" to hearing. The musical experience is not the same as with our ears, warns the professor. She compares the sensation of the gloves to the feeling you get when you place your hands on a speaker and feel the vibrations through your body.


The gloves could be useful for people who are hard of hearing as a supplement to hearing aids or cochlear implants. "It is sometimes difficult for them to clearly understand speech or music over background noise, even with their devices," reports Professor Sharp, who is an audiologist by training and a musician by passion.

Musicians could also benefit from the gloves to enhance their performance. "We could help them coordinate with each other using vibrotactile cues when the environment is noisy, in a bar for example," points out the researcher. She adds that many instrumentalists already use vibrations in their practice, such as for the violin with the strings under the fingers.

In a recent study, the team looked at the perception of sound timbre with the gloves. This characteristic of sound allows us to distinguish voices, musical instruments, or sound sources like an air conditioner or a barking dog. Another project is currently testing whether training to use the gloves improves sound perception. "We are not used to using touch to perceive sounds, it's a bit disorienting at first," adds the researcher.

What happens in the brain

The perception of vibrations uses receptors in the skin that are very different from those activated when touching a hot object, for example. To understand how the brain processes sounds sent to the hands, the team created a glove that can be used inside a magnetic resonance imaging (MRI) scanner, where any metal object is prohibited.

The research team also uses electrodes, placed directly on the brain, in people with epilepsy who are awaiting surgery. "During the pre-surgical period, we have them listen to their favorite music with the gloves, and that allows us to see which areas are activated in the brain," reports Andréanne Sharp.

The results show a continuity between the sense of touch and that of hearing. "The areas related to the tactile system are activated, followed by those related to the auditory system," specifies the researcher.

A technology under development

For now, the gloves are not portable since they are connected to external devices. However, the research team received a grant from the New Frontiers in Research Fund to make the gloves portable. The challenge is to miniaturize the components, such as the speakers, and power the gloves, using a battery for example.

Eventually, the gloves could be directly connected to a smartphone, as is the case for most hearing aids. "The sound from the environment would be picked up by a microphone, either from the hearing aid or the cell phone, the signal would then be processed to remove background noise and sent to the gloves." The same principle could be used to watch videos or take phone calls.

The current prototype cannot transmit the signal in its entirety, as our fingers do not perceive as many sound frequencies as our ears. High-pitched sounds, high frequencies, are lost. This is not problematic, according to Professor Sharp, since the fundamental information for recognizing sounds, like a voice or a musical instrument, is in a frequency range that is captured. However, signal processing algorithms could allow the information from high-pitched sounds to be transposed into lower frequencies, and the research team will need to test whether performance is truly improved.