Using skin to "listen" to music, netizens wear this device to listen to concerts: it's like living in a piano

Yige sent from Aofei Temple
Quantum Bit | Public Account QbitAI

I believe everyone has heard the story of Beethoven who continued to create after he became deaf.

Works such as "Für Alice", "Heroic Symphony", "Pastoral Symphony" and so on are all classics.

Have you ever wondered how he did it?

In junior high school physics class, the teacher told us: Beethoven, who lost his hearing, bit one end of a wooden stick with his teeth and pressed the other end against the piano to feel the vibration of the piano and persist in composing.

And now, scientists' latest research happens to confirm this: music can not only be heard, but also touched!

Even deaf musicians can understand the complex emotions conveyed by music through touch.

Enjoy music even if you can’t hear

For people with normal hearing, we hear sounds when vibrations of specific frequencies (20Hz-20,000Hz) travel through the air and are picked up by tiny sensory cells in the cochlea.

But in fact, sound can also be felt through touch.

If you have ever been to a wild concert, you will feel it: the exciting music not only impacts your eardrums, but can even make your body sway.

That's right, the skin can directly feel the pressure generated by sound waves. At the same time, sound waves can also be transmitted to people's hands, feet, and body through intermediaries such as guitar picks and stage floors, and can even penetrate deep into the membranes between bones and inner cavity walls (such as the lungs and chest).

Therefore, when some hearing-impaired people listen to concerts, they will fasten the surface of the balloon with their hands to better feel the music through the vibration of the thin rubber.

Frank Russo, a psychology professor and musician at City University of Toronto, also found in his research that people who are deaf tend to be more sensitive to touch.

In the hearing-impaired brain, the auditory cortex is often repurposed in other ways, rewiring signals from other senses, such as touch and vision.

Particularly for those with acquired deafness, the entire brain recalibrates to adjust to the hearing loss. For example, a neural network involved in attention was found to be more closely connected to neural networks that coordinate motor responses, the visual system, and memory.

The benefit of feeling music through touch is that touch can effectively transmit rhythm. Russo found through the comparison of EEG that the rhythm of music felt by both ears and touch is basically the same.

Not only that, touch can also sense other information in the music, such as feeling the volume changes through the amplitude.

As for whether he can feel the complex emotions in the music, Russo said: Most of them can be felt. Specifically, low-frequency vibrations convey more emotion, while high-frequency vibrations sound less emotional.

Byron Remache-Vinueza from the University of Malaga describes this feeling vividly:

Switching music from sound to vibration is like switching one of your mom’s most delicious dishes to soup; it’s just a different experience.

It is worth mentioning that, to date, the method of experiencing music through touch has changed the lives of many hearing-impaired people.

Life-changing technology

Musical Vibrations at the University of Liverpool, an organisation that specialises in devices that convert music into vibrations, has provided their technology to deaf rapper SignKid.

The team fed one of his songs into the device, which converted the sounds of a guitar, kick drum and snare drum into vibrations, and had SignKid step barefoot on two vibrators to feel the music through his heels and foreshoes.

"I love this," SignKid told the Music Vibration team, "it really gives me confidence."

Finally, SignKid used the equipment lent to him by the team to produce his personal EP "Visual Experience".

Vibrators are still simple to operate. A healthcare company has designed a wearable device that straps a series of sensors to a person's back, shoulders, waist, ankles and wrists. It looks like a combination of a seat belt and a backpack. , can be used to amplify and transmit vibrations of different frequencies.

The equipment has been tested at a concert in downtown Las Vegas, where hearing-impaired participants said they felt as if their bodies had become instruments and music was being played through them.

△ Caption: From YouTube@Not Impossible

After wearing the device to experience the third movement of Beethoven's "Moonlight Sonata", a hearing-impaired person said it was like "living in the piano".

But while the ability to experience music through touch has been proven, there are some problems along the way.

Still needs improvement

The first problem you encounter when you want to feel music through your sense of touch is that the frequency of the music and the frequency of the vibration do not completely correspond.

Compared to our ears which are most sensitive to sounds between 2000Hz and 5000Hz, the sound range transmitted by vibrations is usually between 5Hz and 1000Hz.

This means that many high-pitched sounds that can be heard by the ear cannot be captured by the sense of touch, or cannot be accurately identified. Conversely, we can feel certain low-pitched sounds with our sense of touch even if we cannot hear them.

The second hurdle in converting existing music into vibrations is distinguishing the different sounds.

Mark Fletcher of the Institute of Sound and Vibration at the University of Southampton said that many current technologies can only process music that has been divided into independent tracks for different instruments, and it is difficult to "decompose" complex music well.

Even the most advanced algorithms currently struggle with issues such as stronger vibrations masking other vibrations and the volume of different instruments changing over the course of a song.

Without a powerful way to separate the different instruments and compile a balanced signal in the form of vibrations, it will be impossible to identify the nuances of music from vibrations alone.

To be honest, although these technical difficulties still need to be solved, the invention of this technology has brought real changes to the lives of the hearing-impaired.

The Liverpool University research team once took their vibration device to a school for the deaf.

Teachers responded this way:

The students were more engaged in the music lessons, the kids really enjoyed it, and one boy always came back during break time and knocked on the door asking, "Can I play again?"

Reference links:
[1] https://nautil.us/how-the-brain-allows-the-deaf-to-feel-music-22197/
[2] https://direct.mit.edu/jocn/article -abstract/33/4/635/97411/Neural-and-Behavioral-Evidence-for-Vibrotactile?
[3] https://www.sohu.com/a/277841760_100163655
[4] https://www.musicalvibrations. com/signkid/
[5] https://www.researchgate.net/publication/348669717_Neural_and
[6] https://www.youtube.com/channel/UCeBCj0wtkliFdOqEN4jC_4Q