Google has created a new machine learning tool called Tone Transfer that allows people to transform every sound like whistling into something like a violin, or a saxophone, or a flute.
“We created a tool called Tone Transfer to allow musicians and amateurs alike to tap into DDSP (Differentiable Digital Signal Processing) as a delightful creative tool,” Nida Zada, UX Lead, Google Research, wrote in a blog post on Thursday.
An open-source technology, DDSP is a new approach to machine learning that enables models to learn the characteristics of a musical instrument and map them to a different sound.
Developed by Google Research’s Magenta team, the process can lead to many creative, quirky results.
For example, one can try replacing a capella singing with a saxophone solo, or a dog barking with a trumpet performance.
“This development is important because it enables music technologies to become more inclusive,” Zada said.
Machine learning models inherent biases from the datasets they are trained on, and music models are no different.
Many are trained on the structure of western musical scores, which excludes much of the music from the rest of the world.
Rather than following the formal rules of western music, like the 12 notes on a piano, DDSP transforms sound by modeling frequencies in the audio itself.
This opens up machine learning technologies to a wider range of musical cultures, Zada said.
