With the speedy advance in artificial intelligence (AI) technology that has already seen it replace many musicians and their works on popular streaming platforms, one jazz trio has decided to use AI to its advantage, bringing it in as the fourth member of the group.
As it happens, the jazz trio Sveið, which includes Dr. Federico Reuben, Emil Karlsen, and James Mainwaring, has decided to improvise live music with AI-generated sound, taking part in one of the biggest innovations since the introduction of sampling, according to a report by the University of York on June 19.
The band, which is about to release a free jazz improvisation album called ‘Latent Imprints,’ has deployed an emerging technology called neural audio synthesis (NAS), which allows them to improvise in a live setting with AI-generated sounds – in other words, ‘jam’ onstage with AI.
How AI complements the jazz trio
As Dr. Reuben, an Associate Professor at the School of Arts and Creative Technologies at the University of York, who is also a “laptop improviser and live coder” in the trio where Mainwaring is the saxophonist and Karlsen is the drummer, explained:
“NAS employs deep learning, an AI technique where programs are trained on large datasets – in this case, collections of sound recordings – to find features and patterns in the data that enable the generation of new sounds resembling those in the original dataset.”
One of the particular techniques this AI model uses is called ‘timbre transfer,’ where an AI model “trained on a database of recorded speech can respond in real-time to inputs from a microphone placed in front of a drum kit.” This way, when the drummer plays, AI “generates vocal sounds mimicking the drums.”
The result of its use, in Reuben’s words, is “mind-boggling” because the AI will try to approximate the rhythms and characteristics of the drums but with vocal sounds “creating an effect similar to beatboxing.”
Meanwhile, fake bands have flooded Spotify and YouTube with AI songs, including one called ‘Concubanas’ that the description claims formed in Havana in 1971 and played an original fusion of Cuban and Congolese music, before disbanding in 1992.
ReadNext
