Inspiring Creativity With VoIP

Welcome to the Sound On Sound podcast about electronic music and all things synth. After more than a hundred years of electronic music, is there still room for innovation? Are there still new sounds waiting to be discovered? New techniques waiting to be explored? The album U. I. by Olivier Allery and Johannes Malfatti shows that there are. We invited Olivier and Johannes to tell us about it in their own words. Hi. My name is Olivier Alary. And my name is Johannes Malfatti. We are here to present you the story behind the sound of UI, our collaborative album, which has just been released on 130701, a sub label of Fat Cat Records, focusing on post classical music. Johannes and I are composers often working together on different projects such as music albums and film music. I live and work in Montreal, and I live in Berlin. For 20 years, we've been using Skype as a tool to collaborate. And so for this record, we decided to use voice over IP technology, or VoIP, as a creative audio tool and a conceptual frame. So to put it plainly, we use Skype as an effect unit, basically. And in this podcast, we're going to talk about the process behind the album. Johannes and I are friends and we've been working together for the past 20 years. As I'm based in Montreal and Johannes in Berlin, it's quite rare for us to be in the same physical space. So, from the moment we met, we used internet as a tool to make music together. At first, we used email, chat, and then 15 years ago, when Skype appeared, we used it right away. It was the first time we could see each other and exchange musical ideas across the Atlantic. in real time. It thus allowed us to cement a friendship and our musical collaboration. And at the time, this made us dream of a network based studio bridging our two cities together. But really quickly, we noticed more the limitations of VoIP technology and started to poke fun at its distorted and glitchy sound quality. The way Skype or other VoIP software work is that they encode the audio in function of the available internet bandwidth. So when the internet signal is weak, the encoder will drop as much information as it can get away with. Therefore, the audio quality can become quite bad and in summer of 2010, while working on a film project together, we had Skype running in the background and suddenly the video dropped and Olivier's voice started to sound really robot like as if it was going through some sort of vocoder or something. And we were used to the audio latency problems and Skype dropouts, but that was some next level. All the transients were completely unrecognizable, and it barely sounded like a human being. I told Olivier I couldn't understand anything he was saying, and as a joke, he started doing some sort of body percussion, and it sounded completely alien. So I did a quick recording setup and asked him to play again the patterns he was doing with his hands on his thighs and foot on the floor. This is the recording of the body percussion through the Skype distortion. So we just laughed about it and left the file on our hard drive somewhere, not thinking that it was anything worth using. But it was quite funny. Seven years later, while working on another project, we had a conversation on Skype. And Johannes, this time, was playing accordion through his studio setup. Then suddenly, the network started becoming unstable again. The sound of the Skype sounded so mangled that I decided to capture it on the spot. Here's the recording. In this case, the sound of the accordion had lost all original characteristics. It felt it was spontaneously emerging from the void, and it had so much added character. In our music, we often use different forms of sonic degradation. It can be tape distortion, like using dictaphone. physical manipulation, such as prepared instruments, or it could be esoteric microphone techniques. So it felt that this time we encountered something new and special that could be used musically, but the outcome was not really clear at all. It was at one point when I was working on the mix of one of Olivier's film scores, that the sounds produced by Skype started to evoke the idea of doing an album using these artifacts. So while we were on Skype, Olivier played a composition in his studio through his speakers and the audio got picked up by his webcam's microphone and the result played back on my laptop speakers was hardly recognizable as the piece that I knew. So seemingly Skype had trouble differentiating the acoustic instruments present in the piece and the diffusion through my laptop added even more blur. As I felt this was quite special, I tried to capture the actual auditive experience I had by placing a pair of stereo mics in front of my laptop and then re recorded the laptop speakers. I added some reverb later and this is the result. Here's the original piece written for piano, clarinets, and slide guitar. And the VoIPified version recorded off of my laptop speaker. Having discovered this haunting sound by accident, the concept of the album started to emerge gradually. We came to realize that, due to the distance between us, we spend way more time online together than in the same physical space. And looking around us at the explosion of personal video feeds online, we felt that a lot of people were sharing the same reality. Our lives are increasingly atomized and our social interaction is becoming more and more mediated. And we thought that the sound of VoIP degradation through faulty networks, that we both found so haunting and nostalgic, Could represent the distance between us and by extension the distance between all these other voices on the internet trying to connect to one another. This gave us the idea that it might be possible to build an album narrative of a person's whole life via disconnected voice fragments that we found posted online. When the concept of the album became clear, we had to find a way to replicate the accidents we witnessed using Skype, so that we could incorporate them into our compositions. These artifacts occurred because of the instability of our internet connection. So we had to recreate some form of bandwidth reduction to process sound. Our first idea was to broadcast sound between our studios while exchanging large files and streaming too many online videos. Such a system was quite esoteric and hard to quantify. We wrote down the number of minimum files and videos needed to get an interesting sonic result. But it felt quite arbitrary and random. After experimenting for a while, we found ways of directly controlling our internet bandwidth with software simulating adverse networking environments. Usually this software are used to recreate a wide variety of networks to test apps for smartphones. With them, you can change the network environment by limiting uplink or download bandwidth, latency, etc. So now that we could artificially control our bandwidth, we recorded more performances and reamped existing recordings through Skype. We sensed the sound in real time from one studio to another, while playing with bandwidth reduction in order to generate accidents. It was like using Skype as a sound plugin. And the beauty of such process is how unique each take is. Every time we ran audio through a Skype loop, the sound was more or less the same, but new and unexpected artifacts appeared differently. It was almost as if you were using a unique piece of mangled tape each time, for each take. So, here's an example of one of the voices we found online, and then processed using our system. Here's the dry signal. I'm super bored, I can't even see myself. I'm lying, I got a shitload of hectares. Hold on to hope if you can, and they say that dreams I can't sing though. Then the processed version. I'm super blind, I can't even see myself. Am I a god or shit, what the heck? Am I a god? Hold on to a hope Maybe say that to me I can't sing Here's another example of a detuned organ take we did, then processed using Skype. Here's the dry signal, then the processed version. What you hear in these examples Is the sonic degradation caused by the codec, used by Skype, WhatsApp, and other VoIP software. And for those who don't know, the function of a code deck is to be a coder decoder. A coder encodes a signal for transmission, and the decoder function reverses the encoding for playback. The Skype codec can scale from low bitrate, narrowband speech to high quality audio according to the nature and health of the network. But more importantly, the codec used in Skype is intended for speech, not music. It's based on the model of the human voice. This specialization allows It has high compression rates and is fairly good for speech, but can sound weird when processing other sounds, such as music, as it's not designed for it. Another issue is how the codec deals with the difference between loud and quiet moments in music. Gentle dynamics may be treated as noise and filtered out, or distorted by a denoise filter, or partially cut out by a voice activity detection algorithm that basically stops transmission. If the signal is below a certain level to minimize bandwidth usage. Beside the Skype degradation techniques we did, we also incorporated our webcam microphones in our sessions in order to add a certain color or spatial information. As the Skype audio feed is in mono, in order to obtain some stereo information, we reamped some of our processed sounds using guitars and microphones to add phase distortion and phase information. So for example, this is a piano recording sent through the Skype loop and then to a guitar amp, and then it's miked with a pair of condensers. At some point, we also built a feedback delay system using two computers and two internet phones, sending audio back and forth to our studios in Montreal and Berlin. The delay time was fixed and determined by the speed of transmission, and the feedback amount was controlled on my side of the loop. What you hear is the sound of a piano, passing through countless servers, running through endless undersea cables and reappearing on the other side of the Atlantic, going this time through Olivier's phone, then returning back to my studio after thousands of miles of technology, just to be sent down the same path over and over again. Beside using webcam microphones. and VoIP codecs to distort the sound of our instruments. We also try to incorporate the idea of these artifacts into the writing of the compositions themselves. We ask ourselves how to recreate the warbling, the dropouts, the unsteady pitch of VoIP for strings or wind instruments. For strings, we ask the musicians to change technique while playing to get a thinner or more fragile sound. For example, to shift from a bow position over the fingerboard to a position close to the bridge, thus creating an unsteady texture. And for the winds, we ask the musicians to incorporate a lot of breath in their sound, and to exaggerate gliding between notes at certain times. With this approach, we try to blur what was processed and what was generated acoustically on this album. The idea was to create a fragile meshing. Between the acoustic and the digital world. A sort of sonic distant utopia. Ten years ago, when we laughed at the sound of body percussion going through Skype, we had no idea we would base an entire album on that same process. So as a nod, we decided to incorporate the body percussion accident. In the final piece we composed for the album, thank you for listening and be sure to check out the show notes page for this episode where you'll find further information along with web links and details of all the other episodes. And just before you go, let me point you to the soundonsound. com forward slash podcasts website page, where you can explore what's playing on our other channels.