IBC2021 Accelerator: Immersive audio and sound imagery | Industry trends


It’s very tempting to think of immersive audio as a done deal in the industry. Great strides have been made in audio reproduction in recent years, with consumer deployments of 5.1 and Dolby Atmos increasingly becoming a mass market and no longer just the preserve of audiovisual enthusiasts.

However, as this fascinating accelerator focused on immersive audio and sound imagery proves, there is still a long way to go. New object-oriented audio technologies not only raise the bar for high-end audio reproduction, but also offer the potential to dramatically improve a range of more pragmatic areas of interest to broadcasters.

“We see the delivery of the object audio not for the purpose of flying it around the living room, but for viewers to make their own mixes and be able to choose from a list of languages,” comments Karl Petermichl, Technische Direktion of Austrian broadcaster ORF, one of the co-champions of the project.

Wide support

The mix of pragmatic solution and top-of-the-line innovation is, of course, one of the factors that make IBC accelerators so compelling and thus attract champions and participants from such a wide variety of industries. This one in particular has a wide range of champions from industry and academia working together to produce three binaural POCs for the show that will showcase some of the new possibilities brought about by immersive audio from live music performance to music. 360 video and oral dramatizations. .

“We see the delivery of object audio not for the purpose of flying it around the living room, but so that viewers can make their own mixes and be able to choose from a list of languages,” Karl Petermichl, ORF.

Originally featured at the IBC Accelerators Kickstart event by independent media consultant and project manager Benjamin Schwartz, his DIY soundscape experience “Being There” helped inspire the wider scope of this challenge. “I was stranded at my mom’s house in March 2020 and deeply disappointed to miss a concert due to the lockdown,” he explains, that’s when he realized he could take advantage of his experience of working with the Ultra HD Forum and the 8K association.

“I was looking for a solution to address the millions of people stuck like me in a house with just a normal HDTV and stereo sound. I wanted a solution that would make a difference for everyone. Knowing that the sound mix of the clips is permanently fixed during a performance, even though there are varying camera angles, I started working on the ‘Being There’ project: breaking this unspoken rule of a static audio mix.

To further explore where cutting-edge audio innovation can be applied to this core concept, one area of ​​the POC will focus on a fully immersive musical experience, featuring six degrees of freedom, room spatialization, and 360VR elements. The idea is that dynamic soundscapes will accompany the camera and, thanks to the contributions of the Participants Omnilive (video) and MagicBeans (audio) project, the viewer will be able to select the angle of the camera and even move through the space in 360 °. degrees. By doing this, the audio perspective will match their movement thanks in part to the 6DOF volumetric audio feature pre-release of MagicBeans’ upcoming RoundHead software.

After hearing an ideal piece of tango-based music that could demonstrate this at a concert, Schwartz approached composer and cellist, Soizic Chevrant Breton, to re-orchestrate it for cello, violin, flute and bass clarinet, which she did so with complacency. “We got a bunch of professional musicians together and recorded it,” he says, “so I guess you could call it a world premiere of ‘Like a Tango’. “

The key software introduced into the accelerator for the express purpose of reaching the space mix is ​​Braud.io, a spin-off from co-champion Kings College London, which has developed a subset of technologies that it hopes it, will be able to support the next generation of sound. In addition to its application within the musical R&D of this accelerator, it will also be implemented in the use case of the oral creation of the POC. This will focus on a multi-narrative audio piece, specially commissioned by another Audible Champion and recorded at Twickenham Film Studios, another co-champion of this challenge. The co-founder of Braudi.io is Zoran Cvetkovic, professor of signal processing at KCL, and he lists two main challenges in the field of immersive audio broadcasting.

Read more IBC accelerators: 5G LBXR

The first, how to provide a stable auditory perspective to a listener moving through the listening environment, goes beyond this accelerator; hence the decision to present the POCs in binaural to the IBC. The second is however very discussed and concerns the techniques of capturing the acoustics of the room.

“When you observe an acoustic event in enclosed spaces, what you hear is the initial wave, then the first order reflections and the second order reflections and so on from walls and objects,” explains Cvetkovic. “It is by processing the relationships between these different reflections that the human auditory system creates a perception of where objects are in space. The more of these reflections we render the better, but this is numerically prohibitive to be done with proper precision in real time and is made more difficult by moving objects. However, we have developed a technology which makes it possible to synthesize most of them in real time using modest computing resources. “

Broadcast use case

The team behind Braud.io initially developed the technology with classical music in mind before realizing its potential in immersive environments. And it was the prospect of taking that potential, and object-oriented audio in general, and applying it to the broadcast environment that got ORF’s Petermichl excited.

“How it automatically adapts to the actual listening environment is very important to us,” he says. “In the age of smartphones and large TVs, you never know where you are listening to your program, on the subway with headphones or on a 100-inch Smart TV in a large living room in a cinematic experience. Synthesizing and adapting a given setup to the actual listening point is a very important task for us.

Immersive audio and sound images

Champions CTOI, ORF, BBC, Audible, Twickenham Film Studios, Kings College London, The Audio Engineering Society (AES), University of Surrey, University of Lethbridge, MuseumTV

Participants Omnilive, Magic Beans

  • More information on the Accelerator Media Innovation program, supported by Nvidia, is available here

Petermichl was involved in two recent EBU recommendations, R153 and R154, which carefully considered the subject of object-oriented audio in the context of broadcast production audio models for UHD / HDR file interchange. Building on this work, he sees object-oriented audio as a more complex offering that will actually result in a more streamlined and efficient living room experience.

It should also hopefully address four key issues for broadcasters. The ongoing problem of volume variation and the inability of viewers to hear the center channel; improvement of linguistic versatility; providing interactivity for storytelling purposes and being able to provide stems to provide a more immersive mix.

The importance of being there

Immersive Audio and Sound Imagery is an active accelerator that is constantly targeting itself. While originally seeking to provide dynamic soundscapes that will accompany the movement of the camera, as Schwartz reports, the list of innovations grew with the number of active participants. Testing a new approach to sound source re-specialization as well as a six-degree-of-freedom approach to sound are now both on the cards. At its core though, despite all the multiple benefits that can come from it, from solving accessibility issues to new forms of storytelling, fundamentally it remains an experiential POC.

“We see our work as a stepping stone towards immersive technologies, which may require specific equipment at the customer’s premises,” he concludes. “We don’t know if our current iteration of the technology will simply play a proof of concept role in moving the immersive agenda forward or if it will be used for several years, maybe even commercially. This was not our original goal. We just want to rediscover that wonderful feeling of being there when the audience understands the artistic intent of a live artist.

IBC Digital offers industry insight and the opportunity to engage with exhibitors before and during IBC2021. register here

Leave A Reply

Your email address will not be published.