What is Soundscape Explorer?

Soundscape Explorer is a web application that plays back mixes made using the BBC Sound Effects library mixer tool. The mixer tool lets anyone add sounds and do some simple mixing (like setting volumes, start times, and loop points). The mix can then be shared as a link. Our Soundscape Explorer tool decodes that shared link to grab the sounds from the Sound Effects library and then positions them (randomly) in 3D space to be explored. If you use the app on a mobile device, moving the device can change the viewpoint. You can use the arrow keys to move around the scene on a computer.

You can use Soundscape Explorer with mixes you've made yourself, but we launched with some ready-made soundscapes to try out.

• Winter Outside is a poem written specially for this app by BBC Research & Development's published poet, Philip Robinson. Philip's poem was read by Mary Ann Hobbs (BBC Radio 6 Music) and set to a wintry soundscape designed by Emma Young.
• A Visit from St. Nicholas is one of the best-known Christmas poems. It was read by Nick Bright (BBC Radio 1Xtra, BBC Radio 5 Live) and set to narrative sound design by Jon Francombe.
• Tropical Beach, Spring Awakening, and Coastal Winds are short natural soundscapes designed by Sian Eleri (BBC Radio 1), Jack Reynolds (BBC R&D), and Emma Young (BBC R&D), respectively.

As well as 3D audio, the soundscapes have 360-degree background images (including one that was captured by Florian Schweiger with BBC Research & Development's lightfield capture rig). A sphere represents each sound, and you can interact with the sounds to move them or change their volume.

We used the Sound Effects Mixer to find audio clips from the archive and set the timing for each effect. We then imported the mix into Soundscape Explorer to listen in 3D and position each sound source relative to the background image.

How did this work come about?

This work was inspired by our conversations with the BBC Sound Effects archive team to explore how spatial audio and augmented reality (AR) could lead to different uses of the collection and the Soundscapes for Wellbeing project, which investigates the wellbeing benefits of nature sounds.

We ran a short co-design study with our colleagues to better understand how we could use augmented reality audio with soundscapes. We hoped this would also lead to a user-led design process, ultimately producing a prototype that would interest our audiences. In the co-design study, we aimed to investigate how we could use natural soundscapes to enhance wellbeing, specifically how we could use augmented reality audio techniques to do this. We asked our participants about spatial audio, graphics, and interaction, how and if they could enhance soundscapes.

Responses from these interviews were analysed for common themes, with immersion, relaxation, and connection emerging as the key themes. Based on these themes, we chose a set of features that fulfilled the desires expressed by our participants. These features include binaural sound and visual augmented reality to enhance immersion; pre-curated soundscapes to deliver easily accessed relaxation; and the ability to share a link directly to soundscapes to improve participants' sense of connection with others. Additionally, the participants expressed a desire to customise and create their own soundscapes, leading us to add interaction with the soundscape to our set of desired features.

We created a prototype implementing these features and using visual augmented reality; a visual representation of the sounds in the mix was layered on top of the camera view of the space (so-called "magic window" AR). We decided to work in the browser and to prototype ideas from the workshop quickly we relied on existing open-source software: Google's Resonance library for dynamic spatial audio rendering and the AFrame toolkit for describing relatively simple virtual and augmented reality scenes. We could quickly create an explorable 3D scene visualising the sound effect positions on top of the real world camera view and render them to a binaural headphone signal. However, there were some stumbling blocks: the experimental WebXR API is not yet available on Apple devices, and enabling the camera view on our Android phones tended to interfere with the performance of the audio rendering.

In developing the app, we first prototyped with the augmented reality view (left), then added user interface elements (centre), and finally found the audio performance to be much better when using a static 360 image instead, especially with many sound sources (right).

So, we took the project in two directions. For a public trial on BBC Taster, we wanted the app to work as well as possible with as many devices as possible. Given the problems described above, we decided to scale back our ambitions for the augmented reality view. Instead, we represented our sound sources on a 360-degree background image while still using the phone's sensors to rotate the view. But we didn't drop the idea of audio augmented reality altogether, and we continued to build a slightly scaled-back version for some focus group testing and evaluation. That's happening now, and we should have some findings in early 2022.

Find out more

We hope you enjoy using Soundscape Explorer—either to escape into one of our authored soundscapes or to make your own. We'll use feedback from the BBC Taster pilot to inform our thinking on immersive audio experiences and add findings from this release (and the earlier prototyping) to add to a growing body of research on audio augmented reality (for example, you can read about earlier trials with augmented reality audio on the blog).

Using the sound effects mixer and positioning sound sources in a virtual space is much like the work of professional sound designers, albeit using slightly different tools (for example, digital audio workstation plugins like our EAR Production Suite) and often working with composers and original recordings.

If you're inspired to take your mixes further, why not check out some of the sessions from our recent Sounds Amazing event, including a talk on the soundscapes of BBC drama Vigil and insight from producer Benbrick into the detail of his work. If you'd like to hear how binaural sound is used in professional mixes, we have some examples in a collection of binaural audio on the BBC.