Handbook of audio technology – www.audio-technology.info

Chapter: Binaural Technology

_{Broken links? Missing or wrong information? Tell Fabian.}

Software for dynamic real time rendering

Open source / free

Spatial Audio Real-time Applications (SPARTA): Plug-Ins for binaural rendering, Ambisonics rendering to loudspeakers and headphones, and parametric audio reproduction (cross platform, support for custom SOFA files). [1]
IEM Plug-in suite: Plug-ins for rendering Ambisonics signals to loudspeakers and headphones audio effects for Ambisonics (cross platform, support for custom SOFA files).
Sound Scape Renderer (SSR): Stand-alone software for sound-field synthesis and rendering binaural impulse responses (Unix based operating systems) [2]
ambiX: Ambisonics plug-in suite (cross platform).
3D Tune-In Toolkit: VST/Unity Plugin and standalone application for rendering binaural impulse responses, binaural reverberation, and simulating hearing loss and hearing aids (cross-platform, support for custom SOFA files, [3]).
Anaglyph: VST and Unity Plug-in for HRIR based spatialization including near-field rendering and ITD cosutomization (cross platform).
Ear production suite: Plug-ins for object oriented audio production based on the Audio Definition Model with a soon to be released binaural rendering plug-in (cross platform)
Steam Audio Unity Plug-in Plugin: 3D-Audio framework for real-time room simulation and binaural rendering in game engines (cross platform, including C-API).
Oculus Spatializer: 3D-Audio framework for real-time room simulation and binaural rendering in game engines and audio workstations (cross platform).
Google Resonance Audio: 3D-Audio framework for real-time room simulation and binaural rendering in game engines, audio workstations, web application, and on mobile devices (cross platform).
Virtual Acoustics: Auralization framework for research purposes (Windows, can be used on other platforms).
Virtual Acoustic Spaces Unity Spatializer: Unity plug in for the auralization of binaural impulse responses and ray tracing (cross platform).

Proprietary

Dear Reality: Audio workstation and unity plug-ins for 3D audio production.
Waves: Plug-ins for 3D audio production.
L-ISA Studio: Plug-ins for 3D audio reproduction
Magic Leap Soundfield Audio: Binaural rendering and acoustic simulation for VR/AR, gaming, and 3D video (cross platform).
IRCAM Spat: Spatialization for musical creation, post-production, and live performances in Max/MSP including binaural rendering, reverberation, Ambisonics panning and sound field synthesis.

Note that many digital audio workstations already include basic plug-ins for binaural rendering and that a variety of spatial audio modules are available for PureData and Max/MSP. A more extensive list or real time applications can be found at https://en.everybodywiki.com/List_of_Ambisonic_software.

VR/AR audio content

Symphonic orchestra: Anechoic recordings of a symphonic orchestra. [4]
Italian opera: Anechoic Italian opera recordings. [5]
Small ensemble: Anechoic audio and 3D video recordings of small ensemble performances. [6]
Guitar and vocals: Anechoic recordings of guitar and vocal performances. [7]
Castanets: Anechoic castanet rhythm produced with samples from the Vienna Symphonic Library.
Drama: Three object-based audio drama scenes. The source movements and metadata required to render the scenes are stored using the Audio Definition Model.
Dramatic speech: Anechoic recording of Cicero’s 3rd Cataline oration. [8]
Ambisonics Recordings of Typical Environments (ARTE): Multi-channel higher-order ambisoncs recordings of 13 realistic environments. [9]

Binaural and array impulse responses

SOFA conventions: Collection of binaural impulse responses including HRIRs, BRIRs, and HpIRs stored in the SOFA file format for which APIs for a variety of languages exists (C, Python, and Matlab/Octave, etc.).

VR/AR headphones and head tracker

Apple AirPods Max: Head tracked headphones for MAC OS and iOS.
Samsung Galaxy Buds Pro: Head tracked earphones for Android.
Supperware Head Tracker 1: Head tracker that works with the SPARTA and IEM plug-ins including and open and flexible API
Waves NX head tracker: Head tracker for Waves audio plug-ins.
Mr. Head Tracker: Open manual and code for a cross platform head tracker.
Headrot: Open manual and code for a cross platform head tracker.
Virtual Acoustic Spaces head tracker (VAStracker): Open manual and code for a cross platform head tracker.

Binaural microphones

Sennheiser AMBEO Smart Headset: Microphones for individual recordings. es not capture all spatial cues from the outer ear because they partly obstruct the conchae. Similar products are available from other manufacturers.
3DIO Binaural Microphone: Microphones for non-individual binaural recordings. Does not capture all spatial cues due to the missing acoustic shadow of the head.
miniDSP Binaural Microphone: Microphones for non-individual binaural recordings. Does not capture all spatial cues due to the missing acoustic shadow of the head.
Binaural Enthusiast: Artificial head for non-individual binaural recordings including all spatial cues.
Neumann KU 100: Artificial head for non-individual binaural recordings including all spatial cues.
PIRATE: Open manual for building 3D printed in-ear microphones capturing all spatial cues. [10]

Microphone arrays

Sennheiser AMBEO VR Microphone: First order ambisoncs microphone with four capsules. Similar products are available from other manufacturers.
Core Sound Octomic: Second order Ambisonics microphone and VST plug-ins
Zylia ZM 1: Third order ambisoncs microphone with 19 capsules.
mh acoustics EigenMike: Fourth order ambisoncs mirophone with 32 capsules

Other Hardware

Klang: Immersive and real-time in ear mixing for stage performances

Data and formats

Spatially Oriented Format for Acoustics (SOFA): Container format for saving data such as HRIRs/HRTFs, BRIRs/BRTFs, or musical instrument directivities including versatile Python and Matlab/Octave APIs and basic C-support.
Audio Definition Model (ADM): Container format for saving static and dynamic meta-data such as room acoustic parameters and source positions.
MPEG-H: Standard for converting between a variety of 3D audio input and output formats. [11]

Toolboxes and APIs

Spatial Audio Framework (SAF): C/C++ framework for ambisonics encoding/decoding, spherical array processing, amplitude-panning, HRIR processing.
3D Tune-In Toolkit: C++ library for spatialization and simulation of hearing loss and hearing aids. [3]
atmoky: Web and native SDKs for spatial audio rendering
Versatile Interactive Scene Renderer (VISR) C/C++ framework with Python binding for spatialization based on VBAP. [12]
Sound Field Analysis Toolbox (SOFiA): Python/Matlab Toolbox for analyzing spherical microphone array signals. [13]
Real-time Spherical Array Renderer (ReTiSar): Python package for spatialization of spherical microphone array signals. [14]
(Binaural) Spatial Decomposition Method (SDM, BSDM): Matlab toolbox for auralization of spatial room impulse responses. [15] & [16]
Spatial Upsampling by Directional Equalization (SUpDEq): Matlab toolbox for processing sparse HRTF data sets using SUpDEq and other methods. [17]
pyBinSim: Python package for auralization of binaural impulse responses. [18]
pyfar: Python packages for acoustic research.
AKtoools: Matlab Toolbox for acquisition, inspection, and processing of acoustic signals. [19]
ITA-Toolbox: Matlab Toolbox for acquisition, inspection, and processing of acoustic signals. [20]

Bibliography

Tervo, S., Pätynen, J., Kuusinen, A., & Lokki, T. (2013). Spatial decomposition method for room impulse responses. J. Audio Eng. Soc., 61(1/2), 17–28.

Lindau, A., & Weinzierl, S. (2012). Assessing the Plausibility of Virtual Acoustic Environments. Acta Acustica United with Acustica, 98(5), 804–810. https://doi.org/10.3813/AAA.918562

Franck, A., Costantini, G., Pike, C., & Fazi, F. M. (2018). An Open Realtime Binaural Synthesis Toolkit for Audio Research. 144th AES Convention.

Geier, M., & Spors, S. (2012, November). Spatial Audio Reproduction with the SoundScape Renderer. 27th Tonmeistertagung - VDT International Convention. https://www.int.uni-rostock.de/fileadmin/user_upload/publications/spors/2012/Geier_TMT2012_SSR.pdf

Böhm, C., Fiedler, F., Weinzierl, S., Holter, E., Muth, S., Schaefer, U., & Schwesinger, S. (2019). An Anechoic Recording of Cicero’s 3rd Cataline Oration: Italian, Latin and German. Technical University of Berlin. https://doi.org/10.14279/depositonce-8536

Weisser, A., Buchholz, J. M., Oreinos, C., Badajoz-Davila, J., Galloway, J., Beechey, T., & Keidser, G. (2019). The ambisonics recordings of typical environments (ARTE) database. Acta Acust. United Ac., 105(4), 695–713. https://doi.org/10.3813/AAA.919349

Amengual Garí, S. V., Arend, J. M., Calamia, P. T., & Robinson, P. W. (2020). Optimizations of the Spatial Decomposition Method for Binaural Reproduction. J. Audio Eng. Soc., 68(12), 959–976. https://doi.org/10.17743/jaes.2020.0063

Bernschütz, Benjamin, Pörschmann, C., Spors, S., & Weinzierl, S. (2011). SOFiA Sound field analysis toolbox. Int. Conf. Spatial Audio (ICSA).

Woirgardt, M., Stade, P., Amankwor, J., Bernschütz, B., & Arend, J. M. (2012). Cologne University of Appliedd Sciences - Anechoic recordings. https://audiogroup.web.th-koeln.de/anechoic.html

Thery, D., & Katz, B. F. G. (2019). Anechoic audio and 3D-video content database of small ensemble performances for virtual concerts. 23rd Int. Congress on Acoustics, 739–746.

Helmholz, H., Lübeck, T., Ahrens, J., Amengual Garí, S. V., Alon, D. L., & Mehra, R. (2020). Updates on the Real-Time Spherical Array Renderer (ReTiSAR). Fortschritte Der Akustik – DAGA 2020, 1169–1172.

Böhm, C., Ackermann, D., & Weinzierl, S. (2021). A Multi-channel Anechoic Orchestra Recording of Beethoven’s Symphony No. 8 op. 93. J. Audio Eng. Soc., 68(12), 977–984.

D’Orazio, D. (2020). Anechoic recordings of Italian opera played by orchestra, choir, and soloists. J. Acoust. Soc. Am., 147(2), EL157–EL163. https://doi.org/10.1121/10.0000739

McCormack, L., Pulkki, V., Politis, A., Scheuregger, O., & Marschall, M. (2020). Higher-Order Spatial Impulse Response Rendering: Investigating the Perceived Effects of Spherical Order, Dedicated Diffuse Rendering, and Frequency Resolution. J. Audio Eng. Soc., 68(5), 338–354. https://doi.org/10.17743/jaes.2020.0026

Brinkmann, F., & Weinzierl, S. (2017, May). AKtools – An Open Software Toolbox for Signal Acquisition, Processing, and Inspection in Acoustics. 142nd AES Convention, Convention e-Brief 309.

Berzborn, M., Bomhardt, R., Klein, J., Richter, J.-G., & Vorländer, M. (2017). The ITA-Toolbox: An open source MATLAB toolbox for acoustic measurements and signal processing. Fortschritte Der Akustik – DAGA 2017, 222–225.

AES Standards Comittee. (2020). AES69-2020: AES standard for file exchange - Spatial acoustic data file format. Audio Engineering Society, Inc.

Denk, F., Brinkmann, F., Stirnemann, A., & Kollmeier, B. (2019). The PIRATE: An anthropometric earPlug with exchangeable microphones for Individual Relaiable Acquisition of Transfer functions at the Ear canal entrance. Fortschritte Der Akustik – DAGA 2019, 635–638. https://doi.org/10.5281/zenodo.2574395

Cuevas-Rodríguez, M., Picinali, L., González-Toledo, D., Garre, C., de la Rubia-Cuestas, E., Molina-Tanco, L., & Reyes-Lecuona, A. (2019). 3D Tune-In Toolkit: An open-source library for real-time binaural spatialisation. PLoS ONE, 14(3), 1–37. https://doi.org/10.1371/journal.pone.0211899

Neidhardt, A., Klein, F., Knoop, N., & Köllmer, T. (2017, May). Flexible Python Tool for Dynamic Binaural Synthesis Applications. 142nd AES Convention, e-Brief 346.