Augmented/Mixed Reality Audio for Hearables: Sensing, control, and rendering

Rishabh Gupta; Jianjun He; Rishabh Ranjan; Woon-Seng Gan; Annika Neidhardt; Florian Klein; Christian Schneiderwind; Vesa Välimäki; Karlheinz Brandenburg

doi:10.1109/MSP.2021.3110108

Back

Augmented/Mixed Reality Audio for Hearables: Sensing, control, and rendering

Journal article

Peer reviewed

Augmented/Mixed Reality Audio for Hearables: Sensing, control, and rendering

Rishabh Gupta, Jianjun He, Rishabh Ranjan, Woon-Seng Gan, Annika Neidhardt, Florian Klein, Christian Schneiderwind, Vesa Välimäki and Karlheinz Brandenburg

IEEE Signal Processing Magazine, Vol.39(3), pp.63-89

05/2022

DOI: https://doi.org/10.1109/MSP.2021.3110108

Abstract

Extended Reality

Virtual Acoustics

Headphones

Hearables

Binaural Synthesis

6DOF

Augmented Reality

Augmented or mixed reality (AR/MR) is emerging as one of the key technologies in the future of computing. Audio cues are critical for maintaining a high degree of realism, social connection, and spatial awareness for various AR/MR applications, such as education and training, gaming, remote work, and virtual social gatherings to transport the user to an alternate world called the metaverse. Motivated by a wide variety of AR/MR listening experiences delivered over hearables, this article systematically reviews the integration of fundamental and advanced signal processing techniques for AR/MR audio to equip researchers and engineers in the signal processing community for the next wave of AR/MR.

Files and links (1)

url

https://ieeexplore.ieee.org/document/9770284View

IEEE Xplore

Metrics

1 Record Views

Details

Title: Augmented/Mixed Reality Audio for Hearables: Sensing, control, and rendering
Creators: Rishabh Gupta (Corresponding Author) - Nanyang Technological University
Jianjun He (Author) - Nanyang Technological University
Rishabh Ranjan (Author) - Nanyang Technological University
Woon-Seng Gan (Author) - Nanyang Technological University
Annika Neidhardt (Author) - Technische Universität Ilmenau
Florian Klein (Author) - Technische Universität Ilmenau
Christian Schneiderwind (Author) - Technische Universität Ilmenau
Vesa Välimäki (Author) - Aalto University
Karlheinz Brandenburg (Author) - Technische Universität Ilmenau
Publication Details: IEEE Signal Processing Magazine, Vol.39(3), pp.63-89
Publisher: Institute of Electrical and Electronics Engineers (IEEE)
Number of pages: 27
First online publication date: 06/05/2022
Publication Date: 05/2022
Grants: Automatic Characterization of Acoustic Environments, 5575/10-16, Thüringer Ministerium für Wirtschaft, Wissenschaft und Digitale Gesellschaft (Germany, Erfurt)
Influence of translatory motion of listeners on the perception of virtual acoustic environments, BR 1333/18-1, Deutsche Forschungsgemeinschaft (Germany, Bonn) - DFG
Grant note: Rishabh Gupta and Woon-Seng Gan would like to acknowledge the support by the Singapore Ministry of Education Academic Research Fund Tier 2, under research grant MOE2017-T2-2-060. The work of Vesa Välimäki is part of the activities of the Nordic Sound and Music Computing Network funded by NordForsk (Aalto University project 86892). The work of Annika Neidhardt, Christian Schneiderwind, Florian Klein, and Karlheinz Brandenburg was funded by the German Research Foundation (projects BR1333/14-1 and BR1333/18-1) and the Free State of Thuringia, Germany (FKZ: 5575/10-16).
Identifiers: 99944466302346; WOS:000803108800010
Academic Unit: Department of Music and Media
Language: English
Resource Type: Journal article

Augmented/Mixed Reality Audio for Hearables: Sensing, control, and rendering

Abstract

Files and links (1)

Metrics

Details

Usage Policy