SeeHear: Signer Diarisation and a New Dataset

Samuel Albanie; Gul Varol; Liliane Momeni; Triantafyllos Afouras; Andrew Brown; Chuhan Zhang; Ernesto Coto; Necati Cihan Camgoz; Ben Saunders; Abhishek Dutta; Neil Fox; Richard Bowden; Bencie Woll; Andrew Zisserman

doi:10.1109/ICASSP39728.2021.9414856

Back

SeeHear: Signer Diarisation and a New Dataset

Conference proceeding

Open access

SeeHear: Signer Diarisation and a New Dataset

Samuel Albanie, Gul Varol, Liliane Momeni, Triantafyllos Afouras, Andrew Brown, Chuhan Zhang, Ernesto Coto, Necati Cihan Camgoz, Ben Saunders, Abhishek Dutta, …

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp.2280-2284

2021 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2021) (Toronto, Ontario, Canada, 06/06/2021–11/06/2021)

07/2021

DOI: https://doi.org/10.1109/ICASSP39728.2021.9414856

Abstract

Assistive technology

Benchmark testing

Conferences

Gesture recognition

Sign Language Datasets

Signer Diarisation

Speech processing

Acoustics

Signal Processing

In this work, we propose a framework to collect a large-scale, diverse sign language dataset that can be used to train automatic sign language recognition models.The first contribution of this work is SDTrack, a generic method for signer tracking and diarisation in the wild. Our second contribution is SeeHear, a dataset of 90 hours of British Sign Language (BSL) content featuring more than 1000 signers, and including interviews, monologues and debates. Using SDTrack, the SeeHear dataset is annotated with 35K active signing tracks, with corresponding signer identities and subtitles, and 40K automatically localised sign labels. As a third contribution, we provide benchmarks for signer diarisation and sign recognition on SeeHear.

Files and links (2)

pdf

albanie2021icassp5.30 MBDownload View

Open Access

url

https://www.2021.ieeeicassp.org/2021.ieeeicassp.org/index.htmlView

Event WebsiteConference website

Metrics

314 File views/ downloads

83 Record Views

See more details

Details

Title: SeeHear: Signer Diarisation and a New Dataset
Creators: Samuel Albanie - Oxford Research Group
Gul Varol - Oxford Research Group
Liliane Momeni - University of Oxford,Visual Geometry Group,UK
Triantafyllos Afouras - University of Oxford,Visual Geometry Group,UK
Andrew Brown - University of Oxford,Visual Geometry Group,UK
Chuhan Zhang - University of Oxford,Visual Geometry Group,UK
Ernesto Coto - University of Oxford,Visual Geometry Group,UK
Necati Cihan Camgoz - University of Surrey, School of Computer Science and Electronic Engineering
Ben Saunders - University of Surrey, School of Computer Science and Electronic Engineering
Abhishek Dutta - University of Oxford,Visual Geometry Group,UK
Neil Fox - University College London
Richard Bowden - University of Surrey, School of Computer Science and Electronic Engineering
Bencie Woll - University College,Deafness, Cognition and Language Research Centre,London,UK
Andrew Zisserman - Oxford Research Group
Publication Details: ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp.2280-2284
Conference: 2021 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2021) (Toronto, Ontario, Canada, 06/06/2021–11/06/2021)
Publisher: Institute of Electrical and Electronics Engineers (IEEE)
Number of pages: 5
First online publication date: 13/05/2021
Publication Date: 07/2021
Date accepted for publication: 29/01/2021
Grants: Engineering and Physical Sciences Research Council (United Kingdom, Swindon) - EPSRC
Grant note: This work was supported by EPSRC grant ExTol. The authors would like to thank Himel Chowdhury for his assistance with sign annotation.
Identifiers: 99550422002346; WOS:000704288402106
Academic Unit: School of Computer Science and Electronic Engineering
Language: English
Resource Type: Conference proceeding

SeeHear: Signer Diarisation and a New Dataset

Abstract

Files and links (2)

Metrics

Details

Usage Policy