Abstract
Automated Audio captioning (AAC) is a cross-modal translation task that aims
to use natural language to describe the content of an audio clip. As shown in
the submissions received for Task 6 of the DCASE 2021 Challenges, this problem
has received increasing interest in the community. The existing AAC systems are
usually based on an encoder-decoder architecture, where the audio signal is
encoded into a latent representation, and aligned with its corresponding text
descriptions, then a decoder is used to generate the captions. However,
training of an AAC system often encounters the problem of data scarcity, which
may lead to inaccurate representation and audio-text alignment. To address this
problem, we propose a novel encoder-decoder framework called Contrastive Loss
for Audio Captioning (CL4AC). In CL4AC, the self-supervision signals derived
from the original audio-text paired data are used to exploit the
correspondences between audio and texts by contrasting samples, which can
improve the quality of latent representation and the alignment between audio
and texts, while trained with limited data. Experiments are performed on the
Clotho dataset to show the effectiveness of our proposed approach.