Clustering of acoustic environments with variational autoencoders

Abstract Traditional acoustic environment classification relies on (i) classical signal processing algorithms, which are unable to extract meaningful representations of high-dimensional data; or on (ii) supervised learning, limited by the availability of labels. Knowing that human-imposed labels do not always reflect the true structure of acoustic scenes, we explore the potential of (unsupervised) clustering of acoustic environments using variational autoencoders (VAEs). We employ a VAE model for categorical latent clustering with a Gumbel-Softmax reparameterization which can operate with a time-context windowing scheme for lower memory requirements, relevant to resource constrained applications. Additionally, general adaptations on VAE architectures for audio clustering are also proposed. The approaches are validated through the clustering of spoken digits, a simpler task where labels are meaningful, and urban acoustic scenes, where the recordings present strong overlap in time and frequency. While all variational methods succeeded when clustering spoken digits, only the proposed model achieved effective clustering performance on urban acoustic scenes, given its categorical nature.