Deep learning methods have achieved high performance in sound recognition tasks. Deciding how to feed the training data is important for further performance improvement. We propose a novel learning method for deep sound recognition: Between-Class learning (BC learning). Our strategy is to learn a discriminative feature space by recognizing the between-class sounds as between-class sounds. We generate between-class sounds by mixing two sounds belonging to different classes with a random ratio. We then input the mixed sound to the model and train the model to output the mixing ratio. The advantages of BC learning are not limited only to the increase in variation of the training data; BC learning leads to an enlargement of Fisher’s criterion in the feature space and a regularization of the positional relationship among the feature distributions of the classes. The experimental results show that BC learning improves the performance on various sound recognition networks, datasets, and data augmentation schemes, in which BC learning proves to be always beneficial. Furthermore, we construct a new deep sound recognition network (EnvNet-v2) and train it with BC learning. As a result, we achieved a performance surpasses the human level. Sound recognition has been conventionally conducted by applying classifiers such as SVM to local features such as MFCC or log-mel features BID13 BID27 BID14 . Convolutional neural networks (CNNs) BID12 , which have achieved success in image recognition tasks BID11 BID22 BID8 , have recently proven to be effective in tasks related to series data, such as speech recognition BID0 BID18 BID4 and natural language processing BID10 . Some researchers applied CNNs to sound recognition tasks and achieved high performance BID2 BID3 BID25 .The amount and quality of training data and how to feed it are important for machine learning, particularly for deep learning. Various approaches have been proposed to improve the sound recognition performance. The first approach is to efficiently use limited training data with data augmentation. Researchers proposed increasing the training data variation by altering the shape or property of sounds or adding a background noise BID25 BID20 . Researchers also proposed using additional training data created by mixing multiple training examples BID15 BID24 . The second approach is to use external data or knowledge. BID2 proposed learning rich sound representations using a large amount of unlabeled video datasets and pre-trained image recognition networks. The sound dataset expansion was also conducted BID21 BID17 BID6 .In this paper, as a novel third approach we propose a learning method for deep sound recognition: Between-Class learning (BC learning). Our strategy is to learn a discriminative feature space by recognizing the between-class sounds as between-class sounds. We generate between-class sounds by mixing two sounds belonging to different classes with a random ratio. We then input the mixed sound to the model and train the network to output the mixing ratio. Our method focuses on the characteristic of the sound, from which we can generate a new sound simply by adding the waveform data of two sounds. The advantages of BC learning are not limited only to the increase in variation of the training data; BC learning leads to an enlargement of Fisher's criterion BID5 (i.e., the ratio of the between-class distance to the within-class variance) in the feature space, and a regularization of the positional relationship among the feature distributions of the classes.The experimental results show that BC learning improves the performance on various sound recognition networks, datasets, and data augmentation schemes, in which BC learning proves to be always beneficial. Furthermore, we constructed a new deep sound recognition network (EnvNet-v2) and trained it with BC learning. As a result, we achieved a 15.1% error rate on a benchmark dataset ESC-50 (Piczak, 2015b), which surpasses the human level.We argue that our BC learning is different from the so-called data augmentation methods we introduced above. Although BC learning can be regarded as a data augmentation method from the viewpoint of using augmented data, the novelty or key point of our method is not mixing multiple sounds, but rather learning method of training the model to output the mixing ratio. This is a fundamentally different idea from previous data augmentation methods. In general, data augmentation methods aim to improve the generalization ability by generating additional training data which is likely to appear in testing phase. Thus, the problem to be solved is the same in both training and testing phase. On the other hand, BC learning uses only mixed data and labels for training, while mixed data does not appear in testing phase. BC learning is a method to improve the classification performance by solving a problem of predicting the mixing ratio between two different classes. To the best of our knowledge, this is the first time a learning method that employs a mixing ratio between different classes has been proposed. We intuitively describe why such a learning method is effective and demonstrate the effectiveness of BC learning through wide-ranging experiments. We proposed a novel learning method for deep sound recognition, called BC learning. Our method improved the performance on various networks, datasets, and data augmentation schemes. Moreover, we achieved a performance surpasses the human level by constructing a deeper network named EnvNet-v2 and training it with BC learning. BC learning is a simple and powerful method that improves various sound recognition methods and elicits the true value of large-scale networks. Furthermore, BC learning is innovative in that a discriminative feature space can be learned from betweenclass examples, without inputting pure examples. We assume that the core idea of BC learning is generic and could contribute to the improvement of the performance of tasks of other modalities.
We propose an novel learning method for deep sound recognition named BC learning. Authors defined a new learning task that requires a DNN to predict mixing ratio between sounds from two different classes to increase discriminitive power of the final learned network. Proposes a method to improve the performance of a generic learning method by generating "in between class" training samples and presents the basic intuition and necessity of the proposed technique.