Journal Article

Speech emotion classification using feature-level and classifier-level fusion

Mishra, Siba Prasad, Deb, Suman, Warule, Pankaj

2024

Overview

Emotion plays a vital role in every living being. Understanding emotion is a very complex task for everyone, but if possible, it will work like a miracle to solve thousands of problems and save many lives. Emotion is reflected not only in the gesture but also in work and in producing an efficient result. Hence, the recognition of emotion using speech has been a topic of interest for many researchers for the last three decades. In our study, we used three features, mel frequency cepstral coefficient (MFCC), spectrogram, and mel-spectrogram, as a one-dimensional input vector to the convolutional neural network (CNN) and deep neural network (DNN) for speech emotion classification. We evaluated the accuracy of SER using the features individually and in combination with the deep learning classifiers CNN and DNN. For both CNN and DNN classifiers, the combination of features performed better than the individual features. The combination of features using the DNN classifier achieved an accuracy of 76.60%, 87.10%, 79.79%, and 100%, and using the CNN classifier achieved classification accuracy of 75%, 84.11%, 78.13%, and 100% for the RAVDESS, EMO-DB, SAVEE, and TESS datasets,respectively. Then we applied a proposed feature and classifier-level fusion method using CNN and DNN to improve emotion classification performance and achieved classification accuracy of 80.42%, 87.48%, and 80.99% on the RAVDESS, EMO-DB, and SAVEE datasets, respectively. The performance of the proposed feature and classifier-level fusion method was compared with the other methods, and it was found that the proposed method performed better than the state-of-the-art methods. Graphical abstract

• Publisher: Springer Berlin Heidelberg,Springer Nature B.V
• Subject: Accuracy / Acoustics / Algorithms / Artificial Intelligence / Artificial neural networks / Classification / Classifiers / Complex Systems / Complexity / Datasets / Deep learning / Emotions / Engineering / Machine learning / Neural networks / Original Paper / Signal processing / Speech / Support vector machines