Nathan Schilaty

Speech emotion recognition (SER) is an important application in Affective Computing and Artificial Intelligence. Recently, there has been a significant interest in Deep Neural Networks using speech spectrograms. As the two-dimensional representation of the spectrogram includes more speech characteristics, research interest in convolution neural networks (CNNs) or advanced image recognition models is leveraged to learn deep patterns in a spectrogram to effectively perform SER. Accordingly, in this study, we propose a novel SER model based on the learning of the utterance-level spectrogram. First, we use the Spatial Pyramid Pooling (SPP) strategy to remove the size constraint associated with the CNN-based image recognition task. Then, the SPP layer is deployed to extract both the global-level prominent feature vector and multi-local-level feature vector, followed by an attention model to weigh the feature vectors. Finally, we apply the ArcFace layer, typically used for face recognition, to the SER task, thereby obtaining improved SER performance. Our model achieved an unweighted accuracy of 67.9% on IEMOCAP and 77.6% on EMODB datasets.

Speech signal analysis to support objective clinical decision-making has gained immense interest, especially in neurological disorders. This research assessed the feasibility of speech analysis on the detection of concussions. Using a speech dataset from 82 concussed and 82 healthy participants, we extracted two speech feature sets focusing on Mel Frequency Cepstral Coefficients (MFCCs) to characterize speech articulation. A machine learning pipeline was developed to discriminate concussion speech from healthy speech by applying Support Vector Machine (SVM), K-Nearest Neighbors (KNN), and Decision Tree (DT) classifiers. All three classifiers trained on the MFCC-based feature set achieved Matthew's correlation coefficient score above 0.5 on the holdout data set. DT model achieved a 78% sensitivity and 75% specificity. The findings of this research serve as proof-of-concept for speech analysis of concussion detection.