Human Emotion Detection Through Hybrid Approach

Improper synchronisation, data correlation and relationship between human emotions are common issues in emotion detection via facial expressions in speech processing. These issues are more critical in real-world environment due to different intensity of emotions issue, image resolution, random additive noise and data masking factors. Moreover, emotion detection, through speech and facial expressions by conventional techniques seem to hamper emotion detection accuracy and robustness in the real-world environment. An efficient emotion detection technique should consider minimising the issues mentioned above, by eliminating the input noise from the real-world conditions, as well as dealing with facial expressions during speech. Current literature lacks suggestions for emotion detection mechanisms that can solve the aforementioned issues in a combinatory way. Henceforth, this research addresses these emotion detection issues (under real-world conditions) collectively without affecting each other negatively. To maximise emotion detection accuracy and robustness, this research proposes two new feature extraction techniques for speech and facial expressions. Experiments were conducted using standard databases (DaFEx, ENTERFACE, Cohn-Kanade + CSC corpus, IITK + Emo-Db) to validate the proposed technique. The proposed hybrid approach offered higher emotion detection accuracy than other techniques by producing an average overall accuracy between the range of 82 and 87. Furthermore, it also offered higher robustness against the real-world conditions by maintaining a lower average overall error than other related emotion detection techniques. © 2022, The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.