Emotion Recognition using PIZAM-ANFIS by Considering Partial Occlusion and Behind the Mask

Abstract

Emotional expressions, comprising both verbal and non-verbal cues, communicate an individual's emotional state or attitude to others. To understand the complex human behavior, it is essential to analyze physical features across multiple modalities. Recent research has extensively focused on spontaneous multi-modal emotion recognition for human behavior analysis. Nonetheless, accurate Facial Emotion Recognition (FER) is hindered by challenges such as partial facial occlusions from random objects and mask-wearing. This paper proposes a novel classification method, Pizam-ANFIS-based FER, which addresses these issues by incorporating Occlusions and Masks (PAFEROM). The process begins with pre-processing the input image, followed by face detection and cropping using the Viola-Jones Algorithm (VJA). Skin tone analysis and segmentation of facial parts are performed using Local Structural Weighted K-Means Clustering (LSW-KCM). Subsequently, contour formation and edge detection via CGED are conducted, leading to feature extraction. The retrieved features' dimensionality is reduced using PIGA before being processed by CSE for Action Unit (AU) identification. Finally, PizMamdani-Adaptive Neuro Fuzzy Interference System (Pizam ANFIS) classifies the identified AUs, and reduced-dimensionality features to determine human emotions. Experimental results indicate that the proposed model surpasses existing techniques in both efficacy and accuracy, providing a robust solution for FER in the presence of occlusions and masks