A Comprehensive Study of Various Modalities Used for Deception Detection

Abstract

Deception detection is an investigative method to determine if someone is telling the truth or fabricating information. It has attracted a lot of study interest because of its potential to be helpful in a variety of real-life problems, including healthcare, law enforcement, internet fraud, criminal investigation, and national security systems. Conventional methods such as the polygraph, demeanor observation, electroencephalogram, and functional magnetic resonance imaging (fMRI) are available to detect deception. These methods are unreliable because they require human interaction and training. They are also time-consuming and costly. Therefore, researchers developed machine learning-driven algorithms to remove human dependency. They have explored thermal imaging, acoustic analysis, eye tracking, facial micro-expression processing, and linguistic analysis to detect deception using machine learning. These techniques may produce better results because they are human-independent and unaffected by race or ethnicity. One can achieve a more reliable automatic deception detection system using features from multiple modalities. This study investigates the feasibility of using linguistic, speech, thermal, and video modality for automatic deception detection. This paper intends to present a detailed analysis of various deception detection data sets, modalities, and possible directions for the field’s development in the future.