|
| ORIGINAL ARTICLE |
|
| Year : 2016 | Volume
: 25
| Issue : 1 | Page : 78-85 |
 |
|
Quantitative analysis of Euclidean distance to complement qualitative analysis of facial expression during deception
Ananya Mondal1, Pritha Mukhopadhyay1, Nabanita Basu2, Samir Kumar Bandyopadhyay2, Tanima Chatterjee1
1 Department of Psychology, University of Calcutta, 92, A.P.C. Rd, JD-2, Salt Lake City, Kolkata, West Bengal, India
2 Department of Computer Science and Engineering, University of Calcutta, JD-2, Salt Lake City, Kolkata, West Bengal, India
| Date of Web Publication | 19-Dec-2016 |
Correspondence Address:
Ananya Mondal
Research Scholar, Department of Psychology, University of Calcutta, 92, A.P.C Road, Kolkata, West Bengal - 700 009
India
 Source of Support: None, Conflict of Interest: None  | Check |
DOI: 10.4103/0972-6748.196048
 Abstract | | |
Background: Accurate evaluation of an individuals' veracity is a fundamental aspect of social functioning that allows individuals to act in adaptive ways. The domain of deception detection ability is still young, and many components in this field are yet to be touched which demands more research in this field. Aims: The present study aims at deciphering the structural composition of face during felt, posed, and deceived emotions in facial expression unique to Indian culture, using Facial Action Coding System (FACS). Quantitative analysis of Euclidean distance has been done to complement qualitative FACS analysis. Methods: In this study, thirty female, young adults with age range of 23–27 years were chosen randomly for portraying their (felt, posed, and deceived) facial expression. All facial expressions were captured through instruction, and videos were converted into static images. The static images were coded on the basis of FACS to decipher the felt, posed, and deceived expressions. Quantitative analysis of the data has been done using MATLAB to meet the objectives of the study and to complement the qualitative analysis. Results: Felt and posed emotions differ in terms of intensity of the expression and subjective experience. Posed emotional and deceived expressions differ in intent. Facial asymmetry is an important indicator for detecting deception. Keywords: Deception, Euclidean distance analysis, Facial Action Coding System, felt emotion, posed emotion
How to cite this article:
Mondal A, Mukhopadhyay P, Basu N, Bandyopadhyay SK, Chatterjee T. Quantitative analysis of Euclidean distance to complement qualitative analysis of facial expression during deception. Ind Psychiatry J 2016;25:78-85 |
How to cite this URL:
Mondal A, Mukhopadhyay P, Basu N, Bandyopadhyay SK, Chatterjee T. Quantitative analysis of Euclidean distance to complement qualitative analysis of facial expression during deception. Ind Psychiatry J [serial online] 2016 [cited 2021 Jan 17];25:78-85. Available from: https://www.industrialpsychiatry.org/text.asp?2016/25/1/78/196048 |
Communication is the heart of human social interaction. Perceiving the emotional state of the interacting partner accurately is a fundamental aspect of effective communication. Emotions have expressive and observable components that allow perceiver to know about the emotional state of the interacting partner. However, not all emotional expressions of the interacting partner reflect actual emotional experiences. Telling a little white lie on occasion may smooth social interaction, however, when these expressions intend to transmit misleading information are called deception. The foundation of this evaluative process to detect misleading information originates in our evolutionary past when the basic discrimination of friend and foe was one of the significant interpersonal judgments.[1] In spite of the evolutionary significance, deception detection ability has not reached very high level of accuracy. A meta-analysis examines the results of 108 studies that attempted to determine if individual differences exist in the ability to detect deception. It has been found that 40.41% parole officers and 64%–70% teachers could able to detect deception and another bulk of 156 studies where students were asked to detect deception; they scored average of 54.22% on detecting deception.[2],[3]
Most of the time, convincing expressions which are false may occur at one moment and concealed expressions leak at the very next moment. According to Ekman, Facial Action Coding System (FACS) tells us that genuine emotional expressions involve both voluntary and involuntary muscle movements of the face. Activation of involuntary muscle movement is inherently associated with genuine expression since it is spontaneous and occurs without planned thought or intention, whereas the false ones predominantly use voluntary muscle movement of the face because the voluntary control of muscles allows people to portray the false emotion altering the felt emotion.[4] Deception is a complex cognitive activity.[5] It is assumed to be a conscious effort of central nervous system with the physiological reactions of autonomic nervous system which are not under conscious control.[6]
However, intentional manipulation of emotional facial expressions has three major ways of manifesting emotions:[7]
- Expression is simulated when it is not accompanied by any real emotion
- In masked expression, deceiver withholds true information but presents false information as if it were true
- Emotional expression is neutralized when the expression of a genuine felt emotion is inhibited.
These deceptive displays can be highly convincing; despite high self-reported confidence in their deception detection,[8] observers often are unable to distinguish genuine versus faked expressions.[4],[9]
Much of the research has been conducted on posed facial expressions using FACS.[10] In most of the researches, the posed and deceived emotions are not differentiated. The present study aims at deciphering the structural composition of face during felt, posed, and deceived emotions, using FACS. Quantitative analysis of Euclidean distance has been done to complement qualitative FACS analysis. The study however does not aim at examining cross-cultural differences in these expressions. Therefore, the study is conducted with Indian encoders of facial expressions only.
Objective
The general aims of the investigation were:
- To determine the difference, if any, between parameters of expression of felt emotion, posed emotion, and deceived emotional expression
- To determine whether there is any specificity in parameters of expression of each emotion (happy and sad) while posing and deceiving.
| Methods | |  |
Sample
In this study, the sample consisted of normal healthy thirty female young adults with age range of 23–27 years. Their minimum educational qualification was undergraduation and all of them were right handed. Adults were chosen randomly for portraying their facial expression. Skin texture of the adults was normal and without makeup. Adults were also asked to uncover their forehead to fully show their eyebrows. For photographs, adults who have rough skin texture, eye glasses, and anatomical facial asymmetry were not selected. Adults who are not naive to purpose of the experiment were not considered.
Ethic statement
Informed written consent had been collected from each of the adults before the experiment. The consent was obtained twice: first one while participants were being invited to take part in the experiment and second one after the experimental sessions through debriefing. In case of first consent, the purpose of the research was explained to the adults and was assured of confidentiality of their photographs. They were also certain of their right to withdraw from the study at any stage. In case of second one, semi-structured conversation between researcher and the adult had been conducted. This research had been conducted based on the ethical guidelines set by the University of Calcutta.
Tools
FACS coding manual has been used for facial action coding of portrayers' facial expression. FACS [11] is more comprehensive than Maximally Discriminative Facial Movement Coding System.[12] Coders can manually code all possible facial expressions using FACS. Facial expressions are decomposed into thirty action units (AUs) and 14 miscellaneous actions. FACS codes the major actions of individual or groups of muscles of emotional facial expressions. The fundamental actions of individual muscles or groups of muscles are AUs.
Design of the research
Procedure
Individuals have been selected according to the inclusion and exclusion criteria. Written consent was taken from all the participating individuals.
In the first phase, thirty young adults (female only) were selected. Each adult was seated 4 feet away from a Nikon D3200 camera and was asked to face it frontally during emotion recollection. At first, static pictures have been taken for neutral and posed emotion. Next, video clips were shot to capture felt to deceived emotion. Photography and video clips were shot in a diffuse light condition. There are various types of methods for eliciting emotions; for example, using films, emotional stories, voluntary facial movements, imagery technique. In this experiment, imagery technique has been used to elicit emotion. The primary benefit of imagery is that one can draw on intense personally relevant situations. Adults were given instruction to imagine oneself in an emotional situation or recalling any personally significant emotional life event and portray their facial expression in their own way. After the video recording started, the experimenter left the room so that they could feel free to portray their felt emotion. After few minutes during recording when experimenter entered the room, the adult started to deceive her emotions deliberately. This strategy has been used to capture the deceived expression. Later, the adults were debriefed.
Neutral expression
Adults were asked to give a static photograph without posing or feeling any emotion.
Posed emotion
Adults were instructed to portray emotional expression without thinking about any incident associated with that emotion. It is nonfelt posed emotion.
Felt emotional expression
Adults were given instruction to imagine or recalling any personally significant emotional life event and were asked to express in their own way (in the absence of the experimenter).
Deceived expression
while social display rules are consciously used to conceal emotions for effective communication, it can be termed as deception. It misleads person by deliberate squelched expression; neutralizing or masking their true felt emotion can be defined as deceived expressions. In this experiment, adults were compelled to deceive their felt emotion by sudden presence of the experimenter during their recalling of emotional situation. No explicit instruction was provided to the adult.
Brightness and contrast of the pictures were held constant. Size of the stimulus was held constant at 6/6 inch. Inter-session interval, lighting, and head position were held constant during video shoot. Noise was minimized as far possible. Background of photographs was kept white. Mental set was kept constant through instruction. The expressions have been videotaped. Within adult design has been performed in 2 days session (1 day session: one emotion).
All videos were converted into 1000 frames of static images. The static images of the moment of intensified felt emotion and that of the very moment of deceived expression were taken for analysis. In the second phase, coding was done on the basis of FACS to decipher the felt emotional, posed, and deceived expressions. The pictures of facial expressions had been edited by using FaceGenModeller (Version 3.4) (Computer software: Free Trial Version)[13] for confidentiality purposes.
Quantitative analysis of the data has been done using MATLAB to meet the objectives of the study and to complement the qualitative analysis. Euclidean distance value of eyebrows apart, lips apart, and orbicularis oculi has been calculated for further analysis. Paired t-test has been used for statistical analysis; P = 0.01 level of significance has been fixed as critical level.
| Results | | |
Action units analysis
The following AUs' analyses were adopted from FACS coding manual.[11] [Table 1] shows the consistent AUs, present in 83.33% (25 out of 30) of the adults in each of the emotion categories. Distinctive inter-adult variabilities are given in brackets. These AUs had been observed in some but not in all individuals. The percentages within the bracket represent the proportion of adults who use this AU while expressing these emotions during recollection of emotion and while deceiving emotion.
The study [Table 1] reveals that the reliable AUs for felt smile [Figure 1] are lip corner pullers, cheek raise, lips apart, nasolabial furrow deepener. The smiling action itself intensified the zygomaticus major muscles (lip corner puller), which in turn raises the cheeks, gathers the skin below the eyes, and produces crow's feet wrinkles. This bagged skin below the eyes is actually the activation of orbicularis oculi muscle. Whereas, in case of nonfelt posed smile [Figure 2], the reliable AUs are same as felt smile but disintensified and there is no sign of activation of orbicularis oculi muscle.
In case of expressing happiness, adults deceived by [Figure 3] using deliberate squelched expression of lower face (lips pressure). The false smile did not accompany the involvement of the muscles around the eyes. Even other reliable measures of felt emotion become less intensified during deception. | Figure 3: Deceived expression (happy) Facial asymmetry: (a) Deceived expression Right right (happy) (b) Deceived expression (Left left (happy)
Click here to view |
In case of negative emotion, the reliable AUs for felt sadness [Figure 4] are brow lowerer, lip corner down, wrinkles in chin boss, and lips pressure. Another clue for felt emotion is reflection of eyes due to tears. Although reflection of eye is not included in the FACS coding system, the eyes are thought to be the windows of the soul to reveal the innermost feelings. In comparison to felt smile, felt sadness has many subtle AUs such as infra-orbital, cheek raise, and nasolabial furrow deepener.
The reliable AUs for posed nonfelt sadness [Figure 5] are disintensified brow lowerer, lip corner down, but there are no signs of wrinkles in chin boss, lips pressure, and reflection of eyes.
Sadness is a negative sedimentary feeling; adults deceived using masking (lips apart, lips stretcher) and deintensification of reliable AUs.
Another important clue to detect deception is facial asymmetry or lateralization of deliberate actions. Earlier theories predicted the right hemispheric specialization for negative emotion and positive or approach emotions.[14],[15] From the composite figures it can be observed that deceived emotional expression is the blend of false emotional expression and concealed felt emotional expression. In the composite figures it can be observed that the left side of the face portrayed felt emotional expression and right side portrayed deceived emotional expression (deamplified or absence of indicators genuine emotional expression).
From [Table 2], it can be interpreted that felt emotional expressions were defined by the presence of AUs, which receive intensity rating of “b” using Friesen's and Ekman's updated 5-point “a” to “e” intensity scale of it.[16] | Table 2: Intensity of action units (agreed responses on action units intensity)
Click here to view |
Whereas deceived emotional expressions were defined by the absence of certain reliable features (AUs) or neutralization and deamplification of AUs. This was evident by an intensity rating of “a” using Friesen's and Ekman's updated 5-point “a” to “e” intensity scale of it.[16]
| Discussion | | |
The results [Table 1] reveal that there exists difference in facial muscle movements across the situations of nonfelt posed condition (second), emotionally aroused condition (third), and deceived condition (fourth) in case of happy and sad emotional expression.
Difference between felt and deceived emotion
Positive emotion
In case of felt smile [Figure 1] (third), the reliable AUs are lip corner pullers, cheek raise, lips apart, nasolabial furrow deepener. Compared to felt emotion, the adults used deamplification in deceived emotion [Figure 3] which is evident from the indices of deintensified lip corner pullers, nasolabial furrow deepener, and absence of cheek raise, lips apart, [Table 1] and [Table 2]. This result indicates that in deception, one manipulates the emotion consciously by reducing the activity levels of the muscles which are involved in felt smile that is zygomatic major and orbicularis oculi.[16] However, the observation of lip pressure in deception of positive emotion to suppress felt smile while the activation of the orbicularis oris muscle is still in their face [Table 1] and [Table 2] indicates the conscious effort to withhold felt emotion.[16]
The resulting output in deception, i.e., reduced activation of zygomaticus major and activation of orbicularis oris to suppress felt smile, may occur because neurobiological disposition of the lower face is more voluntarily controlled than the upper face.[17] As opposed to false smiles, felt smile involves contraction of a muscle near the eyes. Since zygomaticus branch innervates the orbicularis oculi muscle and upper facial motor nerves are almost automatically related to felt smile and less voluntarily controlled, conscious suppression of smile may aid to put effort in reduced activation of zygomaticus major muscle. This conscious suppression of happiness leaves an inhibitory effect on orbicularis oculi during deception. Complementary quantitative analysis [Table 3] suggests that the distance of two edges of lips, which can be voluntary controlled, is less in deception than in genuine condition.
Negative emotion
In case of negative emotion, the reliable AUs for felt sadness [Figure 4] are amplified brow lowerer, lip corner down, wrinkles in chin boss, and lips pressure. Another clue for felt emotion is reflection of eyes due to tears. In deception of sadness [Figure 6]a and [Figure 6]b, adults used deamplification and masking. Deamplification [Figure 6]a is characterized by deintensified wrinkles in chin boss, lips pressure, and absence of tears, brow lowerer, lip corner depressor, nasolabial furrow deepener [Table 1] and [Table 2]. This result implies that in deception, activation of the mentalis and orbicularis oris muscles which are involved in felt sadness [17] get consciously reduced. Complementary quantitative analysis [Table 3] also suggests that felt sadness is discriminated from suppressed sadness by reduced parting of eyebrows (AU4), which is monitored by mentalis muscle. Masking [Figure 6]b is characterized by absence of tears, brow lowerer, lip corner depressor, nasolabial furrow deepener, deamplified lip corner puller: zygomaticus major.
Composite photographs analysis
Note: UMN = Upper motor neuron, LMN = Lower motor neuron, R = Right side, L = Left side
Interestingly, from composite [Figure 3]a, [Figure 3]b and [Figure 7]a, [Figure 7]b, it has been revealed more intensified emotional expression in the left hemifacial than the right counterpart, especially in lower face, regardless of positive and negative emotion. Left side of the face was portraying felt emotion while the other side was portraying deceived expression. From [Figure 7]a (right hemiface), it can be seen that lower right hemiface showed the absence of lip corner depressor: reduced activation of depressor anguli oris which is the indicator of suppressed sadness. In contrary, [Figure 7]b (left hemiface) shows lip corner depressor in lower left hemiface, a reliable indicator of felt sadness. Moreover, from [Figure 3]a (right hemiface), it can be seen that lower portion of the right hemiface showed deamplified lip corner puller: zygomaticus major which is the indicator of deceived smile. In contrast to [Figure 3]a, [Figure 3]b (left hemiface) shows amplified lip corner puller: zygomaticus major which is a reliable indicator of felt smile. | Figure 7: Deceived expression (sad) Facial asymmetry: (a) Deceived expression Right right (sad) (b) Deceived expression Left left (sad)
Click here to view |
These results provide support for the right hemisphere specialization hypothesis in terms of leakage phenomenon of felt emotion at lower left hemiface and also support left hemispheric inhibitory control mechanism over lower right hemiface. It implies that upper motor neuron controls lower face only in contra lateral way.
From previous researches on Hemiplegia and Bell's palsy that are supranuclear lesion and infranuclear lesions, respectively, it has been found that upper motor neuron controls upper face through lower motor neurons in ipsilateral way as well as contralateral way whereas the neuron controls lower face only in contralateral way. The lower motor neurons in the dorsal aspect of the facial motor nucleus receive inputs from both sides of the cortex, while those in the ventral aspect mainly receive contralateral inputs (i.e., from the opposite side of the cortex).
Rinn suggested that the findings showing right hemisphere lateralization for emotion may actually be due to a left hemispheric superiority in the inhibition of emotion.[18] He suggested that the left hemisphere, due to its linguistic and propositional thought capacities, is better equipped to inhibit emotional episodes, resulting in better inhibition of the right face and subsequently greater expression in the left face. Since left hemisphere exerts more contralateral control over lower portion of right hemiface indicated by the absence of lip corner depressor: depressor anguli oris for deceived sadness and deamplified lip corner puller: Zygomaticus major for deceived happiness, it revealed more conscious effort of controlling emotion than upper face and lower left hemiface.
Difference between posed and deceived emotion
Masking and facial asymmetry are evident in deception. However, posed emotion which is also not genuine, is it identical to deception? In case of posed emotion only, disintensification has been observed. According to Ekman,[19] the posed emotion is defined as an expression of the facial configuration without felt emotional experience. Whereas in case of deceived emotion, the facial configuration provides two types of information:
- False but convincing emotional expression
- Concealed felt emotional expression.
This requires more conscious effort than posed emotion.
Positive emotion
In case of nonfelt posed smile [Figure 2], the reliable AUs are same as felt smile but disintensified (disintensified lip corner pullers, nasolabial furrow deepener, cheek raise, lips apart) and there is no sign of activation of orbicularis oculi muscle. The Quantitative analysis [Table 3] also implies that there is a significant difference between posed and deception [Figure 3] regarding lips apart. In case of posed emotional expression, adults used deamplified zygomaticus major, whereas in case of deception, adults used orbicularis oris to suppress smile. However, no significant difference exists between posed and deception regarding orbicularis oculi measure since there is no sign of orbicularis oculi in case of posed and deceived expressions.
Negative emotion
In case of negative emotion, the reliable AUs for posed nonfelt sadness [Figure 5] are disintensified brow lowerer, lip corner down, but there are no signs of wrinkles in chin boss, lips pressure, and reflection of eyes. And there are also no signs of lips apart and lips stretcher as indicator of masking felt emotion during deception. Quantitative analysis [Table 3] of eyebrows apart which is a reliable indicator of felt sadness also suggests that there is a significant difference between posed [Figure 5] and deception [Figure 7].
These findings of AUs reveal that posed emotional expression and deceived expression differ in intent. In case of posed expression while someone poses, their intent is to replicate the expression of felt emotion through voluntary movement of the facial muscle that is why posed expression is more or less similar to felt emotion, but it loses its intricacies in terms of intensity and subjective experience. In case of deception, the intent is to hide their felt emotion by masking or suppressing. This negative intent to reduce activation of facial muscles for felt emotion and mask it with other emotion requires more conscious effort than posed emotion.
| Conclusion | | |
From these findings, it can be observed that felt and posed emotions differ in terms of intensity of the expression and subjective experience whereas posed emotional and deceived expressions differ in intent, and facial asymmetry is an important indicator for detecting deception. However, before any inference is made, more such replication of this study is required.
Financial support and sponsorship
Nil.
Conflicts of interest
There are no conflicts of interest.
| References | | |
| 1. | Williams MA, Mattingley JB. Do angry men get noticed? Curr Biol 2006;16:R402-4.  |
| 2. | Aamodt M, Custer H. Who can best catch a liar? Forensic Exam 2006;15:6-11. |
| 3. | Enos F, Benus S, Cautin RL, Graciarena M, Hirschberg J, Shriberg E. Personality factors in human deception detection: Comparing human to machine performance. Interspeech Pittsburgh, Pennsylvania: Proceedings of Interspeech; 2006. p. 17-21. |
| 4. | Porter S, ten Brinke L. Reading between the lies: Identifying concealed and falsified emotions in universal facial expressions. Psychol Sci 2008;19:508-14. |
| 5. | Ganis G, Kosslyn SM, Stose S, Thompson WL, Yurgelun-Todd DA. Neural correlates of different types of deception: An fMRI investigation. Cereb Cortex 2003;13:830-6. |
| 6. | Sharma M, Modi S, Khushu S, Mandal MK. Neural Activation Pattern In Self-deceivers. Psychological Studies; 2009. |
| 7. | Ekman P, Friesen WV. Unmasking the Face: A Guide to Recognizing Emotions from Facial Clues. Oxford, England: Prentice-Hall; 1975. |
| 8. | Vrij A, Mann S. Who killed my relative? Police officers' ability to detect real-life high-stakelies. Psychol Crime Law 2001;7:119-32. |
| 9. | Porter S, Juodis M, Brinke L, Klein R, Wilson K. Evaluation of the effectiveness of abrief deception detection training program. J Forensic Psychiatry Psychol 2010;21:66-76. |
| 10. | Rosenberg E. Introduction: The study of spontaneous facial expressions in psychology. In: Ekman P, Rosenberg E, editors. What the face reveals: Basic and applied studies of spontaneous expressions using the Facial Action Coding System (FACS). New York: Oxford University Press; 1997. p. 3-18. |
| 11. | Ekman P, Friesen WV. Facial Action Coding System. Palo Alto: Consulting Psychologist Press; 1978. |
| 12. | Oster H, Hegle D, Nagel L. Adult judgments and fine-grained analysis of infant facial expressions: Testing the validity of a priori coding formulas. Dev Psychol 1992;28:1115-31. |
| 13. | FaceGenModeller (Version 3.4) [Computer Software]. Toronto, ON, Canada: Singular Inversions Inc. [Free Trial Version]. |
| 14. | Mandal MK, Borod JC, Asthana JC, Mohanty A, Mohanty S, Koff E. Effects of lesion variables and emotion type on the perception of facial emotion. J Nerv Ment Dis 1999;187:603-9. |
| 15. | Bryden MP. Laterality: Functional Asymmetry in the Intact Brain. New York: Academic Press; 1982. |
| 16. | Friesen WV, Ekman P. Changes in FACS scoring. Unpublished manuscript, University of California, San Francisco; 1992. |
| 17. | Cohn J, Ambadar Z, Ekman, P. Observer based measurement of facial expression with the facial action coding system. In: Coan JA, Allen JB, editor. The Handbook of Emotion Elicitation and Assessment. [Chapter: 13]: Oxford University Press Series in Affective Science; 2006. p. 203-21. |
| 18. | Rinn WE. The neuropsychology of facial expression: A review of the neurological and psychological mechanisms for producing facial expressions. Psychol Bull 1984;95:52-77. |
| 19. | Ekman P. Telling Lies: Clues to Deceit in the Marketplace, Politics, and Marriage. New York: Norton; 1992. |
[Figure 1], [Figure 2], [Figure 3], [Figure 4], [Figure 5], [Figure 6], [Figure 7]
[Table 1], [Table 2], [Table 3]
|
