ORIGINAL PAPER

ORIGINAL PAPER

Do you see what I see? Learning to detect micro expressions of emotion

Carolyn M. Hurley

Published online: 11 November 2011

� Springer Science+Business Media, LLC 2011

Abstract The ability to detect micro expressions is an

important skill for understanding a person’s true emotional

state, however, these quick expressions are often difficult

to detect. This is the first study to examine the effects of

boundary factors such as training format, exposure, moti-

vation, and reinforcement on the detection of micro

expressions of emotion. A 3 (training type) by 3 (rein-

forcement) fixed factor design with three control groups

was conducted, in which 306 participants were trained and

evaluated immediately after exposure and at 3 and 6 weeks

post-training. Training improved the recognition of micro

expressions and the greatest success was found when a

knowledgeable instructor facilitated the training and

employed diverse training techniques such as description,

practice and feedback (d’s [ .30). Recommendations are offered for future training of micro expressions, which can

be used in security, health, business, and intercultural

contexts.

Keywords Micro expression � Facial expression � Emotion � Training

Introduction

If facial expressions of emotion were delivered uniformly

each and every time an emotion was elicited, eventually all

of us would be near perfect perceivers of others. However,

pressures to conceal or mask one’s true feelings may result

in emotional displays that are quick or fragmented (called

micro momentary expressions, Haggard and Isaacs 1966;

or micro expressions, Ekman and Friesen 1969). Since

daily life features many pressures to conceal or mask one’s

emotions, as a function of status, culture, context, polite-

ness, and so forth (Ekman 1972), the ability to accurately

perceive and interpret these quick expressions would

improve our interpersonal skills, allowing us to better

understand individuals’ true emotional states.

The ability to ‘‘read’’ others is advantageous for the

average person, but in particular for clinicians and security

practitioners where the ability to understand others can

result in more informed judgments regarding threats to

oneself and others. Practitioners are already utilizing web-

based micro expression (ME) training in security (e.g.,

Department of State, Department of Homeland Security,

Department of Defense) and health contexts, although

testing of these efforts has been largely limited to clinical

populations (e.g., Marsh et al. 2010; Russell et al. 2006,

2008). Identifying effective training methods is imperative,

especially in these critical situations where a superior

understanding of emotion can significantly improve our

national security and quality of life.

The best available research in concealment of emotion

suggests that these masked emotional signals, particularly

MEs, are very difficult to detect (Ekman and Friesen 1969,

1974a; Etcoff et al. 2000; Porter and ten Brinke 2008).

Recent research has found that it is possible to train these

skills in a short period (Matsumoto and Hwang, in press),

This work was submitted in partial fulfillment of a Doctor of

Philosophy degree at the University at Buffalo by the author. Any

opinions, findings, and conclusions or recommendations expressed in

this material are those of the author and do not necessarily reflect the

views of the Transportation Security Administration, the Department

of Homeland Security, or the United States of America. The author

would like to thank Dr.’s Mark Frank and David Matsumoto for loan

of the Micro Expression Training Tool, second edition.

C. M. Hurley (&) Transportation Security Administration, 601 South 12th street,

Arlington, VA 22202, USA

e-mail: Carolyn.Hurley@tsa.dhs.gov

123

Motiv Emot (2012) 36:371–381

DOI 10.1007/s11031-011-9257-2

 

 

yet few boundary factors that may affect training success

have been explored. This manuscript examines the train-

ability of MEs of emotion, the optimal method of training,

the role of motivating factors, the effect of reinforcement,

and the retention of training materials over a 6-week per-

iod. This will help identify more effective training meth-

ods, which can be used to train individuals—such as those

in national security contexts—who may encounter con-

cealed emotions like MEs.

Background

Micro expressions of emotion

Emotions can be defined as ‘‘short-lived psychological-

physiological phenomena that represent efficient modes of

adaptation to changing environmental demands’’ (Levenson

1994, p. 123). Emotions are automatic responses that are

triggered—aroused in a fraction of a second—by environ-

mental stimuli that alter our attention and organize biological

responses, preparing us to react. Emotions are complex and

involve a number of bodily response systems such as

expression, muscular tonus, voice, and autonomic nervous

system activity (Levenson 1994).

Besides unique internal signals, emotions also generate

external signals—such as facial expressions—that provide

clues of these internal changes. A significant body of lit-

erature has examined the basic emotions of anger, con-

tempt, disgust, fear, happiness, sadness, and surprise,

revealing that each appears to have a characteristic

expression that is universal across cultures (e.g., Ekman

2003; Elfenbein and Ambady 2002). The universal pro-

duction of these facial signals suggests that these emotional

expressions are genetically determined and biology is lar-

gely responsible for establishing which facial movements

are associated with certain emotions (DeJong 1979;

DeMyer 1980).

A ME is a special case of the basic emotional expression,

which was first discovered by Haggard and Isaacs (1966)

while studying clinical interviews. They believed MEs were

caused by an unconscious repression of conflict and that

those expressions occurred too quickly to be seen in real

time. Ekman and Friesen (1969, 1974b) undertook a more

rigorous program of study that fully articulated the nature of

MEs. After examining recorded psychiatric interviews

frame-by-frame they found that MEs were emotional

expressions that ‘‘leaked’’ out when individuals attempted to

inhibit or manage their facial displays (Ekman, 2003). They

concluded that these quick expressions represented signs of

concealed emotion, as uninhibited or naturally occurring

emotional expressions generally last several seconds in

length or more (Hess and Kleck 1990).

The existence of MEs has been verified in studies of

concealment (Porter and ten Brinke 2008) and is relevant to

high-stakes contexts like law enforcement and national

security. For example, if someone is transiting a security

checkpoint and is in possession of illegal drugs, he may

have a fear of discovery. He will in all likelihood try to

hide these feelings, so any emotional clues he produces

may be more subtle then in a context where he is not trying

to manage his behavior. Research has shown that the

ability to detect MEs is related to skill at detecting

deception in high-stakes scenarios (Ekman and O’Sullivan

1991, 2006; Ekman et al. 1999), likely because it is easier

to judge veracity when an observer is able to accurately

understand how the target is feeling. This research

emphasizes the importance of ME recognition skills for

any individual whose profession requires interpersonal

interaction or deception detection.

Facial and micro expression training

Scientists have long endeavored to train people to better

recognize facial expressions. As early as the 1920s

researchers had students study pictorals or verbal descrip-

tions of facial expressions (Allport 1924; Guilford 1929;

Jarden and Fernberger 1926; Jenness 1932). However, the

absence of clear stimulus materials (drawings versus pho-

tographs) and clear identification of expressions limited

this training research. After researchers began to system-

atically study and define the muscle movements inherent in

emotional expressions they were able to create detailed

facial coding systems (e.g., Ekman and Friesen 1978; Izard

1979). This allowed researchers to create standardized sets

of valid emotion training and testing materials (e.g.,

BART, Ekman and Friesen 1974b; PoFA, Ekman and

Friesen 1975; JACFEE, Matsumoto and Ekman 1988;

JACBART, Matsumoto et al. 2000).

The Japanese and Caucasian Brief Affect Recognition

Test (JACBART) was the first published test of micro

expression recognition accuracy (MERA) that was rigor-

ously evaluated (Matsumoto et al. 2000). The JACBART

created the appearance of more dynamic expressions, as

each poser’s neutral face was imposed before and after the

emotional expression face, reducing the after effects of the

stimuli. All expression images were scored with the Facial

Action Coding System (FACS; Ekman and Friesen 1978)

to ensure the same muscle actions occurred for each

emotion and were consistent with universally recognized

expressions (Ekman 2003). Additionally, these images

were tested with an international audience to ensure cross-

cultural agreement (Biehl et al. 1997). Matsumoto and

colleagues provided evidence of internal and temporal

reliability and convergent and concurrent validity for this

test across five studies and found similar accuracy patterns

372 Motiv Emot (2012) 36:371–381

123

 

 

even with the differences made to presentation speed and

judgment task (Matsumoto et al. 2000).

This ME testing procedure evolved into a self-instruc-

tional training tool, originally called the Micro Expression

Training Tool (METT; now available as the METT

Advanced at face.paulekman.com and the Microexpression

Recognition Tool [MiX] at www.humintell.com). The

METT is presented as a stand-alone training tool; it offers a

pre-test, a training section, practice examples with feed-

back, a review section, and a post-test. The stimuli used in

these training tools are laboratory produced which provides

the necessary consistency and reliability of expression,

poser, intensity, angle and so forth to provide scientific test

of MERA. However, use of this type of materials limits the

ability to generalize to naturally occurring spontaneous

expression, which have more dynamic features (Naab and

Russell 2007).

Researchers have used versions of the METT to

train department store employees and trial consultants

(Matsumoto and Hwang, in press) and individuals with

Schizophrenia (Marsh et al. 2010; Russell et al. 2006,

2008) to detect MEs. A 2-h instructor led session using the

MiX not only significantly improved Korean department

store employees’ ability to identify MEs (N = 81, 18%

increase), but also led to higher social and communication

skills scores (Matsumoto and Hwang, in press). A similar

experiment using a small group of trial consultants also

showed improvements in accuracy (N = 25, 18%

increase). Further analyses revealed no skill decay over a

2-week period for both groups (Matsumoto and Hwang, in

press).

The METT has also been used to train clinical patients

with emotion recognition deficiencies to more accurately

recognize emotion (Marsh et al. 2010; Russell et al. 2006,

2008). Training individuals with Schizophrenia to read

facial expressions using the METT resulted in a significant

improvement in ME recognition at the post-test (9%

increase, Russell et al. 2006; 18% increase, Russell et al.

2008), illustrating the tool’s robustness to different popu-

lations. These studies support a meaningful training-accu-

racy relationship for identifying MEs, as well as, highlight

some possible social benefits.

Researchers have used other materials to teach others

about facial expressions. Stickle and Pellegreno (1982) and

Elfenbein (2006) used the Pictures of Facial Affect (PoFA,

Ekman and Friesen 1975) to train American students to

recognize emotional expressions (Elfenbein also used a

subset of Chinese posing facial expressions Wang and

Markham 1999). Although both studies reported success

for training, the authors did not report either the pre and

post accuracy scores and within subjects change (Stickle

and Pellegreno 1982) or the baseline recognition accuracy

(Elfenbein 2006). Those limitations inhibit interpretation of

these data. These studies also did not examine the ability to

detect quick expressions—such as MEs—further limiting

the ability to compare these methods to standardized tools

such as the METT or MiX.

Boundary factors to training

While research demonstrates the validity of using

commercial ME training tools to train recognition skills

(Matsumoto and Hwang, in press; Russell et al. 2006,

2008), little research has analyzed the underlying factors

associated with these skill improvements. Training formats

such as simple feedback (Elfenbein 2006), lecture and

practice (Stickle and Pellegreno 1982), and the METT/MiX

(Matsumoto and Hwang, in press) have all improved

expression recognition; but it is unknown which methods

have produced the greatest improvements or had the

greatest retention, due to differences in both testing mate-

rials and measures of effectiveness. It is also unknown

which format and materials are optimal for training indi-

viduals to detect MEs.

These studies revealed that individuals can be trained to

recognize laboratory produced MEs fairly quickly and

effectively, however, retention has only been examined in

one study and only at 2 weeks (Matsumoto and Hwang, in

press). Although training with the METT can improve

individuals’ recognition in as little as a few hours, the

length that this training outlasts the post-test is unknown.

Skill decay is an important variable to examine as many

military or government employees may only be able to

receive ME training once a year or once in a career span.

Another factor to consider is that understanding emo-

tional expressions is a skill that may improve with practice.

People who have repeated exposure to individuals who try

to conceal their emotions or who scrutinize nonverbal

behavior for their jobs—such as law enforcement officers,

judges, clinical psychologists, and secret service person-

nel—are often more accurate judges of how others are

feeling (Ekman and O’Sullivan 1991; Ekman et al. 1999).

Studies that have repeatedly tested the same participants

have found they improved without training (Matsumoto

et al. 2000). This suggests that repeated exposure to the

task or stimuli may serve as a training function as well and

should be examined.

Motivation can also influence a person’s ability to learn

material. Even though micro expression training may

improve MERA for all individuals, those who are more

motivated may learn and retain more material. Motivation

to learn is positively related to skill acquisition (Colquitt

et al. 2000), deeming it an important area for investigation.

It is important to examine individuals’ motivation to learn

Motiv Emot (2012) 36:371–381 373

123

 

 

both at the start and completion of each testing phase, as

motivation may be affected by external factors such as the

quality or content of the training or assignment to the

training or control group. Any differences must be con-

trolled for to insure that any gains made post-training can

be properly attributed to the training.

Overall, the previously published studies raise questions

regarding the optimal method of training, the role of

exposure and motivating factors, and the persistence of

training effects over time. It is important to examine these

boundary factors that may reduce skill loss so that

researchers can identify more effective training techniques.

The METT is an ideal instructional tool for testing these

differences. This training can be self-administered or

administered by an instructor in a group setting and pro-

vides enough stimulus materials to examine skill retention.

This will allow us to assess these factors in an existing and

well-used training.

Based on the above literature review, which found sig-

nificant improvements in MERA with different iterations

of the METT training (Matsumoto and Hwang, in press;

Russell et al. 2006, 2008), the following set of specific

hypotheses are proposed:

H1 ME Training will significantly improve participants’

MERA and result in greater skill retention, opposed to the

control conditions, which will experience no change in

MERA.

Although training by feedback alone has significantly

improved expression recognition skills (Elfenbein 2006),

ME recognition is an advanced skill which requires

understanding of subtle differences among expressions.

Thus,

H2 An instructor-led, multi-faceted ME training condi-

tion will produce the greatest increases in MERA, opposed

to ME training conditions that are self-led, or only provide

feedback to participants.

Any increased exposure to training material should also

provide an advantage to the exposed group. Thus,

H3 Reinforcement will significantly improve retention of

MERA.

Previous studies have assumed that a comparison group

assigned to do nothing during the training time serves as an

adequate control for examining training effects. Factors

such as mere exposure to stimuli or motivation to learn

could affect ME post-test scores or moderate effectiveness

of training. Thus, three control groups will also be exam-

ined to answer the following research question:

RQ1 What is the effect of motivation and simple expo-

sure on MERA?

Method

Participants

Three hundred thirty four (334) participants were recruited

from large introductory communication courses. An in-

class announcement advertised the study as ‘‘an evaluation

of students’ nonverbal communication skills’’ and inter-

ested students signed up for three 1-h appointments through

an online sign-up system. Participants who completed the

study received 3 h of research credit in partial fulfillment

of their 5 h departmental requirement.

Design

The study employed a 3 (training type—instructor feed-

back; instructor feedback plus description; or self led) by 3

(reinforcement—none; at time 2 only; or at time 3 only)

fixed factor design with three control groups (traditional

control; control with additional exposure of items; or

control with a motivating lecture). The four times at which

participants’ accuracy at judging MEs was assessed (pre-

training, immediately after training, 3 weeks later, and

6 weeks later) was treated as a within-subject independent

variable. The dependent variable was the participants’

accuracy on the various ME tests. Participants were ran-

domly assigned to each condition.

Conditions

Participants in the control conditions received no training

to serve as comparison groups to the training manipula-

tions. Participants in the ‘‘traditional’’ control condition

occupied themselves for the length of the manipulation and

were not exposed to any other emotional expression items.

Participants in the ‘‘exposure’’ control condition were

exposed to the same stimulus items (photographs of facial

expressions) as the training conditions during the manip-

ulation period, but received no feedback or other infor-

mation to facilitate their judgment. Participants in the

‘‘motivating lecture’’ control condition were provided with

a lecture on the importance of accurately perceiving and

interpreting human emotion based on the work of Ekman

(2001, 2003), but were not exposed to any other facial

expression material.

Training techniques previously published were com-

bined to allow for a fair comparison and evaluation of these

different training methods. Participants in both the ‘‘feed-

back only’’ training and ‘‘full instruction’’ training condi-

tions received the METT training led by an instructor

highly knowledgeable in the area of facial expressions of

emotion (the author). The difference was that the feedback

only training manipulation consisted solely of feedback

374 Motiv Emot (2012) 36:371–381

123

 

 

regarding the MEs of emotion (available in the practice

section of the METT), whereas in the full instruction

training manipulation the instructor also discussed subtle

differences among expressions (according to the ‘‘training’’

and ‘‘review’’ sections of METT) and answered questions

raised by participants. Participants in the ‘‘self-led’’ train-

ing condition also received training via the METT. These

participants led themselves through the training, feedback,

and review sections of the METT on a personal computer

(monitored by the instructor). The self-led training group

was exposed to the same materials as the full instruction

training group except the instructor was not allowed to

answer questions or discuss subtle differences to mirror a

"Is this question part of your assignment? We can help"

ORDER NOW