To direct emotion in AI video, stop writing the feeling and start writing what the body physically does. "She looks sad" gives you a generic sad face. "Her inner brows lift and pull together, the corners of her mouth drop, her chin tightens and she breathes out slowly through her nose" gives you a performance. Describe the brow, eyes, eyelids, mouth, jaw, breath, shoulders and hands, in the order they move, and support it with a close camera and light that shows the skin. I call this method Direct The Emotion, and it works in any current video model because it is a description of what the camera sees.
Key takeaways
- A model doesn't know what "sad" means to your story. It knows what a face looks like when the brow drops and the jaw tightens.
- Real expressions are combinations of specific muscle movements, mapped by researchers decades ago. You can borrow that map.
- Order matters. An emotion unfolds over a second or two. Write it as a sequence, not a single state.
- The eyes are where a real expression shows and where a posed one gives itself away.
- Camera and light do half the work. A locked-off close-up with light that shows skin texture lets small movements read.
What does it mean to direct emotion?
Directing emotion means telling a performer what to do physically so the audience feels something, without ever naming the feeling. On a set, a good director rarely says "be sad". They say "don't look at her yet", "hold the breath", "let it hit you before you answer".
With AI, the model is the performer. And it has the same problem a nervous actor has when you say "be sad": it reaches for the most obvious version. Glassy eyes, a pout, a tear. The audience has seen that face a thousand times and feels nothing.
When you describe the mechanics instead, the model has something specific to simulate. The result looks like someone reacting, not someone posing.
Why do emotion words give you generic AI faces?
Because an emotion word is a label, and a label gives the model nothing to build on. "Angry" covers a clenched jaw, a cold stare, a shout and a slammed door. The model picks the average, and the average is a cliché.
Kling's own prompt guide points the same way. It tells you to describe "facial expression, shoulder movement, timing, and emotional weight" for emotional gestures, to describe gesture and expression together "so the motion and emotion stay connected", and above all to "keep the language grounded in what the viewer sees". Google's Veo guide says the same thing in its own words: specify what the subject is doing, and add detail to get a realistic expression.
Nobody sees "sad". You see a brow, a mouth and a breath.
What does the science of facial expression tell us?
More than you'd think, and it is very usable for prompting.
In 1862, the French neurologist Guillaume Duchenne published Mécanisme de la physionomie humaine. He stimulated individual facial muscles with electrical current and photographed the results, mapping which muscle produced which look. His name stuck to one finding in particular: a real smile involves the muscle around the eyes, not just the mouth.
In 1978, psychologists Paul Ekman and Wallace Friesen published the Facial Action Coding System, known as FACS, building on earlier work by the Swedish anatomist Carl-Herman Hjortsjö. FACS breaks every visible facial movement into numbered "action units", each tied to specific muscles. It is still widely used in psychology research.
Researchers using FACS have also mapped the combinations that typically make up the basic emotions. A few of them:
- Happiness: cheek raiser plus lip corner puller. The muscle around the eye (orbicularis oculi) lifts the cheeks and narrows the eyes, and the zygomatic major pulls the mouth corners up.
- Sadness: inner brow raiser, brow lowerer and lip corner depressor. The inner ends of the brows lift and pull together, and the mouth corners pull down.
- Surprise: brows raised, upper lids raised, jaw dropped.
- Fear: brows raised and pulled together, upper lids raised, lower lids tightened, lips stretched sideways, jaw dropped.
You don't need to memorise action unit numbers, and you don't need to name the muscles in your prompt. What matters is the visible movement each one creates. That list is the start of a vocabulary the model can actually simulate.
Why are the eyes so important?
Because the eyes are where a real expression shows and where a forced one fails.
A smile that only moves the mouth is the one you give a cashier. Researchers call it a social smile. The smile Duchenne identified also lifts the cheeks and narrows the eyes, with creases at the outer corners. It is the difference between a smile you believe and one you don't.
AI faces fail in exactly that spot. A mouth pulled into a smile above dead, unchanged eyes reads as uncanny. So every emotion I direct starts by deciding what the eyes and the brow do, and only then the mouth.
How do you write an emotion as biomechanics?
This is the Direct The Emotion method in practice. For every emotional beat, I work through the body in this order:
- Brow. Up, down, pulled together, inner ends lifting.
- Eyes and eyelids. Widening, narrowing, lids tightening, gaze holding or dropping, a blink held a little too long.
- Mouth corners and lips. Pulling up, pulling down, stretching sideways, pressing thin, parting.
- Jaw and chin. Clenching, dropping, the chin tightening and dimpling.
- Breath. A sharp intake, a held breath, a slow exhale through the nose. Breath is what makes a face look alive between expressions.
- Shoulders and neck. Rising, dropping, a swallow.
- Hands. Gripping, loosening, a thumb moving, fingers going still.
Then I write it as a sequence: what the face is doing at the start, what changes and in what order, and where it settles. An emotion that appears all at once looks like a switch. An emotion that unfolds looks like a person.
Two more rules make a big difference:
- One beat per shot. A face going from joy to doubt to grief in five seconds is too much for a model to hold. Give each shot one change.
- Small is stronger. "The corners of her mouth lift slightly" lands better than "she smiles widely". If anything, under-direct. You can always push it on the next take.
What does this look like in a real prompt?
Same character, same setup, one moment: a man in his late forties at a kitchen window, phone to his ear, hearing news. First, the way most people write it:
Close-up of a man at a kitchen window on the phone. He hears sad news and becomes very emotional and heartbroken. Cinematic.
That gives you a stock photo of grief. Now the same moment directed four ways. The setting, camera and light stay identical so you can see the performance doing the work.
Joy
Close-up of a man in his late forties standing at a kitchen window in soft morning light from the left, a phone pressed to his right ear, listening. Camera locked off. His face starts still and neutral. Then his eyebrows lift slightly, his cheeks rise and his eyes narrow, with creases forming at their outer corners. The corners of his mouth pull up slowly, lips still closed. He breathes out a short laugh through his nose and his shoulders drop. His free hand comes up and presses flat against his chest. Visible skin texture, shallow depth of field.
Grief
Close-up of a man in his late forties standing at a kitchen window in soft morning light from the left, a phone pressed to his right ear, listening. Camera locked off. His face starts still and neutral. The inner ends of his eyebrows lift and pull together. His gaze drops to the windowsill and stays there. The corners of his mouth pull down and his chin tightens and dimples. He swallows. His eyes become wet but no tear falls. He breathes out slowly through his nose and his shoulders sink. The hand holding the phone lowers a few centimetres. Visible skin texture, shallow depth of field.
Fear
Close-up of a man in his late forties standing at a kitchen window in soft morning light from the left, a phone pressed to his right ear, listening. Camera locked off. His face starts still and neutral. His eyebrows lift and draw together, his upper eyelids open wide so white shows above the iris, and his lower lids tighten. His lips stretch sideways and part slightly. He stops breathing. His free hand grips the edge of the counter, knuckles whitening. He doesn't blink. Visible skin texture, shallow depth of field.
Relief
Close-up of a man in his late forties standing at a kitchen window in soft morning light from the left, a phone pressed to his right ear, listening. Camera locked off. His jaw is clenched, his eyebrows pulled down and together, his breathing shallow. Then his jaw unclenches and his brows slowly release. His eyes close. He exhales long and slow through his mouth and his shoulders drop a few centimetres. His head tilts forward until his forehead almost touches the glass. Visible skin texture, shallow depth of field.
Notice what the four prompts have in common. None of them names the emotion. Each one has a starting state, a sequence of changes and an end. And each one uses the breath and one hand, because the face alone is rarely enough.
Which movements go with which emotion?
| Emotion | What the face and body do | What to avoid |
|---|---|---|
| Joy | Cheeks lift, eyes narrow with creases at the corners, mouth corners pull up, breath out through the nose | A wide smile with unchanged eyes |
| Grief | Inner brows lift and pull together, gaze drops, mouth corners pull down, chin tightens, slow exhale | Streaming tears from the first frame |
| Fear | Brows lift and pull together, upper lids wide, lower lids tight, lips stretch sideways, breath stops | Screaming, unless the scene needs it |
| Surprise | Brows lift, upper lids open, jaw drops, a sharp intake of breath | Holding it too long. Surprise is brief |
| Anger | Brows pull down and together, upper lids lift into a hard stare, lower lids tighten, lips press thin, jaw clenches | Shouting or bared teeth for a quiet moment |
| Relief | Jaw unclenches, brows release, eyes close, long exhale, shoulders drop | Starting relaxed. Relief needs tension first |
The face movements in the first five rows come from the FACS combinations; the breath, jaw and hands are what I add on top. Relief isn't a FACS basic emotion, which is a good reminder that the method works for any feeling. If you can describe what a body does, you can direct it.
How do camera and light support the performance?
A great performance can disappear in the wrong shot. Three things I always set alongside the emotion:
The frame. Emotion lives in the close-up. A medium shot loses the eyes, and a wide shot loses everything but posture. If the beat is about the face, frame the face.
The camera. Keep it simple so the face is the only thing moving. A locked-off camera is the safest choice. A very slow push-in during the change is the classic way to tell the audience that something is happening inside the character. I go through which moves hold up in AI video in my post on AI video camera movements.
The light. Flat, even light kills small movements because there are no shadows to show them. A hard raking light from the side shows skin texture, creases and the catchlight in the eye, which is exactly where emotion reads. More on this in how to prompt light in AI images and video.
Should you start from an image or go straight to video?
I almost always start from an image. I generate or edit a still of the character where the starting state of the face is exactly right, then animate it with the video prompt describing the change.
Two reasons. First, you lock the character's face, lighting and framing before spending video credits. Second, the video model only has to handle the change, not invent the whole person at the same time. If the same face has to hold across several emotional shots, start every shot from the same reference images. I cover that workflow in how to keep a character consistent across AI video shots.
For the still, describe the start of the beat, not the peak. If the image already shows the peak of grief, the video has nowhere to go.
What mistakes kill an emotional AI shot?
- Naming the emotion. "Heartbroken", "furious" and "overjoyed" are the fastest way to a cliché.
- Peak from frame one. If the expression is fully formed at the start, there is no performance, only a face.
- Mouth without eyes. A smile or a frown that never reaches the eyes looks wrong immediately.
- Too many beats. One shot, one change.
- Overacting. Write the smaller version first. It is easier to push a performance up than to calm it down.
- Tears as a shortcut. Wet eyes that don't spill are usually more powerful than a tear rolling down a cheek.
- Flat light. No shadow, no texture, no readable expression.
- A busy camera. If the camera is orbiting while the face changes, the viewer watches the camera.
Checklist before you generate
- Did you remove every emotion word from the prompt?
- Is there a clear starting state, a sequence of changes and an end?
- Do the brow and the eyes move, not only the mouth?
- Is there breath in the shot?
- Is there one thing the body does beyond the face, like a hand or the shoulders?
- Is the shot a close-up with a simple camera?
- Does the light come from one side so the skin shows texture?
- Does the still you're starting from show the start of the beat, not the peak?
FAQ
Can AI video really do subtle expressions? Yes, if you ask for them. Current models respond to specific physical descriptions of the face and body. In my experience, flat AI performances almost always come from prompts that name the feeling and leave the model to guess.
Do I need to name the muscles in my prompt? No. Describe what the viewer would see: the inner brows lifting, the chin tightening. Muscle names can add precision, but the visible movement is what matters.
What is the Direct The Emotion method? It is my method for directing performances in AI video. You describe what the brow, eyes, mouth, jaw, breath, shoulders and hands physically do, in the order they move, instead of naming the emotion, and you support it with a close frame, a simple camera and light that shows texture.
How long should an emotional shot be? Long enough for one change to unfold. Plan a still starting state, the change, and a moment where the face settles, all in one clip.
Does this work for ads, not just short films? Yes. A customer's face reacting to a product is an emotional beat like any other. A real reaction sells far better than a model smiling at the camera.
Should I start from an image or from text? Start from an image when you can. Lock the character and the starting expression in a still, then use the video prompt only to describe the change.
His free guide, Direct The Emotion, covers ten emotions with an image prompt and a video prompt for each, using the method in this post.
Sources
- Facial Action Coding System overview: publication history, action units, muscles and emotion combinations (Ekman and Friesen, 1978)
- iMotions, "The Duchenne Smile": action units of genuine versus social smiles
- National Gallery of Art, Guillaume Duchenne de Boulogne, plate from Mécanisme de la physionomie humaine (1862)
- Kling AI, motion prompts guide: describing facial expression, gesture and timing
- Google Cloud, Vertex AI video generation prompt guide (Veo)
Want to direct AI video like this?
AI Unchained is my community for marketers, brand owners and filmmakers who want to direct AI video, not just prompt it. Every week we break down a film technique and put it into practice, there are three live calls a week, members get our prompt GPTs, and I reply to every piece posted in the Cinema Room the same day.
Running a brand or a content team? I also run in-company AI video workshops, so your team learns to make ads and social content in house.
