Can a video model deliver the shot a cinematographer asks for?
LTX and Kling lead the first round, graded by working cinematographers. No model reliably delivers the direction: across all models, graders judged the note delivered on 42% of fresh shots and 22% of revisions.
delivered the direction, averaged across all models.
Revisions22%
delivered the note. Taking a note is the harder skill for every model.
Unrequested changes44%
of revisions changed something the note didn't ask for.
Leaders
Fresh shots Kling
69.6
Revisions Wan
60.2
The two tests rank models differently.
Preliminary · updated Nov 19, 2026
01Leaderboard
Score is 100 if a model is always ranked first for a question and 0 if always last. Overall shows a 95% range from resampling questions.
#
Model
Score
Range
Took the note
01
LTX *Lightricks
58.9
39 to 77
54%
02
KlingKuaishou
58.2
46 to 70
45%
03
SeedanceByteDance
49.7
40 to 61
27%
04
WanAlibaba
49.5
37 to 62
31%
05
Grok ImaginexAI
48.0
36 to 60
35%
06
Gemini OmniGoogle
43.9
35 to 53
24%
07
Hailuo *MiniMax
42.3
29 to 56
29%
01
KlingKuaishou
69.6
02
LTXLightricks
58.9
03
Gemini OmniGoogle
55.9
04
HailuoMiniMax
42.3
05
SeedanceByteDance
42.3
06
Grok ImaginexAI
41.7
07
WanAlibaba
39.3
01
WanAlibaba
60.2
02
SeedanceByteDance
57.4
03
Grok ImaginexAI
54.6
04
KlingKuaishou
46.3
05
Gemini OmniGoogle
31.5
01
KlingKuaishou
59.3
02
Grok ImaginexAI
56.2
03
SeedanceByteDance
55.2
04
HailuoMiniMax
50.0
05
WanAlibaba
44.8
06
Gemini OmniGoogle
42.6
07
LTXLightricks
34.5
01
LTXLightricks
83.3
02
KlingKuaishou
57.1
03
WanAlibaba
54.2
04
Gemini OmniGoogle
45.2
05
SeedanceByteDance
44.3
06
Grok ImaginexAI
40.2
07
HailuoMiniMax
34.5
With 3 graders the ranges overlap: the top and bottom of the table are separated, the middle is statistically tied. "Took the note" is how often graders judged the clip delivered the direction. * LTX and Hailuo were tested on fresh shots only, on half as many questions. Numbers will move as more graders finish.
02What we tested
All 20 questions in this round, 10 fresh shots and 10 revisions, each written as a single direction in the language of the set. Every model got the same questions. "Delivered" is how often evaluators judged a clip delivered the direction, across all models; "ranked best" is the model ranked highest on that question.
Slow pull back from him to reveal the studyA man in his sixties sits by a window in a quiet study, thinking.Pull back
Fresh shot
95%
Gemini Omni
Tracking alongside him as he carves the wallA skateboarder in his twenties carves the wall of an empty swimming pool.Track / crab
Fresh shot
38%
Grok Imagine
Locked off on the barber, then a whip pan to the customer waiting by the door, landing clean and holdingA barber in his fifties shaves a customer with a straight razor.Whip pan
Fresh shot
33%
Kling
Locked off, she walks into the frameA nurse in her forties walks toward camera down a long hospital corridor at the end of a night shift.Locked off
Fresh shot
24%
Kling
Handheld, following her from behind at the same distanceA woman in her thirties walks toward a parked car along a wet terraced street at night, streetlights overhead.Handheld
Fresh shot
21%
Seedance
Slow tilt up from her hands to her faceSource: A woman in her forties shapes a bowl at a pottery wheel in a small studio.Tilt
Revision
47%
Wan
Go handheldSource: A man in his thirties walks along a row of shop fronts, left to right, at a steady pace.Handheld
Revision
33%
Seedance
Slow push in on himSource: A man in his thirties walks along a row of shop fronts, left to right, at a steady pace.Push in
Revision
27%
Grok Imagine
Pan with himSource: A man in his thirties walks along a row of shop fronts, left to right, at a steady pace.Pan
Revision
20%
Grok Imagine
Slow push in on him and rest on a frame where he fills itSource: A barista makes a latte behind a cafe counter.Push in
Revision
0%
Kling
Focus and depth of field
Where focus sits and how it moves: shallow and deep focus, wide open, focus presets and rack focus. 5 fresh shots, 5 revisions.
29%delivered on average
Direction, as written
Test
Delivered
Ranked best
Rack focus from the near player to the far player as he moves a pieceTwo men play chess in a park, one close to camera, the other across the board.Rack focus
Fresh shot
67%
Kling
Very shallow focus, the lights behind him fall to large soft discsA cyclist waits at a red light on a wet city street at night, shop signs and traffic behind him.Shallow focus
Fresh shot
57%
LTX
Razor-thin focus: only the bolt head and fingertips are sharpExtreme close-up of a mechanic's hand turning a wrench on a bolt in an engine bay.Shallow focus
Fresh shot
29%
Grok Imagine
Deep focus, sharp front to backA young chef plates a dish at the pass of a busy restaurant kitchen, cooks working behind her.Deep focus
Fresh shot
29%
LTX
Focus preset on the near lockers where she stops; she runs into focusA girl runs down a school hallway toward camera and stops just short of the lens.Focus preset
Fresh shot
24%
Kling
Throw the background out of focus, keep him sharpSource: A man in his thirties stands at a bus stop on a city street, traffic and storefronts behind him.Shallow focus
Revision
27%
Wan
Go wide open: keep him sharp, let the glass in front and the bar behind fall offSource: A man in his fifties sits alone at a hotel bar, a whiskey glass on the bar close to camera.Wide open
Revision
20%
Kling
Rack focus to the cup, let her go softSource: A woman in her thirties sits at a cafe table by the window, a coffee cup on the table close to camera.Rack focus
Revision
20%
Wan
Shift focus to the shelves behind him, let him go softSource: A man in his fifties stands still in a hardware store aisle, shelves of products stretching behind him.Rack focus
Revision
13%
Seedance
Deep focus: bring the office behind her into focus tooSource: A woman in her forties sits at a desk in an open-plan office, coworkers working at desks behind her.Deep focus
Revision
0%
Gemini Omni
03Task coverage and results
Every question in the round, grouped by control, with each model's score on each one.
Figure · Round 1 questions by direction
Shallow focus315%
Rack focus315%
Handheld210%
Push in210%
Deep focus210%
Locked off15%
Track / crab15%
Pull back15%
Pan15%
Focus preset15%
Whip pan15%
Tilt15%
Wide open15%
20 questions across 13 directions. Each question tests a single direction, as a fresh shot or as a revision.
Figure · Task-by-task results
Rank score by question, not a pass rate. The shaded row shows each model's average for the control; expand a control for every question.
Camera movement10 questions · Kling leads34595742564549
"Handheld, following her from behind at the same distance"Handheld · Fresh shot
25421008385042
"Locked off, she walks into the frame"Locked off · Fresh shot
281006739221778
"Slow pull back from him to reveal the study"Pull back · Fresh shot
78333383393944
"Tracking alongside him as he carves the wall"Track / crab · Fresh shot
22782244947811
"Locked off on the barber, then a whip pan to the customer waiting by the door, landing clean and holding"Whip pan · Fresh shot
17783361444472
"Go handheld"Handheld · Revision
n/a678307525n/a
"Pan with him"Pan · Revision
n/a50501210038n/a
"Slow push in on him"Push in · Revision
n/a585850758n/a
"Slow push in on him and rest on a frame where he fills it"Push in · Revision
n/a886205050n/a
"Slow tilt up from her hands to her face"Tilt · Revision
n/a0584250100n/a
Focus and depth of field10 questions · LTX leads84574546415333
"Deep focus, sharp front to back"Deep focus · Fresh shot
94891139285633
"Focus preset on the near lockers where she stops; she runs into focus"Focus preset · Fresh shot
83892267282239
"Rack focus from the near player to the far player as he moves a piece"Rack focus · Fresh shot
78833944225033
"Very shallow focus, the lights behind him fall to large soft discs"Shallow focus · Fresh shot
10058675833258
"Razor-thin focus: only the bolt head and fingertips are sharp"Shallow focus · Fresh shot
67335650831150
"Deep focus: bring the office behind her into focus too"Deep focus · Revision
n/a3838626250n/a
"Rack focus to the cup, let her go soft"Rack focus · Revision
n/a2542177592n/a
"Shift focus to the shelves behind him, let him go soft"Rack focus · Revision
n/a3310042075n/a
"Throw the background out of focus, keep him sharp"Shallow focus · Revision
n/a4242424283n/a
"Go wide open: keep him sharp, let the glass in front and the bar behind fall off"Wide open · Revision
n/a7533423367n/a
75 to 10050 to 7525 to 500 to 25n/a: model does not support revisions
04Score distributions
How each model scored question by question, grouped into bands. A model with a high average can still have a long tail of misses.
Figure · Questions per score band
0 to 2525 to 5050 to 7575 to 100
Kling
1748
20
LTX
2215
10
Grok Imagine
4736
20
Wan
4655
20
Seedance
3773
20
Gemini Omni
4862
20
Hailuo
2521
10
Each question is one model's average rank score across evaluators. Number at right is questions taken; LTX and Hailuo took fresh shots only.
05Two instruments, one answer
Rankings and yes/no checks are collected independently. Where they agree, the result is measuring the work, not taste.
Rank score
LTX
58.9
Kling
58.2
Seedance
49.7
Wan
49.5
Grok Imagine
48.0
Gemini Omni
43.9
Hailuo
42.3
Checks passed
LTX
65%
Kling
59%
Seedance
59%
Wan
48%
Grok Imagine
57%
Gemini Omni
55%
Hailuo
65%
Rank score: 100 if always ranked first, 0 if always last. Checks passed: share of yes answers across the three checks for each direction. Across the round, checks and rank correlate at 0.50.
06Case studies
Three questions from the round, shown in full: the scene, the direction, what evaluators checked, how every model did, and what evaluators wrote. Notes are paraphrased; evaluators always stay anonymous.
Case 01 · Pull back · Fresh shot · 3 evaluators
Nearly every clip delivered it. Rank came down to precision.
95%delivered
Scene
A man in his sixties sits by a window in a quiet study, thinking.
Direction
"Slow pull back from him to reveal the study"
What evaluators checked
The camera travels away from him, it is not a zoom
He gets smaller in frame as the shot goes on
More of the study comes into view by the end
Model
Rank score
Verdict
Checks
Gemini Omni
83
Delivered 3/3
LTX
78
Delivered 3/3
Hailuo
44
Delivered 3/3
Wan
39
Delivered 3/3
Grok Imagine
39
Delivered 3/3
Kling
33
Delivered 3/3
Seedance
33
Delivered 2/3
What evaluators saw
Several clips moved faster than "slow" and read as a pull back combined with a zoom out.
With few objects between camera and subject, some moves read as a zoom rather than a physical move: there was nothing to show parallax.
One clip rendered a wall clock with three hands, none of them a second hand.
The test separates models on how well they execute a move, not just whether it happens.
Case 02 · Rack focus · Fresh shot · 3 evaluators
Two models clearly led; one never racked at all.
67%delivered
Scene
Two men play chess in a park, one close to camera, the other across the board.
Direction
"Rack focus from the near player to the far player as he moves a piece"
What evaluators checked
The near player starts sharp and the far player soft
Focus moves to the far player
The rack happens as the far player moves a piece
Model
Rank score
Verdict
Checks
Kling
83
Delivered 3/3
LTX
78
Delivered 3/3
Wan
50
Delivered 2/3
Gemini Omni
44
Delivered 2/3
Seedance
39
Delivered 2/3
Hailuo
33
Delivered 2/3
Grok Imagine
22
Missed 0/3
What evaluators saw
The most common miss was timing: focus moved before the far player reached for a piece.
In one clip focus never started on the near player, so there was no rack.
Scene continuity broke in several clips: a pawn turned into a bishop, a piece vanished after it was placed, the players were seated at different tables.
Focus pulls are judged on when they land, not only where. Timing is where models fall short.
Case 03 · Push in · Revision · 2 evaluators
No clip delivered the note. Every model misread its subject.
0%delivered
Source clip
A barista makes a latte behind a cafe counter.
Note
"Slow push in on him and rest on a frame where he fills it"
What evaluators checked
The camera moves slowly toward him
The move ends and holds on a frame he fills
Parallax: the counter and background shift at different rates
Other than the note, nothing else has changed
Model
Rank score
Verdict
Checks
Kling
88
Missed 0/2
Seedance
62
Missed 0/2
Wan
50
Missed 0/2
Grok Imagine
50
Missed 0/2
Gemini Omni
0
Missed 0/2
What evaluators saw
Every model pushed toward the coffee cup on the counter instead of his face.
No clip ended on a frame he fills, which was the point of the note.
One revision read as a digital zoom rather than a camera move.
On set, "him" means the face. Models do not reliably resolve who or what a note is about.
07Hardest and easiest directions
Real directions from the round, with how often graders judged the note delivered across all models, and the model that ranked best on each.
Hardest
Delivered
Best
"Slow push in on him and rest on a frame where he fills it." Push in · Revision
0%
Kling
"Deep focus: bring the office behind her into focus too." Deep focus · Revision
0%
Gemini Omni
"Shift focus to the shelves behind him, let him go soft." Rack focus · Revision
13%
Seedance
"Pan with him." Pan · Revision
20%
Grok Imagine
"Go wide open: keep him sharp, let the glass in front and the bar behind fall off." Wide open · Revision
20%
Kling
Easiest
Delivered
Best
"Slow pull back from him to reveal the study" Pull back · Fresh shot
95%
Gemini Omni
"Rack focus from the near player to the far player as he moves a piece" Rack focus · Fresh shot
67%
Kling
"Very shallow focus, the lights behind him fall to large soft discs" Shallow focus · Fresh shot
57%
LTX
08Two test types
Every control is tested two ways, because a model that makes a good shot from scratch may still fail to take a note.
Dailies
A fresh shot from a written setup
The model gets a scene and one direction. All seven models are compared side by side.
42%
delivered the direction · leader: Kling
Notes
A revision to an existing clip
The model gets a source clip and one note, the way a director gives it on set. Everything else must hold. Five models support revisions and are compared side by side; no model revises its own footage.
22%
delivered the note · 44% changed something else · leader: Wan
09The panel
A benchmark is only as credible as the people grading it. FinalGrade-Camera is graded by cinematographers and camera operators at the top of the profession, including members of the International Cinematographers Guild, IATSE Local 600, with 19 to 25 years behind the camera and credits on features, series and commercials.
These are the people productions call when the shot has to be right, and whose eye the rest of the industry calibrates against. When they say a model delivered the direction, that is the standard a working set would hold it to.
The first graded round drew on 3 panelists. More are joining as the round continues, and every grader is held to the same bar.
Invitation onlyGraders are invited on their credits and their standing with peers. There is no open sign-up.
Working at the top of the craftEvery grader is working now, on features, series and commercials, not retired from it or adjacent to it.
The people who set the standardMembers of the International Cinematographers Guild, IATSE Local 600: the guild whose members define how a professional image is made.
Independent by designGraders are anonymous, paid for every hour, blind to the model and to each other, and free to work with anyone.
Local 600International Cinematographers Guild
19 to 25years in the craft
251written notes, marked on the frame
0.32agreement between graders (0 is chance)
10Evaluation effort
What went into this round. Every number here is expert time, not crowd time.
20questions, each a single direction
331blind clip rankings
251written notes, marked on the frame
417minutes of expert viewing
3evaluators, about 2.7 per question
11Methodology
01Build
Prompts
Written by working professionals in the craft. Each question carries one direction in the craft's own terms, with a plain scene and everything else held as simple as possible. Three yes/no checks define "delivered" before any clip exists.
Generation
Every model receives the same prompt through its public interface, with no tuning per model and no cherry-picking. Revisions start from AI-generated source clips rotated across labs; no model revises its own footage.
02Evaluate
Blind ranking
Evaluators see every model's clip for a question side by side, labels hidden and order shuffled. They rank the clips from best to worst at delivering the direction, then draw a line between the clips that delivered it and the ones that didn't.
Checks and notes
For each clip, evaluators answer the three checks, plus a check on revisions that nothing else changed. They write improvement and defect notes, marked on the frame at the moment and place they describe.
03Score
Score and range
A clip ranked first scores 100 and last scores 0, averaged across questions and evaluators. The 95% range comes from resampling questions. The checks agree with the rankings (correlation 0.50 across the round).
Agreement
We report how closely evaluators' rankings match on every question. Questions where they don't converge are flagged and rewritten for the next round.
Exclusions
The test designer's answers and practice questions are excluded from every score. Models are only compared on questions they actually took.
12Reliability
How closely evaluators agreed on each question. Questions on the left of zero are flagged and rewritten for the next round.
Figure · Rank agreement per question
Mean pairwise rank correlation between evaluators on each question: 0 is chance, 1 is identical rankings. Rubric v1.0, round 1, updated Nov 19, 2026.
13The full rubric: 13 dimensions
The complete benchmark covers 13 dimensions. Each prompt tests one and holds everything else fixed. Select a dimension to see sample directions, what evaluators check, and how models did in round 1.
Background
Camera language is how a director talks to a model. When a prompt asks for a slow push-in and gets a static frame, or asks for a rack focus and gets everything sharp, the model has failed the most basic instruction on a set. FinalGrade-Camera is built from the vocabulary cinematographers use on set, and each question isolates one control so a failure points somewhere specific.
Full per-question results, the clips and the panel's written notes, shared under confidentiality, or an evaluation of an unreleased model. Experts always stay anonymous.