FINALGRADE
CinematographyLivePreliminary · Nov 19, 2026

FinalGrade-Lighting

Can a video model light a scene the way a gaffer would?

LTX and Seedance lead the first round, graded by working cinematographers. No model reliably delivers the direction: across all models, graders judged the note delivered on 45% of fresh shots and 25% of revisions.

Preliminary · 3 graders · 10 questions · 163 clip rankingsMethodology

00Key findings

Fresh shots45%

delivered the direction, averaged across all models.

Revisions25%

delivered the note. Taking a note is the harder skill for every model.

Unrequested changes38%

of revisions changed something the note didn't ask for.

Leaders
Fresh shots
LTX
70.2
Revisions
Kling
59.6

The two tests rank models differently.

Preliminary · updated Nov 19, 2026

01Leaderboard

Score is 100 if a model is always ranked first for a question and 0 if always last. Overall shows a 95% range from resampling questions.

#ModelScoreRangeTook the note
01LTX *Lightricks70.2
51 to 8986%
02SeedanceByteDance60.5
47 to 7148%
03KlingKuaishou58.3
45 to 7133%
04Gemini OmniGoogle54.0
40 to 6852%
05Grok ImaginexAI43.5
30 to 6222%
06WanAlibaba36.1
21 to 5411%
07Hailuo *MiniMax25.0
0 to 4521%

With 3 graders the ranges overlap: the top and bottom of the table are separated, the middle is statistically tied. "Took the note" is how often graders judged the clip delivered the direction. * LTX and Hailuo were tested on fresh shots only, on half as many questions. Numbers will move as more graders finish.

02What we tested

All 10 questions in this round, 5 fresh shots and 5 revisions, each written as a single direction in the language of the set. Every model got the same questions. "Delivered" is how often evaluators judged a clip delivered the direction, across all models; "ranked best" is the model ranked highest on that question.

Lighting

Named portrait patterns, motivated and environmental light: Rembrandt, split, loop, butterfly, rim, firelight, golden hour, practicals and hard light. 5 fresh shots, 5 revisions.

35%delivered on average
Direction, as writtenTestDeliveredRanked best
The work light as an underlightA mechanic in his forties slides under a car in a garage, a work light on the floor beside him.UnderlightFresh shot
64%
Grok Imagine
Late-afternoon sun as a hard rim from the window behind him, no key on his faceA man in his fifties talks on his phone, facing camera, in front of a floor-to-ceiling office window.Backlight / rimFresh shot
52%
Seedance
Hard daylight through the blindsTwo men in their fifties shake hands in a law office with venetian blinds.Cookie / blinds patternFresh shot
48%
Kling
Late afternoon. Hard low sun through the window panes as a cookie pattern across her, three-quarter backlight, bounce off the wooden floor as the only fillA ballet teacher in her fifties watches a class from the studio doorway.Cookie / blinds patternFresh shot
38%
Gemini Omni
Firelight onlyThree people in period dress sit in silence around a hearth in a farmhouse kitchen.FirelightFresh shot
29%
LTX
Make it golden hour: low warm sun pouring through the window behind her, rim light on her hair, long shadows across the tableSource: A woman in her thirties sits at a kitchen table with a cup of coffee, a window behind her.Golden hourRevision
47%
Kling
Light him RembrandtSource: A man in his forties sits still on a stool facing camera in a plain gray room, looking just past the lens.RembrandtRevision
40%
Grok Imagine
Make it hard light. Same positionSource: A man in his forties sits still on a stool facing camera in a plain gray room, looking just past the lens.Hard lightRevision
27%
Seedance
Split light himSource: A man in his forties sits still on a stool facing camera in a plain gray room, looking just past the lens.SplitRevision
7%
Wan
Loop light himSource: A man in his forties sits still on a stool facing camera in a plain gray room, looking just past the lens.LoopRevision
0%
Grok Imagine

03Task coverage and results

Every question in the round, grouped by control, with each model's score on each one.

Figure · Round 1 questions by direction
10questions
Cookie / blinds pattern220%
Backlight / rim110%
Firelight110%
Underlight110%
Rembrandt110%
Split110%
Hard light110%
Loop110%
Golden hour110%

10 questions across 9 directions. Each question tests a single direction, as a fresh shot or as a revision.

Figure · Task-by-task results

Rank score by question, not a pass rate. The shaded row shows each model's average for the control; expand a control for every question.

DirectionLTXKlingSeedanceGemini OmniGrok ImagineWanHailuo
Lighting10 questions · LTX leads69575953483623
"Late-afternoon sun as a hard rim from the window behind him, no key on his face"Backlight / rim · Fresh shot
7844836128056
"Hard daylight through the blinds"Cookie / blinds pattern · Fresh shot
44837867171744
"Late afternoon. Hard low sun through the window panes as a cookie pattern across her, three-quarter backlight, bounce off the wooden floor as the only fill"Cookie / blinds pattern · Fresh shot
6761339439560
"Firelight only"Firelight · Fresh shot
100567828502217
"The work light as an underlight"Underlight · Fresh shot
58335842100580
"Make it golden hour: low warm sun pouring through the window behind her, rim light on her hair, long shadows across the table"Golden hour · Revision
n/a10067421725n/a
"Make it hard light. Same position"Hard light · Revision
n/a4275503350n/a
"Loop light him"Loop · Revision
n/a7525128850n/a
"Light him Rembrandt"Rembrandt · Revision
n/a255088880n/a
"Split light him"Split · Revision
n/a5042502583n/a
75 to 10050 to 7525 to 500 to 25n/a: model does not support revisions

04Score distributions

How each model scored question by question, grouped into bands. A model with a high average can still have a long tail of misses.

Figure · Questions per score band
0 to 2525 to 5050 to 7575 to 100
LTX
122
5
Seedance
334
10
Kling
433
10
Gemini Omni
1342
10
Grok Imagine
2413
10
Wan
4141
10
Hailuo
311
5

Each question is one model's average rank score across evaluators. Number at right is questions taken; LTX and Hailuo took fresh shots only.

05Two instruments, one answer

Rankings and yes/no checks are collected independently. Where they agree, the result is measuring the work, not taste.

Rank score
LTX
70.2
Seedance
60.5
Kling
58.3
Gemini Omni
54.0
Grok Imagine
43.5
Wan
36.1
Hailuo
25.0
Checks passed
LTX
62%
Seedance
70%
Kling
52%
Gemini Omni
53%
Grok Imagine
47%
Wan
45%
Hailuo
26%

Rank score: 100 if always ranked first, 0 if always last. Checks passed: share of yes answers across the three checks for each direction. Across the round, checks and rank correlate at 0.50.

06Case studies

Three questions from the round, shown in full: the scene, the direction, what evaluators checked, how every model did, and what evaluators wrote. Notes are paraphrased; evaluators always stay anonymous.

Case 01 · Underlight · Fresh shot · 2 evaluators

A clear winner, and a long tail of near misses.

64%delivered
Scene

A mechanic in his forties slides under a car in a garage, a work light on the floor beside him.

Direction

"The work light as an underlight"

What evaluators checked
  1. Light comes from below his face
  2. Shadows run upward on his face
  3. The work light on the floor is the source
ModelRank scoreVerdictChecks
Grok Imagine
100
Delivered 2/2
Wan
58
Delivered 1/2
LTX
58
Delivered 2/2
Seedance
58
Delivered 1/2
Gemini Omni
42
Delivered 2/2
Kling
33
Delivered 1/2
Hailuo
0
Missed 0/2
What evaluators saw
  • One model was ranked first by both evaluators: light from below, shadows running up the face, the work light as the source.
  • Several clips placed the light but pointed it the wrong way.
  • One clip showed the work light without its effect on his face; another lost the face entirely.

Motivated, environmental setups are the most reliable lighting directions so far.

Case 02 · Golden hour · Revision · 3 evaluators

Three models were judged acceptable, with caveats from every evaluator.

47%delivered
Source clip

A woman in her thirties sits at a kitchen table with a cup of coffee, a window behind her.

Note

"Make it golden hour: low warm sun pouring through the window behind her, rim light on her hair, long shadows across the table"

What evaluators checked
  1. Low warm sun comes through the window behind her
  2. Rim light on her hair
  3. Long shadows fall across the table
  4. Other than the note, nothing else has changed
ModelRank scoreVerdictChecks
Kling
100
Delivered 3/3
Seedance
67
Delivered 2/3
Gemini Omni
42
Delivered 2/3
Wan
25
Missed 0/3
Grok Imagine
17
Missed 0/3
What evaluators saw
  • Evaluators noted that no clip put the sun itself behind her, even when the window was.
  • Two revisions drifted into a stylized, animated look, with the sun moving unrealistically fast.
  • One revision removed the set dressing from the background entirely.

Atmospheric notes are easier than geometric ones, but revisions still change more than the light.

Case 03 · Split · Revision · 3 evaluators

One of the hardest notes in the round: split light was almost never delivered.

7%delivered
Source clip

A man in his forties sits still on a stool facing camera in a plain gray room, looking just past the lens.

Note

"Split light him"

What evaluators checked
  1. One half of his face is lit, the other black
  2. The light falls off at the line of the nose
  3. Key comes from 90 degrees to the side
  4. Other than the note, nothing else has changed
ModelRank scoreVerdictChecks
Wan
83
Missed 1/3
Kling
50
Missed 0/3
Gemini Omni
50
Missed 0/3
Seedance
42
Missed 0/3
Grok Imagine
25
Missed 0/3
What evaluators saw
  • The key was usually too high and too frontal, which reads as Rembrandt rather than split light.
  • In most revisions the light moved during the shot, which the note did not ask for.
  • Several revisions also darkened the background, cooled the color temperature or added camera movement.

Portrait patterns are geometric specifications. Models treat them as a mood.

07Hardest and easiest directions

Real directions from the round, with how often graders judged the note delivered across all models, and the model that ranked best on each.

HardestDeliveredBest
"Loop light him."
Loop · Revision
0%Grok Imagine
"Split light him."
Split · Revision
7%Wan
"Make it hard light. Same position."
Hard light · Revision
27%Seedance
"Firelight only"
Firelight · Fresh shot
29%LTX
"Late afternoon. Hard low sun through the window panes as a cookie pattern across her, three-quarter backlight, bounce off the wooden floor as the only fill"
Cookie / blinds pattern · Fresh shot
38%Gemini Omni
EasiestDeliveredBest
"The work light as an underlight"
Underlight · Fresh shot
64%Grok Imagine
"Late-afternoon sun as a hard rim from the window behind him, no key on his face"
Backlight / rim · Fresh shot
52%Seedance
"Hard daylight through the blinds"
Cookie / blinds pattern · Fresh shot
48%Kling

08Two test types

Every control is tested two ways, because a model that makes a good shot from scratch may still fail to take a note.

Dailies

A fresh shot from a written setup

DIRECTIONKey from left 12345677 MODELS, RANKED BLINDTHE LINEDELIVERED

The model gets a scene and one direction. All seven models are compared side by side.

45%

delivered the direction · leader: LTX

Notes

A revision to an existing clip

SOURCE CLIP + Warmer key REVISION NOTEHELD EVERYTHING EXCEPT THE NOTE MUST HOLD

The model gets a source clip and one note, the way a director gives it on set. Everything else must hold. Five models support revisions and are compared side by side; no model revises its own footage.

25%

delivered the note · 38% changed something else · leader: Kling

09The panel

A benchmark is only as credible as the people grading it. FinalGrade-Lighting is graded by cinematographers at the top of the profession, including members of the International Cinematographers Guild, IATSE Local 600, with 19 to 25 years behind the camera and credits on features, series and commercials.

These are the people productions call when the shot has to be right, and whose eye the rest of the industry calibrates against. When they say a model delivered the direction, that is the standard a working set would hold it to.

The first graded round drew on 3 panelists. More are joining as the round continues, and every grader is held to the same bar.

  • Invitation onlyGraders are invited on their credits and their standing with peers. There is no open sign-up.
  • Working at the top of the craftEvery grader is working now, on features, series and commercials, not retired from it or adjacent to it.
  • The people who set the standardMembers of the International Cinematographers Guild, IATSE Local 600: the guild whose members define how a professional image is made.
  • Independent by designGraders are anonymous, paid for every hour, blind to the model and to each other, and free to work with anyone.
Local 600International Cinematographers Guild
19 to 25years in the craft
118written notes, marked on the frame
0.38agreement between graders (0 is chance)

10Evaluation effort

What went into this round. Every number here is expert time, not crowd time.

10questions, each a single direction
163blind clip rankings
118written notes, marked on the frame
124minutes of expert viewing
3evaluators, about 2.7 per question

11Methodology

01Build
Prompts

Written by working professionals in the craft. Each question carries one direction in the craft's own terms, with a plain scene and everything else held as simple as possible. Three yes/no checks define "delivered" before any clip exists.

Generation

Every model receives the same prompt through its public interface, with no tuning per model and no cherry-picking. Revisions start from AI-generated source clips rotated across labs; no model revises its own footage.

02Evaluate
Blind ranking

Evaluators see every model's clip for a question side by side, labels hidden and order shuffled. They rank the clips from best to worst at delivering the direction, then draw a line between the clips that delivered it and the ones that didn't.

Checks and notes

For each clip, evaluators answer the three checks, plus a check on revisions that nothing else changed. They write improvement and defect notes, marked on the frame at the moment and place they describe.

03Score
Score and range

A clip ranked first scores 100 and last scores 0, averaged across questions and evaluators. The 95% range comes from resampling questions. The checks agree with the rankings (correlation 0.50 across the round).

Agreement

We report how closely evaluators' rankings match on every question. Questions where they don't converge are flagged and rewritten for the next round.

Exclusions

The test designer's answers and practice questions are excluded from every score. Models are only compared on questions they actually took.

12Reliability

How closely evaluators agreed on each question. Questions on the left of zero are flagged and rewritten for the next round.

Figure · Rank agreement per questionFRESHREVISIONAVERAGE 0.38Make it hard light. Same position. (-0.27)Split light him. (-0.07)Loop light him. (0.30)Hard daylight through the blinds (0.36)The work light as an underlight (0.43)Late-afternoon sun as a hard rim from the window behind him, no key on his face (0.48)Late afternoon. Hard low sun through the window panes as a cookie pattern across her, three-quarter backlight, bounce off the wooden floor as the only fill (0.52)Firelight only (0.59)Make it golden hour: low warm sun pouring through the window behind her, rim light on her hair, long shadows across the table. (0.60)Light him Rembrandt. (0.90)-1-0.500.51

Mean pairwise rank correlation between evaluators on each question: 0 is chance, 1 is identical rankings. Rubric v1.0, round 1, updated Nov 19, 2026.

13The full rubric: 13 dimensions

The complete benchmark covers 13 dimensions. Each prompt tests one and holds everything else fixed. Select a dimension to see sample directions, what evaluators check, and how models did in round 1.

Background

Lighting is where AI video looks most convincing and fails most often. A model can make a beautiful image and still ignore the setup it was given: the wrong pattern on the face, a hard light that came out soft, a practical that lights nothing. FinalGrade-Lighting writes each question as a lighting direction in the terms a gaffer uses, and checks whether the light follows it.

Related

For labs, studios and tool makers

See where your model stands with professionals.

Full per-question results, the clips and the panel's written notes, shared under confidentiality, or an evaluation of an unreleased model. Experts always stay anonymous.