Introducing FinalGrade-Lighting · FinalGrade
FINALGRADE
Benchmarks

Introducing FinalGrade-Lighting

Key side, ratio, color temperature, motivation and practicals. Graded in the terms a gaffer uses on set.

Lighting is where AI video looks most convincing and fails most often. A model can make a beautiful image and still put the key on the wrong side, flatten a 4:1 ratio to 1:1, or light a face from a lamp that isn't on.

The reason is that a good-looking image and a correctly lit image are different things. Lighting on a set is a plan: where each source sits, how hard it is, how much it falls off, what color it is and what in the scene justifies it. A gaffer can look at a frame and reverse-engineer the plan. If the plan is wrong, the frame is wrong, however pretty it is.

A lighting plot, read as a rubric

FinalGrade-Lighting turns a lighting plot into prompts. Lighting professionals grade whether the light follows the plot, one dimension at a time.

Figure 1A lighting plot, read as a rubric
BACKGROUND WALLBACKGROUND LEVEL WindowMOTIVATION SUBJECT KeyKEY DIRECTION, HEIGHTHARD VERSUS SOFT, RATIO FillFILL LEVEL BacklightBACKLIGHT AND RIM Lamp in framePRACTICALS, COLOR TEMP CAMERA SHADOW PLACEMENT AND CONTINUITY ARE GRADED ACROSS THE WHOLE SETUP
Each element of a standard setup maps to one dimension in FinalGrade-Lighting. A prompt tests one at a time.
Dimension What the panel checks Example direction
Key direction The key comes from the requested side Key from camera left
Key height The key sits at the right height for the setup High key, 45 degrees above
Lighting ratio Contrast between key and fill is as asked 4:1 ratio
Fill level The shadow side is filled the right amount No fill, let it fall off
Backlight and rim The subject separates from the background as asked Soft rim from behind right
Hard versus soft Light quality is hard or soft as specified Hard source, crisp shadow edges
Color temperature Each source has the right color temperature Tungsten practicals, daylight through the window
Mixed sources Different sources read as different sources Cool moonlight, warm lamp, kept separate
Practicals Lamps in frame read as sources The desk lamp lights his face
Motivation Light comes from a believable source in the scene Motivate it from the window
Shadow placement Shadows fall where the setup says Nose shadow straight down, butterfly
Background level Background exposure holds where asked Background two stops under
Lighting continuity Lighting holds across a move or a cut Same setup on the reverse

How a lighting question is checked

The checks are written the way a gaffer would verify the setup. For "split light him," the panel checks that one half of his face is lit and the other falls to black, that the light falls off at the line of the nose, and that the key comes from 90 degrees to the side. On a revision there is a fourth check: other than the note, nothing else changed.

What round 1 found

Round 1 covered 10 lighting questions, evaluated blind.

Figure 2Round 1: how often models delivered each lighting direction
Underlight
64%
Backlight / rim
52%
Golden hour
47%
Cookie / blinds pattern
43%
Rembrandt
40%
Firelight
29%
Hard light
27%
Split
7%
Loop
0%
Share of graded clips ranked above the grader's line ("took the note"), round 1, averaged across fresh shots and revisions. 10 questions, 3 graders. Preliminary.

Environmental light does better than portrait patterns. An underlight from a work lamp was delivered in 64% of graded clips, and a backlight or rim from a window in 52%. These are setups where the scene itself suggests the light.

Named portrait patterns are hard, especially as a note. Rembrandt as a revision was delivered in 40% of clips, split light in 7%, and loop light in none. These patterns are basic vocabulary on any set. They are also precise: the key has to sit at a specific angle and height, and the shadow has to land in a specific place. "Make it moodier" is easy. "Loop light him" is a specification.

The leaderboard is close at the top. On fresh shots, the top models were separated by a few points, with ranges that overlap. On revisions, the order changes. The results page shows both, with 95% ranges.

Why portrait patterns are so hard

Loop, Rembrandt, split and butterfly are defined by where the key sits and where its shadow falls. Rembrandt needs a small triangle of light on the shadow-side cheek. Loop needs the nose shadow to fall down and to the side, without touching the cheek shadow. Split needs the light to stop at the bridge of the nose. These are geometric specifications, and a model has to place a light source in three dimensions to meet them.

Models appear to treat lighting as a style rather than a setup. They can make a face look "dramatic" or "moody," but they do not reliably place a key at a specific angle and height. Our evaluators' defect notes describe the result: light too high and too soft, sun angle wrong for the window, light that moves during the shot.

Reading the lighting leaderboard

On fresh shots, LTX led lighting at 70.2, followed closely by Seedance at 66.7, with overlapping ranges. On revisions, which LTX doesn't support, Kling led at 59.6. Lighting had fewer questions than camera in round 1, so its ranges are wider, and the order at the top should be read as close rather than settled.

What comes next

Later rounds add lighting ratio, fill level, mixed sources and continuity, and more evaluators per question. Preliminary results are live on the FinalGrade-Lighting page, alongside FinalGrade-Camera.

Keep reading