3 people 3389e6065f fix(nescapture): tag encoded streams full-range to match the samples written (#313) ...

## The bug

`nescapture` sets the colour converter full-range unconditionally, but
the video usability information carried pixelforge's **default
limited-range flag**. A compliant decoder then expanded 16–235 out of
samples that already covered 0–255 — darkening midtones and clipping
both ends.

pixelforge keeps two separate flags for this, one on the converter and
one on the colour description, and its own documentation says they must
agree. Only the first was being set.

The two lines are about forty apart, each is correct on its own, and the
comment above the second states the right intent while the call below it
does the opposite:

```rust
// GPU framebuffer captures are always full-range — use BT.709 full-range
// so the decoder doesn't apply limited-range expansion.
enc_cfg = enc_cfg.with_color_description(ColorDescription::bt709());
//                                       ^ this constructor is limited-range
```

## Evidence

Measured on a Radeon RX 9060 XT, comparing the encoded result against
the compositor's own readback of the same frames:

| ground truth = `51` | before | after |
|---|---|---|
| flat background, decoded | **`38`** | `49–51` |
| mean luma, capture path vs readback | **10.41 apart** | **0.55 apart**
|
| luma histogram intersection | **0.090** | **0.913** |
| declared `color_range` | `tv` | `pc` |

**The encoded luma is byte-identical before and after** — `Y = 51.00`,
standard deviation `0.00` on both runs. Only the tag changed, which is
what identifies this as a signalling bug rather than a conversion one,
and why nothing short of a comparison against ground truth could see it:
the stream was valid, the frame rate was right, the picture was
recognisable, and every liveness check passed.

The HDR arm (`bt2020_pq`) carried the same defect and is fixed the same
way, but **has not been run** — no 10-bit verification here.

## `scripts/verify-chain.sh`

Runs a Vulkan workload under `nescope` with the layer active and
compares the encoded output against `nescope-shot`'s readback of the
same frames. Two paths that share almost no code see the same content,
so disagreement localises the fault; a single path cannot tell a correct
frame from a plausible-looking wrong one.

**Confirmed it fails when this change is reverted** — both the tag check
and the brightness-agreement check fire.

One note on its thresholds, since it is easy to get backwards: the
not-blank check is a low absolute floor plus a comparison against the
readback's own structure, rather than a fixed number. A fixed number was
tried first and was wrong in the worst way — the **broken** build scored
20.49 on it and the **fixed** build 17.74, because the range defect
stretched contrast and that reads as more detail. How much structure a
correct frame carries depends on what the workload drew, so the only
stable reference is ground truth measured in the same run.

## Not covered

`vkcube` rather than a real workload; 720p, H.264, 8-bit; one card, one
driver. XWayland, HUD detection and real swapchain formats are
untouched.










<!-- greptile_comment -->

<h3>Greptile Summary</h3>

The PR aligns encoded-stream color metadata with the full-range samples
produced by nescapture and updates the CPU fallback to BT.709 full-range
conversion.
- Updates pixelforge and configures matching converter color space,
range, and SDR reference white.
- Corrects Vulkan color-space mapping and adds regression tests for SDR,
HDR, and CPU fallback behavior.
- Adds SDR capture-chain and HDR comparison verification scripts.

<h3>Confidence Score: 5/5</h3>

The PR appears safe to merge.

No blocking failure remains.

<h3>Important Files Changed</h3>




| Filename | Overview |
|----------|----------|
| apps/nescapture/src/encode.rs | Aligns GPU and CPU conversion output
with encoded color metadata and adds focused regression coverage. |
| apps/nescapture/scripts/verify-chain.sh | Adds an end-to-end SDR
verifier using a static corner patch to avoid the previously reported
temporal mismatch. |
| apps/nescapture/scripts/verify-hdr.sh | Adds an HDR comparison harness
for inspecting conversion behavior across builds. |
| apps/nescapture/Cargo.toml | Advances pixelforge to the revision
providing the required color-conversion configuration. |
| Cargo.lock | Records the pixelforge update and resulting transitive
dependency refresh. |

<sub>Reviews (5): Last reviewed commit: ["test(nescapture): check the
HDR
conversi..."](https://github.com/nestrilabs/nestri/commit/2f9773c4b7ded7774b9396648de26423afe926fa)
| [Re-trigger
Greptile](https://app.greptile.com/api/retrigger?id=60147076)</sub>

**Context used:**

- Knowledge Base — [Vulkan capture
layer](https://app.greptile.com/nestri/-/custom-context/knowledge-base/nestrilabs/nestri/-/docs/capture-layer.md)

<!-- /greptile_comment -->

---------

Co-authored-by: DatCaptainHorse <DatCaptainHorse@users.noreply.github.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>

2026-09-04 19:03:13 +03:00

7.2 KiB

Executable File

Raw Blame History

View Raw