# Validation status

As of 2026-08-18, the catalog contains **57 cases: 46 pass, 6 fail, 3 blocked, and 2 intentionally skipped**. The added passing case is a local fitted-Q nanobeam objective: a fresh literal-cylinder solve reaches Q=187,133, or 5.33× the unchanged cavity, after one exact-checkpointed adjoint update. A matched Tidy3D control/final pair with 20 holes in both scenes independently confirms that the shape update raises Q from 84,247 to 146,678 (1.741×), although FDTDX overpredicts the fixed-count gain as 2.023× at 25 nm resolution. Separately, an independent Tidy3D forward run of each available Tidy3D-derived binary inverse design invalidated the optimistic local WDM score and moved WDM and bandpass out of the passing set. The S-matrix crossing is the only such design that passes the user's 10% trust check, but its Tidy3D score is 93.75% of published and therefore misses the repository's stricter 95% external gate. The compact grating coupler is reproduced at its exact 20 nm control resolution and fails honestly below the published target.

```{figure} ../_static/generated/validation_status.svg
:alt: Stacked bar of passing, blocked, skipped, and failing cases
```

## What is demonstrated

| Area | Evidence in the passing suite |
|---|---|
| Core FDTD | phase velocity, impedance, finite energy, PML decay, temporal profiles |
| Grids/boundaries | uniform and rectilinear metrics, PML, periodic, TFSF, symmetry tests in smoke |
| Materials | dielectric and lossy interfaces, Lorentz/Drude/CCPR, anisotropy, rotated complex tensors |
| Sources/detectors | plane, Gaussian, dipole, mode, TFSF; fields, phasors, energy, flux, overlaps |
| Scattering | dielectric cylinder against Mie theory and Tidy3D |
| PIC | straight guide, S-parameters, Y junction, ring, directional coupler, O-band crossing, Euler bend, edge coupler, 1×4 MMI, Bragg grating, nanobeam cavity, polarization splitter-rotator |
| Free-space / periodic | dielectric THz metasurface absorber and geometric-phase metalens |
| Inverse design | reusable filters, projection, erosion/dilation, symmetry handling, binary validation, automatic pulse duration, memory-planned optimization, and differentiable fitted-Q shape controls; the local high-Q cavity passes, while no Tidy3D-derived final artifact yet clears the external 95% gate |

Every passing case has a page in the [example atlas](../examples/index.md) with its exact current metric.

The [real-world device ladder](../examples/real_world_devices.md) collects the reconstructions in one illustrated walkthrough. All ten forward entries, the new high-Q cavity, and the inverse audits have executed notebooks. The high-Q page includes parameter setup, automatic mirror completion, exact gradient, final binary cylinders, fresh fields, ringdowns, spectra, and two independent fit paths. The Tidy3D-derived inverse pages add fabrication corners and independent Tidy3D scores where available. Only the S-matrix crossing is within 10% of published performance; none clears 95% under the reference solver. No cloud simulation runs in the development loop; the earlier external final-design validation used 0.19759 FlexCredits under the user's separate 2 FC authorization.

## Blocked references

These are not counted as solver failures because the cached external reference is not trustworthy enough to decide correctness:

`parity_ring_drop`
: The cached Tidy3D spectrum did not converge and changed materially across reruns.

`parity_grating_coupler`
: The Tidy3D transmission spectrum was unstable across runs and needs a newly converged golden.

`parity_mmi_splitter`
: A cached Tidy3D run did not converge. Repeated finite-time reference results are not reproducible; the corrected local scene matches the first reference run rather than the quarantined artifact.

Resolving these requires a carefully reviewed, one-time external run. Ordinary development must not spend FlexCredits.

## Deliberate skips

Two placeholder cases are superseded or lack a real reference: an early coarse Y-junction placeholder and an inverse mode-converter placeholder. They are kept visible so catalog history is not rewritten.

## What is not yet claimed

- complete Tidy3D API compatibility;
- generally converged high-$Q$ resonator spectroscopy across grids, runtimes,
  process corners, and multi-pole scenes;
- a full material database or tabulated-$n,k$ fitting workflow;
- general near-to-far/diffraction parity;
- multi-port component-modeler ergonomics;
- nonlinear, thermal, charge, or EME parity;
- identical inverse-designed geometries or optimization speed.
- universal device-level inverse-design parity at the current local resolution; independent forward checks show one 10%-tolerance pass that still misses the 95% gate, three larger performance failures, and one invalid local objective among the five tested final designs.

The nanobeam forward case validates a specific $Q\approx3.5\times10^4$ pole,
and the inverse case independently validates a nominal $Q\approx1.87\times10^5$
literal design. The limitation above concerns general, systematically converged
high-Q workflows rather than these scoped benchmarks.

A held-out 20 nm replay with the same 0.30 µm physical PML thickness gives
$Q=127{,}282$ for the optimized geometry and $Q=40{,}862$ for its matched
baseline (3.12×). Because the 500 nm beam spans an odd number of cells on that
mirror-reduced refinement, this supports robustness but is not presented as a
formal grid-convergence result.

```{admonition} A pass is scoped
:class: important

A passing benchmark establishes the observable, geometry class, discretization, and tolerance encoded by that case. It does not certify every scene built from the same API objects. Run grid, boundary, and runtime convergence for scientific results.
```