Source

Engine quan_loncar_v1 · package package/quan_loncar_optimizer · assembled 2026-07-29 15:57 UTC.

Quan–Lončar trustworthy optimizer 2.0.0¶

This package restarts the optimization from the original 15-pair Quan–Lončar nanobeam and preserves the solver lineage that produced the only clear analytical-to-FDTD optimization gains in this project.

The default optimizer is deliberately conservative. It uses the proven projected gradient direction, solves a fresh complex pole for every trial, recomputes the complete 45-coordinate custom VJP after every accepted step, and stops for a matched FDTD comparison after two accepted steps.

It does not claim that the analytical model is universally trustworthy. It is a self-checking optimizer for this fixed Quan–Lončar device family. FDTD remains the authority.

What is preserved¶

The entire Phase-14 through Phase-57 Quan solver lineage and the supporting Phase-1 through Phase-6 waveguide/cavity code are copied unchanged under engine/. The production path is the Phase-55/57 configuration:

P2-z1 hole basis;
y3-z1 waveguide cross-section basis;
outgoing pole-subtracted radiation continuum;
the original (+1, -1, +1) Ey-like symmetry sector;
15 symmetric hole pairs and 45 geometry coordinates;
the original exact geometry pullbacks and custom VJP.

engine_manifest.json pins SHA-256 hashes for the important preserved source files. run_preflight.py and the regression suite fail if any of them changes.

The new code under src/quan_optimizer/ is a trust and restart layer around that engine. It does not replace the Maxwell operator.

Evidence carried with the release¶

Campaign	FDTD baseline Q	FDTD optimized Q	Improvement
Phase 57, 12 ps	15,468.56	42,792.17	2.766×
Phase 58 step 3, 24 ps	15,449.47	59,337.16	3.841×

The later Phase-58 analytical design with reported analytical Q=30,307 has no matching FDTD result. It is included for audit but is explicitly marked unverified and cannot be selected accidentally.

See VALIDATION_EVIDENCE.md for the exact interpretation.

Installation¶

Python 3.11 or newer is required.

cd .\quan_loncar_optimizer
python -m pip install -r .\requirements.txt
python -m unittest discover -s .\tests -v

The analytical optimizer itself needs only NumPy and SciPy. Tidy3D is separated into requirements-fdtd.txt.

Preflight¶

The fast preflight checks every pinned source hash and the selected seed:

python .\run_preflight.py --seed baseline

The physical preflight also solves the production complex pole and recomputes the full 45-coordinate derivative:

python .\run_preflight.py `
  --seed baseline `
  --physical `
  --output .\physical_preflight.json

On the release machine this took about 13 minutes. The pole part converged to an operator residual of \(2.29\times10^{-10}\); the derivative gate passed. This is a numerical self-consistency test, not an FDTD validation.

Recommended first campaign¶

Start from the original baseline, use the method that already produced the FDTD-positive Phase-57/58 geometries, and stop after two accepted steps:

python .\run_optimizer.py `
  --seed baseline `
  --output .\quan_baseline_restart `
  --method gradient `
  --new-accepted-steps 2 `
  --checkpoint-every 2

The run is restartable. It writes:

state.json, using atomic replacement;
complete accepted-step records under checkpoints/;
all rejected line-search attempts and their physical gates;
the current complex right eigenvector and complete gradient;
an immutable accepted-FDTD geometry and append-only FDTD history;
REPORT.html;
matched incumbent and candidate designs in fdtd_checkpoint_step_XXXX/.

The analytical engine is intentionally slow. A full pole plus VJP evaluation took approximately 13 minutes on the release machine. Each line-search trial also solves a fresh pole.

Run the matched FDTD checkpoint¶

Install the pinned FDTD dependency:

python -m pip install -r .\requirements-fdtd.txt

Validate the generated designs locally first:

python .\run_fdtd.py `
  .\quan_baseline_restart\fdtd_checkpoint_step_0002\baseline_incumbent.json `
  .\quan_baseline_restart\fdtd_checkpoint_step_0002\candidate.json `
  --output .\quan_baseline_restart\fdtd_step_0002 `
  --convergence `
  --validate-only

Then submit the matched convergence panel:

python .\run_fdtd.py `
  .\quan_baseline_restart\fdtd_checkpoint_step_0002\baseline_incumbent.json `
  .\quan_baseline_restart\fdtd_checkpoint_step_0002\candidate.json `
  --output .\quan_baseline_restart\fdtd_step_0002 `
  --convergence `
  --run-time-ps 60 `
  --submit

This generates baseline, mesh, runtime, boundary, and combined refinements for both geometries. Review both __convergence.json files. A passing ledger entry requires explicit confirmation and both matched result files:

python .\record_fdtd_checkpoint.py `
  --run .\quan_baseline_restart `
  --decision auto `
  --result-json .\quan_baseline_restart\fdtd_step_0002\results\quan_candidate_step_0002__combined.json `
  --incumbent-result-json .\quan_baseline_restart\fdtd_step_0002\results\quan_fdtd_incumbent_step_0002__combined.json `
  --convergence-passed `
  --note "Both matched convergence gates passed"

If either convergence gate fails, record the failure instead:

python .\record_fdtd_checkpoint.py `
  --run .\quan_baseline_restart `
  --decision reject `
  --result-json .\quan_baseline_restart\fdtd_step_0002\results\quan_candidate_step_0002__baseline.json `
  --convergence-failed `
  --note "FDTD convergence gate failed"

A rejected candidate is retained as diagnostic evidence, but it cannot replace the accepted incumbent. The geometry rolls back, BFGS state is cleared, and the trust radius is halved.

Resume¶

After recording the checkpoint, repeat the original optimizer command with the same settings:

python .\run_optimizer.py `
  --seed baseline `
  --output .\quan_baseline_restart `
  --method gradient `
  --new-accepted-steps 2 `
  --checkpoint-every 2

The saved state owns the seed after initialization. The --seed argument is ignored on resume, but all optimizer settings must match the saved run.

BFGS¶

BFGS is implemented, but it is not the default because the new optimizer should first reproduce the known gradient/FDTD correlation.

Start BFGS only as a separate campaign after at least one new two-step gradient checkpoint passes:

python .\run_optimizer.py `
  --seed baseline `
  --output .\quan_baseline_bfgs `
  --method bfgs `
  --bfgs-warmup-steps 2 `
  --new-accepted-steps 2 `
  --checkpoint-every 2

BFGS only proposes a direction. It cannot bypass geometry, pole, branch, reciprocity, residual, Armijo, trust-radius, or FDTD gates. Invalid curvature, loss of positive definiteness, non-descent directions, discontinuous gradients, failed line searches, and FDTD rollback all skip or reset its state.

Additional qualification tools¶

Run a stricter basis/continuum audit:

python .\run_basis_audit.py `
  --seed baseline `
  --output .\basis_audit.json

This compares production order-16/D16-A48 with order-24/D20-A64 and requires:

Q agreement within 2%;
full gradient cosine at least 0.99;
projected optimization-direction cosine at least 0.99.

Generate a local FDTD falsification panel from any initialized run:

python .\make_validation_panel.py `
  --run .\quan_baseline_restart `
  --output .\quan_baseline_restart\validation_panel

The panel contains the gradient, negative gradient, dominant coordinates, and random directions at 0.25 and 0.5 nm scaled L2 displacements. Use it before a long campaign to test sign and rank correlation against FDTD.

Seeds¶

baseline: original Quan geometry and recommended starting point; verified FDTD Q=15,449.47 in the 24 ps comparison.
phase58_step3_verified: best independently verified geometry; FDTD Q=59,337.16.
phase58_q30k_unverified: later analytical continuation with no matching FDTD. It requires --allow-unverified-seed and an immediate FDTD gate.

Scope and limits¶

This code supports the established Quan–Lončar family only: the same cross-section, material model, symmetry sector, elliptical-hole parameterization, and 15-pair layout. It does not automatically validate a new cavity family and does not prove that a high analytical Q is physically real.

The success criterion is sustained, matched FDTD improvement. If the local perturbation panel or successive checkpoints lose directional correlation, stop. Do not loosen the physical gates to make the run continue.

Read TRUST_CONTRACT.md before changing tolerances or checkpoint cadence.