Source

Engine design23_v1 · package package/design23_recentered_optimizer · assembled 2026-07-29 15:57 UTC.

Design23 self-consistent optimizer v1.4.0¶

This is the production replacement for the invalidated v1.3 campaign loop. It starts from the best verified geometry from the July 23 gradient gate:

FDTD Q: 20,942.06
FDTD wavelength: 791.593 nm
geometry: plus_0p5nm in tidy3d_validation_batch.json

The optimizer changes the 126 positive-side hole gap/radius coordinates. Beam width and thickness remain frozen.

The important correction¶

Version 1.3 reused the original frequency center and frequency derivative while the geometry and resonance moved. Its reported exponential Q growth was an extrapolation artifact.

Version 1.4 does all of the following before it returns Q or a gradient:

Rebuilds the complete pole-subtracted outgoing radiation operator at the current geometry and real frequency.
Re-enumerates all closed strip poles and re-evaluates the radiation continuum.
Builds operators at the center and at center ± \(10^{-4}\).
Newton-recenters the expansion until abs(real(tracked_eigenvalue)) <= 5e-5.
Recomputes the local frequency derivative at that center.
Requires a passive pole, branch overlap ≥ 0.999, side overlap ≥ 0.999, and reciprocity error ≤ \(10^{-10}\).
Uses the 412-node operator only to screen a step.
Uses the converged 812-node operator to accept the step and calculate the gradient.

There is no global stored frequency derivative in this implementation.

Optimization method¶

The objective is -log(Q) in dimensionless coordinates where one coordinate unit is 1 nm. The first direction is steepest descent. After one accepted secant pair, full BFGS proposes directions for all 126 variables.

BFGS cannot accept a design. Every proposed direction is:

limited to a 0.5 nm maximum-coordinate trust radius;
checked against the coupled geometry constraints;
line searched by halving the radius;
self-consistently recentered at both continuum resolutions;
rejected if the 412- and 812-node Q ratios disagree by more than 5%;
checked with the Armijo condition;
rejected if one step predicts more than a 0.25 log-Q increase.

The inverse Hessian is updated only after full acceptance. It is reset on bad curvature, a non-descent direction, a failed analytical iteration, or an FDTD rollback.

Install and verify¶

From PowerShell:

cd .\design23_recentered_optimizer
python -m pip install -r requirements.txt
python -m unittest discover -s tests -v

The offline suite should report 24 passing tests.

Run the first two analytical steps¶

python .\run_recentered_bfgs.py `
  --seed .\seed_best_verified.json `
  --output .\recentered_bfgs_run `
  --new-accepted-steps 2

This is intentionally much slower than v1.3. Each Newton correction requires three rebuilt real-frequency operators, and each accepted step is checked at both 412 and 812 nodes. The terminal prints one newton_recenter JSON line after every center/minus/plus triple so the residual collapse is visible.

The command is restartable. state.json is written atomically after every accepted step. Running the same command again resumes the same state.

After two accepted analytical steps, the command stops with status awaiting_fdtd and writes a uniquely named batch such as:

recentered_bfgs_run/pending_fdtd_batch_step_0002.json

Run the FDTD checkpoint¶

Preflight:

python .\run_pending_fdtd.py `
  --state .\recentered_bfgs_run\state.json `
  --preflight

Estimate cost:

python .\run_pending_fdtd.py `
  --state .\recentered_bfgs_run\state.json `
  --estimate

Submit the incumbent and candidate:

python .\run_pending_fdtd.py `
  --state .\recentered_bfgs_run\state.json `
  --submit

Record the result:

python .\run_pending_fdtd.py `
  --state .\recentered_bfgs_run\state.json `
  --record

The candidate is promoted only if its FDTD Q exceeds the best previously accepted FDTD Q. If it fails:

the failed result is retained in fdtd_history;
the incumbent Q and geometry are unchanged;
the analytical geometry is rolled back to the incumbent;
the trust radius is halved;
BFGS is reset.

A rejected FDTD result can never become the comparison baseline.

Then resume:

python .\run_recentered_bfgs.py `
  --seed .\seed_best_verified.json `
  --output .\recentered_bfgs_run `
  --new-accepted-steps 2

Output¶

The important files are:

state.json: complete restart state, immutable best-FDTD ledger, BFGS inverse Hessian, current gradients, branch vectors, and all decisions.
iterations/iteration_XXXX.json: every line-search attempt and physical rejection reason.
pending_fdtd_batch_step_XXXX.json: exact incumbent and candidate geometries for the next checkpoint.
fdtd_step_XXXX/gradient_gate_results.json: analyzed Tidy3D result.

Safety boundary¶

No analytical optimizer can guarantee in advance that every accepted geometry will improve a separate FDTD solver. This package instead guarantees the engineering behavior that matters:

it will not report Q from an unrecentered local extrapolation;
it will not silently change radiation resolution between a comparison;
it will not accept a branch jump or non-passive pole;
it will not replace a good FDTD incumbent with a failed checkpoint;
it will stop rather than manufacture progress when its gates fail.

run_autonomous.py is deliberately disabled in this release because it is the old fixed-linearization campaign driver.