Skip to content

Source

Engine majumdar_lab · package package/multilayer_autonomous_optimizer · assembled 2026-07-29 15:57 UTC.

Autonomous multilayer nanobeam optimizer

This is a standalone, resumable loop for the uploaded Majumdar SiN cavity with the experimentally relevant film stack:

  • substrate: n=1.45;
  • SiN beam: 500 nm wide, 300 nm thick, n=2.0;
  • anthracene: 200 nm, n=1.8;
  • PVA: 200 nm, n=1.5;
  • air side and upper cladding.

It performs the requested sequence:

  1. run a 24 ps Tidy3D FDTD checkpoint;
  2. report maximum estimated and actual billed FlexCredit cost, Q, peak and anthracene-referenced mode volumes, fundamental-TE0 feed-waveguide external Q and decay fraction, and save both the cavity and TE0 port-mode plots;
  3. perform five guarded analytical geometry iterations;
  4. run and report the next 24 ps FDTD checkpoint;
  5. repeat until Ctrl+C, the next job would cross the budget, or 25 FlexCredits have been accounted.

REPORT.md, REPORT.html, and run_state.json are written atomically after every optimizer iteration and every FDTD build, estimate, start, download, analysis, completion, error, or interruption. REPORT.html refreshes itself every 20 seconds.

Install

Python 3.10 or newer is required.

unzip multilayer_autonomous_optimizer_v0.3.1.zip
cd multilayer_autonomous_optimizer
python -m venv .venv

# Linux/macOS
source .venv/bin/activate

# Windows PowerShell instead:
# .venv\Scripts\Activate.ps1

python -m pip install --upgrade pip
python -m pip install -r requirements.txt
tidy3d configure --apikey=YOUR_API_KEY

Continue from the confirmation you already ran

Do not pay to repeat the baseline and phase-16 simulations. Point the new package at the directory containing these two files:

majumdar_multilayer_baseline.hdf5
majumdar_multilayer_phase16.hdf5

They may either be directly in that directory or in its data/ subdirectory. The following command copies them into a fresh resumable run, repairs the old permittivity-interpolation post-processing bug, regenerates Q/V and total-flux diagnostics and plots, records the analytical-to-FDTD calibration, and builds the next simulation locally. It does not upload or spend anything:

python run_autonomous.py \
  --output autonomous_run \
  --bootstrap-confirmation /path/to/multilayer_fdtd_output \
  --prior-spend-flexcredits 0.212

0.212 is the conservative sum of the two displayed 0.106 maximum estimates. If the Tidy3D workbench gives you the two real billed costs, use their sum instead. The imported cost counts toward the hard 25-FlexCredit ledger.

The repaired confirmation is:

Metric Baseline Phase 16 Change
FDTD Q 5383.02 5601.21 +4.053%
wavelength (nm) 778.4620 778.4663 +0.0043 nm
peak V / (lambda/nSiN)^3 3.4581 3.4618 +0.107%
core-center Ey V / (lambda/nSiN)^3 7.7803 7.7151 -0.838%
anthracene-center V / (lambda/nAnth)^3 11.7751 11.6986 -0.650%
total x-plane flux decay fraction (not mode-resolved) 0.8583 0.8722 +0.0139
directional-Q / fitted-Q 1.0141 1.0215 —

The FDTD log-Q response is 0.716 times the analytical prediction and has the same sign. The newly matched core-center FDTD volume also changes in the same direction as the analytical core-point volume. This validates the local Q/V direction, but not the analytical TE0 coupling prediction: the two imported HDF5 files predate the new mode monitors and cannot distinguish TE0 from other x-directed modes or radiation.

Generic preflight without spending credits

This builds and serializes the complete initial Tidy3D simulation locally. It does not upload anything:

python run_autonomous.py --output autonomous_run

Open autonomous_run/REPORT.html to verify the live report layout and inspect autonomous_run/preflight_simulation.json if desired.

Start or resume the loop

python run_autonomous.py --output autonomous_run --run --analytical-workers 2

The same command resumes from autonomous_run/run_state.json. Do not start two copies against the same output directory. Press Ctrl+C once to stop. If a Tidy3D task is active, the program attempts to abort it, accounts its observed cost (or conservatively its maximum estimate if actual cost is unavailable), and refreshes both reports before exiting.

The hard cap defaults to 25 FlexCredits and cannot be configured above 25:

python run_autonomous.py --output autonomous_run --run \
  --flexcredit-budget 25 \
  --run-time-ps 24 \
  --steps-per-wavelength 10

For a one-checkpoint smoke test:

python run_autonomous.py --output smoke_test --run --max-fdtd-checkpoints 1

That limit stops after the initial checkpoint, before analytical optimization. For a bootstrapped run, the two imported simulations already count as checkpoints, so use --max-fdtd-checkpoints 3 to stop after one new FDTD job.

Optimization objective

The default scalar score is

log(Q/Q0) - 0.35 log(V/V0) + 0.10 log(beta_feed/beta_feed,0)

where V is the analytical core-point QNM mode volume and beta_feed is the fraction of cavity decay into the continued fundamental Ey-like mode of the two inline x-directed waveguide ports. The Q gradient is the validated implicit all-geometry derivative with respect to all 25 gaps, 25 longitudinal radii, and 25 transverse radii. Mode-volume and coupling gradients are learned as minimum-norm secants from accepted steps. Every proposal is accepted only after a nonlinear pole solve and fresh exact QNM mode-volume and guided-coupling evaluation. Wavelength drift is removed to first order and branch overlap must remain at least 0.98.

Weights and the per-coordinate trust radius can be changed only when starting a new output directory:

python run_autonomous.py --output stronger_v --run \
  --mode-volume-weight 0.6 \
  --coupling-weight 0.1 \
  --trust-radius-nm 0.2

The analytical iterations are CPU-heavy but do not consume FlexCredits. Version 0.3 shares the base/plus/minus QNM linearization and contour samples among Q, V, coupling, and the geometry gradient. On the production phase-16 problem, the first exact Q/V/coupling evaluation fell from about 450 s to 179 s, reuse of an already-built state took 0.39 s, and the implicit gradient after that state fell from about 300 s to 86 s. The candidate pole solve remains sequential, but now exits immediately if the gradient-predicted pole already meets tolerance.

Two analytical workers parallelize the independent plus/minus frequency evaluations. More than two does not help this call graph. To avoid BLAS oversubscription on Linux/macOS, a useful launch pattern is:

OMP_NUM_THREADS=1 OPENBLAS_NUM_THREADS=1 MKL_NUM_THREADS=1 \
python run_autonomous.py --output autonomous_run --run --analytical-workers 2

The worker count is a performance setting and may be changed when resuming an existing output directory. See docs/PERFORMANCE_AND_TE0.md for the measured breakdown and remaining bottleneck.

Future checkpoints use field_interval_space=3 (the confirmation used 4) to make relative mode-volume changes more trustworthy without changing the FDTD mesh. Override it only when starting a fresh output directory.

Output layout

autonomous_run/
  REPORT.md                 continuously updated text report
  REPORT.html               continuously updated browser report
  run_state.json            exact resumable state and cost ledger
  designs/                  geometry vector at every FDTD checkpoint
  simulations/              loadable Tidy3D Simulation JSON
  data/                     downloaded SimulationData HDF5
  results/                  detailed JSON and resonance tables
  plots/                    geometry, ringdown, cavity-field, and TE0 port plots

The analytical feed-waveguide coupling follows the continued fundamental Ey-like feed pole. New FDTD checkpoints independently measure the same channel with a Tidy3D mode decomposition at the straight +x feed port. The selected branch must be the highest-real-neff propagating candidate with x-plane TE fraction at least 0.5; for an x-normal plane this is the Ey-like fraction |Ey|^2 / (|Ey|^2 + |Ez|^2). The code also checks outgoing direction, incoming contamination, mode-power closure, and resonance detuning. X symmetry supplies the identical -x contribution.

The resulting cavity-decay metric is beta_TE0 = Q_total / Q_TE0. It is not driven on-resonance transmission or the fraction launched from an external input into the cavity; a separate port-excited S-parameter calculation would be required for those observables. Imported v0.2 files report only the legacy all-x-plane flux fraction and leave TE0 values blank.

Cost safety and recovery

Each simulation is uploaded first so web.estimate_cost(task_id) can provide the maximum job estimate. It is started only when

actual_spent + maximum_job_estimate <= configured_budget <= 25

After completion, web.real_cost(task_id) is recorded. If actual cost cannot be retrieved, the maximum estimate is charged to the local ledger. This is intentionally conservative.

An ordinary Ctrl+C is handled. A power failure or forced process kill can leave active_task in run_state.json. In that case the program refuses to resume automatically because blindly resubmitting could double-spend. Inspect the named task in the Tidy3D workbench and reconcile its actual cost/result before editing or starting a fresh output directory.

The 200 nm PVA thickness is fixed because that is the explicit value in the benchmark call that produced Q=5793.918446 at 780.443613 nm. A 300 nm PVA run requires rebuilding the analytical layered background, not merely changing the FDTD geometry.

Confidence gate before turning this into the long-lived repository

The phase-16 result passes the first gate: correct branch, correct Q direction, stable wavelength, and consistent six-face loss accounting. The next run is the second gate: five more guarded analytical steps followed by one new FDTD checkpoint. We should require the composite FDTD objective to improve and the analytical/FDTD Q directions to agree again. After that, run one mesh/monitor convergence check and one finite-difference gradient audit before treating the optimizer as production-ready. The intended repository layout is documented in docs/REPOSITORY_BLUEPRINT.md.