3D air-strip nanocavity

96 GB fine-grid convergence runbook

Run from the project root on the 96 GB Linux workstation. This sequence uses the included transferred checkpoint and stops at each scientific gate. Do not launch the PML campaign or optimizer merely because a resonance peak is visible.

Status on 2026-08-17: this initial reacquisition sequence is complete on the exact y/z backend. The baseline sheet-local spectral and PML gates pass, and three atom-LDOS/Q steps are independently accepted, including one shifted constrained-MMA step. That latest topology passes its own checkpoint-bound PML campaign. Use UNATTENDED_RUNBOOK.md for the current multi-step command and its explicit adaptive PML cadence. Deferred topology checkpoints are never described as PML converged.

0. Unpack, create the environment, and record the machine

cd ~/github/periodic_mirror_3d
python3 -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip
python -m pip install -r requirements.txt
export OPENBLAS_NUM_THREADS=4
export OMP_NUM_THREADS=4
export MKL_NUM_THREADS=4
mkdir -p outputs_96gb_machine
uname -a | tee outputs_96gb_machine/uname.txt
lscpu | tee outputs_96gb_machine/lscpu.txt
free -h | tee outputs_96gb_machine/memory.txt
python --version | tee outputs_96gb_machine/python.txt
python -m pip freeze | tee outputs_96gb_machine/pip-freeze.txt

If virtual-environment creation fails on Debian/Ubuntu:

sudo apt update
sudo apt install python3-venv python3-full unzip time

Do not use --break-system-packages.

1. Verify package integrity and tests

If SHA256SUMS.txt is present, run:

sha256sum -c SHA256SUMS.txt

Then:

python -m unittest discover -v 2>&1 | tee outputs_96gb_machine/tests.log

Expected result after the shifted-MMA and parallel-gradient additions: 72 tests pass. Stop if any test fails.

Re-run the existing controlled-grid combined validator. This requires no new Maxwell sweep and confirms that the transferred numerical artifacts still produce the accepted result:

python combined_rational_validation.py \
  --wide-data outputs_refined_multiprobe/multiprobe_data.npz \
  --local-data outputs_refined_multiprobe_mode_b/multiprobe_data.npz \
  --reference-real 1.0794616875213896 \
  --reference-imag -0.000585613385409534 \
  --output-dir outputs_96gb_controlled_validator_recheck \
  2>&1 | tee outputs_96gb_machine/controlled-validator.log
python -m json.tool outputs_96gb_controlled_validator_recheck/metrics.json

Expected accepted: true and Q approximately 921.6505. Stop if this changes.

2. Inspect and calibrate the fine profile

The included fine checkpoint is:

outputs_refined_fine_workstation64_checkpoint/run_data.npz
outputs_refined_fine_workstation64_checkpoint/config.json

Its grid is 40 x 26 x 22, with 68,640 electric unknowns and a 1,716-unknown surface plane. The conservative analytic estimate is 16.16 GiB, but it is not a substitute for measured RSS.

Print the estimate:

python vector3d_optimizer.py \
  --profile workstation64-fine \
  --estimate-only \
  2>&1 | tee outputs_96gb_machine/fine-estimate.log

Run the expensive boundary/adjoint acceptance check once under the system timer:

/usr/bin/time -v python vector3d_optimizer.py \
  --profile workstation64-fine \
  --steps 0 \
  --output-dir outputs_96gb_fine_acceptance \
  2>&1 | tee outputs_96gb_machine/fine-acceptance.log

This check uses the profile's standard initial topology, not the transferred candidate; its purpose is to validate the fine-grid operator, boundary, and adjoint implementation. Stop if:

  • the process swaps or approaches roughly 75 GiB RSS;
  • the complex-symmetry, condensation, coupling-support, or Dyson residual is much worse than 1e-9;
  • the two-period translation error is much worse than 1e-8; or
  • the directional LDOS-gradient relative error is worse than 1e-3.

The 75 GiB operational ceiling leaves headroom for the OS, filesystem cache, BLAS/SuperLU variability, and Codex. It is a safety margin, not a solver requirement.

3. Reacquire the fine-grid spectrum broadly

A grid transfer carries no frequency or Q. Begin with a sparse broad search:

/usr/bin/time -v python frequency_search.py \
  --warmstart outputs_refined_fine_workstation64_checkpoint/run_data.npz \
  --frequency-min 0.976 \
  --frequency-max 1.176 \
  --points 17 \
  --output-dir outputs_96gb_fine_broad_search \
  2>&1 | tee outputs_96gb_machine/fine-broad-search.log
python -m json.tool outputs_96gb_fine_broad_search/metrics.json

Inspect frequency_search.png, both real and imaginary response, and all curvature candidates. The controlled-grid pole 1.07946 is only a tracking hint. If the 17-point search has no credible feature, run an interleaved 33-point search over the same interval rather than assuming the mode vanished:

/usr/bin/time -v python frequency_search.py \
  --warmstart outputs_refined_fine_workstation64_checkpoint/run_data.npz \
  --frequency-min 0.976 \
  --frequency-max 1.176 \
  --points 33 \
  --output-dir outputs_96gb_fine_broad_search33 \
  2>&1 | tee outputs_96gb_machine/fine-broad-search33.log

If the feature lies outside the interval, expand the interval symmetrically. Do not select a target using maximum LDOS alone; retain every plausible nearby feature for the wide multi-probe model.

4. Acquire wide and dense-local seven-probe data

Choose a wide window enclosing the complete fine-grid cluster plus smooth background on both sides. Set these project-specific shell variables from the broad search (example values below are the controlled-grid values and must be changed if the fine-grid cluster moves):

export NANOCAVITY_WIDE_CENTER=1.076
export NANOCAVITY_WIDE_HALFSPAN=0.007

Run the restartable 41-point sweep:

/usr/bin/time -v python multiprobe_sweep.py \
  --warmstart outputs_refined_fine_workstation64_checkpoint/run_data.npz \
  --center "$NANOCAVITY_WIDE_CENTER" \
  --half-span "$NANOCAVITY_WIDE_HALFSPAN" \
  --sample-points 41 \
  --output-dir outputs_96gb_fine_multiprobe_wide \
  2>&1 | tee outputs_96gb_machine/fine-multiprobe-wide.log

The script saves multiprobe_partial.npz after every new frequency and resumes by default. If interrupted, rerun exactly the same command.

Use the wide response to estimate the upper target pole and its half-width \(\gamma=|\Im f_p|\). Choose a dense local window extending approximately four half-widths on either side, with at least three samples per half-width. For a controlled-grid-like cluster:

export NANOCAVITY_LOCAL_CENTER=1.0795
export NANOCAVITY_LOCAL_HALFSPAN=0.0023
/usr/bin/time -v python multiprobe_sweep.py \
  --warmstart outputs_refined_fine_workstation64_checkpoint/run_data.npz \
  --center "$NANOCAVITY_LOCAL_CENTER" \
  --half-span "$NANOCAVITY_LOCAL_HALFSPAN" \
  --sample-points 29 \
  --output-dir outputs_96gb_fine_multiprobe_local \
  2>&1 | tee outputs_96gb_machine/fine-multiprobe-local.log

If the fine-grid pole is narrower, increase sample density. Never infer a million-Q pole from a window whose sample spacing is wider than its linewidth.

5. Apply the spectral acceptance gate

Run the wide and local diagnostic model families, then the authoritative combined gate:

python multipole_validation.py \
  --data outputs_96gb_fine_multiprobe_wide/multiprobe_data.npz \
  --output-dir outputs_96gb_fine_multipole_wide

python local_upper_validation.py \
  --data outputs_96gb_fine_multiprobe_local/multiprobe_data.npz \
  --output-dir outputs_96gb_fine_upper_local

python combined_rational_validation.py \
  --wide-data outputs_96gb_fine_multiprobe_wide/multiprobe_data.npz \
  --local-data outputs_96gb_fine_multiprobe_local/multiprobe_data.npz \
  --reference-real REACQUIRED_POLE_REAL \
  --reference-imag REACQUIRED_POLE_IMAG \
  --output-dir outputs_96gb_fine_combined_validation

python -m json.tool outputs_96gb_fine_combined_validation/metrics.json

The combined metrics.json is authoritative. Stop if accepted is false. Widen, recenter, or densify sampling according to the failed gate; do not tune the topology to make a bad spectral fit look better.

6. Apply the combined transverse-PML gate

Each worker constructs an independent Maxwell model. Begin with one worker; only use two after the single-sweep RSS demonstrates sufficient headroom:

/usr/bin/time -v python multipole_pml_convergence.py \
  --warmstart outputs_refined_fine_workstation64_checkpoint/run_data.npz \
  --baseline-data outputs_96gb_fine_multiprobe_wide/multiprobe_data.npz \
  --local-data outputs_96gb_fine_multiprobe_local/multiprobe_data.npz \
  --validator combined \
  --reference-real ACCEPTED_POLE_REAL \
  --reference-imag ACCEPTED_POLE_IMAG \
  --workers 1 \
  --output-dir outputs_96gb_fine_combined_pml \
  2>&1 | tee outputs_96gb_machine/fine-combined-pml.log

python -m json.tool outputs_96gb_fine_combined_pml/metrics.json

This recomputes six PML perturbations for both frequency windows and may take a long time. Stop unless multipole_pml_gate_passed and every_case_passed_spectral_gate are both true. A failed case must be diagnosed as absorber sensitivity, inadequate windowing, or mode switching before optimization.

7. Take exactly one atom-coupling topology step

Only after Sections 5 and 6 pass:

For the currently qualified shifted checkpoint, use the one-step controller documented in UNATTENDED_RUNBOOK.md. The legacy scripts/run_fine_cooperativity_step.sh is retained only to reproduce the earlier common-ascent result. For a different grid or topology, first rebuild its checkpoint-matched wide, local, confirmatory, and PML inputs and update the runner rather than reusing these filenames.

Read the generated report and metrics. Confirm:

  • the step was accepted by the rebuilt combined validator;
  • normalized fixed-atom LDOS, moving-pole LDOS, and Q all increased according to independent direct and rational estimates, not only their differentiable surrogates;
  • the target pole remains the upper member of the same cluster;
  • air-strip maximum density is zero; and
  • reflection symmetry remains at roundoff.

Then follow the adaptive cadence in UNATTENDED_RUNBOOK.md. Reacquire the combined spectral gate on every new checkpoint; run the unchanged combined PML gate whenever scheduled or triggered. Keep each spectrally accepted step in a new directory and label only fully tested milestones PML converged.

8. What to report back

For each completed phase preserve or report:

  • output directory/archive;
  • config.json, metrics.json, and relevant .npz data;
  • maximum resident set size and elapsed time;
  • broad-search candidate frequencies;
  • combined pole/Q and every gate boolean;
  • PML case table and aggregate gate;
  • maximum air-strip density; and
  • any code changes plus regression results.

The first decisive report is the fine-grid combined spectral result. The second is the combined PML result. Only those two results determine whether optimization should resume.