# From-scratch component-aware Yee atom-strip campaign Campaign: `sin100_sio2clad_xyz_y_atom_strip_16x2_from_scratch_yee_temporal_qv_beta_v1` ## Purpose and convergence-driven stage sequence This campaign repeats the successful SiN/SiO2 atom-strip discovery sequence with the component-aware Yee material operator from the first solve. It resumes the immutable accepted update-50 temporal checkpoint without importing any later Q/V curvature or rejected trial. Each stage hands off only after its active objective reaches a stage-local plateau: 1. continue the unchanged absolute temporal-trace Adam optimization until its temporal objective plateaus; 2. optimize Q/V with only the unfiltered 50 nm density pixels until the pixel-only Q/V objective plateaus; 3. enable both SiN and SiO2 thickness coordinates and optimize Q/V at each fixed projection beta until that fixed-beta Q/V plateau is reached, then raise beta by one; 4. once binary readiness and a Q/V plateau coincide, replay the density as a smooth spline level-set boundary and continue optimizing Q/V over a topology-free absolute phase field and both thicknesses indefinitely. Rejected trust-region proposals never advance an update counter. No total iteration number causes an objective or parameterization change. For stage-local objective values `J_k`, a least-squares trend is fit over two optimizer-memory times: 20 samples for Stage A (twice the Adam first-moment time constant) and 20 accepted samples later (twice the ten-pair L-BFGS memory). With fitted slope `s` and slope standard error `sigma_s`, the stage is settled when ```text s <= max(2 sigma_s, 64 eps_float32 max(1, max |J_k|)). ``` The second term is the differentiated solver's numerical resolution. In a trust-region stage, two complete trust-floor curvature restarts without an intervening accepted improvement are independent plateau evidence. ## Geometry and source - The unfolded design region is 16 µm along x and 2 µm along y on a 50 nm grid. Full x/y/z symmetry leaves one octant. - A y-oriented atom is at the origin inside an immutable vacuum strip `|x| < 0.5 µm` spanning the full y width and complete material stack. - Trainable pixels outside the strip start at density 0.5. The initial SiN core is 100 nm thick and has index 2.0. Symmetric SiO2 cladding has index 1.45 and initially extends 300 nm per side. - The fundamental y-polarized feedthrough mode is injected as the coherent symmetry-reduced bright input. A fixed 15 THz Gaussian pulse excites nascent low-Q poles and remains broad enough for shifted poles. - The non-PML z span is 3.5 µm, followed by ten 50 nm PML cells on each full side. ## Stationary component-aware material map Let `rho(x,y)` be the topology coordinate, `F_c(z)` the SiN-core fill, and `F_s(z)` the complete core-plus-cladding fill. Inside the core, ```text epsilon_c(x,y) = epsilon_SiO2 + rho(x,y) (epsilon_SiN - epsilon_SiO2). ``` Outside the hard vacuum strip, the tangential cell precursor is ```text epsilon_t = epsilon_air + F_s (epsilon_SiO2 - epsilon_air) + F_c (epsilon_c - epsilon_SiO2), ``` and the z-normal precursor is the layered harmonic mixture ```text 1/epsilon_n = (1-F_s)/epsilon_air + (F_s-F_c)/epsilon_SiO2 + F_c/epsilon_c. ``` The diagonal cell averages are composed first and registered once onto the staggered Ex/Ey/Ez dual faces. Forward and adjoint solves use the same differentiable material operator. Spatial grid, subpixel rule, source waveform, and monitor definitions never change inside a gradient/proposal comparison. ## Stage A — absolute temporal trace until convergence The exact discrete target is the y-field at the atom produced by the injected pulse driving one ideal pole with Q=10,000, V=0.1 µm³, beta=1, perfect dipole alignment, and the correct causal prompt plus resonant response: ```text J_time = - sum_t ||E_sim(t)-E_target(t)||² / sum_t |E_target,y(t)|². ``` Raw Adam with learning rate 0.01 changes every unfiltered density pixel. Both thickness coordinates remain nominal. The temporal tape is fixed at 2 ps and the source-off fit begins at 0.25 ps. The handoff occurs only when the temporal trend passes the stage-local plateau test and an independent field/energy pole fit is trustworthy with harmonic Q at least 100. The update-50 Adam moments are preserved when resuming, so this remains one continuous trajectory. ## Stage B — pixel-only Q/V until convergence At handoff, the broad pulse is centered on the accepted pole. Field amplitude and total energy are independently fit, and ```text Q_h = 2 Q_field Q_energy / (Q_field + Q_energy) V_n = V_atom / (lambda_pole/n_air)^3 J_QV = log(Q_h) - log(V_n). ``` The atom-oriented volume is the geometric mean of three source-free window estimates for the same fitted pole. Only density pixels are trainable; SiN and SiO2 thicknesses are exactly frozen. Adaptive trust L-BFGS-B forms one direction from one fresh forward/adjoint evaluation. The forward proposal is accepted only for a trustworthy strict Q/V increase. Rejected proposals shrink the radius and can reuse the cached direction; well-predicted accepted boundary-reaching moves enlarge it. The next stage starts only after the accepted Q/V trend plateaus or the trust-region plateau evidence is exhausted. ## Lifetime-conditioned Q/V tapes For fitted frequency `f` and harmonic Q, the amplitude lifetime is ```text tau_a = Q / (pi f). ``` The Q/V fit does not start at the first nominally source-free sample. It starts after the analytic source-clear time plus three maximum port flight times, rounded upward to the next 0.05 ps. For this geometry that is 0.35 ps. This separates the pole fit from the short prompt/energy redistribution transient without paying for a long ringdown. Each Q/V epoch then uses three equal windows. A window is the power-of-two multiple of 0.05 ps immediately above one eighth of `tau_a`. The three windows therefore cover between three eighths and three quarters of an amplitude lifetime, except at the 0.05 ps low-Q floor. The tape length can only grow as Q increases; a noisy lower-Q update cannot shorten it or cause recompilation thrash. The tape is immutable while differentiating and replaying a proposal. It may change only after an accepted checkpoint, at which point stale curvature and cached fields are cleared. The compact tape must also pass a precision gate: field and energy Q agree within 10%, both normalized fit residuals are at most 0.02, and the three atom-volume estimates span at most 10% of their mean. Both final field-power and energy fractions must remain above 32 times float32 epsilon. If a finite, still-observable incumbent misses the precision gate, the next epoch doubles the window and clears stale curvature. This recovery can grow as far as one measured amplitude lifetime; disagreement beyond that is treated as a pole or model problem rather than hidden by ever-longer simulations. The policy was selected from a frozen-prefix replay at accepted update 472 (projected trial 336, beta 7, long-tape harmonic Q 829.21). With the old 0.25 ps fit start, a 0.85 ps run underestimated Q/V by about 5.7%, while a 1.45 ps run was within 1.5% of the 2.65 ps result. Moving the fit start to 0.35 ps removed the early energy-fit bias: a 0.65 ps prefix (three 0.10 ps windows) gave Q=824.31 and normalized V=3.2106, while the same trace fit over long late windows approached Q=833.92 and V=3.1938. The compact policy is thus about 1.7% low in Q/V on this checkpoint while using about one quarter of the former tape. The first proposal after handoff, after a tape-length change, and after a trust floor reset is capped at coordinate radius 1e-4. Its normal forward replay is also a finite-step directional-derivative audit: predicted and measured Q/V changes must have the same positive sign before the audit is marked verified. ## Continuous pole branch and zero-extra-solve prediction The broadband source carrier and the pole-fit center are separate state. The source is recentered only when the power of its actual finite sampled waveform at the accepted pole falls below 95% of the power at the physical source carrier. This ratio is not normalized at the pole predictor: doing that would make a detuned source appear artificially well centered. Recentring happens only at an accepted checkpoint, invalidates the cached proposal and prevents a new L-BFGS secant from spanning the waveform change. The pole-fit center advances after every accepted geometry replay, so a broad pulse cannot permit the translated-window estimator to drift across an alias. The source carrier and visualization frequency are also separate. The `xy`, `xz`, and outgoing-mode phasors demodulate at the accepted pole while the broadband source remains fixed. The monitor frequency is frozen throughout an incumbent-adjoint/proposal comparison and refreshed only at an accepted checkpoint when its late-window phase drift would exceed one quarter cycle. This changes only diagnostics: Q, atom-oriented V, and Q/V continue to come from the atom time trace and total-energy trace. The dashboard records source, monitor, and fitted-pole wavelengths independently, and marks saved field arrays stale during the short checkpoint restart before their replacement. For an isolated Maxwell eigenvalue, analytic eigenvalue continuation gives `f(p+dp) = f(p) + O(||dp||)`. The controller uses the last two accepted pole frequencies as a first-order secant prediction, ```text f_predict = f_k + clip(f_k-f_(k-1), -Delta_f/4, +Delta_f/4), Delta_f = 1 / T_window. ``` This scalar calculation uses measurements already produced by accepted FDTD solves and costs no additional FDTD run. The prediction is a stop-gradient controller decision and is frozen for the incumbent adjoint and every trust- radius replay. A fitted candidate must remain within `0.40 Delta_f` of that prediction. If it does not, the existing time trace—not a new FDTD run—is Fourier searched for the dominant branch, curvature is cleared, and the new branch is frozen before differentiation resumes. This guard also applies to beta changes and spline conversion. The physical resonance may shift under binarization, but phase wrapping by an integer multiple of `Delta_f` cannot silently change the atom-field amplitude, mode volume, or Purcell objective. ## Stage C — joint thickness and plateau-paced beta continuation After the pixel-only Q/V plateau, the inverse centered-tanh map at beta=1 reproduces the accepted physical density to numerical precision: ```text rho(p; beta) = [tanh(beta/2) + tanh(beta(p-1/2))] / [2 tanh(beta/2)]. ``` There is no conic filter and no separate binarity term. The sole objective remains `J_QV`. Each fixed-beta problem continues until its own Q/V plateau; then beta rises by 1. Rejected replays do not directly advance beta. This continues toward beta=32. L-BFGS pairs are retained across a beta increment, but no secant pair is formed across the small change in parameterization. The fixed-beta stages also use a practical diminishing-returns test. One full ten-update L-BFGS memory must be observed. If that memory produces less than 1% cumulative Q/V gain, the per-update gains are not accelerating, and the adaptive trust radius is no longer expanding, the fixed-beta stage has yielded enough of its accessible performance to advance. This is deliberately earlier than waiting for a numerically zero gradient, while an expanding step size or an improving marginal return prevents premature continuation. One exact incumbent forward/adjoint evaluation is cached while the geometry, projection beta, predicted pole branch, and Q/V tape remain unchanged. A rejected trust proposal therefore shrinks the radius and performs only another forward candidate replay. Acceptance, a beta increment, a tape-length change, a source recenter, or a trust-floor restart invalidates the cache. This changes neither the proposal sequence nor the differentiated objective; it only removes repeated adjoints of the identical incumbent. Both stack thicknesses become trainable at this point. Their search direction uses a fixed physical metric: a normalized thickness coordinate is scaled by the number of 50 nm cells it spans. This keeps global thickness motion commensurate with local pixel motion while leaving the objective and exact adjoint gradient unchanged. The parameterization always enforces `t_SiO2 >= t_SiN + 0.050 µm`. ## Stage D — smooth boundary after projected Q/V converges Boundary conversion is eligible when the current fixed-beta Q/V objective has plateaued, beta is at least 16, the bulk binarity score is at least 0.95, and at most 10% of trainable pixels remain gray. An absolute tensor-product cubic spline phase field is fit directly to the projected cell fractions without an FDTD solve. Its zero contour is smooth and is integrated by subcell quadrature: bulk cells remain exactly binary and only cut-cell fill fractions are gray. The replay must be trustworthy and must preserve the fixed Q/V floor recorded at the start of Stage C. It need not exceed the last pixel value before boundary optimization has had a chance to move the fitted surface. The phase field has no immutable reference contour, sign constraint, or component-lineage gate. Every spline control may cross zero. Two approaching material islands can therefore fuse; a shrinking island or hole can vanish; and a new component can form when the Q/V gradient supports it. These topology changes are recorded as component and enclosed-void counts, but acceptance is decided by the same trusted pole-branch and Q/V replay—not by preserving the pixel raster's topology. The topology-free spline controls and both stack thicknesses continue with adaptive-trust L-BFGS on the same `J_QV` objective and the same lifetime-conditioned measurement epochs. ## Stage E — rollback-safe pure-Q boundary fork The Stage-D trajectory reached update 1080 at harmonic Q=1217.36, normalized atom volume=3.10912, and Q/V=391.544. Its exact artifact is retained before an objective-only fork. At that state the trust radius had contracted to 3.56e-6, and the accumulated Q/V L-BFGS direction was nearly tangent to the current Q/V gradient. Two fresh adjoint directions and forward-only replays at the same pole separated objective choice from stale curvature. At coordinate radius 1e-4, the memory-free Q/V direction changed Q by +0.029, normalized V by -0.00090, and Q/V by +0.122. The memory-free pure-Q direction changed Q by +0.256, normalized V by +0.00024, and Q/V by +0.053. Thus pure Q was not the larger raw gradient—it had 0.374 times the Q/V gradient norm—but it supplied about nine times the immediate Q increase while retaining positive local Q/V progress. The rollback-safe pure-Q boundary fork optimized ```text J_Q = log(2 Q_field Q_energy / (Q_field + Q_energy)). ``` Geometry, stack coordinates, topology-free spline chart, pole predictor, time-window precision gates, and forward trust replay are unchanged. Because the scalar objective changed, all objective-specific L-BFGS pairs and the cached direction are cleared, and the adaptive trust radius restarts at 1e-4. Q/V and atom mode volume remain measured on every trial but are not acceptance constraints. The exact pre-fork checkpoint is recorded in the artifact as `pure_q_rollback_snapshot`, so the Q/V branch can be restored without reconstructing the geometry. Six accepted pure-Q updates raised Q from 1217.36 to 1222.10, but normalized V rose from 3.10912 to 3.12093 and Q/V ended at 391.584 after briefly reaching 391.847. The experiment therefore isolated a useful Q direction but did not outperform the current Q/V gradient on the metric this campaign is intended to maximize. The live branch returns to the exact update-1080 Q/V checkpoint. ## Stage F — safeguarded adaptive Q/V curvature The optimizer still uses L-BFGS when its curvature is informative, but it no longer assumes every positive secant is numerically useful. A pair `(s,y)` is retained only when its normalized curvature satisfies ```text s.y / (||s|| ||y||) > sqrt(float32 epsilon). ``` For every fresh adjoint, the full limited-memory proposal and the current Q/V gradient are mapped through the same component metric, active-bound projection, and peak-coordinate normalization used by the actual trust step. Their linear predictions are ```text A_LBFGS = grad(J_QV).(d_LBFGS / ||d_LBFGS||_infinity) A_grad = grad(J_QV).(D grad(J_QV) / ||D grad(J_QV)||_infinity). ``` If the peak-normalized predicted ascent ratio `A_LBFGS/A_grad` is below the ordinary trust acceptance ratio, the oldest curvature pairs are removed until a healthy suffix remains. If no suffix is healthy, the proposal is simply the current Q/V gradient in the same diagonal component metric. This test is architecture-independent and costs only host vector operations over at most ten stored pairs; it requires no extra FDTD solve. The forward replay remains authoritative. If a trustworthy replay shows that a curvature-accelerated proposal actually decreases Q/V, or repeatedly fails the trust agreement test, the stored curvature is cleared immediately and the same already-computed adjoint is reused for a current-gradient retry at the reduced radius. Accepted steps then rebuild curvature normally. Thus the controller can accelerate along reliable curvature, cannot collapse indefinitely along a known-bad direction, and does not alter the Q/V objective, Maxwell solve, pole tracking, or differentiated gradient. ## Automatic fast gradient backend The former fixed 48-checkpoint reverse pass was much more expensive than the forward solve because reverse-mode differentiation repeatedly reconstructed the long time trajectory. It was not merely a second Maxwell solve with source and detector exchanged. The optimizer now asks the solver to choose one of two exact gradient implementations after every scene/tape epoch is placed: 1. for a nondispersive scene, use reversible FDTD when the exact PML-interface history fits a fixed fraction of accelerator memory; 2. otherwise, use a lean checkpointed reverse pass whose checkpoint count is selected from the time-step count, evolving-state size, and memory capacity. For `T` time steps, let `b_pml` be the bytes in all electric and magnetic PML interface samples at one step. The reversible choice requires ```text T b_pml <= 0.42 M_allocator. ``` The checkpoint fallback stores only the time-varying fields, detector state, and recorder state. Material arrays, Yee coefficients, and other immutable scene data are closure-captured once rather than copied into every checkpoint. If one lean state occupies `b_state`, its count is ```text C = min(C_treeverse(T), floor(0.25 M_device / b_state), T), ``` where `C_treeverse(T)` is the smallest count in the optimal online treeverse regime. No density, field, objective, Q, V, gradient, cavity type, or hand-set geometry constant enters either decision. The plan is therefore immutable while an optimizer epoch moves the dielectric and automatically re-evaluates when a different geometry or time window is placed. Dispersive scenes and devices whose memory capacity cannot be queried fail safely to checkpointing. The production objective calls the public solver dispatcher, and the complete resolved plan is saved in the campaign artifact and dashboard provenance. On the frozen accepted update-574 geometry and its 0.95 ps/9,966-step tape, the PML history is 3.840099120 GB and the automatic policy selects reversible FDTD. Against the exact cached production checkpointed result, it gave: ```text objective relative error 2.21e-7 full-gradient cosine 0.999999999999 full-gradient relative L2 2.07e-6 active-coordinate sign match 100% steady value+gradient time 5.42 s (formerly 28.48 s; 5.25x faster) ``` Adding 15 interior field resets did not improve the gradient and reduced the speedup to 2.24x, so the lossless campaign uses zero resets. The same planner selects reversible for the measured 0.65 ps tape, while a 2.65 ps tape would require 10.712 GB of PML history and therefore switches automatically to lean checkpointing (235 checkpoints on the measured 12 GB device). The in-place migration retains the accepted geometry, projection beta, stage, pole/tape state, adaptive trust radius, counters, and completed L-BFGS pairs. Only the cached old-backend direction and pending cross-backend secant are invalidated. The first new proposal uses a fresh gradient and must pass the ordinary forward trust replay/directional check before normal continuation. An immutable byte-identical pre-migration checkpoint provides rollback. ## Persistence and dashboard The worker, watchdog, and dashboard publisher run indefinitely. State is written atomically, and an immutable full restart snapshot is saved every 50 accepted parameter updates. The dashboard reports stage/update counters, adaptive tape length, Q, normalized V, Q/V, Purcell estimate, pole wavelength, field/energy fit agreement and observability, stack thicknesses, geometry and difference maps, fields, temporal response, binarity, and trust behavior. The stale-checkpoint watchdog does not interrupt a long high-Q FDTD step while the GPU is actively computing.