CMPS Theory & User Reference Manual › NeuralFlow Recurrent Correction, Rollout and Training

NeuralFlow Recurrent Correction, Rollout and Training

Training target. NeuralFlow learns a recurrent solver update from intermediate CMPS states. A library of completed CFD fields is not required as the primary label source. Each rollout generates new predicted states, CMPS evaluates their physical imbalance, and the resulting residual/Jacobian feedback trains the shared GNO correction operator.

Learning the correction process

For one NeuralFlow pass,

\[\mathbf q^{(r+1)}=\mathbf q^{(r)}+\Delta\mathbf q_\theta^{(r)}.\]

The same local operator is reused at every recurrent pass. Training therefore optimizes the behavior of a repeated nonlinear update, rather than only the accuracy of a single final-field prediction.

Physics and network gradients

The training loop combines two distinct differentiation mechanisms:

\[\underbrace{\mathbf q\xrightarrow{\text{CMPS finite-volume physics}}\mathbf B,\ \mathbf J_B}_{\text{forward differentiated solver}}\qquad\text{and}\qquad\underbrace{\theta\xrightarrow{\text{recurrent GNO}}\mathbf q^{(R)}}_{\text{reverse differentiated network}}.\]

CMPS supplies the state-space physical gradient from its residual/Jacobian machinery. Reverse-mode differentiation of the GNO rollout then converts that gradient into parameter derivatives. This is fundamentally different from supervised regression against stored completed solutions.

Shared recurrent graph stage

A forward application first encodes node inputs to a latent state and then applies one shared graph-update block repeatedly. The parameters are reused; the formulation does not contain 60 independently parameterized graph layers.

\[\mathbf h^{(\ell+1)}=\mathbf h^{(\ell)}+\alpha_h\,\Delta\mathbf h_\theta^{(\ell)},\qquad \alpha_h=0.5\ \text{by default}.\]

With the default 60 repeated graph stages and 20 learned rollout steps, one full training rollout can contain

\[N_{stage}=60\times20=1200\]

applications of the shared graph-update operator before the final physical objective is differentiated.

Near-identity correction initialization

The correction head is explicitly initialized near zero so the first network behaves close to the identity map rather than imposing a large arbitrary flow-field change:

\[\mathbf W_{\Delta}\leftarrow10^{-3}\mathbf W_{\Delta},\qquad \mathbf b_{\Delta}=\mathbf 0.\]

Primitive correction map

Velocity is corrected additively, while pressure and temperature are corrected in logarithmic variables:

\[\mathbf u^{+}=\mathbf u+\omega_u\,\Delta\mathbf u_\theta,\]

\[p^{+}=p\exp(\omega_p\,\Delta\ell_{p,\theta}),\qquad T^{+}=T\exp(\omega_T\,\Delta\ell_{T,\theta}).\]

The thermodynamic log-increments are bounded before relaxation,

\[\Delta\ell_p=\Delta\ell_{p,max}\tanh(\widehat{\Delta\ell}_p),\qquad \Delta\ell_T=\Delta\ell_{T,max}\tanh(\widehat{\Delta\ell}_T).\]

With the current defaults \(\Delta\ell_{p,max}=0.2\), \(\Delta\ell_{T,max}=0.08\), \(\omega_p=\omega_T=0.05\), the effective per-pass limits are

\[|\Delta\ln p|\le0.01,\qquad |\Delta\ln T|\le0.004.\]

The velocity branch has a relaxation authority but no corresponding independent tanh magnitude cap in the current decoder.

Optional learned relaxation head

The learned relaxation/gating head is optional and disabled by default. In the default configuration the configured maximum correction authorities are applied directly. When enabled, the head predicts bounded group-wise modulation for velocity, pressure and temperature.

Optional residual teacher gate

NeuralFlow also provides an optional residual-based teacher gate that can reduce or emphasize correction authority according to local and global residual difficulty. It is disabled by default. When enabled, the gate influences how strongly velocity, pressure and temperature corrections are applied during training.

\[\chi_i=\frac{r_i}{\max(r_{rms},\epsilon)},\]

followed by bounded gate construction and epoch-dependent warmup/ramp blending. The gate is an auxiliary training mechanism; it does not change the definition of the CMPS physical residual.

Gate imitation loss

When teacher-gate imitation is active, an auxiliary loss penalizes the difference between predicted and teacher correction gates. It is mixed with the physical objective according to the configured training schedule.

Differentiable recurrent rollout

Starting from a training anchor \(\mathbf q^{(0)}\),

\[\mathbf q^{(r+1)}=\Phi_\theta(\mathbf q^{(r)}),\qquad r=0,\ldots,R-1.\]

After each learned correction the primitive state can be committed back to the CMPS cells and boundary data refreshed. The final—and, when required by auxiliary training logic, intermediate—states are evaluated by the solver physics. The training graph therefore spans the repeated correction chain.

Training-anchor semantics

The anchor is a physical CMPS state from which a rollout starts. Frozen physical metrics, boundary-derived scales and other case-dependent training quantities must remain consistent with the accepted anchor. When the anchor is updated, metric data that depend on it must be rebuilt according to their validity rules.

Episode training

Episode mode advances the current solver state across training iterations so the GNO sees a sequence of intermediate nonlinear states rather than repeatedly training from one identical field. This supports the intended learned-iterative-solver interpretation: the network is exposed to the trajectory it creates.

Optional completed-solution auxiliary objective

A converged-solution objective exists as an optional auxiliary term and is disabled by default. Its presence does not turn NeuralFlow into a supervised completed-field surrogate: the primary physical learning signal remains CMPS residual/Jacobian feedback generated at the predicted states.

Optimizer and parameter update

The current default optimizer is adaptive decoupled-weight-decay optimization with

\[\eta=10^{-3},\qquad \lambda_{wd}=10^{-6},\qquad \beta_1=0.9,\qquad \beta_2=0.999,\qquad \epsilon=10^{-8}.\]

Global gradient clipping is enabled by default. For parameter gradient \(\mathbf g_\theta\) and \(g_{max}=1\),

\[\mathbf g_\theta\leftarrow\mathbf g_\theta\min\!\left(1,\frac{g_{max}}{\|\mathbf g_\theta\|_2+\epsilon}\right).\]

If a non-finite parameter gradient is detected, the optimizer step is skipped. Learning-rate scheduler options are stored in configuration, but the current runner does not currently apply the configured Step or exponential scheduler during training.

Training memory and execution

Optional activation checkpointing groups repeated shared graph stages and recomputes their activations during back-propagation to reduce memory use. It changes memory/runtime cost, not the learned operator itself, and does not truncate the recurrent gradient chain. Training currently runs through the single training process, while the GNO tensor operations can use CPU or NVIDIA CUDA and single- or double-precision arithmetic.