|
MFC
Exascale flow solver
|
Contains module m_amr_transfer. More...
Go to the source code of this file.
Modules | |
| module | m_amr_transfer |
| Prolongation, restriction and Berger-Colella reflux, including the restrict/reflux/freg waves. | |
Functions/Subroutines | |
| pure logical function | m_amr_transfer::f_amr_reflux_participates (r) |
| True iff rank r refluxes some face of the current block (its unclipped subdomain); the participation map and both exchange paths gate freg receives on it. | |
| pure subroutine | m_amr_transfer::s_amr_reflux_faces_for (r, s_lo, s_hi) |
| Faces of the current block that rank r applies (seam-clipped); sender and receiver derive the identical set from replicated data. | |
| impure subroutine, public | m_amr_transfer::s_amr_reflux_faces_wave () |
| Per-stage level-1 reflux-face exchange as one wave: faces are packed on the owner's device into one pool, cross as one message per peer and unpack on the device. Both ends list a peer's faces by ascending block, dim, low before high, from the same predicates, so layouts agree with no handshake. Nothing may post into freg before s_amr_reg_prepare (the apply can reallocate the registers). | |
| impure subroutine | m_amr_transfer::s_amr_face_xfers (w, peer, k, f_lo, f_hi) |
| Append block k's faces f_lo/f_hi with peer as transfers (bl = dim, side 1 low / 2 high, register slot). | |
| impure subroutine | m_amr_transfer::s_amr_freg_pack (w, pack) |
| Copy a side's freg faces into (pack) or out of the device pool, one kernel per face direction. CCE 19 packs garbage when one procedure holds several of these loop nests (as in s_amr_apply_reflux), so each direction has its own procedure. | |
| impure subroutine | m_amr_transfer::s_amr_freg_pack_d1 (w, pack) |
| s_amr_freg_pack for the faces normal to direction 1, both sides in one kernel. | |
| impure subroutine | m_amr_transfer::s_amr_freg_pack_d2 (w, pack) |
| s_amr_freg_pack for the faces normal to direction 2, both sides in one kernel. | |
| impure subroutine | m_amr_transfer::s_amr_freg_pack_d3 (w, pack) |
| s_amr_freg_pack for the faces normal to direction 3, both sides in one kernel. | |
| impure subroutine, public | m_amr_transfer::s_amr_freg_wave () |
| Split-ownership level>=2 freg exchange as one wave, run once before the reflux fold; zero-copy like the faces wave, on tag band 1 (faces use band 0). | |
| impure subroutine | m_amr_transfer::s_amr_freg_faces_req (dir, peer, f_lo, f_hi) |
| Post one zero-copy request per flagged face register of the current slot (block k): dir 1 sends to / 2 receives from peer, in (direction, face) order. | |
| impure subroutine | m_amr_transfer::s_amr_freg_faces_update (to_device, f_lo, f_hi) |
| Move the flagged face registers of the current slot to the device (to_device) or to the host. | |
| impure subroutine | m_amr_transfer::s_amr_freg_faces_poison (f_lo, f_hi) |
| Debug builds: NaN-poison the unflagged face registers of the current slot before a receive lands, so a stray read of an unshipped face NaNs. | |
| impure subroutine, public | m_amr_transfer::s_set_amr_fine_geometry (lo, hi) |
| Set the fine level's geometry (region, intersection, extents, bounds, coordinates) for the box lo:hi; arrays are preallocated, this updates metadata and coords. Collective. Also refreshes the amr_xchg_coarse_ghosts flag. A level-l block's fine extent is amr_ref_ratio**l * (coarse-region width) - 1, not amr_ref_ratio*width. | |
| impure subroutine | m_amr_transfer::s_amr_check_block_geometry (k, pblk, fext) |
| Abort unless block k's fine extent is amr_ref_ratio**level*(region width) - 1 and, above level 1, its region lies inside parent pblk. | |
| impure subroutine, public | m_amr_transfer::s_amr_reduce_xchg_flag () |
| Close a geometry scan: one allreduce of the accumulated flag, then reset. Call after every s_set_amr_fine_geometry loop; the flag is read by s_amr_exchange_coarse_cons_halo. | |
| impure subroutine | m_amr_transfer::s_amr_build_cfl_map () |
| amr_cfl_rf from the block table: amr_ref_ratio**level of the deepest block over each of this rank's coarse cells. | |
| impure subroutine | m_amr_transfer::s_prolong_one_var (qc, loc, ivar) |
| Conservative prolongation of one variable from the patch qc into the fine interior: minmod-limited linear, or the ghost fill's limited quartic with amr_prolong = 2. | |
| impure subroutine | m_amr_transfer::s_prolong_guard (qc, loc) |
| Quartic safety net (amr_prolong = 2): a parent whose children leave the admissible state (non-positive density or internal energy) is re-prolonged minmod-limited linear in every variable. | |
| impure subroutine, public | m_amr_transfer::s_interpolate_coarse_to_fine () |
| Conservative prolongation: fill amr_fine interior from coarse (minmod-limited linear, or quartic with amr_prolong = 2); symmetric child offsets make the children average to the coarse value. Multi-fluid volume fractions take the sum- preserving closure path. | |
| impure subroutine | m_amr_transfer::s_prolong_alphas_closure (qc, loc) |
| Sum-preserving volume-fraction prolongation (num_fluids > 1): fluids advbeg..advend-1 are interpolated with minmod slopes under a shared per-cell limiter switch and clamped to [0,1]; the last fluid closes alpha_n = 1 - sum(others). For >2 fluids residual closure undershoot is handled by mpp_lim. Index mapping as s_prolong_one_var. | |
| impure subroutine | m_amr_transfer::s_prolong_species_closure (qc, loc) |
| Species prolongation closure (chemistry): each rho*Y_k is minmod-prolonged and clamped non-negative, then rescaled so sum_k(rho*Y_k) equals the prolonged continuity density. Index mapping as s_prolong_one_var; cont is prolonged before this runs. | |
| impure subroutine, public | m_amr_transfer::s_populate_amr_fine (q_cons_base) |
| Disblock prolongation. Guard: no-op unless amr. | |
| impure subroutine | m_amr_transfer::s_amr_build_static_multilevel () |
| Build the static multi-level hierarchy (amr_regrid_int = 0): nest one level-2 block inside level-1 block 1 by a fixed geometric inset, prolong the parent state into it and keep it persistent. | |
| impure subroutine, public | m_amr_transfer::s_amr_sync_level (lev, dt_reflux) |
| Sync level lev >= 2 into its parents: restrict its blocks into the covered parent cells, then state-reflux each into its parent (step-integral fluxes scaled by dt_reflux; 1 when registers carry dt). | |
| impure subroutine, public | m_amr_transfer::s_amr_restrict_wave (coarse_tgt, dt_reflux) |
| Lock-step restrict fold as plan-based waves : per level, finest first, the split-ownership child->parent folds (s_amr_restrict_parent_wave) then the per-box reflux-to-parent applies (freg already exchanged by s_amr_freg_wave); then the level-1 -> L0 scatter (s_amr_restrict_l1_wave). Both sides derive identical transfer lists from replicated metadata; one rank has no peers. Within a level the covered targets are disjoint. | |
| impure subroutine, public | m_amr_transfer::s_amr_restrict_fold (coarse_tgt) |
| Restrict every level into its parent, finest first, with no reflux: for a cell-local update of the fine blocks made after the step's fold (phase-change relaxation). | |
| impure subroutine | m_amr_transfer::s_amr_restrict_parent_wave (lev) |
| One per-level wave of the split-ownership level>=2 child->parent restrict folds. Co-located folds run inline on the child owner; cross-rank folds ship the parent-frame covered box, one message per (child-owner, parent-owner) peer. Both sides walk the same block list ascending with per-peer running offsets. | |
| impure subroutine | m_amr_transfer::s_amr_restrict_l1_wave (coarse_tgt) |
| Level-1 -> L0 covered-cell scatter as one wave: every owned level-1 block's covered slabs ship to each listed coarse owner in one message per peer; the owner-local overwrite stays per block during the pack walk. The receiver plan (my interior x region(k) over level-1 blocks I do not own) equals the sender's list membership. | |
| impure subroutine | m_amr_transfer::s_amr_restrict_l1_batched (coarse_tgt, milo, mihi) |
| Rest of the level-1 restrict wave (receives posted): owned-block overwrite, send packs and received unpacks as one launch each over slab rows. | |
| impure subroutine | m_amr_transfer::s_amr_sibling_face_weights (kb, pblk, w_lo, w_hi) |
| Sibling-seam face weights for level>=2 block kb under parent pblk: 0 on a face shared with a same-parent sibling tile (fine-fine; refluxing there double-writes), 1 otherwise. Uses replicated metadata only, and both the reflux apply and the freg wire-skip call it, so shipped and consumed faces agree. | |
| impure subroutine, public | m_amr_transfer::s_amr_reflux_to_parent (dt_reflux) |
| Multi-level reflux: apply the Berger-Colella C/F flux correction from the current level>=2 block into its parent block's cells just outside the block footprint, in the parent-fine frame (mirror of the L0 s_amr_apply_reflux targeted at the parent; "the coarse" is level l-1). State form: q_parent(outside) += dt*(F_coarse - Fbar_fine)/dxf on the low face and += dt*(Fbar_fine - F_coarse)/dxf on the high face, where Fbar_fine is the child-averaged fine register. creg/freg key off this block's slot. Per-face parent-fine dx (stretched-grid safe). | |
| impure subroutine | m_amr_transfer::s_amr_restrict_device_st (ctloc, loc, bl, bh, rlo, rr, o1, o2, o3) |
| Device-native restriction of the fine block (flat store) to coarse averages over the covered cells [bl:bh] (global; block origin rlo, ratio rr); child-sum order ddk, ddj, ddi, /nchild. One body for two destinations so results match bit-for-bit: a parent block in the flat store (_st, local origin o) or the wire buffer buf (_wire; ci fastest, then cj, ck, i; wp). Never push the whole coarse array to the device. | |
| impure subroutine | m_amr_transfer::s_amr_restrict_device_wire (buf, loc, bl, bh, rlo, rr) |
Contains module m_amr_transfer.
Definition in file m_amr_transfer.fpp.f90.