|
MFC
Exascale flow solver
|
Parent/child data exchange: gather plans, pack/unpack and the fill waves. More...
Functions/Subroutines | |
| pure real(wp) function | f_amr_par_val (g1, g2, g3, i, qp, th) |
| Parent value for a level >= 2 ghost fill: the current state, or its blend with the substep-start backup when th < 1. | |
| impure subroutine | s_amr_pack_parent_box_device (qp, bl, bh, buf) |
| Sub-box variants of the parent-patch pack/copy for the ring-clipped parent-fill wave; bl:bh are patch-local, the buffer layout matches the full-patch kernels. | |
| impure subroutine | s_amr_copy_parent_box (qp, bl, bh) |
| impure subroutine | s_amr_pack_box_device (q_coarse, bl, bh, o1, o2, o3, buf) |
| Pack (pack = T) the global box [bl:bh] of q_coarse into the wire buffer buf, or unpack it into amr_cg at patch-local index g. | |
| impure subroutine | s_amr_unpack_box_device (bl, bh, off, buf) |
| impure subroutine | s_amr_shell_slabs (plo, phi, clo, chi, ns, sb, se, cells) |
| Ring clip: split the hollow shell of patch [plo:phi] minus open core [clo:chi] into at most 6 disjoint slabs in fixed order (x, y within core x, z within core x and y). Collapsed dims pass 0; the width-1 double cover is clamped. Both sides derive the list from replicated metadata. | |
| impure subroutine | s_amr_shell_clip (ns, sb, se, bl, bh, ms, tb, te, cells) |
| Intersect the shell slabs with box [bl:bh], keeping order; also returns the total cell count. | |
| impure subroutine | s_amr_poison_patch_device (w1, w2, w3) |
| Debug only: NaN-flood amr_cg before a clipped gather so any read of an unshipped cell NaNs the ghost fill. | |
| impure subroutine | s_amr_gather_own_rows (q_coarse, cg, o1, o2, o3) |
| Ring-clipped own-box copy on device: queue shell-slab/own-box intersections as slab rows and copy q_coarse into amr_cg in one fused kernel. | |
| impure subroutine | s_amr_fx_plan (w) |
| Fused-pack plan for transfers 1:nt: slab corner, extents, absolute pool payload offset and exclusive element prefix amr_fx_pre. Rows 8:11 are left to the parent-store pack site. | |
| pure subroutine | s_amr_fx_run (k, blk, nx, t0, t1) |
| Longest run t0:t1 of one box's recv transfers that is contiguous in the wire pool (unpacks in one launch). | |
| impure subroutine | s_amr_fx_pack_box (q_coarse, t0, t1, o1, o2, o3, pl, pre, buf) |
| Fused coarse-field pack: the wave's whole send list in one launch; inverts the linear index s_amr_pack_box_device writes, so wire bytes match the per-transfer packs. | |
| impure subroutine | s_amr_fx_pack_parent (t0, t1, pl, pre, buf) |
| Fused parent-store pack: as s_amr_fx_pack_box, with the source the flat store at parent slot pl(8,:) and child frame pl(9:11,:). | |
| impure subroutine | s_amr_fx_unpack (t0, t1, base, pl, pre, buf, cg) |
| Fused unpack of the contiguous transfer run t0:t1 into amr_cg in one launch; serves both packs via the per-transfer frame in plan rows 9:11. | |
| impure subroutine, public | s_amr_l1_fill_exchange (q_cons_coarse, full) |
| Level-1 fill wave, exchange phase: each level-1 box's padded coarse patch (region +/- amr_cpat_mar) is assembled on its owner from the coarse owners, one message per peer; both sides derive the transfers from replicated metadata. Slices are ring-clipped to the hollow shell unless full (init, regrid). Consume with s_amr_l1_fill_consume per owned box ascending. | |
| impure subroutine | s_amr_l1_slice_slabs (k, plo, phi, bl, bh, full, msl, tb, te) |
| A contributor's slice [bl, bh] of box k's patch as transfer slabs: the whole slice (full) or its hollow-shell intersection. | |
| impure subroutine | s_amr_fill_drain (k, off) |
| Drain block k's received transfers at the wave cursor into amr_cg (patch origin off): one fused unpack per contiguous run under amr_device_pack, else one unpack per transfer. | |
| impure subroutine, public | s_amr_l1_fill_consume (q_cons_coarse, k, full) |
| Level-1 fill wave, consume for owned box k: set the patch frame, copy the own slice, unpack k's transfers. Consume in ascending order: transfers are box-major, so k's are the next run at the cursor. | |
| impure subroutine, public | s_amr_fill_wave_done () |
| Every receive transfer of the open fill wave has been consumed. | |
| impure subroutine, public | s_amr_stage_fill_wave (q_cons_coarse) |
| Per-stage level-1 fill: the wave, then every owned level-1 box's consume and ghost fill at once. | |
| impure subroutine, public | s_amr_unpack_rows (buf, cg) |
| Unpack the queued slab rows (row 9: payload offset in buf) into the pooled amr_cg patches: a whole fill wave in one launch. | |
| impure subroutine, public | s_amr_restrict_rows_sf (coarse_tgt, o1, o2, o3) |
| Restriction over the queued slab rows (row 9 store column, 10:12 region lo, 13 wire payload offset): children average into the coarse field (sf, frame o) or into buf (wire layout of s_amr_unpack_rows). | |
| impure subroutine, public | s_amr_restrict_rows_wire (buf) |
| impure subroutine | s_amr_cg_reserve (np) |
| Hold np coarse patches in amr_cg stacked along dim 3 (patch p at p*(amr_cpat_hi(3) + 1)); grows only, contents do not survive a fill. | |
| impure subroutine | s_amr_l1_fill_batched (q_cons_coarse) |
| Level-1 consume and ghost fill for all owned boxes at once: patches pooled in amr_cg (s_amr_cg_reserve), one kernel each. | |
| impure subroutine, public | s_amr_l1_pool_gather (q_cons_coarse) |
| Consume the open level-1 fill wave into the pooled amr_cg: box k at dim-3 offset amr_cg_pz(k), amr_cg_nuse patches. | |
| impure subroutine, public | s_amr_l1_pool_fill () |
| Ghost-fill every owned level-1 box from its pooled patch (s_amr_l1_pool_gather's layout) in one kernel. | |
| pure subroutine | s_amr_patch_width (plo, phi, w1, w2, w3) |
| Padded parent-patch extents of a child with parent footprint [plo, phi] (amr_cpat_mar coarse cells each side). | |
| impure subroutine | s_amr_parent_slabs (w1, w2, w3, full, msl, tb, te) |
| Parent-fill transfer list of a child with padded patch (w1, w2, w3), patch-local: the whole patch (full) or its hollow shell. Send, receive and consume share it. | |
| impure subroutine | s_amr_parent_frame (k, plo, phi, w1, w2, w3) |
| A child's patch frame: the global parent-fine origin of its padded parent patch (region footprint - amr_cpat_mar). | |
| impure subroutine, public | s_amr_parent_fill_exchange (lev, full) |
| Level-lev parent-fill wave, exchange phase: each block's parent patch from its parent's fine array (parent-fine frame); split children ship from the parent's owner, co-located parents copy at consume. Called per level ascending so parents are complete. Consume with s_amr_parent_fill_consume per owned box. | |
| impure subroutine, public | s_amr_parent_fill_consume (k, full) |
| The parent-fill wave, consume phase for owned box k (the current slot): set the patch frame, then a co-located parent's own-box device copies or k's received transfers (box-major: the next contiguous run at the cursor). | |
| impure subroutine, public | s_amr_parent_fill_wave (lev) |
| Per-stage level-lev fill: the wave, then per owned level-lev box the consume and the ghost fill. | |
| impure subroutine, public | s_amr_pack_unpack_block_st (loc, o1, o2, o3, fm1, fm2, fm3, buf, to_buf) |
| Device pack (to_buf=T) / unpack (F) of the interior block [o+0:o+fm] of the flat store at column loc <-> contiguous MPI buffer buf for P2P migration; wp buffer cast to/from stp. Only buf is mapped (flang miscomputes strided sf sections in map clauses). | |
Parent/child data exchange: gather plans, pack/unpack and the fill waves.
|
private |
Parent value for a level >= 2 ghost fill: the current state, or its blend with the substep-start backup when th < 1.
Definition at line 395 of file m_amr_exchange.fpp.f90.
|
private |
Hold np coarse patches in amr_cg stacked along dim 3 (patch p at p*(amr_cpat_hi(3) + 1)); grows only, contents do not survive a fill.
Definition at line 1513 of file m_amr_exchange.fpp.f90.
|
private |
| [in] | qp | parent's flat-store slot |
Definition at line 484 of file m_amr_exchange.fpp.f90.
|
private |
Drain block k's received transfers at the wave cursor into amr_cg (patch origin off): one fused unpack per contiguous run under amr_device_pack, else one unpack per transfer.
Definition at line 1195 of file m_amr_exchange.fpp.f90.
| impure subroutine, public m_amr_exchange::s_amr_fill_wave_done |
Every receive transfer of the open fill wave has been consumed.
Definition at line 1254 of file m_amr_exchange.fpp.f90.
|
private |
Fused coarse-field pack: the wave's whole send list in one launch; inverts the linear index s_amr_pack_box_device writes, so wire bytes match the per-transfer packs.
Definition at line 913 of file m_amr_exchange.fpp.f90.
|
private |
Fused parent-store pack: as s_amr_fx_pack_box, with the source the flat store at parent slot pl(8,:) and child frame pl(9:11,:).
Definition at line 975 of file m_amr_exchange.fpp.f90.
|
private |
Fused-pack plan for transfers 1:nt: slab corner, extents, absolute pool payload offset and exclusive element prefix amr_fx_pre. Rows 8:11 are left to the parent-store pack site.
Definition at line 874 of file m_amr_exchange.fpp.f90.
|
private |
Longest run t0:t1 of one box's recv transfers that is contiguous in the wire pool (unpacks in one launch).
Definition at line 897 of file m_amr_exchange.fpp.f90.
|
private |
Fused unpack of the contiguous transfer run t0:t1 into amr_cg in one launch; serves both packs via the per-transfer frame in plan rows 9:11.
| [in,out] | cg | amr_cg, passed so a regrown pool maps as an argument |
Definition at line 1038 of file m_amr_exchange.fpp.f90.
|
private |
Ring-clipped own-box copy on device: queue shell-slab/own-box intersections as slab rows and copy q_coarse into amr_cg in one fused kernel.
| [in,out] | cg | amr_cg, passed so a regrown pool maps as an argument |
Definition at line 807 of file m_amr_exchange.fpp.f90.
|
private |
Level-1 consume and ghost fill for all owned boxes at once: patches pooled in amr_cg (s_amr_cg_reserve), one kernel each.
Definition at line 1705 of file m_amr_exchange.fpp.f90.
| impure subroutine, public m_amr_exchange::s_amr_l1_fill_consume | ( | type(scalar_field), dimension(sys_size), intent(in) | q_cons_coarse, |
| integer, intent(in) | k, | ||
| logical, intent(in) | full ) |
Level-1 fill wave, consume for owned box k: set the patch frame, copy the own slice, unpack k's transfers. Consume in ascending order: transfers are box-major, so k's are the next run at the cursor.
Definition at line 1222 of file m_amr_exchange.fpp.f90.
| impure subroutine, public m_amr_exchange::s_amr_l1_fill_exchange | ( | type(scalar_field), dimension(sys_size), intent(inout) | q_cons_coarse, |
| logical, intent(in) | full ) |
Level-1 fill wave, exchange phase: each level-1 box's padded coarse patch (region +/- amr_cpat_mar) is assembled on its owner from the coarse owners, one message per peer; both sides derive the transfers from replicated metadata. Slices are ring-clipped to the hollow shell unless full (init, regrid). Consume with s_amr_l1_fill_consume per owned box ascending.
Definition at line 1101 of file m_amr_exchange.fpp.f90.
| impure subroutine, public m_amr_exchange::s_amr_l1_pool_fill |
Ghost-fill every owned level-1 box from its pooled patch (s_amr_l1_pool_gather's layout) in one kernel.
Definition at line 1757 of file m_amr_exchange.fpp.f90.
| impure subroutine, public m_amr_exchange::s_amr_l1_pool_gather | ( | type(scalar_field), dimension(sys_size), intent(in) | q_cons_coarse | ) |
Consume the open level-1 fill wave into the pooled amr_cg: box k at dim-3 offset amr_cg_pz(k), amr_cg_nuse patches.
Definition at line 1715 of file m_amr_exchange.fpp.f90.
|
private |
A contributor's slice [bl, bh] of box k's patch as transfer slabs: the whole slice (full) or its hollow-shell intersection.
Definition at line 1176 of file m_amr_exchange.fpp.f90.
|
private |
Pack (pack = T) the global box [bl:bh] of q_coarse into the wire buffer buf, or unpack it into amr_cg at patch-local index g.
Definition at line 544 of file m_amr_exchange.fpp.f90.
|
private |
Sub-box variants of the parent-patch pack/copy for the ring-clipped parent-fill wave; bl:bh are patch-local, the buffer layout matches the full-patch kernels.
| [in] | qp | parent's flat-store slot |
Definition at line 425 of file m_amr_exchange.fpp.f90.
| impure subroutine, public m_amr_exchange::s_amr_pack_unpack_block_st | ( | integer, intent(in) | loc, |
| integer, intent(in) | o1, | ||
| integer, intent(in) | o2, | ||
| integer, intent(in) | o3, | ||
| integer, intent(in) | fm1, | ||
| integer, intent(in) | fm2, | ||
| integer, intent(in) | fm3, | ||
| real(wp), dimension(:), intent(inout), contiguous | buf, | ||
| logical, intent(in) | to_buf ) |
Device pack (to_buf=T) / unpack (F) of the interior block [o+0:o+fm] of the flat store at column loc <-> contiguous MPI buffer buf for P2P migration; wp buffer cast to/from stp. Only buf is mapped (flang miscomputes strided sf sections in map clauses).
Definition at line 1962 of file m_amr_exchange.fpp.f90.
| impure subroutine, public m_amr_exchange::s_amr_parent_fill_consume | ( | integer, intent(in) | k, |
| logical, intent(in) | full ) |
The parent-fill wave, consume phase for owned box k (the current slot): set the patch frame, then a co-located parent's own-box device copies or k's received transfers (box-major: the next contiguous run at the cursor).
Definition at line 1900 of file m_amr_exchange.fpp.f90.
| impure subroutine, public m_amr_exchange::s_amr_parent_fill_exchange | ( | integer, intent(in) | lev, |
| logical, intent(in) | full ) |
Level-lev parent-fill wave, exchange phase: each block's parent patch from its parent's fine array (parent-fine frame); split children ship from the parent's owner, co-located parents copy at consume. Called per level ascending so parents are complete. Consume with s_amr_parent_fill_consume per owned box.
Definition at line 1820 of file m_amr_exchange.fpp.f90.
| impure subroutine, public m_amr_exchange::s_amr_parent_fill_wave | ( | integer, intent(in) | lev | ) |
Per-stage level-lev fill: the wave, then per owned level-lev box the consume and the ghost fill.
Definition at line 1937 of file m_amr_exchange.fpp.f90.
|
private |
A child's patch frame: the global parent-fine origin of its padded parent patch (region footprint - amr_cpat_mar).
Definition at line 1806 of file m_amr_exchange.fpp.f90.
|
private |
Parent-fill transfer list of a child with padded patch (w1, w2, w3), patch-local: the whole patch (full) or its hollow shell. Send, receive and consume share it.
Definition at line 1788 of file m_amr_exchange.fpp.f90.
|
private |
Padded parent-patch extents of a child with parent footprint [plo, phi] (amr_cpat_mar coarse cells each side).
Definition at line 1776 of file m_amr_exchange.fpp.f90.
|
private |
Debug only: NaN-flood amr_cg before a clipped gather so any read of an unshipped cell NaNs the ghost fill.
Definition at line 750 of file m_amr_exchange.fpp.f90.
| impure subroutine, public m_amr_exchange::s_amr_restrict_rows_sf | ( | type(scalar_field), dimension(sys_size), intent(inout) | coarse_tgt, |
| integer, intent(in) | o1, | ||
| integer, intent(in) | o2, | ||
| integer, intent(in) | o3 ) |
Restriction over the queued slab rows (row 9 store column, 10:12 region lo, 13 wire payload offset): children average into the coarse field (sf, frame o) or into buf (wire layout of s_amr_unpack_rows).
Definition at line 1345 of file m_amr_exchange.fpp.f90.
| impure subroutine, public m_amr_exchange::s_amr_restrict_rows_wire | ( | real(wp), dimension(:), intent(inout), contiguous | buf | ) |
Definition at line 1428 of file m_amr_exchange.fpp.f90.
|
private |
Intersect the shell slabs with box [bl:bh], keeping order; also returns the total cell count.
Definition at line 731 of file m_amr_exchange.fpp.f90.
|
private |
Ring clip: split the hollow shell of patch [plo:phi] minus open core [clo:chi] into at most 6 disjoint slabs in fixed order (x, y within core x, z within core x and y). Collapsed dims pass 0; the width-1 double cover is clamped. Both sides derive the list from replicated metadata.
Definition at line 683 of file m_amr_exchange.fpp.f90.
| impure subroutine, public m_amr_exchange::s_amr_stage_fill_wave | ( | type(scalar_field), dimension(sys_size), intent(inout) | q_cons_coarse | ) |
Per-stage level-1 fill: the wave, then every owned level-1 box's consume and ghost fill at once.
Definition at line 1268 of file m_amr_exchange.fpp.f90.
|
private |
| impure subroutine, public m_amr_exchange::s_amr_unpack_rows | ( | real(wp), dimension(:), intent(in), contiguous | buf, |
| type(scalar_field), dimension(sys_size), intent(inout) | cg ) |
Unpack the queued slab rows (row 9: payload offset in buf) into the pooled amr_cg patches: a whole fill wave in one launch.
Definition at line 1278 of file m_amr_exchange.fpp.f90.