|
MFC
Exascale flow solver
|
Dynamic regrid for the AMR level set: density-gradient tagging, clustering (m_amr_cluster), box shaping, hierarchical child nesting (m_amr_nest), and slot rebuild with cross-rank fine-state migration. More...
Functions/Subroutines | |
| impure subroutine, public | s_amr_check_seam_topology () |
| Abort on same-level seam topologies no halo reconciles: partial-transverse face adjacency or box intersection (both arise only from IB body-bbox expansion). Each rank tests its own blocks against all blocks. | |
| impure subroutine | s_amr_check_box_caps (boxes, nboxes, box_level) |
| Abort if a box exceeds its level's slot cap, amr_maxc_fit/amr_ref_ratio**(lev-1) coarse cells (it would overrun the fine coord arrays); floor of 1 as in s_amr_tile_box. | |
| impure subroutine | s_amr_check_box_disjoint (boxes, nboxes, box_level) |
| Abort if two same-level boxes overlap (the overlap carry-forward and exchange plans assume disjointness). All levels share the coarse index space; an x-lo sweep keeps it cheap. | |
| pure logical function | f_in_acoustic_support (gi, gj, gk) |
| True iff global level-0 cell (gi, gj, gk) lies inside any acoustic source support bbox. | |
| impure subroutine | s_amr_clip_box_from_sources (lo, hi) |
| Clip a box (global indices) clear of every acoustic source support; may empty it (hi < lo). | |
| impure subroutine | s_amr_compute_lag_supp (pad_cells) |
| Collectively recompute the global bubble-cloud exclusion bbox, padded by pad_cells; lag_supp_on = false when no rank holds a bubble. | |
| pure logical function | f_in_lag_support (gi, gj, gk) |
| True iff global level-0 cell (gi, gj, gk) lies inside the Lagrangian-cloud exclusion bbox. | |
| pure subroutine | s_amr_clip_box_from_supp (lo, hi, slo, shi) |
| Clip a box (global indices) clear of one support bbox along the axis/side keeping the most extent (ties: lower axis, begin side); may empty it. | |
| impure subroutine | s_amr_clip_box_clear (lo, hi) |
| Clip a box clear of the acoustic source and bubble-cloud supports. | |
| impure subroutine | s_amr_local_tags (tag_grid, sidx, tags, ntag) |
| This rank's tagged cells as global level-0 coordinates; rank interiors are disjoint, so a SUM reduction counts each cell once. | |
| impure subroutine, public | s_amr_regrid (q_cons_base) |
| Regrid: tag by density gradient, cluster, shape, nest and rebuild the slots, carrying old data into the overlaps. Called between steps; no-op if nothing is tagged or the box set is unchanged. | |
| impure subroutine | s_amr_report_coverage (boxes, nboxes, box_level) |
| Print the share of the base domain covered by each level's boxes (rank 0, rank_time_wrt). | |
| impure subroutine | s_amr_regrid_tag_cells (q_cons_base, tag_grid, sidx) |
| Regrid phase 1: density-gradient tag field, skipping the global boundary cells; tags over acoustic and bubble-cloud supports are suppressed. sidx = this rank's global start offsets. | |
| impure subroutine | s_amr_regrid_cluster_tags (tag_grid, sidx, boxes, nboxes) |
| Regrid phase 2: cluster this rank's tags (rank-invariant, via signature reductions) into separated boxes; nboxes = 0 if nothing is tagged. | |
| impure subroutine | s_amr_regrid_dormant_seed (boxes, nboxes) |
| One coarse cell at the domain center, which shaping pads into the dormant block that replaces the hierarchy. | |
| pure logical function | f_amr_boxes_compatible (alo, ahi, blo, bhi) |
| True iff two same-level blocks can coexist: disjoint, and either >= 2 empty coarse cells apart or face-adjacent with an exact transverse match. Mirrors s_amr_check_seam_topology. | |
| impure subroutine | s_amr_regrid_snap_boxes (boxes, nboxes, box_level) |
| Hysteresis (amr_snap > 0): a box within amr_snap cells per face of a live same-level block takes that block's box. All-or-none: the snapped set must stay disjoint, compatible and singly nested (within amr_cpat_mar), else the fresh boxes stand. | |
| pure subroutine | s_amr_box_carve (box, glo, ghi, gap, out, n) |
| Append to out(n+1:) the pieces of box that stay gap empty cells clear of [glo:ghi] and of each other, cutting dim by dim; the part inside the block's span is dropped. The caller guarantees box overlaps the gap-grown block. | |
| impure subroutine | s_amr_check_boxes_agree (boxes, nboxes) |
| Abort if ranks hold different box lists (each rank sees only nearby IB bodies; a mismatch hangs the ownership collectives). | |
| pure subroutine | s_amr_merge_body_groups (glo, ghi, clo, chi, ng, i, j) |
| Fold body group j into group i (bbox and body-extent unions) and move the last group into slot j. | |
| impure subroutine | s_amr_gather_body_groups (glo, ghi, clo, chi, ng, mrg, nbody) |
| One margin-padded box (glo:ghi) and body extent (clo:chi) per immersed body, identical on every rank. | |
| impure subroutine | s_amr_resolve_body_groups (glo, ghi, clo, chi, ng, gap) |
| Separate body groups that face each other across < gap empty cells or overlap, reject groups over the block cap, and order them canonically. | |
| impure subroutine | s_amr_carve_tile_around_bodies (boxes, nboxes, out, nt, capt) |
| Regrid phase 3a (IB): tile the boxes around the immersed bodies. Each body (bbox + margin; close bodies share one) owns a block; boxes give up that block plus a gap ring, the remainder tiles to <= amr_maxc_fit, and body blocks are appended last. The ring keeps body blocks off other blocks, since the seam halo and reflux need exact-match transverse faces. | |
| impure subroutine | s_amr_regrid_shape_boxes (boxes, nboxes) |
| Regrid phase 3: pad and clamp each box, clip it clear of the acoustic/bubble supports, then tile oversized boxes (IB: after carving out the body blocks). | |
| impure subroutine | s_amr_regrid_boxes_unchanged (boxes, nboxes, box_level, same) |
| True iff the box set and levels equal the live slots (a rebuild would reproduce them). | |
| impure subroutine | s_amr_mig_pack_device (loc, l1, l2, l3, h1, h2, h3, buf) |
| Pack fine window [l, h] of an owned old block's stash (device store) into its wire slice buf; gi fastest, then gj, gk, ii. | |
| pure subroutine | s_amr_mig_window (wlo, whi, ilo, lev, flo, fhi) |
| Fine-index window [flo, fhi] in level-1 old block ilo's stash of the coarse box [wlo, whi] (0:0 in collapsed dims). | |
| impure subroutine | s_amr_mig_free (pool) |
| Free a migration pool (device-resident when amr_fw_dev). | |
| impure subroutine | s_amr_stash_copy_device (loc, e1, e2, e3) |
| Device cons->stor stash copy of one owned old block's fine interior (the store is device-authoritative; no host staging). | |
| impure subroutine | s_amr_overlap_copy_st (vloc, vm, vn, vp, sh, l1, l2, l3, h1, h2, h3, |
| Overwrite the new block's prolonged cons with the covering old block's stashed detail (old fine index = new + sh) over the old window [l, h], from store column vold or the received wire slice buf. | |
| impure subroutine | s_amr_overlap_copy_wire (vloc, vm, vn, vp, sh, l1, l2, l3, h1, h2, h3, |
| impure subroutine | s_amr_stash_old_blocks (old_np, old_ilo, old_chi, old_ext, old_owner, old_level, old_owns) |
| Record the live slots' fine interiors on the device and the old block set (old_*; old_chi = replicated coarse hi corner). | |
| impure subroutine | s_amr_commit_boxes (boxes, nboxes, box_level) |
| Commit the new regions, levels and owners, after checking proper nesting. | |
| impure subroutine | s_amr_migrate_old_blocks (boxes, nboxes, box_level, old_np, old_ilo, old_chi, old_owner, old_level, old_owns) |
| Cross-rank migration as one wave: each old owner ships the bounding box of its overlaps to every other rank owning an overlapping new block of the same level. Both sides enumerate old blocks ascending, so the transfers line up; received windows stay in amr_mig_rq. No-op at np=1. | |
| impure subroutine | s_amr_regrid_stash_migrate (boxes, nboxes, box_level, old_np, old_ilo, old_ext, old_level, old_owns) |
| Regrid phase 5: stash the old block set, commit the new one, and migrate stashed old blocks point-to-point to the ranks owning overlapping new blocks. | |
| impure subroutine | s_amr_plan_rebuild_visits (boxes, nboxes, old_np, old_ilo, old_ext, old_level, vpos, last_use, held, nh) |
| Visit order of the owned new boxes (level-major, vpos) and, per held old slot, the last visit that reads it (last_use). | |
| impure subroutine | s_amr_carry_old_into_slot (ks, boxes, old_np, old_ilo, old_ext, old_level) |
| Overwrite the freshly prolonged slot ks with the overlapping old blocks of its level: from the migration pool when another rank owned them, else from the local stash. | |
| impure subroutine | s_amr_regrid_rebuild_slots (q_cons_base, boxes, nboxes, old_np, old_ilo, old_ext, old_level) |
| Regrid phase 6: build each new slot level by level (prolong from the coarse/parent patch, overwrite the overlap from covering stashed old blocks), then reconcile the slot pool, rebuild the fine IB state and re-check the seam topology. Old-only slots are freed after their last reader so device memory does not peak. | |
Variables | |
| integer, dimension(3) | lag_supp_lo |
| Padded global coarse-index bbox of the Lagrangian bubble cloud; blocks and regrid boxes stay clear of it. | |
| integer, dimension(3) | lag_supp_hi |
| logical | lag_supp_on = .false. |
| real(wp), dimension(:), allocatable | amr_mig_sq |
| Migration send/receive pools (freed after the regrid); amr_mig_win(:,old block) = payload start in amr_mig_rq, fine window lo(3), hi(3) (start 0 = none). | |
| real(wp), dimension(:), allocatable | amr_mig_rq |
| integer, dimension(:,:), allocatable | amr_mig_win |
Dynamic regrid for the AMR level set: density-gradient tagging, clustering (m_amr_cluster), box shaping, hierarchical child nesting (m_amr_nest), and slot rebuild with cross-rank fine-state migration.
|
private |
True iff two same-level blocks can coexist: disjoint, and either >= 2 empty coarse cells apart or face-adjacent with an exact transverse match. Mirrors s_amr_check_seam_topology.
Definition at line 805 of file m_amr_regrid.fpp.f90.
|
private |
True iff global level-0 cell (gi, gj, gk) lies inside any acoustic source support bbox.
Definition at line 512 of file m_amr_regrid.fpp.f90.
|
private |
True iff global level-0 cell (gi, gj, gk) lies inside the Lagrangian-cloud exclusion bbox.
Definition at line 565 of file m_amr_regrid.fpp.f90.
|
private |
Append to out(n+1:) the pieces of box that stay gap empty cells clear of [glo:ghi] and of each other, cutting dim by dim; the part inside the block's span is dropped. The caller guarantees box overlaps the gap-grown block.
Definition at line 890 of file m_amr_regrid.fpp.f90.
|
private |
Overwrite the freshly prolonged slot ks with the overlapping old blocks of its level: from the migration pool when another rank owned them, else from the local stash.
Definition at line 1653 of file m_amr_regrid.fpp.f90.
|
private |
Regrid phase 3a (IB): tile the boxes around the immersed bodies. Each body (bbox + margin; close bodies share one) owns a block; boxes give up that block plus a gap ring, the remainder tiles to <= amr_maxc_fit, and body blocks are appended last. The ring keeps body blocks off other blocks, since the seam halo and reflux need exact-match transverse faces.
Definition at line 1058 of file m_amr_regrid.fpp.f90.
|
private |
Abort if a box exceeds its level's slot cap, amr_maxc_fit/amr_ref_ratio**(lev-1) coarse cells (it would overrun the fine coord arrays); floor of 1 as in s_amr_tile_box.
Definition at line 458 of file m_amr_regrid.fpp.f90.
|
private |
Abort if two same-level boxes overlap (the overlap carry-forward and exchange plans assume disjointness). All levels share the coarse index space; an x-lo sweep keeps it cheap.
Definition at line 483 of file m_amr_regrid.fpp.f90.
|
private |
Abort if ranks hold different box lists (each rank sees only nearby IB bodies; a mismatch hangs the ownership collectives).
Definition at line 915 of file m_amr_regrid.fpp.f90.
| impure subroutine, public m_amr_regrid::s_amr_check_seam_topology |
Abort on same-level seam topologies no halo reconciles: partial-transverse face adjacency or box intersection (both arise only from IB body-bbox expansion). Each rank tests its own blocks against all blocks.
Definition at line 415 of file m_amr_regrid.fpp.f90.
|
private |
Clip a box clear of the acoustic source and bubble-cloud supports.
Definition at line 603 of file m_amr_regrid.fpp.f90.
|
private |
Clip a box (global indices) clear of every acoustic source support; may empty it (hi < lo).
Definition at line 529 of file m_amr_regrid.fpp.f90.
|
private |
Clip a box (global indices) clear of one support bbox along the axis/side keeping the most extent (ties: lower axis, begin side); may empty it.
Definition at line 579 of file m_amr_regrid.fpp.f90.
|
private |
Commit the new regions, levels and owners, after checking proper nesting.
Definition at line 1504 of file m_amr_regrid.fpp.f90.
|
private |
Collectively recompute the global bubble-cloud exclusion bbox, padded by pad_cells; lag_supp_on = false when no rank holds a bubble.
Definition at line 547 of file m_amr_regrid.fpp.f90.
|
private |
One margin-padded box (glo:ghi) and body extent (clo:chi) per immersed body, identical on every rank.
Definition at line 949 of file m_amr_regrid.fpp.f90.
|
private |
This rank's tagged cells as global level-0 coordinates; rank interiors are disjoint, so a SUM reduction counts each cell once.
Definition at line 614 of file m_amr_regrid.fpp.f90.
|
private |
Fold body group j into group i (bbox and body-extent unions) and move the last group into slot j.
Definition at line 937 of file m_amr_regrid.fpp.f90.
|
private |
Free a migration pool (device-resident when amr_fw_dev).
Definition at line 1249 of file m_amr_regrid.fpp.f90.
|
private |
Pack fine window [l, h] of an owned old block's stash (device store) into its wire slice buf; gi fastest, then gj, gk, ii.
Definition at line 1184 of file m_amr_regrid.fpp.f90.
|
private |
Fine-index window [flo, fhi] in level-1 old block ilo's stash of the coarse box [wlo, whi] (0:0 in collapsed dims).
Definition at line 1238 of file m_amr_regrid.fpp.f90.
|
private |
Cross-rank migration as one wave: each old owner ships the bounding box of its overlaps to every other rank owning an overlapping new block of the same level. Both sides enumerate old blocks ascending, so the transfers line up; received windows stay in amr_mig_rq. No-op at np=1.
Definition at line 1540 of file m_amr_regrid.fpp.f90.
|
private |
Overwrite the new block's prolonged cons with the covering old block's stashed detail (old fine index = new + sh) over the old window [l, h], from store column vold or the received wire slice buf.
Definition at line 1327 of file m_amr_regrid.fpp.f90.
|
private |
|
private |
Visit order of the owned new boxes (level-major, vpos) and, per held old slot, the last visit that reads it (last_use).
Definition at line 1620 of file m_amr_regrid.fpp.f90.
| impure subroutine, public m_amr_regrid::s_amr_regrid | ( | type(scalar_field), dimension(sys_size), intent(inout) | q_cons_base | ) |
Regrid: tag by density gradient, cluster, shape, nest and rebuild the slots, carrying old data into the overlaps. Called between steps; no-op if nothing is tagged or the box set is unchanged.
Definition at line 643 of file m_amr_regrid.fpp.f90.
|
private |
True iff the box set and levels equal the live slots (a rebuild would reproduce them).
Definition at line 1165 of file m_amr_regrid.fpp.f90.
|
private |
Regrid phase 2: cluster this rank's tags (rank-invariant, via signature reductions) into separated boxes; nboxes = 0 if nothing is tagged.
Definition at line 773 of file m_amr_regrid.fpp.f90.
|
private |
One coarse cell at the domain center, which shaping pads into the dormant block that replaces the hierarchy.
Definition at line 790 of file m_amr_regrid.fpp.f90.
|
private |
Regrid phase 6: build each new slot level by level (prolong from the coarse/parent patch, overwrite the overlap from covering stashed old blocks), then reconcile the slot pool, rebuild the fine IB state and re-check the seam topology. Old-only slots are freed after their last reader so device memory does not peak.
Definition at line 1692 of file m_amr_regrid.fpp.f90.
|
private |
Regrid phase 3: pad and clamp each box, clip it clear of the acoustic/bubble supports, then tile oversized boxes (IB: after carving out the body blocks).
Definition at line 1114 of file m_amr_regrid.fpp.f90.
|
private |
Hysteresis (amr_snap > 0): a box within amr_snap cells per face of a live same-level block takes that block's box. All-or-none: the snapped set must stay disjoint, compatible and singly nested (within amr_cpat_mar), else the fresh boxes stand.
Definition at line 831 of file m_amr_regrid.fpp.f90.
|
private |
Regrid phase 5: stash the old block set, commit the new one, and migrate stashed old blocks point-to-point to the ranks owning overlapping new blocks.
Definition at line 1601 of file m_amr_regrid.fpp.f90.
|
private |
Regrid phase 1: density-gradient tag field, skipping the global boundary cells; tags over acoustic and bubble-cloud supports are suppressed. sidx = this rank's global start offsets.
Definition at line 727 of file m_amr_regrid.fpp.f90.
|
private |
Print the share of the base domain covered by each level's boxes (rank 0, rank_time_wrt).
Definition at line 708 of file m_amr_regrid.fpp.f90.
|
private |
Separate body groups that face each other across < gap empty cells or overlap, reject groups over the block cap, and order them canonically.
Definition at line 988 of file m_amr_regrid.fpp.f90.
|
private |
Device cons->stor stash copy of one owned old block's fine interior (the store is device-authoritative; no host staging).
Definition at line 1273 of file m_amr_regrid.fpp.f90.
|
private |
Record the live slots' fine interiors on the device and the old block set (old_*; old_chi = replicated coarse hi corner).
Definition at line 1472 of file m_amr_regrid.fpp.f90.
|
private |
Definition at line 408 of file m_amr_regrid.fpp.f90.
|
private |
Migration send/receive pools (freed after the regrid); amr_mig_win(:,old block) = payload start in amr_mig_rq, fine window lo(3), hi(3) (start 0 = none).
Definition at line 408 of file m_amr_regrid.fpp.f90.
|
private |
Definition at line 409 of file m_amr_regrid.fpp.f90.
|
private |
Definition at line 404 of file m_amr_regrid.fpp.f90.
|
private |
Padded global coarse-index bbox of the Lagrangian bubble cloud; blocks and regrid boxes stay clear of it.
Definition at line 404 of file m_amr_regrid.fpp.f90.
|
private |
Definition at line 405 of file m_amr_regrid.fpp.f90.