|
MFC
Exascale flow solver
|
MPI point-to-point waves for AMR exchange: post all receives, pack, post all sends, wait once. Pooled waves list transfers on a send/receive side and s_amr_wave_close lays them out contiguously per peer; both ranks enumerate the same transfers in the same order, so no metadata is exchanged. Pools are caller-owned (device-resident under rdma_mpi). More...
Data Types | |
| type | t_amr_wave_side |
| One side (send or receive) of a pooled wave. More... | |
| type | t_amr_wave |
| An open wave: tag band and requests; reqw is -1 for sends, the planned word count for receives. More... | |
Functions/Subroutines | |
| impure subroutine, public | s_amr_wave_open (w, b) |
| Open a wave on tag band b. | |
| pure integer function, public | f_amr_wave_nreq (w) |
| impure subroutine, public | s_amr_wave_req (w, dir, buf, n, peer, dev) |
| Post one request of n words between buf (device-resident when dev) and peer: dir = 1 sends, 2 receives. | |
| impure subroutine, public | s_amr_wave_req_raw (w, dir, buf, n, peer) |
| Host-array variant of s_amr_wave_req (assumed-size, any-rank sections, no device address). | |
| impure subroutine | s_amr_tag_wave_open (band) |
| Open a keyed-tag wave: clear the touched per-peer seq counters and bump the band's generation. Call at wave entry on every rank. | |
| impure integer function | f_amr_wave_seq (peer, idir) |
| Next seq on the (peer, dir) channel of the open wave. dir: 1 = send, 2 = recv (separate counters). | |
| pure integer function | f_amr_wave_tag_of (band, sq) |
| impure integer function | f_amr_wave_tag (w, dir, peer) |
| Keyed tag of the next request to peer; grows the request list (wnreq becomes the new request's index). | |
| impure subroutine, public | s_amr_wave_wait (w) |
| Wait for all requests; under MFC_DEBUG each receive length must equal its plan. | |
| impure subroutine, public | s_amr_wave_reset (s) |
| Start enumerating a side's transfers. | |
| impure subroutine, public | s_amr_wave_add (s, peer, blk, bl, bh, words) |
| Append one transfer of words payload words for block blk over box [bl, bh] with peer. | |
| impure subroutine, public | s_amr_wave_add_slabs (s, peer, blk, msl, tb, te) |
| Add the msl slabs [tb:te] of block blk as transfers with peer. | |
| impure subroutine, public | s_amr_wave_close (s, pool, dev) |
| Lay the side out in its pool (peers in first-appearance order, transfers contiguous per peer); grows pool to the words in use. | |
| pure subroutine, public | s_amr_wave_slice (s, t, lo, hi) |
| Pool bounds [lo, hi] of transfer t's payload. | |
| impure subroutine | s_amr_wave_peers (w, dir, s, pool, dev) |
| One message per peer (dir = 1 sends, 2 receives) over the side's pool runs. | |
| impure subroutine, public | s_amr_wave_post (w, s, pool, dev) |
| One receive per peer into the side's pool runs. | |
| impure subroutine, public | s_amr_wave_send (w, s, pool, dev) |
| One send per peer (the caller has packed every slice). | |
Variables | |
| type(t_amr_wave_side), public | amr_wsend |
| Fill and restrict waves share amr_wsend/amr_wrecv; the seam wave stays open across the fills, so it has its own pair. | |
| type(t_amr_wave_side), public | amr_wrecv |
| type(t_amr_wave_side), public | amr_wseam_snd |
| type(t_amr_wave_side), public | amr_wseam_rcv |
| type(t_amr_wave), target, public | amr_wave |
| type(t_amr_wave), target, public | amr_wave_seam |
| integer, dimension(:), allocatable | bmap |
| Rank-indexed scratch for s_amr_wave_close; all-zero between calls. | |
| integer, dimension(:), allocatable | bnx |
| integer, dimension(:), allocatable | bwords |
MPI point-to-point waves for AMR exchange: post all receives, pack, post all sends, wait once. Pooled waves list transfers on a send/receive side and s_amr_wave_close lays them out contiguously per peer; both ranks enumerate the same transfers in the same order, so no metadata is exchanged. Pools are caller-owned (device-resident under rdma_mpi).
| pure integer function, public m_amr_wave::f_amr_wave_nreq | ( | type(t_amr_wave), intent(in) | w | ) |
|
private |
Next seq on the (peer, dir) channel of the open wave. dir: 1 = send, 2 = recv (separate counters).
Definition at line 545 of file m_amr_wave.fpp.f90.
|
private |
Keyed tag of the next request to peer; grows the request list (wnreq becomes the new request's index).
Definition at line 575 of file m_amr_wave.fpp.f90.
|
private |
|
private |
Open a keyed-tag wave: clear the touched per-peer seq counters and bump the band's generation. Call at wave entry on every rank.
Definition at line 525 of file m_amr_wave.fpp.f90.
| impure subroutine, public m_amr_wave::s_amr_wave_add | ( | type(t_amr_wave_side), intent(inout) | s, |
| integer, intent(in) | peer, | ||
| integer, intent(in) | blk, | ||
| integer, dimension(3), intent(in) | bl, | ||
| integer, dimension(3), intent(in) | bh, | ||
| integer, intent(in) | words ) |
Append one transfer of words payload words for block blk over box [bl, bh] with peer.
Definition at line 630 of file m_amr_wave.fpp.f90.
| impure subroutine, public m_amr_wave::s_amr_wave_add_slabs | ( | type(t_amr_wave_side), intent(inout) | s, |
| integer, intent(in) | peer, | ||
| integer, intent(in) | blk, | ||
| integer, intent(in) | msl, | ||
| integer, dimension(3, 6), intent(in) | tb, | ||
| integer, dimension(3, 6), intent(in) | te ) |
Add the msl slabs [tb:te] of block blk as transfers with peer.
Definition at line 644 of file m_amr_wave.fpp.f90.
| impure subroutine, public m_amr_wave::s_amr_wave_close | ( | type(t_amr_wave_side), intent(inout) | s, |
| real(wp), dimension(:), intent(inout), allocatable | pool, | ||
| logical, intent(in) | dev ) |
Lay the side out in its pool (peers in first-appearance order, transfers contiguous per peer); grows pool to the words in use.
Definition at line 658 of file m_amr_wave.fpp.f90.
| impure subroutine, public m_amr_wave::s_amr_wave_open | ( | type(t_amr_wave), intent(inout) | w, |
| integer, intent(in) | b ) |
Open a wave on tag band b.
Definition at line 415 of file m_amr_wave.fpp.f90.
|
private |
One message per peer (dir = 1 sends, 2 receives) over the side's pool runs.
Definition at line 711 of file m_amr_wave.fpp.f90.
| impure subroutine, public m_amr_wave::s_amr_wave_post | ( | type(t_amr_wave), intent(inout) | w, |
| type(t_amr_wave_side), intent(in) | s, | ||
| real(wp), dimension(:), intent(inout), contiguous | pool, | ||
| logical, intent(in) | dev ) |
One receive per peer into the side's pool runs.
Definition at line 727 of file m_amr_wave.fpp.f90.
| impure subroutine, public m_amr_wave::s_amr_wave_req | ( | type(t_amr_wave), intent(inout) | w, |
| integer, intent(in) | dir, | ||
| real(wp), dimension(:), intent(inout), contiguous | buf, | ||
| integer, intent(in) | n, | ||
| integer, intent(in) | peer, | ||
| logical, intent(in) | dev ) |
Post one request of n words between buf (device-resident when dev) and peer: dir = 1 sends, 2 receives.
Definition at line 434 of file m_amr_wave.fpp.f90.
| impure subroutine, public m_amr_wave::s_amr_wave_req_raw | ( | type(t_amr_wave), intent(inout) | w, |
| integer, intent(in) | dir, | ||
| real(wp), dimension(*), intent(inout) | buf, | ||
| integer, intent(in) | n, | ||
| integer, intent(in) | peer ) |
Host-array variant of s_amr_wave_req (assumed-size, any-rank sections, no device address).
Definition at line 504 of file m_amr_wave.fpp.f90.
| impure subroutine, public m_amr_wave::s_amr_wave_reset | ( | type(t_amr_wave_side), intent(inout) | s | ) |
Start enumerating a side's transfers.
Definition at line 621 of file m_amr_wave.fpp.f90.
| impure subroutine, public m_amr_wave::s_amr_wave_send | ( | type(t_amr_wave), intent(inout) | w, |
| type(t_amr_wave_side), intent(in) | s, | ||
| real(wp), dimension(:), intent(inout), contiguous | pool, | ||
| logical, intent(in) | dev ) |
One send per peer (the caller has packed every slice).
Definition at line 739 of file m_amr_wave.fpp.f90.
| pure subroutine, public m_amr_wave::s_amr_wave_slice | ( | type(t_amr_wave_side), intent(in) | s, |
| integer, intent(in) | t, | ||
| integer, intent(out) | lo, | ||
| integer, intent(out) | hi ) |
Pool bounds [lo, hi] of transfer t's payload.
Definition at line 700 of file m_amr_wave.fpp.f90.
| impure subroutine, public m_amr_wave::s_amr_wave_wait | ( | type(t_amr_wave), intent(inout) | w | ) |
Wait for all requests; under MFC_DEBUG each receive length must equal its plan.
Definition at line 588 of file m_amr_wave.fpp.f90.
| type(t_amr_wave), target, public m_amr_wave::amr_wave |
Definition at line 408 of file m_amr_wave.fpp.f90.
| type(t_amr_wave), target, public m_amr_wave::amr_wave_seam |
Definition at line 408 of file m_amr_wave.fpp.f90.
| type(t_amr_wave_side), public m_amr_wave::amr_wrecv |
Definition at line 401 of file m_amr_wave.fpp.f90.
| type(t_amr_wave_side), public m_amr_wave::amr_wseam_rcv |
Definition at line 401 of file m_amr_wave.fpp.f90.
| type(t_amr_wave_side), public m_amr_wave::amr_wseam_snd |
Definition at line 401 of file m_amr_wave.fpp.f90.
| type(t_amr_wave_side), public m_amr_wave::amr_wsend |
Fill and restrict waves share amr_wsend/amr_wrecv; the seam wave stays open across the fills, so it has its own pair.
Definition at line 401 of file m_amr_wave.fpp.f90.
|
private |
Rank-indexed scratch for s_amr_wave_close; all-zero between calls.
Definition at line 410 of file m_amr_wave.fpp.f90.
|
private |
Definition at line 410 of file m_amr_wave.fpp.f90.
|
private |
Definition at line 410 of file m_amr_wave.fpp.f90.