MFC
Exascale flow solver
Loading...
Searching...
No Matches
m_amr_wave Module Reference

MPI point-to-point waves for AMR exchange: post all receives, pack, post all sends, wait once. Pooled waves list transfers on a send/receive side and s_amr_wave_close lays them out contiguously per peer; both ranks enumerate the same transfers in the same order, so no metadata is exchanged. Pools are caller-owned (device-resident under rdma_mpi). More...

Data Types

type  t_amr_wave_side
 One side (send or receive) of a pooled wave. More...
type  t_amr_wave
 An open wave: tag band and requests; reqw is -1 for sends, the planned word count for receives. More...

Functions/Subroutines

impure subroutine, public s_amr_wave_open (w, b)
 Open a wave on tag band b.
pure integer function, public f_amr_wave_nreq (w)
impure subroutine, public s_amr_wave_req (w, dir, buf, n, peer, dev)
 Post one request of n words between buf (device-resident when dev) and peer: dir = 1 sends, 2 receives.
impure subroutine, public s_amr_wave_req_raw (w, dir, buf, n, peer)
 Host-array variant of s_amr_wave_req (assumed-size, any-rank sections, no device address).
impure subroutine s_amr_tag_wave_open (band)
 Open a keyed-tag wave: clear the touched per-peer seq counters and bump the band's generation. Call at wave entry on every rank.
impure integer function f_amr_wave_seq (peer, idir)
 Next seq on the (peer, dir) channel of the open wave. dir: 1 = send, 2 = recv (separate counters).
pure integer function f_amr_wave_tag_of (band, sq)
impure integer function f_amr_wave_tag (w, dir, peer)
 Keyed tag of the next request to peer; grows the request list (wnreq becomes the new request's index).
impure subroutine, public s_amr_wave_wait (w)
 Wait for all requests; under MFC_DEBUG each receive length must equal its plan.
impure subroutine, public s_amr_wave_reset (s)
 Start enumerating a side's transfers.
impure subroutine, public s_amr_wave_add (s, peer, blk, bl, bh, words)
 Append one transfer of words payload words for block blk over box [bl, bh] with peer.
impure subroutine, public s_amr_wave_add_slabs (s, peer, blk, msl, tb, te)
 Add the msl slabs [tb:te] of block blk as transfers with peer.
impure subroutine, public s_amr_wave_close (s, pool, dev)
 Lay the side out in its pool (peers in first-appearance order, transfers contiguous per peer); grows pool to the words in use.
pure subroutine, public s_amr_wave_slice (s, t, lo, hi)
 Pool bounds [lo, hi] of transfer t's payload.
impure subroutine s_amr_wave_peers (w, dir, s, pool, dev)
 One message per peer (dir = 1 sends, 2 receives) over the side's pool runs.
impure subroutine, public s_amr_wave_post (w, s, pool, dev)
 One receive per peer into the side's pool runs.
impure subroutine, public s_amr_wave_send (w, s, pool, dev)
 One send per peer (the caller has packed every slice).

Variables

type(t_amr_wave_side), public amr_wsend
 Fill and restrict waves share amr_wsend/amr_wrecv; the seam wave stays open across the fills, so it has its own pair.
type(t_amr_wave_side), public amr_wrecv
type(t_amr_wave_side), public amr_wseam_snd
type(t_amr_wave_side), public amr_wseam_rcv
type(t_amr_wave), target, public amr_wave
type(t_amr_wave), target, public amr_wave_seam
integer, dimension(:), allocatable bmap
 Rank-indexed scratch for s_amr_wave_close; all-zero between calls.
integer, dimension(:), allocatable bnx
integer, dimension(:), allocatable bwords

Detailed Description

MPI point-to-point waves for AMR exchange: post all receives, pack, post all sends, wait once. Pooled waves list transfers on a send/receive side and s_amr_wave_close lays them out contiguously per peer; both ranks enumerate the same transfers in the same order, so no metadata is exchanged. Pools are caller-owned (device-resident under rdma_mpi).

Function/Subroutine Documentation

◆ f_amr_wave_nreq()

pure integer function, public m_amr_wave::f_amr_wave_nreq ( type(t_amr_wave), intent(in) w)

Definition at line 425 of file m_amr_wave.fpp.f90.

Here is the caller graph for this function:

◆ f_amr_wave_seq()

impure integer function m_amr_wave::f_amr_wave_seq ( integer, intent(in) peer,
integer, intent(in) idir )
private

Next seq on the (peer, dir) channel of the open wave. dir: 1 = send, 2 = recv (separate counters).

Definition at line 545 of file m_amr_wave.fpp.f90.

Here is the caller graph for this function:

◆ f_amr_wave_tag()

impure integer function m_amr_wave::f_amr_wave_tag ( type(t_amr_wave), intent(inout) w,
integer, intent(in) dir,
integer, intent(in) peer )
private

Keyed tag of the next request to peer; grows the request list (wnreq becomes the new request's index).

Definition at line 575 of file m_amr_wave.fpp.f90.

Here is the call graph for this function:
Here is the caller graph for this function:

◆ f_amr_wave_tag_of()

pure integer function m_amr_wave::f_amr_wave_tag_of ( integer, intent(in) band,
integer, intent(in) sq )
private

Definition at line 566 of file m_amr_wave.fpp.f90.

Here is the caller graph for this function:

◆ s_amr_tag_wave_open()

impure subroutine m_amr_wave::s_amr_tag_wave_open ( integer, intent(in) band)
private

Open a keyed-tag wave: clear the touched per-peer seq counters and bump the band's generation. Call at wave entry on every rank.

Definition at line 525 of file m_amr_wave.fpp.f90.

Here is the caller graph for this function:

◆ s_amr_wave_add()

impure subroutine, public m_amr_wave::s_amr_wave_add ( type(t_amr_wave_side), intent(inout) s,
integer, intent(in) peer,
integer, intent(in) blk,
integer, dimension(3), intent(in) bl,
integer, dimension(3), intent(in) bh,
integer, intent(in) words )

Append one transfer of words payload words for block blk over box [bl, bh] with peer.

Definition at line 630 of file m_amr_wave.fpp.f90.

Here is the caller graph for this function:

◆ s_amr_wave_add_slabs()

impure subroutine, public m_amr_wave::s_amr_wave_add_slabs ( type(t_amr_wave_side), intent(inout) s,
integer, intent(in) peer,
integer, intent(in) blk,
integer, intent(in) msl,
integer, dimension(3, 6), intent(in) tb,
integer, dimension(3, 6), intent(in) te )

Add the msl slabs [tb:te] of block blk as transfers with peer.

Definition at line 644 of file m_amr_wave.fpp.f90.

Here is the call graph for this function:
Here is the caller graph for this function:

◆ s_amr_wave_close()

impure subroutine, public m_amr_wave::s_amr_wave_close ( type(t_amr_wave_side), intent(inout) s,
real(wp), dimension(:), intent(inout), allocatable pool,
logical, intent(in) dev )

Lay the side out in its pool (peers in first-appearance order, transfers contiguous per peer); grows pool to the words in use.

Definition at line 658 of file m_amr_wave.fpp.f90.

Here is the caller graph for this function:

◆ s_amr_wave_open()

impure subroutine, public m_amr_wave::s_amr_wave_open ( type(t_amr_wave), intent(inout) w,
integer, intent(in) b )

Open a wave on tag band b.

Definition at line 415 of file m_amr_wave.fpp.f90.

Here is the call graph for this function:
Here is the caller graph for this function:

◆ s_amr_wave_peers()

impure subroutine m_amr_wave::s_amr_wave_peers ( type(t_amr_wave), intent(inout) w,
integer, intent(in) dir,
type(t_amr_wave_side), intent(in) s,
real(wp), dimension(:), intent(inout), contiguous pool,
logical, intent(in) dev )
private

One message per peer (dir = 1 sends, 2 receives) over the side's pool runs.

Definition at line 711 of file m_amr_wave.fpp.f90.

Here is the call graph for this function:
Here is the caller graph for this function:

◆ s_amr_wave_post()

impure subroutine, public m_amr_wave::s_amr_wave_post ( type(t_amr_wave), intent(inout) w,
type(t_amr_wave_side), intent(in) s,
real(wp), dimension(:), intent(inout), contiguous pool,
logical, intent(in) dev )

One receive per peer into the side's pool runs.

Definition at line 727 of file m_amr_wave.fpp.f90.

Here is the call graph for this function:
Here is the caller graph for this function:

◆ s_amr_wave_req()

impure subroutine, public m_amr_wave::s_amr_wave_req ( type(t_amr_wave), intent(inout) w,
integer, intent(in) dir,
real(wp), dimension(:), intent(inout), contiguous buf,
integer, intent(in) n,
integer, intent(in) peer,
logical, intent(in) dev )

Post one request of n words between buf (device-resident when dev) and peer: dir = 1 sends, 2 receives.

Definition at line 434 of file m_amr_wave.fpp.f90.

Here is the call graph for this function:
Here is the caller graph for this function:

◆ s_amr_wave_req_raw()

impure subroutine, public m_amr_wave::s_amr_wave_req_raw ( type(t_amr_wave), intent(inout) w,
integer, intent(in) dir,
real(wp), dimension(*), intent(inout) buf,
integer, intent(in) n,
integer, intent(in) peer )

Host-array variant of s_amr_wave_req (assumed-size, any-rank sections, no device address).

Definition at line 504 of file m_amr_wave.fpp.f90.

Here is the call graph for this function:
Here is the caller graph for this function:

◆ s_amr_wave_reset()

impure subroutine, public m_amr_wave::s_amr_wave_reset ( type(t_amr_wave_side), intent(inout) s)

Start enumerating a side's transfers.

Definition at line 621 of file m_amr_wave.fpp.f90.

Here is the caller graph for this function:

◆ s_amr_wave_send()

impure subroutine, public m_amr_wave::s_amr_wave_send ( type(t_amr_wave), intent(inout) w,
type(t_amr_wave_side), intent(in) s,
real(wp), dimension(:), intent(inout), contiguous pool,
logical, intent(in) dev )

One send per peer (the caller has packed every slice).

Definition at line 739 of file m_amr_wave.fpp.f90.

Here is the call graph for this function:
Here is the caller graph for this function:

◆ s_amr_wave_slice()

pure subroutine, public m_amr_wave::s_amr_wave_slice ( type(t_amr_wave_side), intent(in) s,
integer, intent(in) t,
integer, intent(out) lo,
integer, intent(out) hi )

Pool bounds [lo, hi] of transfer t's payload.

Definition at line 700 of file m_amr_wave.fpp.f90.

Here is the caller graph for this function:

◆ s_amr_wave_wait()

impure subroutine, public m_amr_wave::s_amr_wave_wait ( type(t_amr_wave), intent(inout) w)

Wait for all requests; under MFC_DEBUG each receive length must equal its plan.

Definition at line 588 of file m_amr_wave.fpp.f90.

Here is the caller graph for this function:

Variable Documentation

◆ amr_wave

type(t_amr_wave), target, public m_amr_wave::amr_wave

Definition at line 408 of file m_amr_wave.fpp.f90.

◆ amr_wave_seam

type(t_amr_wave), target, public m_amr_wave::amr_wave_seam

Definition at line 408 of file m_amr_wave.fpp.f90.

◆ amr_wrecv

type(t_amr_wave_side), public m_amr_wave::amr_wrecv

Definition at line 401 of file m_amr_wave.fpp.f90.

◆ amr_wseam_rcv

type(t_amr_wave_side), public m_amr_wave::amr_wseam_rcv

Definition at line 401 of file m_amr_wave.fpp.f90.

◆ amr_wseam_snd

type(t_amr_wave_side), public m_amr_wave::amr_wseam_snd

Definition at line 401 of file m_amr_wave.fpp.f90.

◆ amr_wsend

type(t_amr_wave_side), public m_amr_wave::amr_wsend

Fill and restrict waves share amr_wsend/amr_wrecv; the seam wave stays open across the fills, so it has its own pair.

Definition at line 401 of file m_amr_wave.fpp.f90.

◆ bmap

integer, dimension(:), allocatable m_amr_wave::bmap
private

Rank-indexed scratch for s_amr_wave_close; all-zero between calls.

Definition at line 410 of file m_amr_wave.fpp.f90.

◆ bnx

integer, dimension(:), allocatable m_amr_wave::bnx
private

Definition at line 410 of file m_amr_wave.fpp.f90.

◆ bwords

integer, dimension(:), allocatable m_amr_wave::bwords
private

Definition at line 410 of file m_amr_wave.fpp.f90.