MFC
Exascale flow solver
Loading...
Searching...
No Matches
m_start_up.fpp.f90
Go to the documentation of this file.
1# 1 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
2!>
3!! @file
4!! @brief Contains module m_start_up
5
6# 1 "/home/runner/work/MFC/MFC/src/common/include/case.fpp" 1
7! This file exists so that Fypp can be run without generating case.fpp files for
8! each target. This is useful when generating documentation, for example. This
9! should also let MFC be built with CMake directly, without invoking mfc.sh.
10
11! For pre-process.
12# 9 "/home/runner/work/MFC/MFC/src/common/include/case.fpp"
13
14! For moving immersed boundaries in simulation
15# 14 "/home/runner/work/MFC/MFC/src/common/include/case.fpp"
16# 6 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp" 2
17# 1 "/home/runner/work/MFC/MFC/src/common/include/macros.fpp" 1
18# 1 "/home/runner/work/MFC/MFC/src/common/include/parallel_macros.fpp" 1
19# 1 "/home/runner/work/MFC/MFC/src/common/include/shared_parallel_macros.fpp" 1
20# 2 "/home/runner/work/MFC/MFC/src/common/include/shared_parallel_macros.fpp"
21# 3 "/home/runner/work/MFC/MFC/src/common/include/shared_parallel_macros.fpp"
22# 4 "/home/runner/work/MFC/MFC/src/common/include/shared_parallel_macros.fpp"
23# 5 "/home/runner/work/MFC/MFC/src/common/include/shared_parallel_macros.fpp"
24# 6 "/home/runner/work/MFC/MFC/src/common/include/shared_parallel_macros.fpp"
25
26# 8 "/home/runner/work/MFC/MFC/src/common/include/shared_parallel_macros.fpp"
27# 9 "/home/runner/work/MFC/MFC/src/common/include/shared_parallel_macros.fpp"
28# 10 "/home/runner/work/MFC/MFC/src/common/include/shared_parallel_macros.fpp"
29
30# 17 "/home/runner/work/MFC/MFC/src/common/include/shared_parallel_macros.fpp"
31
32# 46 "/home/runner/work/MFC/MFC/src/common/include/shared_parallel_macros.fpp"
33
34# 58 "/home/runner/work/MFC/MFC/src/common/include/shared_parallel_macros.fpp"
35
36# 68 "/home/runner/work/MFC/MFC/src/common/include/shared_parallel_macros.fpp"
37
38# 98 "/home/runner/work/MFC/MFC/src/common/include/shared_parallel_macros.fpp"
39
40# 110 "/home/runner/work/MFC/MFC/src/common/include/shared_parallel_macros.fpp"
41
42# 120 "/home/runner/work/MFC/MFC/src/common/include/shared_parallel_macros.fpp"
43! New line at end of file is required for FYPP
44# 2 "/home/runner/work/MFC/MFC/src/common/include/parallel_macros.fpp" 2
45# 1 "/home/runner/work/MFC/MFC/src/common/include/omp_macros.fpp" 1
46# 1 "/home/runner/work/MFC/MFC/src/common/include/shared_parallel_macros.fpp" 1
47# 2 "/home/runner/work/MFC/MFC/src/common/include/shared_parallel_macros.fpp"
48# 3 "/home/runner/work/MFC/MFC/src/common/include/shared_parallel_macros.fpp"
49# 4 "/home/runner/work/MFC/MFC/src/common/include/shared_parallel_macros.fpp"
50# 5 "/home/runner/work/MFC/MFC/src/common/include/shared_parallel_macros.fpp"
51# 6 "/home/runner/work/MFC/MFC/src/common/include/shared_parallel_macros.fpp"
52
53# 8 "/home/runner/work/MFC/MFC/src/common/include/shared_parallel_macros.fpp"
54# 9 "/home/runner/work/MFC/MFC/src/common/include/shared_parallel_macros.fpp"
55# 10 "/home/runner/work/MFC/MFC/src/common/include/shared_parallel_macros.fpp"
56
57# 17 "/home/runner/work/MFC/MFC/src/common/include/shared_parallel_macros.fpp"
58
59# 46 "/home/runner/work/MFC/MFC/src/common/include/shared_parallel_macros.fpp"
60
61# 58 "/home/runner/work/MFC/MFC/src/common/include/shared_parallel_macros.fpp"
62
63# 68 "/home/runner/work/MFC/MFC/src/common/include/shared_parallel_macros.fpp"
64
65# 98 "/home/runner/work/MFC/MFC/src/common/include/shared_parallel_macros.fpp"
66
67# 110 "/home/runner/work/MFC/MFC/src/common/include/shared_parallel_macros.fpp"
68
69# 120 "/home/runner/work/MFC/MFC/src/common/include/shared_parallel_macros.fpp"
70! New line at end of file is required for FYPP
71# 2 "/home/runner/work/MFC/MFC/src/common/include/omp_macros.fpp" 2
72
73# 4 "/home/runner/work/MFC/MFC/src/common/include/omp_macros.fpp"
74# 5 "/home/runner/work/MFC/MFC/src/common/include/omp_macros.fpp"
75# 6 "/home/runner/work/MFC/MFC/src/common/include/omp_macros.fpp"
76# 7 "/home/runner/work/MFC/MFC/src/common/include/omp_macros.fpp"
77# 8 "/home/runner/work/MFC/MFC/src/common/include/omp_macros.fpp"
78
79# 20 "/home/runner/work/MFC/MFC/src/common/include/omp_macros.fpp"
80
81# 43 "/home/runner/work/MFC/MFC/src/common/include/omp_macros.fpp"
82
83# 48 "/home/runner/work/MFC/MFC/src/common/include/omp_macros.fpp"
84
85# 53 "/home/runner/work/MFC/MFC/src/common/include/omp_macros.fpp"
86
87# 58 "/home/runner/work/MFC/MFC/src/common/include/omp_macros.fpp"
88
89# 63 "/home/runner/work/MFC/MFC/src/common/include/omp_macros.fpp"
90
91# 68 "/home/runner/work/MFC/MFC/src/common/include/omp_macros.fpp"
92
93# 76 "/home/runner/work/MFC/MFC/src/common/include/omp_macros.fpp"
94
95# 81 "/home/runner/work/MFC/MFC/src/common/include/omp_macros.fpp"
96
97# 86 "/home/runner/work/MFC/MFC/src/common/include/omp_macros.fpp"
98
99# 91 "/home/runner/work/MFC/MFC/src/common/include/omp_macros.fpp"
100
101# 96 "/home/runner/work/MFC/MFC/src/common/include/omp_macros.fpp"
102
103# 101 "/home/runner/work/MFC/MFC/src/common/include/omp_macros.fpp"
104
105# 106 "/home/runner/work/MFC/MFC/src/common/include/omp_macros.fpp"
106
107# 111 "/home/runner/work/MFC/MFC/src/common/include/omp_macros.fpp"
108
109# 116 "/home/runner/work/MFC/MFC/src/common/include/omp_macros.fpp"
110
111# 121 "/home/runner/work/MFC/MFC/src/common/include/omp_macros.fpp"
112
113# 151 "/home/runner/work/MFC/MFC/src/common/include/omp_macros.fpp"
114
115# 192 "/home/runner/work/MFC/MFC/src/common/include/omp_macros.fpp"
116
117# 207 "/home/runner/work/MFC/MFC/src/common/include/omp_macros.fpp"
118
119# 232 "/home/runner/work/MFC/MFC/src/common/include/omp_macros.fpp"
120
121# 243 "/home/runner/work/MFC/MFC/src/common/include/omp_macros.fpp"
122
123# 245 "/home/runner/work/MFC/MFC/src/common/include/omp_macros.fpp"
124# 255 "/home/runner/work/MFC/MFC/src/common/include/omp_macros.fpp"
125
126# 283 "/home/runner/work/MFC/MFC/src/common/include/omp_macros.fpp"
127
128# 293 "/home/runner/work/MFC/MFC/src/common/include/omp_macros.fpp"
129
130# 303 "/home/runner/work/MFC/MFC/src/common/include/omp_macros.fpp"
131
132# 312 "/home/runner/work/MFC/MFC/src/common/include/omp_macros.fpp"
133
134# 329 "/home/runner/work/MFC/MFC/src/common/include/omp_macros.fpp"
135
136# 339 "/home/runner/work/MFC/MFC/src/common/include/omp_macros.fpp"
137
138# 346 "/home/runner/work/MFC/MFC/src/common/include/omp_macros.fpp"
139
140# 352 "/home/runner/work/MFC/MFC/src/common/include/omp_macros.fpp"
141
142# 358 "/home/runner/work/MFC/MFC/src/common/include/omp_macros.fpp"
143
144# 364 "/home/runner/work/MFC/MFC/src/common/include/omp_macros.fpp"
145
146# 370 "/home/runner/work/MFC/MFC/src/common/include/omp_macros.fpp"
147
148# 376 "/home/runner/work/MFC/MFC/src/common/include/omp_macros.fpp"
149! New line at end of file is required for FYPP
150# 3 "/home/runner/work/MFC/MFC/src/common/include/parallel_macros.fpp" 2
151# 1 "/home/runner/work/MFC/MFC/src/common/include/acc_macros.fpp" 1
152# 1 "/home/runner/work/MFC/MFC/src/common/include/shared_parallel_macros.fpp" 1
153# 2 "/home/runner/work/MFC/MFC/src/common/include/shared_parallel_macros.fpp"
154# 3 "/home/runner/work/MFC/MFC/src/common/include/shared_parallel_macros.fpp"
155# 4 "/home/runner/work/MFC/MFC/src/common/include/shared_parallel_macros.fpp"
156# 5 "/home/runner/work/MFC/MFC/src/common/include/shared_parallel_macros.fpp"
157# 6 "/home/runner/work/MFC/MFC/src/common/include/shared_parallel_macros.fpp"
158
159# 8 "/home/runner/work/MFC/MFC/src/common/include/shared_parallel_macros.fpp"
160# 9 "/home/runner/work/MFC/MFC/src/common/include/shared_parallel_macros.fpp"
161# 10 "/home/runner/work/MFC/MFC/src/common/include/shared_parallel_macros.fpp"
162
163# 17 "/home/runner/work/MFC/MFC/src/common/include/shared_parallel_macros.fpp"
164
165# 46 "/home/runner/work/MFC/MFC/src/common/include/shared_parallel_macros.fpp"
166
167# 58 "/home/runner/work/MFC/MFC/src/common/include/shared_parallel_macros.fpp"
168
169# 68 "/home/runner/work/MFC/MFC/src/common/include/shared_parallel_macros.fpp"
170
171# 98 "/home/runner/work/MFC/MFC/src/common/include/shared_parallel_macros.fpp"
172
173# 110 "/home/runner/work/MFC/MFC/src/common/include/shared_parallel_macros.fpp"
174
175# 120 "/home/runner/work/MFC/MFC/src/common/include/shared_parallel_macros.fpp"
176! New line at end of file is required for FYPP
177# 2 "/home/runner/work/MFC/MFC/src/common/include/acc_macros.fpp" 2
178
179# 7 "/home/runner/work/MFC/MFC/src/common/include/acc_macros.fpp"
180
181# 17 "/home/runner/work/MFC/MFC/src/common/include/acc_macros.fpp"
182
183# 22 "/home/runner/work/MFC/MFC/src/common/include/acc_macros.fpp"
184
185# 27 "/home/runner/work/MFC/MFC/src/common/include/acc_macros.fpp"
186
187# 32 "/home/runner/work/MFC/MFC/src/common/include/acc_macros.fpp"
188
189# 37 "/home/runner/work/MFC/MFC/src/common/include/acc_macros.fpp"
190
191# 42 "/home/runner/work/MFC/MFC/src/common/include/acc_macros.fpp"
192
193# 47 "/home/runner/work/MFC/MFC/src/common/include/acc_macros.fpp"
194
195# 52 "/home/runner/work/MFC/MFC/src/common/include/acc_macros.fpp"
196
197# 57 "/home/runner/work/MFC/MFC/src/common/include/acc_macros.fpp"
198
199# 62 "/home/runner/work/MFC/MFC/src/common/include/acc_macros.fpp"
200
201# 73 "/home/runner/work/MFC/MFC/src/common/include/acc_macros.fpp"
202
203# 78 "/home/runner/work/MFC/MFC/src/common/include/acc_macros.fpp"
204
205# 83 "/home/runner/work/MFC/MFC/src/common/include/acc_macros.fpp"
206
207# 88 "/home/runner/work/MFC/MFC/src/common/include/acc_macros.fpp"
208
209# 103 "/home/runner/work/MFC/MFC/src/common/include/acc_macros.fpp"
210
211# 131 "/home/runner/work/MFC/MFC/src/common/include/acc_macros.fpp"
212
213# 160 "/home/runner/work/MFC/MFC/src/common/include/acc_macros.fpp"
214
215# 175 "/home/runner/work/MFC/MFC/src/common/include/acc_macros.fpp"
216
217# 192 "/home/runner/work/MFC/MFC/src/common/include/acc_macros.fpp"
218
219# 213 "/home/runner/work/MFC/MFC/src/common/include/acc_macros.fpp"
220
221# 241 "/home/runner/work/MFC/MFC/src/common/include/acc_macros.fpp"
222
223# 256 "/home/runner/work/MFC/MFC/src/common/include/acc_macros.fpp"
224
225# 266 "/home/runner/work/MFC/MFC/src/common/include/acc_macros.fpp"
226
227# 275 "/home/runner/work/MFC/MFC/src/common/include/acc_macros.fpp"
228
229# 291 "/home/runner/work/MFC/MFC/src/common/include/acc_macros.fpp"
230
231# 301 "/home/runner/work/MFC/MFC/src/common/include/acc_macros.fpp"
232
233# 308 "/home/runner/work/MFC/MFC/src/common/include/acc_macros.fpp"
234! New line at end of file is required for FYPP
235# 4 "/home/runner/work/MFC/MFC/src/common/include/parallel_macros.fpp" 2
236
237# 21 "/home/runner/work/MFC/MFC/src/common/include/parallel_macros.fpp"
238
239# 37 "/home/runner/work/MFC/MFC/src/common/include/parallel_macros.fpp"
240
241# 50 "/home/runner/work/MFC/MFC/src/common/include/parallel_macros.fpp"
242
243# 76 "/home/runner/work/MFC/MFC/src/common/include/parallel_macros.fpp"
244
245# 91 "/home/runner/work/MFC/MFC/src/common/include/parallel_macros.fpp"
246
247# 102 "/home/runner/work/MFC/MFC/src/common/include/parallel_macros.fpp"
248
249# 115 "/home/runner/work/MFC/MFC/src/common/include/parallel_macros.fpp"
250
251# 143 "/home/runner/work/MFC/MFC/src/common/include/parallel_macros.fpp"
252
253# 154 "/home/runner/work/MFC/MFC/src/common/include/parallel_macros.fpp"
254
255# 165 "/home/runner/work/MFC/MFC/src/common/include/parallel_macros.fpp"
256
257# 176 "/home/runner/work/MFC/MFC/src/common/include/parallel_macros.fpp"
258
259# 187 "/home/runner/work/MFC/MFC/src/common/include/parallel_macros.fpp"
260
261# 198 "/home/runner/work/MFC/MFC/src/common/include/parallel_macros.fpp"
262
263# 208 "/home/runner/work/MFC/MFC/src/common/include/parallel_macros.fpp"
264
265# 214 "/home/runner/work/MFC/MFC/src/common/include/parallel_macros.fpp"
266
267# 220 "/home/runner/work/MFC/MFC/src/common/include/parallel_macros.fpp"
268
269# 226 "/home/runner/work/MFC/MFC/src/common/include/parallel_macros.fpp"
270
271# 232 "/home/runner/work/MFC/MFC/src/common/include/parallel_macros.fpp"
272
273# 234 "/home/runner/work/MFC/MFC/src/common/include/parallel_macros.fpp"
274# 235 "/home/runner/work/MFC/MFC/src/common/include/parallel_macros.fpp"
275! New line at end of file is required for FYPP
276# 2 "/home/runner/work/MFC/MFC/src/common/include/macros.fpp" 2
277
278# 14 "/home/runner/work/MFC/MFC/src/common/include/macros.fpp"
279
280! Caution:
281! This macro requires the use of a binding script to set CUDA_VISIBLE_DEVICES, such that we have one GPU device per MPI rank.
282! That's because for both cudaMemAdvise (preferred location) and cudaMemPrefetchAsync we use location = device_id = 0.
283! For an example see misc/nvidia_uvm/bind.sh.
284# 63 "/home/runner/work/MFC/MFC/src/common/include/macros.fpp"
285
286# 81 "/home/runner/work/MFC/MFC/src/common/include/macros.fpp"
287
288# 88 "/home/runner/work/MFC/MFC/src/common/include/macros.fpp"
289
290# 111 "/home/runner/work/MFC/MFC/src/common/include/macros.fpp"
291
292# 127 "/home/runner/work/MFC/MFC/src/common/include/macros.fpp"
293
294# 153 "/home/runner/work/MFC/MFC/src/common/include/macros.fpp"
295
296# 159 "/home/runner/work/MFC/MFC/src/common/include/macros.fpp"
297
298# 167 "/home/runner/work/MFC/MFC/src/common/include/macros.fpp"
299! New line at end of file is required for FYPP
300# 7 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp" 2
301
302!> @brief Reads input files, loads initial conditions and grid data, and orchestrates solver initialization and finalization
304
305 use m_derived_types !< definitions of the derived types
306
307 use m_global_parameters !< definitions of the global parameters
308
309 use m_mpi_proxy !< message passing interface (mpi) module proxy
310
311 use m_mpi_common
312
313 use m_variables_conversion !< state variables type conversion procedures
314
315 use m_weno !< weighted and essentially non-oscillatory (weno)
316 !! schemes for spatial reconstruction of variables
317
318 use m_muscl !< monotonic upstream-centered (muscl)
319 !! schemes for convservation laws
320
321 use m_riemann_solvers !< exact and approximate riemann problem solvers
322
323 use m_cbc !< characteristic boundary conditions (cbc)
324
326
327 use m_acoustic_src !< acoustic source calculations
328
329 use m_rhs !< right-hane-side (rhs) evaluation procedures
330
331 use m_chemistry !< chemistry module
332
333 use m_data_output !< run-time info & solution data output procedures
334
335 use m_time_steppers !< time-stepping algorithms
336
337 use m_qbmm !< quadrature mom
338
339 use m_derived_variables !< procedures used to compute quantities derived
340 !! from the conservative and primitive variables
341 use m_hypoelastic
342
344
345 use m_phase_change !< phase-change module
346
347 use m_viscous
348
349 use m_bubbles_ee !< ensemble-averaged bubble dynamics routines
350
351 use m_bubbles_el !< lagrange bubble dynamics routines
352
353 use ieee_arithmetic
354
355 use m_helper_basic !< functions to compare floating point numbers
356
357 use m_helper
358
359
360# 65 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
361#if defined(MFC_OpenACC)
362# 65 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
363 use openacc
364# 65 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
365#elif defined(MFC_OpenMP)
366# 65 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
367 use omp_lib
368# 65 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
369#endif
370# 65 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
371
372
373 use m_nvtx
374
375 use m_ibm
376
378
380
381 use m_checker
382
384
385 use m_body_forces
386
387 use m_sim_helpers
388
389 use m_igr
390
391 implicit none
392
393 private; public :: s_read_input_file, &
403
404 type(scalar_field), allocatable, dimension(:) :: q_cons_temp
405
406 real(wp) :: dt_init
407
408contains
409
410 !> Read data files. Dispatch subroutine that replaces procedure pointer.
411 !! @param q_cons_vf Conservative variables
412 impure subroutine s_read_data_files(q_cons_vf)
413
414 type(scalar_field), &
415 dimension(sys_size), &
416 intent(inout) :: q_cons_vf
417
418 if (.not. parallel_io) then
420 else
422 end if
423
424 end subroutine s_read_data_files
425
426 !> The purpose of this procedure is to first verify that an
427 !! input file has been made available by the user. Provided
428 !! that this is so, the input file is then read in.
429 impure subroutine s_read_input_file
430
431 ! Relative path to the input file provided by the user
432 character(LEN=name_len), parameter :: file_path = './simulation.inp'
433
434 logical :: file_exist !<
435 !! Logical used to check the existence of the input file
436
437 integer :: iostatus
438 !! Integer to check iostat of file read
439
440 character(len=1000) :: line
441
442 ! Namelist of the global parameters which may be specified by user
443 namelist /user_inputs/ case_dir, run_time_info, m, n, p, dt, &
448 bc_x, bc_y, bc_z, &
449 x_a, y_a, z_a, x_b, y_b, z_b, &
452 ib, num_ibs, patch_ib, &
459# 154 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
464# 159 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
465 ca, web, re_inv, &
474 bf_x, bf_y, bf_z, &
475 k_x, k_y, k_z, w_x, w_y, w_z, p_x, p_y, p_z, &
485
486 ! Checking that an input file has been provided by the user. If it
487 ! has, then the input file is read in, otherwise, simulation exits.
488 inquire (file=trim(file_path), exist=file_exist)
489
490 if (file_exist) then
491 open (1, file=trim(file_path), &
492 form='formatted', &
493 action='read', &
494 status='old')
495 read (1, nml=user_inputs, iostat=iostatus)
496
497 if (iostatus /= 0) then
498 backspace(1)
499 read (1, fmt='(A)') line
500 print *, 'Invalid line in namelist: '//trim(line)
501 call s_mpi_abort('Invalid line in simulation.inp. It is '// &
502 'likely due to a datatype mismatch. Exiting.')
503 end if
504
505 close (1)
506
507 if ((bf_x) .or. (bf_y) .or. (bf_z)) then
508 bodyforces = .true.
509 end if
510
511 ! Store m,n,p into global m,n,p
512 m_glb = m
513 n_glb = n
514 p_glb = p
515
517
518 if (cfl_adap_dt .or. cfl_const_dt) cfl_dt = .true.
519
520 if (any((/bc_x%beg, bc_x%end, bc_y%beg, bc_y%end, bc_z%beg, bc_z%end/) == -17) .or. &
521 num_bc_patches > 0) then
522 bc_io = .true.
523 end if
524
525 else
526 call s_mpi_abort(trim(file_path)//' is missing. Exiting.')
527 end if
528
529 end subroutine s_read_input_file
530
531 !> The goal of this procedure is to verify that each of the
532 !! user provided inputs is valid and that their combination
533 !! constitutes a meaningful configuration for the simulation.
534 impure subroutine s_check_input_file
535
536 ! Relative path to the current directory file in the case directory
537 character(LEN=path_len) :: file_path
538
539 ! Logical used to check the existence of the current directory file
540 logical :: file_exist
541
542 ! Logistics
543 file_path = trim(case_dir)//'/.'
544
545 call my_inquire(file_path, file_exist)
546
547 if (file_exist .neqv. .true.) then
548 call s_mpi_abort(trim(file_path)//' is missing. Exiting.')
549 end if
550
552 call s_check_inputs()
553
554 end subroutine s_check_input_file
555
556 !> @brief Reads serial initial condition and grid data files and computes cell-width distributions.
557 !! @param q_cons_vf Cell-averaged conservative variables
558 impure subroutine s_read_serial_data_files(q_cons_vf)
559
560 type(scalar_field), dimension(sys_size), intent(INOUT) :: q_cons_vf
561
562 character(LEN=path_len + 2*name_len) :: t_step_dir !<
563 !! Relative path to the starting time-step directory
564
565 character(LEN=path_len + 3*name_len) :: file_path !<
566 !! Relative path to the grid and conservative variables data files
567
568 logical :: file_exist !<
569 ! Logical used to check the existence of the data files
570
571 integer :: i, r !< Generic loop iterator
572
573 ! Confirming that the directory from which the initial condition and
574 ! the grid data files are to be read in exists and exiting otherwise
575 if (cfl_dt) then
576 write (t_step_dir, '(A,I0,A,I0)') &
577 trim(case_dir)//'/p_all/p', proc_rank, '/', n_start
578 else
579 write (t_step_dir, '(A,I0,A,I0)') &
580 trim(case_dir)//'/p_all/p', proc_rank, '/', t_step_start
581 end if
582
583 file_path = trim(t_step_dir)//'/.'
584 call my_inquire(file_path, file_exist)
585
586 if (file_exist .neqv. .true.) then
587 call s_mpi_abort(trim(file_path)//' is missing. Exiting.')
588 end if
589
590 if (bc_io) then
592 else
594 end if
595
596 ! Cell-boundary Locations in x-direction
597 file_path = trim(t_step_dir)//'/x_cb.dat'
598
599 inquire (file=trim(file_path), exist=file_exist)
600
601 if (file_exist) then
602 open (2, file=trim(file_path), &
603 form='unformatted', &
604 action='read', &
605 status='old')
606 read (2) x_cb(-1:m); close (2)
607 else
608 call s_mpi_abort(trim(file_path)//' is missing. Exiting.')
609 end if
610
611 dx(0:m) = x_cb(0:m) - x_cb(-1:m - 1)
612 x_cc(0:m) = x_cb(-1:m - 1) + dx(0:m)/2._wp
613
614 if (ib) then
615 do i = 1, num_ibs
616 if (patch_ib(i)%c > 0) then
617 np = int((patch_ib(i)%p*patch_ib(i)%c/dx(0))*20) + int(((patch_ib(i)%c - patch_ib(i)%p*patch_ib(i)%c)/dx(0))*20) + 1
618 end if
619 end do
620 end if
621
622 ! Cell-boundary Locations in y-direction
623 if (n > 0) then
624
625 file_path = trim(t_step_dir)//'/y_cb.dat'
626
627 inquire (file=trim(file_path), exist=file_exist)
628
629 if (file_exist) then
630 open (2, file=trim(file_path), &
631 form='unformatted', &
632 action='read', &
633 status='old')
634 read (2) y_cb(-1:n); close (2)
635 else
636 call s_mpi_abort(trim(file_path)//' is missing. Exiting.')
637 end if
638
639 dy(0:n) = y_cb(0:n) - y_cb(-1:n - 1)
640 y_cc(0:n) = y_cb(-1:n - 1) + dy(0:n)/2._wp
641
642 end if
643
644 ! Cell-boundary Locations in z-direction
645 if (p > 0) then
646
647 file_path = trim(t_step_dir)//'/z_cb.dat'
648
649 inquire (file=trim(file_path), exist=file_exist)
650
651 if (file_exist) then
652 open (2, file=trim(file_path), &
653 form='unformatted', &
654 action='read', &
655 status='old')
656 read (2) z_cb(-1:p); close (2)
657 else
658 call s_mpi_abort(trim(file_path)//' is missing. Exiting.')
659 end if
660
661 dz(0:p) = z_cb(0:p) - z_cb(-1:p - 1)
662 z_cc(0:p) = z_cb(-1:p - 1) + dz(0:p)/2._wp
663
664 end if
665
666 do i = 1, sys_size
667 write (file_path, '(A,I0,A)') &
668 trim(t_step_dir)//'/q_cons_vf', i, '.dat'
669 inquire (file=trim(file_path), exist=file_exist)
670 if (file_exist) then
671 open (2, file=trim(file_path), &
672 form='unformatted', &
673 action='read', &
674 status='old')
675 read (2) q_cons_vf(i)%sf(0:m, 0:n, 0:p); close (2)
676 else
677 call s_mpi_abort(trim(file_path)//' is missing. Exiting.')
678 end if
679 end do
680
681 if (bubbles_euler .or. elasticity) then
682 ! Read pb and mv for non-polytropic qbmm
683 if (qbmm .and. .not. polytropic) then
684 do i = 1, nb
685 do r = 1, nnode
686 write (file_path, '(A,I0,A)') &
687 trim(t_step_dir)//'/pb', sys_size + (i - 1)*nnode + r, '.dat'
688 inquire (file=trim(file_path), exist=file_exist)
689 if (file_exist) then
690 open (2, file=trim(file_path), &
691 form='unformatted', &
692 action='read', &
693 status='old')
694 read (2) pb_ts(1)%sf(0:m, 0:n, 0:p, r, i); close (2)
695 else
696 call s_mpi_abort(trim(file_path)//' is missing. Exiting.')
697 end if
698 end do
699 end do
700 do i = 1, nb
701 do r = 1, nnode
702 write (file_path, '(A,I0,A)') &
703 trim(t_step_dir)//'/mv', sys_size + (i - 1)*nnode + r, '.dat'
704 inquire (file=trim(file_path), exist=file_exist)
705 if (file_exist) then
706 open (2, file=trim(file_path), &
707 form='unformatted', &
708 action='read', &
709 status='old')
710 read (2) mv_ts(1)%sf(0:m, 0:n, 0:p, r, i); close (2)
711 else
712 call s_mpi_abort(trim(file_path)//' is missing. Exiting.')
713 end if
714 end do
715 end do
716 end if
717 end if
718
719 end subroutine s_read_serial_data_files
720
721 !> @brief Reads parallel initial condition and grid data files via MPI I/O.
722 !! @param q_cons_vf Conservative variables
723 impure subroutine s_read_parallel_data_files(q_cons_vf)
724
725 type(scalar_field), &
726 dimension(sys_size), &
727 intent(INOUT) :: q_cons_vf
728
729#ifdef MFC_MPI
730
731 real(wp), allocatable, dimension(:) :: x_cb_glb, y_cb_glb, z_cb_glb
732
733 integer :: ifile, ierr, data_size
734 integer, dimension(MPI_STATUS_SIZE) :: status
735 integer(KIND=MPI_OFFSET_KIND) :: disp
736 integer(KIND=MPI_OFFSET_KIND) :: m_mok, n_mok, p_mok
737 integer(KIND=MPI_OFFSET_KIND) :: wp_mok, var_mok, str_mok
738 integer(KIND=MPI_OFFSET_KIND) :: nvars_mok
739 integer(KIND=MPI_OFFSET_KIND) :: mok
740
741 character(LEN=path_len + 2*name_len) :: file_loc
742 logical :: file_exist
743
744 character(len=10) :: t_step_start_string
745
746 integer :: i, j
747
748 ! Downsampled data variables
749 integer :: m_ds, n_ds, p_ds
750 integer :: m_glb_ds, n_glb_ds, p_glb_ds
751 integer :: m_glb_read, n_glb_read, p_glb_read ! data size of read
752
753 allocate (x_cb_glb(-1:m_glb))
754 allocate (y_cb_glb(-1:n_glb))
755 allocate (z_cb_glb(-1:p_glb))
756
757 ! Read in cell boundary locations in x-direction
758 file_loc = trim(case_dir)//'/restart_data'//trim(mpiiofs)//'x_cb.dat'
759 inquire (file=trim(file_loc), exist=file_exist)
760
761 if (down_sample) then
762 m_ds = int((m + 1)/3) - 1
763 n_ds = int((n + 1)/3) - 1
764 p_ds = int((p + 1)/3) - 1
765
766 m_glb_ds = int((m_glb + 1)/3) - 1
767 n_glb_ds = int((n_glb + 1)/3) - 1
768 p_glb_ds = int((p_glb + 1)/3) - 1
769 end if
770
771 if (file_exist) then
772 data_size = m_glb + 2
773 call mpi_file_open(mpi_comm_world, file_loc, mpi_mode_rdonly, mpi_info_int, ifile, ierr)
774 call mpi_file_read(ifile, x_cb_glb, data_size, mpi_p, status, ierr)
775 call mpi_file_close(ifile, ierr)
776 else
777 call s_mpi_abort('File '//trim(file_loc)//' is missing. Exiting.')
778 end if
779
780 ! Assigning local cell boundary locations
781 x_cb(-1:m) = x_cb_glb((start_idx(1) - 1):(start_idx(1) + m))
782 ! Computing the cell width distribution
783 dx(0:m) = x_cb(0:m) - x_cb(-1:m - 1)
784 ! Computing the cell center locations
785 x_cc(0:m) = x_cb(-1:m - 1) + dx(0:m)/2._wp
786
787 if (ib) then
788 do i = 1, num_ibs
789 if (patch_ib(i)%c > 0) then
790 np = int((patch_ib(i)%p*patch_ib(i)%c/dx(0))*20) + int(((patch_ib(i)%c - patch_ib(i)%p*patch_ib(i)%c)/dx(0))*20) + 1
791 allocate (mpi_io_airfoil_ib_data%var(1:2*np))
792 end if
793 end do
794 end if
795
796 if (n > 0) then
797 ! Read in cell boundary locations in y-direction
798 file_loc = trim(case_dir)//'/restart_data'//trim(mpiiofs)//'y_cb.dat'
799 inquire (file=trim(file_loc), exist=file_exist)
800
801 if (file_exist) then
802 data_size = n_glb + 2
803 call mpi_file_open(mpi_comm_world, file_loc, mpi_mode_rdonly, mpi_info_int, ifile, ierr)
804 call mpi_file_read(ifile, y_cb_glb, data_size, mpi_p, status, ierr)
805 call mpi_file_close(ifile, ierr)
806 else
807 call s_mpi_abort('File '//trim(file_loc)//' is missing. Exiting.')
808 end if
809
810 ! Assigning local cell boundary locations
811 y_cb(-1:n) = y_cb_glb((start_idx(2) - 1):(start_idx(2) + n))
812 ! Computing the cell width distribution
813 dy(0:n) = y_cb(0:n) - y_cb(-1:n - 1)
814 ! Computing the cell center locations
815 y_cc(0:n) = y_cb(-1:n - 1) + dy(0:n)/2._wp
816
817 if (p > 0) then
818 ! Read in cell boundary locations in z-direction
819 file_loc = trim(case_dir)//'/restart_data'//trim(mpiiofs)//'z_cb.dat'
820 inquire (file=trim(file_loc), exist=file_exist)
821
822 if (file_exist) then
823 data_size = p_glb + 2
824 call mpi_file_open(mpi_comm_world, file_loc, mpi_mode_rdonly, mpi_info_int, ifile, ierr)
825 call mpi_file_read(ifile, z_cb_glb, data_size, mpi_p, status, ierr)
826 call mpi_file_close(ifile, ierr)
827 else
828 call s_mpi_abort('File '//trim(file_loc)//'is missing. Exiting.')
829 end if
830
831 ! Assigning local cell boundary locations
832 z_cb(-1:p) = z_cb_glb((start_idx(3) - 1):(start_idx(3) + p))
833 ! Computing the cell width distribution
834 dz(0:p) = z_cb(0:p) - z_cb(-1:p - 1)
835 ! Computing the cell center locations
836 z_cc(0:p) = z_cb(-1:p - 1) + dz(0:p)/2._wp
837
838 end if
839 end if
840
841 if (file_per_process) then
842 if (cfl_dt) then
843 call s_int_to_str(n_start, t_step_start_string)
844 write (file_loc, '(I0,A1,I7.7,A)') n_start, '_', proc_rank, '.dat'
845 else
846 call s_int_to_str(t_step_start, t_step_start_string)
847 write (file_loc, '(I0,A1,I7.7,A)') t_step_start, '_', proc_rank, '.dat'
848 end if
849 file_loc = trim(case_dir)//'/restart_data/lustre_'//trim(t_step_start_string)//trim(mpiiofs)//trim(file_loc)
850 inquire (file=trim(file_loc), exist=file_exist)
851
852 if (file_exist) then
853 call mpi_file_open(mpi_comm_self, file_loc, mpi_mode_rdonly, mpi_info_int, ifile, ierr)
854
855 ! Initialize MPI data I/O
856 if (down_sample) then
858 else
859 if (ib) then
861 else
863 end if
864 end if
865
866 if (down_sample) then
867 ! Size of local arrays
868 data_size = (m_ds + 3)*(n_ds + 3)*(p_ds + 3)
869 m_glb_read = m_glb_ds + 1
870 n_glb_read = n_glb_ds + 1
871 p_glb_read = p_glb_ds + 1
872 else
873 ! Size of local arrays
874 data_size = (m + 1)*(n + 1)*(p + 1)
875 m_glb_read = m_glb + 1
876 n_glb_read = n_glb + 1
877 p_glb_read = p_glb + 1
878 end if
879
880 ! Resize some integers so MPI can read even the biggest file
881 m_mok = int(m_glb_read + 1, mpi_offset_kind)
882 n_mok = int(m_glb_read + 1, mpi_offset_kind)
883 p_mok = int(m_glb_read + 1, mpi_offset_kind)
884 wp_mok = int(4._wp, mpi_offset_kind)
885 mok = int(1._wp, mpi_offset_kind)
886 str_mok = int(name_len, mpi_offset_kind)
887 nvars_mok = int(sys_size, mpi_offset_kind)
888
889 ! Read the data for each variable
890 if (bubbles_euler .or. elasticity) then
891 do i = 1, sys_size!adv_idx%end
892 var_mok = int(i, mpi_offset_kind)
893
894 call mpi_file_read(ifile, mpi_io_data%var(i)%sf, data_size*mpi_io_type, &
895 mpi_io_p, status, ierr)
896 end do
897 !Read pb and mv for non-polytropic qbmm
898 if (qbmm .and. .not. polytropic) then
899 do i = sys_size + 1, sys_size + 2*nb*nnode
900 var_mok = int(i, mpi_offset_kind)
901
902 call mpi_file_read(ifile, mpi_io_data%var(i)%sf, data_size*mpi_io_type, &
903 mpi_io_p, status, ierr)
904 end do
905 end if
906 else
907 if (down_sample) then
908 do i = 1, sys_size
909 var_mok = int(i, mpi_offset_kind)
910
911 call mpi_file_read(ifile, q_cons_temp(i)%sf, data_size*mpi_io_type, &
912 mpi_io_p, status, ierr)
913 end do
914 else
915 do i = 1, sys_size
916 var_mok = int(i, mpi_offset_kind)
917
918 call mpi_file_read(ifile, mpi_io_data%var(i)%sf, data_size*mpi_io_type, &
919 mpi_io_p, status, ierr)
920 end do
921 end if
922 end if
923
924 call s_mpi_barrier()
925
926 call mpi_file_close(ifile, ierr)
927
928 else
929 call s_mpi_abort('File '//trim(file_loc)//' is missing. Exiting.')
930 end if
931 else
932 ! Open the file to read conservative variables
933 if (cfl_dt) then
934 write (file_loc, '(I0,A)') n_start, '.dat'
935 else
936 write (file_loc, '(I0,A)') t_step_start, '.dat'
937 end if
938 file_loc = trim(case_dir)//'/restart_data'//trim(mpiiofs)//trim(file_loc)
939 inquire (file=trim(file_loc), exist=file_exist)
940
941 if (file_exist) then
942 call mpi_file_open(mpi_comm_world, file_loc, mpi_mode_rdonly, mpi_info_int, ifile, ierr)
943
944 ! Initialize MPI data I/O
945
946 if (ib) then
948 else
949
951
952 end if
953
954 ! Size of local arrays
955 data_size = (m + 1)*(n + 1)*(p + 1)
956
957 ! Resize some integers so MPI can read even the biggest file
958 m_mok = int(m_glb + 1, mpi_offset_kind)
959 n_mok = int(n_glb + 1, mpi_offset_kind)
960 p_mok = int(p_glb + 1, mpi_offset_kind)
961 wp_mok = int(8._wp, mpi_offset_kind)
962 mok = int(1._wp, mpi_offset_kind)
963 str_mok = int(name_len, mpi_offset_kind)
964 nvars_mok = int(sys_size, mpi_offset_kind)
965
966 ! Read the data for each variable
967 if (bubbles_euler .or. elasticity) then
968 do i = 1, sys_size !adv_idx%end
969 var_mok = int(i, mpi_offset_kind)
970 ! Initial displacement to skip at beginning of file
971 disp = m_mok*max(mok, n_mok)*max(mok, p_mok)*wp_mok*(var_mok - 1)
972
973 call mpi_file_set_view(ifile, disp, mpi_io_p, mpi_io_data%view(i), &
974 'native', mpi_info_int, ierr)
975 call mpi_file_read(ifile, mpi_io_data%var(i)%sf, data_size*mpi_io_type, &
976 mpi_io_p, status, ierr)
977 end do
978 !Read pb and mv for non-polytropic qbmm
979 if (qbmm .and. .not. polytropic) then
980 do i = sys_size + 1, sys_size + 2*nb*nnode
981 var_mok = int(i, mpi_offset_kind)
982 ! Initial displacement to skip at beginning of file
983 disp = m_mok*max(mok, n_mok)*max(mok, p_mok)*wp_mok*(var_mok - 1)
984
985 call mpi_file_set_view(ifile, disp, mpi_io_p, mpi_io_data%view(i), &
986 'native', mpi_info_int, ierr)
987 call mpi_file_read(ifile, mpi_io_data%var(i)%sf, data_size*mpi_io_type, &
988 mpi_io_p, status, ierr)
989 end do
990 end if
991 else
992 do i = 1, sys_size
993 var_mok = int(i, mpi_offset_kind)
994
995 ! Initial displacement to skip at beginning of file
996 disp = m_mok*max(mok, n_mok)*max(mok, p_mok)*wp_mok*(var_mok - 1)
997
998 call mpi_file_set_view(ifile, disp, mpi_io_p, mpi_io_data%view(i), &
999 'native', mpi_info_int, ierr)
1000 call mpi_file_read_all(ifile, mpi_io_data%var(i)%sf, data_size*mpi_io_type, &
1001 mpi_io_p, status, ierr)
1002 end do
1003 end if
1004
1005 call s_mpi_barrier()
1006
1007 call mpi_file_close(ifile, ierr)
1008
1009 else
1010 call s_mpi_abort('File '//trim(file_loc)//' is missing. Exiting.')
1011 end if
1012
1013 end if
1014
1015 deallocate (x_cb_glb, y_cb_glb, z_cb_glb)
1016
1017 if (bc_io) then
1019 else
1021 end if
1022
1023#endif
1024
1025 end subroutine s_read_parallel_data_files
1026
1027 !> The purpose of this procedure is to initialize the
1028 !! values of the internal-energy equations of each phase
1029 !! from the mass of each phase, the mixture momentum and
1030 !! mixture-total-energy equations.
1031 !! @param v_vf conservative variables
1033
1034 type(scalar_field), dimension(sys_size), intent(inout) :: v_vf
1035
1036 real(wp) :: rho
1037 real(wp) :: dyn_pres
1038 real(wp) :: gamma
1039 real(wp) :: pi_inf
1040 real(wp) :: qv
1041 real(wp), dimension(2) :: re
1042 real(wp) :: pres, t
1043
1044 integer :: i, j, k, l, c
1045
1046 real(wp), dimension(num_species) :: rhoyks
1047
1048 real(wp) :: pres_mag
1049
1050 pres_mag = 0._wp
1051
1052 t = dflt_t_guess
1053
1054 do j = 0, m
1055 do k = 0, n
1056 do l = 0, p
1057
1058 call s_convert_to_mixture_variables(v_vf, j, k, l, rho, gamma, pi_inf, qv, re)
1059
1060 dyn_pres = 0._wp
1061 do i = mom_idx%beg, mom_idx%end
1062 dyn_pres = dyn_pres + 5.e-1_wp*v_vf(i)%sf(j, k, l)*v_vf(i)%sf(j, k, l) &
1063 /max(rho, sgm_eps)
1064 end do
1065
1066 if (chemistry) then
1067 do c = 1, num_species
1068 rhoyks(c) = v_vf(chemxb + c - 1)%sf(j, k, l)
1069 end do
1070 end if
1071
1072 if (mhd) then
1073 if (n == 0) then
1074 pres_mag = 0.5_wp*(bx0**2 + v_vf(b_idx%beg)%sf(j, k, l)**2 + v_vf(b_idx%beg + 1)%sf(j, k, l)**2)
1075 else
1076 pres_mag = 0.5_wp*(v_vf(b_idx%beg)%sf(j, k, l)**2 + v_vf(b_idx%beg + 1)%sf(j, k, l)**2 + v_vf(b_idx%beg + 2)%sf(j, k, l)**2)
1077 end if
1078 end if
1079
1080 call s_compute_pressure(v_vf(e_idx)%sf(j, k, l), 0._stp, &
1081 dyn_pres, pi_inf, gamma, rho, qv, rhoyks, pres, t, pres_mag=pres_mag)
1082
1083 do i = 1, num_fluids
1084 v_vf(i + intxb - 1)%sf(j, k, l) = v_vf(i + advxb - 1)%sf(j, k, l)*(gammas(i)*pres + pi_infs(i)) &
1085 + v_vf(i + contxb - 1)%sf(j, k, l)*qvs(i)
1086 end do
1087
1088 end do
1089 end do
1090 end do
1091
1093
1094 !> @brief Advances the simulation by one time step, handling CFL-based dt and time-stepper dispatch.
1095 impure subroutine s_perform_time_step(t_step, time_avg)
1096 integer, intent(inout) :: t_step
1097 real(wp), intent(inout) :: time_avg
1098
1099 integer :: i
1100
1101 if (cfl_dt) then
1102 if (cfl_const_dt .and. t_step == 0) call s_compute_dt()
1103
1104 if (cfl_adap_dt) call s_compute_dt()
1105
1106 if (t_step == 0) dt_init = dt
1107
1108 if (dt < 1.e-3_wp*dt_init .and. cfl_adap_dt .and. proc_rank == 0) then
1109 print *, "Delta t = ", dt
1110 call s_mpi_abort("Delta t has become too small")
1111 end if
1112 end if
1113
1114 if (cfl_dt) then
1115 if ((mytime + dt) >= t_stop) then
1116 dt = t_stop - mytime
1117
1118# 811 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1119#if defined(MFC_OpenACC)
1120# 811 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1121!$acc update device(dt)
1122# 811 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1123#elif defined(MFC_OpenMP)
1124# 811 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1125!$omp target update to(dt)
1126# 811 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1127#endif
1128 end if
1129 else
1130 if ((mytime + dt) >= finaltime) then
1131 dt = finaltime - mytime
1132
1133# 816 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1134#if defined(MFC_OpenACC)
1135# 816 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1136!$acc update device(dt)
1137# 816 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1138#elif defined(MFC_OpenMP)
1139# 816 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1140!$omp target update to(dt)
1141# 816 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1142#endif
1143 end if
1144 end if
1145
1146 if (cfl_dt) then
1147 if (proc_rank == 0 .and. mod(t_step - t_step_start, t_step_print) == 0) then
1148 print '(" [", I3, "%] Time ", ES16.6, " dt = ", ES16.6, " @ Time Step = ", I8, " Time Avg = ", ES16.6, " Time/step = ", ES12.6, "")', &
1149 int(ceiling(100._wp*(mytime/t_stop))), &
1150 mytime, &
1151 dt, &
1152 t_step, &
1153 wall_time_avg, &
1154 wall_time
1155 end if
1156 else
1157 if (proc_rank == 0 .and. mod(t_step - t_step_start, t_step_print) == 0) then
1158 print '(" [", I3, "%] Time step ", I8, " of ", I0, " @ t_step = ", I8, " Time Avg = ", ES12.6, " Time/step= ", ES12.6, "")', &
1159 int(ceiling(100._wp*(real(t_step - t_step_start)/(t_step_stop - t_step_start + 1)))), &
1160 t_step - t_step_start + 1, &
1161 t_step_stop - t_step_start + 1, &
1162 t_step, &
1163 wall_time_avg, &
1164 wall_time
1165 end if
1166 end if
1167
1168 if (probe_wrt) then
1169 do i = 1, sys_size
1170
1171# 844 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1172#if defined(MFC_OpenACC)
1173# 844 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1174!$acc update host(q_cons_ts(1)%vf(i)%sf)
1175# 844 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1176#elif defined(MFC_OpenMP)
1177# 844 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1178!$omp target update from(q_cons_ts(1)%vf(i)%sf)
1179# 844 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1180#endif
1181 end do
1182 end if
1183
1184 mytime = mytime + dt
1185
1186 ! Total-variation-diminishing (TVD) Runge-Kutta (RK) time-steppers
1187 if (any(time_stepper == (/1, 2, 3/))) then
1188 call s_tvd_rk(t_step, time_avg, time_stepper)
1189 end if
1190
1191 if (relax) call s_infinite_relaxation_k(q_cons_ts(1)%vf)
1192
1193 ! Time-stepping loop controls
1194 t_step = t_step + 1
1195
1196 end subroutine s_perform_time_step
1197
1198 !> @brief Collects per-process wall-clock times and writes aggregate performance metrics to file.
1199 impure subroutine s_save_performance_metrics(time_avg, time_final, io_time_avg, io_time_final, proc_time, io_proc_time, file_exists)
1200
1201 real(wp), intent(inout) :: time_avg, time_final
1202 real(wp), intent(inout) :: io_time_avg, io_time_final
1203 real(wp), dimension(:), intent(inout) :: proc_time
1204 real(wp), dimension(:), intent(inout) :: io_proc_time
1205 logical, intent(inout) :: file_exists
1206
1207 real(wp) :: grind_time
1208
1209 call s_mpi_barrier()
1210
1211 if (num_procs > 1) then
1212 call mpi_bcast_time_step_values(proc_time, time_avg)
1213
1214 call mpi_bcast_time_step_values(io_proc_time, io_time_avg)
1215 end if
1216
1217 if (proc_rank == 0) then
1218 time_final = 0._wp
1219 io_time_final = 0._wp
1220 if (num_procs == 1) then
1221 time_final = time_avg
1222 io_time_final = io_time_avg
1223 else
1224 time_final = maxval(proc_time)
1225 io_time_final = maxval(io_proc_time)
1226 end if
1227
1228 grind_time = time_final*1.0e9_wp/ &
1229 (real(sys_size, wp)*real(maxval((/1, m_glb/)), wp)* &
1230 real(maxval((/1, n_glb/)), wp)*real(maxval((/1, p_glb/)), wp))
1231
1232 print *, "Performance:", grind_time, "ns/gp/eq/rhs"
1233 inquire (file='time_data.dat', exist=file_exists)
1234 if (file_exists) then
1235 open (1, file='time_data.dat', position='append', status='old')
1236 else
1237 open (1, file='time_data.dat', status='new')
1238 write (1, '(A10, A15, A15)') "Ranks", "s/step", "ns/gp/eq/rhs"
1239 end if
1240
1241 write (1, '(I10, 2(F15.8))') num_procs, time_final, grind_time
1242
1243 close (1)
1244
1245 inquire (file='io_time_data.dat', exist=file_exists)
1246 if (file_exists) then
1247 open (1, file='io_time_data.dat', position='append', status='old')
1248 else
1249 open (1, file='io_time_data.dat', status='new')
1250 write (1, '(A10, A15)') "Ranks", "s/step"
1251 end if
1252
1253 write (1, '(I10, F15.8)') num_procs, io_time_final
1254 close (1)
1255
1256 end if
1257
1258 end subroutine s_save_performance_metrics
1259
1260 !> @brief Saves conservative variable data to disk at the current time step.
1261 impure subroutine s_save_data(t_step, start, finish, io_time_avg, nt)
1262 integer, intent(inout) :: t_step
1263 real(wp), intent(inout) :: start, finish, io_time_avg
1264 integer, intent(inout) :: nt
1265
1266 integer(kind=8) :: i, j, k, l
1267 integer :: stor
1268
1269 integer :: save_count
1270
1271 if (down_sample) then
1273 end if
1274
1275 stor = 1
1276
1277 if (time_stepper /= 1) then
1278
1279# 942 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1280
1281# 942 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1282#if defined(MFC_OpenACC)
1283# 942 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1284!$acc parallel loop collapse(4) gang vector default(present) copyin(idwbuff)
1285# 942 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1286#elif defined(MFC_OpenMP)
1287# 942 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1288
1289# 942 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1290
1291# 942 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1292
1293# 942 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1294!$omp target teams loop defaultmap(firstprivate:scalar) bind(teams,parallel) collapse(4) defaultmap(tofrom:aggregate) defaultmap(tofrom:allocatable) defaultmap(tofrom:pointer) map(to:idwbuff)
1295# 942 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1296#endif
1297# 942 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1298
1299 do i = 1, sys_size
1300 do l = idwbuff(3)%beg, idwbuff(3)%end
1301 do k = idwbuff(2)%beg, idwbuff(2)%end
1302 do j = idwbuff(1)%beg, idwbuff(1)%end
1303 q_cons_ts(2)%vf(i)%sf(j, k, l) = &
1304 q_cons_ts(1)%vf(i)%sf(j, k, l)
1305 end do
1306 end do
1307 end do
1308 end do
1309
1310# 953 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1311
1312# 953 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1313#if defined(MFC_OpenACC)
1314# 953 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1315!$acc end parallel loop
1316# 953 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1317#elif defined(MFC_OpenMP)
1318# 953 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1319
1320# 953 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1321
1322# 953 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1323!$omp end target teams loop
1324# 953 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1325#endif
1326# 953 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1327
1328 stor = 2
1329 end if
1330
1331 call cpu_time(start)
1332 call nvtxstartrange("SAVE-DATA")
1333 do i = 1, sys_size
1334#ifndef FRONTIER_UNIFIED
1335
1336# 961 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1337#if defined(MFC_OpenACC)
1338# 961 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1339!$acc update host(q_cons_ts(stor)%vf(i)%sf)
1340# 961 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1341#elif defined(MFC_OpenMP)
1342# 961 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1343!$omp target update from(q_cons_ts(stor)%vf(i)%sf)
1344# 961 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1345#endif
1346#endif
1347 do l = 0, p
1348 do k = 0, n
1349 do j = 0, m
1350 if (ieee_is_nan(real(q_cons_ts(stor)%vf(i)%sf(j, k, l), kind=wp))) then
1351 print *, "NaN(s) in timestep output.", j, k, l, i, proc_rank, t_step, m, n, p
1352 error stop "NaN(s) in timestep output."
1353 end if
1354 end do
1355 end do
1356 end do
1357 end do
1358
1359 if (qbmm .and. .not. polytropic) then
1360
1361# 976 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1362#if defined(MFC_OpenACC)
1363# 976 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1364!$acc update host(pb_ts(1)%sf)
1365# 976 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1366#elif defined(MFC_OpenMP)
1367# 976 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1368!$omp target update from(pb_ts(1)%sf)
1369# 976 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1370#endif
1371
1372# 977 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1373#if defined(MFC_OpenACC)
1374# 977 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1375!$acc update host(mv_ts(1)%sf)
1376# 977 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1377#elif defined(MFC_OpenMP)
1378# 977 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1379!$omp target update from(mv_ts(1)%sf)
1380# 977 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1381#endif
1382 end if
1383
1384 if (cfl_dt) then
1385 save_count = int(mytime/t_save)
1386 else
1387 save_count = t_step
1388 end if
1389
1390 if (bubbles_lagrange) then
1391
1392# 987 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1393#if defined(MFC_OpenACC)
1394# 987 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1395!$acc update host(lag_id, mtn_pos, mtn_posPrev, mtn_vel, intfc_rad, intfc_vel, bub_R0, Rmax_stats, Rmin_stats, bub_dphidt, gas_p, gas_mv, gas_mg, gas_betaT, gas_betaC)
1396# 987 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1397#elif defined(MFC_OpenMP)
1398# 987 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1399!$omp target update from(lag_id, mtn_pos, mtn_posPrev, mtn_vel, intfc_rad, intfc_vel, bub_R0, Rmax_stats, Rmin_stats, bub_dphidt, gas_p, gas_mv, gas_mg, gas_betaT, gas_betaC)
1400# 987 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1401#endif
1402# 990 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1403 do i = 1, nbubs
1404 if (ieee_is_nan(intfc_rad(i, 1)) .or. intfc_rad(i, 1) <= 0._wp) then
1405 call s_mpi_abort("Bubble radius is negative or NaN, please reduce dt.")
1406 end if
1407 end do
1408
1409
1410# 996 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1411#if defined(MFC_OpenACC)
1412# 996 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1413!$acc update host(q_beta(1)%sf)
1414# 996 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1415#elif defined(MFC_OpenMP)
1416# 996 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1417!$omp target update from(q_beta(1)%sf)
1418# 996 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1419#endif
1420 call s_write_data_files(q_cons_ts(stor)%vf, q_t_sf, q_prim_vf, save_count, bc_type, q_beta(1))
1421
1422# 998 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1423#if defined(MFC_OpenACC)
1424# 998 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1425!$acc update host(Rmax_stats, Rmin_stats, gas_p, gas_mv, intfc_vel)
1426# 998 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1427#elif defined(MFC_OpenMP)
1428# 998 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1429!$omp target update from(Rmax_stats, Rmin_stats, gas_p, gas_mv, intfc_vel)
1430# 998 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1431#endif
1432 call s_write_restart_lag_bubbles(save_count) !parallel
1433 if (lag_params%write_bubbles_stats) call s_write_lag_bubble_stats()
1434 else
1435 call s_write_data_files(q_cons_ts(stor)%vf, q_t_sf, q_prim_vf, save_count, bc_type)
1436 end if
1437
1438 call nvtxendrange
1439 call cpu_time(finish)
1440 if (cfl_dt) then
1441 nt = mytime/t_save
1442 else
1443 nt = int((t_step - t_step_start)/(t_step_save))
1444 end if
1445
1446 if (nt == 1) then
1447 io_time_avg = abs(finish - start)
1448 else
1449 io_time_avg = (abs(finish - start) + io_time_avg*(nt - 1))/nt
1450 end if
1451
1452 end subroutine s_save_data
1453
1454 !> @brief Initializes all simulation sub-modules in the required dependency order.
1455 impure subroutine s_initialize_modules
1456
1457 integer :: m_ds, n_ds, p_ds
1458 integer :: i, j, k, l, x_id, y_id, z_id, ix, iy, iz
1459 real(wp) :: temp1, temp2, temp3, temp4
1460
1462# 1035 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1463 if (bubbles_euler .or. bubbles_lagrange) then
1465 end if
1469 if (grid_geometry == 3) call s_initialize_fftw_module()
1470
1472 if (ib) call s_initialize_ibm_module()
1473 if (qbmm) call s_initialize_qbmm_module()
1474
1475 if (acoustic_source) then
1477 end if
1478
1479 if (viscous .and. (.not. igr)) then
1481 end if
1482
1484
1486
1488
1492
1494
1495 if (down_sample) then
1496 m_ds = int((m + 1)/3) - 1
1497 n_ds = int((n + 1)/3) - 1
1498 p_ds = int((p + 1)/3) - 1
1499
1500 allocate (q_cons_temp(1:sys_size))
1501 do i = 1, sys_size
1502 allocate (q_cons_temp(i)%sf(-1:m_ds + 1, -1:n_ds + 1, -1:p_ds + 1))
1503 end do
1504 end if
1505
1506 ! Reading in the user provided initial condition and grid data
1507 if (down_sample) then
1510 do i = 1, sys_size
1511
1512# 1083 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1513#if defined(MFC_OpenACC)
1514# 1083 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1515!$acc update device(q_cons_ts(1)%vf(i)%sf)
1516# 1083 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1517#elif defined(MFC_OpenMP)
1518# 1083 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1519!$omp target update to(q_cons_ts(1)%vf(i)%sf)
1520# 1083 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1521#endif
1522 end do
1523 do i = 1, sys_size
1524 deallocate (q_cons_temp(i)%sf)
1525 end do
1526 deallocate (q_cons_temp)
1527 else
1528 call s_read_data_files(q_cons_ts(1)%vf)
1529 end if
1530
1531 ! Populating the buffers of the grid variables using the boundary conditions
1533
1535 if (ib) then
1536 call s_ibm_setup()
1537 call s_write_ib_data_file(0)
1538 end if
1541
1542 ! Initialize the Temperature cache.
1544
1545 ! Computation of parameters, allocation of memory, association of pointers,
1546 ! and/or execution of any other tasks that are needed to properly configure
1547 ! the modules. The preparations below DO DEPEND on the grid being complete.
1548 if (igr .or. dummy) then
1550 end if
1551 if (.not. igr .or. dummy) then
1552 if (recon_type == weno_type) then
1554 elseif (recon_type == muscl_type) then
1556 end if
1559 end if
1560
1563
1566
1567 end subroutine s_initialize_modules
1568
1569 !> @brief Sets up the MPI execution environment, binds GPUs, and decomposes the computational domain.
1570 impure subroutine s_initialize_mpi_domain
1571 integer :: ierr
1572#ifdef MFC_GPU
1573 real(wp) :: starttime, endtime
1574 integer :: num_devices, local_size, num_nodes, ppn, my_device_num
1575 integer :: dev, devnum, local_rank
1576#ifdef MFC_MPI
1577 integer :: local_comm
1578#endif
1579#if defined(MFC_OpenACC)
1580 integer(acc_device_kind) :: devtype
1581#endif
1582#endif
1583
1584 ! Initializing MPI execution environment
1585
1586 call s_mpi_initialize()
1587
1588 ! Bind GPUs if OpenACC is enabled
1589#ifdef MFC_GPU
1590#ifndef MFC_MPI
1591 local_size = 1
1592 local_rank = 0
1593#else
1594 call mpi_comm_split_type(mpi_comm_world, mpi_comm_type_shared, 0, &
1595 mpi_info_null, local_comm, ierr)
1596 call mpi_comm_size(local_comm, local_size, ierr)
1597 call mpi_comm_rank(local_comm, local_rank, ierr)
1598#endif
1599#if defined(MFC_OpenACC)
1600 devtype = acc_get_device_type()
1601 devnum = acc_get_num_devices(devtype)
1602 dev = mod(local_rank, devnum)
1603
1604 call acc_set_device_num(dev, devtype)
1605#elif defined(MFC_OpenMP)
1606 devnum = omp_get_num_devices()
1607 dev = mod(local_rank, devnum)
1608 call omp_set_default_device(dev)
1609#endif
1610#endif
1611
1612 ! The rank 0 processor assigns default values to the user inputs prior to
1613 ! reading them in from the input file. Next, the user inputs are read and
1614 ! their consistency is checked. The identification of any inconsistencies
1615 ! will result in the termination of the simulation.
1616 if (proc_rank == 0) then
1618 call s_read_input_file()
1619 call s_check_input_file()
1620
1621 print '(" Simulating a ", A, " ", I0, "x", I0, "x", I0, " case on ", I0, " rank(s) ", A, ".")', &
1622# 1185 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1623 "regular", &
1624# 1189 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1625 m, n, p, num_procs, &
1626#if defined(MFC_OpenACC)
1627 "with OpenACC offloading"
1628#elif defined(MFC_OpenMP)
1629 "with OpenMP offloading"
1630#else
1631 "on CPUs"
1632#endif
1633 end if
1634
1635 ! Broadcasting the user inputs to all of the processors and performing the
1636 ! parallel computational domain decomposition. Neither procedure has to be
1637 ! carried out if the simulation is in fact not truly executed in parallel.
1638
1640
1642
1644
1645 end subroutine s_initialize_mpi_domain
1646
1647 !> @brief Transfers initial conservative variable and model parameter data to the GPU device.
1649 integer :: i
1650 !Update GPU DATA
1651 if (.not. down_sample) then
1652 do i = 1, sys_size
1653
1654# 1217 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1655#if defined(MFC_OpenACC)
1656# 1217 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1657!$acc update device(q_cons_ts(1)%vf(i)%sf)
1658# 1217 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1659#elif defined(MFC_OpenMP)
1660# 1217 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1661!$omp target update to(q_cons_ts(1)%vf(i)%sf)
1662# 1217 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1663#endif
1664 end do
1665 end if
1666
1667 if (qbmm .and. .not. polytropic) then
1668
1669# 1222 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1670#if defined(MFC_OpenACC)
1671# 1222 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1672!$acc update device(pb_ts(1)%sf, mv_ts(1)%sf)
1673# 1222 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1674#elif defined(MFC_OpenMP)
1675# 1222 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1676!$omp target update to(pb_ts(1)%sf, mv_ts(1)%sf)
1677# 1222 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1678#endif
1679 end if
1680 if (chemistry) then
1681
1682# 1225 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1683#if defined(MFC_OpenACC)
1684# 1225 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1685!$acc update device(q_T_sf%sf)
1686# 1225 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1687#elif defined(MFC_OpenMP)
1688# 1225 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1689!$omp target update to(q_T_sf%sf)
1690# 1225 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1691#endif
1692 end if
1693
1694
1695# 1228 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1696#if defined(MFC_OpenACC)
1697# 1228 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1698!$acc update device(chem_params)
1699# 1228 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1700#elif defined(MFC_OpenMP)
1701# 1228 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1702!$omp target update to(chem_params)
1703# 1228 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1704#endif
1705
1706
1707# 1230 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1708#if defined(MFC_OpenACC)
1709# 1230 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1710!$acc update device(R0ref, p0ref, rho0ref, ss, pv, vd, mu_l, mu_v, mu_g, gam_v, gam_g, M_v, M_g, R_v, R_g, Tw, cp_v, cp_g, k_vl, k_gl, gam, gam_m, Eu, Ca, Web, Re_inv, Pe_c, phi_vg, phi_gv, omegaN, bubbles_euler, polytropic, polydisperse, qbmm, ptil, bubble_model, thermal, poly_sigma, adv_n, adap_dt, adap_dt_tol, adap_dt_max_iters, n_idx, pi_fac, low_Mach)
1711# 1230 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1712#elif defined(MFC_OpenMP)
1713# 1230 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1714!$omp target update to(R0ref, p0ref, rho0ref, ss, pv, vd, mu_l, mu_v, mu_g, gam_v, gam_g, M_v, M_g, R_v, R_g, Tw, cp_v, cp_g, k_vl, k_gl, gam, gam_m, Eu, Ca, Web, Re_inv, Pe_c, phi_vg, phi_gv, omegaN, bubbles_euler, polytropic, polydisperse, qbmm, ptil, bubble_model, thermal, poly_sigma, adv_n, adap_dt, adap_dt_tol, adap_dt_max_iters, n_idx, pi_fac, low_Mach)
1715# 1230 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1716#endif
1717# 1236 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1718
1719 if (bubbles_euler) then
1720
1721# 1238 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1722#if defined(MFC_OpenACC)
1723# 1238 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1724!$acc update device(weight, R0)
1725# 1238 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1726#elif defined(MFC_OpenMP)
1727# 1238 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1728!$omp target update to(weight, R0)
1729# 1238 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1730#endif
1731 if (.not. polytropic) then
1732
1733# 1240 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1734#if defined(MFC_OpenACC)
1735# 1240 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1736!$acc update device(pb0, Pe_T, k_g, k_v, mass_g0, mass_v0, Re_trans_T, Re_trans_c, Im_trans_T, Im_trans_c)
1737# 1240 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1738#elif defined(MFC_OpenMP)
1739# 1240 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1740!$omp target update to(pb0, Pe_T, k_g, k_v, mass_g0, mass_v0, Re_trans_T, Re_trans_c, Im_trans_T, Im_trans_c)
1741# 1240 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1742#endif
1743# 1242 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1744 else if (qbmm) then
1745
1746# 1243 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1747#if defined(MFC_OpenACC)
1748# 1243 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1749!$acc update device(pb0)
1750# 1243 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1751#elif defined(MFC_OpenMP)
1752# 1243 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1753!$omp target update to(pb0)
1754# 1243 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1755#endif
1756 end if
1757 end if
1758
1759
1760# 1247 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1761#if defined(MFC_OpenACC)
1762# 1247 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1763!$acc update device(adv_n, adap_dt, adap_dt_tol, adap_dt_max_iters, n_idx, pi_fac, low_Mach)
1764# 1247 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1765#elif defined(MFC_OpenMP)
1766# 1247 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1767!$omp target update to(adv_n, adap_dt, adap_dt_tol, adap_dt_max_iters, n_idx, pi_fac, low_Mach)
1768# 1247 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1769#endif
1770
1771
1772# 1249 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1773#if defined(MFC_OpenACC)
1774# 1249 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1775!$acc update device(acoustic_source, num_source)
1776# 1249 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1777#elif defined(MFC_OpenMP)
1778# 1249 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1779!$omp target update to(acoustic_source, num_source)
1780# 1249 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1781#endif
1782
1783# 1250 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1784#if defined(MFC_OpenACC)
1785# 1250 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1786!$acc update device(sigma, surface_tension)
1787# 1250 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1788#elif defined(MFC_OpenMP)
1789# 1250 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1790!$omp target update to(sigma, surface_tension)
1791# 1250 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1792#endif
1793
1794
1795# 1252 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1796#if defined(MFC_OpenACC)
1797# 1252 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1798!$acc update device(dx, dy, dz, x_cb, x_cc, y_cb, y_cc, z_cb, z_cc)
1799# 1252 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1800#elif defined(MFC_OpenMP)
1801# 1252 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1802!$omp target update to(dx, dy, dz, x_cb, x_cc, y_cb, y_cc, z_cb, z_cc)
1803# 1252 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1804#endif
1805
1806# 1253 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1807#if defined(MFC_OpenACC)
1808# 1253 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1809!$acc update device(bc_x%vb1, bc_x%vb2, bc_x%vb3, bc_x%ve1, bc_x%ve2, bc_x%ve3)
1810# 1253 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1811#elif defined(MFC_OpenMP)
1812# 1253 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1813!$omp target update to(bc_x%vb1, bc_x%vb2, bc_x%vb3, bc_x%ve1, bc_x%ve2, bc_x%ve3)
1814# 1253 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1815#endif
1816
1817# 1254 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1818#if defined(MFC_OpenACC)
1819# 1254 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1820!$acc update device(bc_y%vb1, bc_y%vb2, bc_y%vb3, bc_y%ve1, bc_y%ve2, bc_y%ve3)
1821# 1254 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1822#elif defined(MFC_OpenMP)
1823# 1254 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1824!$omp target update to(bc_y%vb1, bc_y%vb2, bc_y%vb3, bc_y%ve1, bc_y%ve2, bc_y%ve3)
1825# 1254 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1826#endif
1827
1828# 1255 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1829#if defined(MFC_OpenACC)
1830# 1255 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1831!$acc update device(bc_z%vb1, bc_z%vb2, bc_z%vb3, bc_z%ve1, bc_z%ve2, bc_z%ve3)
1832# 1255 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1833#elif defined(MFC_OpenMP)
1834# 1255 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1835!$omp target update to(bc_z%vb1, bc_z%vb2, bc_z%vb3, bc_z%ve1, bc_z%ve2, bc_z%ve3)
1836# 1255 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1837#endif
1838
1839
1840# 1257 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1841#if defined(MFC_OpenACC)
1842# 1257 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1843!$acc update device(bc_x%grcbc_in, bc_x%grcbc_out, bc_x%grcbc_vel_out)
1844# 1257 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1845#elif defined(MFC_OpenMP)
1846# 1257 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1847!$omp target update to(bc_x%grcbc_in, bc_x%grcbc_out, bc_x%grcbc_vel_out)
1848# 1257 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1849#endif
1850
1851# 1258 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1852#if defined(MFC_OpenACC)
1853# 1258 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1854!$acc update device(bc_y%grcbc_in, bc_y%grcbc_out, bc_y%grcbc_vel_out)
1855# 1258 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1856#elif defined(MFC_OpenMP)
1857# 1258 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1858!$omp target update to(bc_y%grcbc_in, bc_y%grcbc_out, bc_y%grcbc_vel_out)
1859# 1258 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1860#endif
1861
1862# 1259 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1863#if defined(MFC_OpenACC)
1864# 1259 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1865!$acc update device(bc_z%grcbc_in, bc_z%grcbc_out, bc_z%grcbc_vel_out)
1866# 1259 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1867#elif defined(MFC_OpenMP)
1868# 1259 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1869!$omp target update to(bc_z%grcbc_in, bc_z%grcbc_out, bc_z%grcbc_vel_out)
1870# 1259 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1871#endif
1872
1873
1874# 1261 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1875#if defined(MFC_OpenACC)
1876# 1261 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1877!$acc update device(relax, relax_model)
1878# 1261 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1879#elif defined(MFC_OpenMP)
1880# 1261 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1881!$omp target update to(relax, relax_model)
1882# 1261 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1883#endif
1884 if (relax) then
1885
1886# 1263 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1887#if defined(MFC_OpenACC)
1888# 1263 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1889!$acc update device(palpha_eps, ptgalpha_eps)
1890# 1263 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1891#elif defined(MFC_OpenMP)
1892# 1263 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1893!$omp target update to(palpha_eps, ptgalpha_eps)
1894# 1263 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1895#endif
1896 end if
1897
1898 if (ib) then
1899
1900# 1267 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1901#if defined(MFC_OpenACC)
1902# 1267 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1903!$acc update device(ib_markers%sf)
1904# 1267 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1905#elif defined(MFC_OpenMP)
1906# 1267 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1907!$omp target update to(ib_markers%sf)
1908# 1267 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1909#endif
1910 end if
1911# 1270 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1912
1913# 1270 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1914#if defined(MFC_OpenACC)
1915# 1270 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1916!$acc update device(igr, nb, igr_order)
1917# 1270 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1918#elif defined(MFC_OpenMP)
1919# 1270 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1920!$omp target update to(igr, nb, igr_order)
1921# 1270 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1922#endif
1923# 1272 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1924# 1280 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1925
1926 end subroutine s_initialize_gpu_vars
1927
1928 !> @brief Finalizes and deallocates all simulation sub-modules in reverse initialization order.
1929 impure subroutine s_finalize_modules
1930
1937 if (igr) then
1939 else
1942 if (recon_type == weno_type) then
1944 elseif (recon_type == muscl_type) then
1946 end if
1947 end if
1949 if (grid_geometry == 3) call s_finalize_fftw_module
1955 if (viscous .and. (.not. igr)) then
1957 end if
1959
1962
1963 ! Terminating MPI execution environment
1964 call s_mpi_finalize()
1965 end subroutine s_finalize_modules
1966
1967end module m_start_up
type(scalar_field), dimension(sys_size), intent(inout) q_cons_vf
Applies acoustic pressure source terms including focused, planar, and broadband transducers.
impure subroutine, public s_precalculate_acoustic_spatial_sources
This subroutine identifies and precalculates the non-zero acoustic spatial sources before time-steppi...
impure subroutine, public s_initialize_acoustic_src
This subroutine initializes the acoustic source module.
Computes gravitational and user-defined body force source terms for the momentum equations.
impure subroutine, public s_initialize_body_forces_module
This subroutine initializes the module global array of mixture densities in each grid cell.
impure subroutine, public s_finalize_body_forces_module
Deallocates module variables used for body force computations.
Noncharacteristic and processor boundary condition application for ghost cells and buffer regions.
impure subroutine, public s_initialize_boundary_common_module()
Allocates and sets up boundary condition buffer arrays for all coordinate directions.
subroutine, public s_finalize_boundary_common_module()
Deallocates boundary condition buffer arrays allocated during module initialization.
impure subroutine, public s_populate_variables_buffers(bc_type, q_prim_vf, pb_in, mv_in)
The purpose of this procedure is to populate the buffers of the primitive variables,...
subroutine, public s_read_parallel_boundary_condition_files(bc_type)
Reads boundary condition type and buffer data from per-rank parallel files using MPI I/O.
subroutine, public s_populate_grid_variables_buffers
The purpose of this subroutine is to populate the buffers of the grid variables, which are constitute...
subroutine, public s_read_serial_boundary_condition_files(step_dirpath, bc_type)
Reads boundary condition type and buffer data from serial (unformatted) restart files.
subroutine, public s_assign_default_bc_type(bc_type)
Initializes the per-cell boundary condition type arrays with the global default BC values.
Computes ensemble-averaged (Euler–Euler) bubble source terms for radius, velocity,...
impure subroutine s_initialize_bubbles_ee_module
Allocates and initializes arrays for the Euler-Euler bubble model.
Tracks Lagrangian bubbles and couples their dynamics to the Eulerian flow via volume averaging.
impure subroutine s_write_lag_bubble_stats()
Subroutine that writes the maximum and minimum radius of each bubble.
impure subroutine s_write_restart_lag_bubbles(t_step)
Subroutine that writes the restarting files for the particles in the lagrangian solver.
integer nbubs
Number of bubbles in the local domain.
impure subroutine s_initialize_bubbles_el_module(q_cons_vf)
Initializes the lagrangian subgrid bubble solver.
real(wp), dimension(:, :), allocatable intfc_rad
Bubble radius.
type(scalar_field), dimension(:), allocatable q_beta
impure subroutine s_finalize_lagrangian_solver()
The purpose of this subroutine is to deallocate variables.
Characteristic boundary conditions (CBC) for slip walls, non-reflecting subsonic inflow/outflow,...
impure subroutine, public s_initialize_cbc_module
The computation of parameters, the allocation of memory, the association of pointers and/or the execu...
impure subroutine, public s_finalize_cbc_module
Module deallocation and/or disassociation procedures.
Shared input validation checks for grid dimensions and AMD GPU compiler limits.
impure subroutine, public s_check_inputs_common
Checks compatibility of parameters in the input file. Used by all three stages.
Validates simulation input parameters for consistency and supported configurations.
impure subroutine, public s_check_inputs
Checks compatibility of parameters in the input file. Used by the simulation stage.
Multi-species chemistry interface for thermodynamic properties, reaction rates, and transport coeffic...
subroutine s_compute_q_t_sf(q_t_sf, q_cons_vf, bounds)
Initializes the temperature field from conservative variables by inverting the energy equation.
Platform-specific file and directory operations: create, delete, inquire, getcwd, and basename.
impure subroutine my_inquire(fileloc, dircheck)
Inquires on the existence of a directory.
Writes solution data, run-time stability diagnostics (ICFL, VCFL, CCFL, Rc), and probe/center-of-mass...
impure subroutine, public s_initialize_data_output_module
The computation of parameters, the allocation of memory, the association of pointers and/or the execu...
impure subroutine, public s_finalize_data_output_module
Module deallocation and/or disassociation procedures.
impure subroutine, public s_write_data_files(q_cons_vf, q_t_sf, q_prim_vf, t_step, bc_type, beta)
Write data files. Dispatch subroutine that replaces procedure pointer.
subroutine, public s_write_ib_data_file(time_step)
Dispatches immersed boundary data output to the serial or parallel writer.
Shared derived types for field data, patch geometry, bubble dynamics, and MPI I/O structures.
Derives diagnostic flow quantities (vorticity, speed of sound, numerical Schlieren,...
impure subroutine, public s_initialize_derived_variables
Allocate and open derived variables. Computing FD coefficients.
impure subroutine, public s_initialize_derived_variables_module
Computation of parameters, allocation procedures, and/or any other tasks needed to properly setup the...
impure subroutine, public s_finalize_derived_variables_module
Deallocation procedures for the module.
Global parameters for the computational domain, fluid properties, and simulation algorithm configurat...
real(wp) mytime
Current simulation time.
logical null_weights
Null undesired WENO weights.
logical bubbles_euler
Bubbles euler on/off.
real(wp) cont_damage_s
Exponent s for continuum damage modeling.
integer wave_speeds
Wave speeds estimation method.
logical cont_damage
Continuum damage modeling.
logical igr
Use information geometric regularization.
logical hypoelasticity
hypoelasticity modeling
impure subroutine s_assign_default_values_to_user_inputs
Assigns default values to the user inputs before reading them in. This enables for an easier consiste...
impure subroutine s_finalize_global_parameters_module
Module deallocation and/or disassociation procedures.
integer thermal
Thermal behavior. 1 = adiabatic, 2 = isotherm, 3 = transfer.
integer avg_state
Average state evaluation method.
type(int_bounds_info), dimension(1:3) idwint
real(wp), dimension(:), allocatable, target z_cb
integer recon_type
Reconstruction Type.
logical igr_pres_lim
Limit to positive pressures for IGR.
logical, parameter chemistry
Chemistry modeling.
logical acoustic_source
Acoustic source switch.
type(ib_patch_parameters), dimension(num_patches_max) patch_ib
type(int_bounds_info) mom_idx
Indexes of first & last momentum eqns.
integer num_fluids
number of fluids in the simulation
logical polydisperse
Polydisperse bubbles.
type(bubbles_lagrange_parameters) lag_params
Lagrange bubbles' parameters.
integer proc_rank
Rank of the local processor.
real(wp) hyper_cleaning_tau
Hyperbolic cleaning tau.
logical mixture_err
Mixture properties correction.
logical adap_dt
Adaptive step size control.
logical weno_re_flux
WENO reconstruct velocity gradients for viscous stress tensor.
real(wp) hyper_cleaning_speed
Hyperbolic cleaning wave speed (c_h).
integer t_step_print
Number of time-steps between printouts.
logical dummy
AMDFlang workaround: keep a dummy logical to avoid a compiler case-optimization bug when a parameter+...
real(wp) poly_sigma
log normal sigma for polydisperse PDF
character(len=name_len) mpiiofs
integer, dimension(:), allocatable start_idx
Starting cell-center index of local processor in global grid.
type(vec3_dt), dimension(num_probes_max) probe
integer sys_size
Number of unknowns in system of eqns.
integer muscl_order
Order of the MUSCL reconstruction.
logical alt_soundspeed
Alternate mixture sound speed.
real(wp) ptgalpha_eps
trigger parameter for the pTg relaxation procedure, phase change model
integer relax_model
Relaxation model.
integer t_step_old
Existing IC/grid folder.
logical viscous
Viscous effects.
logical run_time_info
Run-time output flag.
type(int_bounds_info) b_idx
Indexes of first and last magnetic field eqns.
real(wp) wenoz_q
Power constant for WENO-Z.
integer riemann_solver
Riemann solver algorithm.
type(int_bounds_info), dimension(1:3) idwbuff
logical int_comp
THINC interface compression.
real(wp) re_inv
Inverse Reynolds number.
real(wp) ic_eps
THINC Epsilon to compress on surface cells.
integer model_eqns
Multicomponent flow model.
integer precision
Precision of output files.
logical hyperelasticity
hyperelasticity modeling
type(physical_parameters), dimension(num_fluids_max) fluid_pp
Database of the physical parameters of each of the fluids that is present in the flow....
real(wp) alf_factor
alpha factor for IGR
impure subroutine s_initialize_global_parameters_module
The computation of parameters, the allocation of memory, the association of pointers and/or the execu...
real(wp), dimension(:), allocatable, target y_cc
type(pres_field), dimension(:), allocatable pb_ts
type(pres_field), dimension(:), allocatable mv_ts
type(chemistry_parameters) chem_params
integer fd_order
The order of the finite-difference (fd) approximations of the first-order derivatives that need to be...
logical bubbles_lagrange
Lagrangian subgrid bubble model switch.
real(wp) ca
Cavitation number.
real(wp) alpha_bar
Damage rate factor for continuum damage modeling.
logical polytropic
Polytropic switch.
logical bf_z
body force toggle in three directions amplitude, frequency, and phase shift sinusoid in each directio...
logical mp_weno
Monotonicity preserving (MP) WENO.
real(wp), dimension(:), allocatable, target z_cc
logical relax
activate phase change
logical qbmm
Quadrature moment method.
real(wp) pi_fac
Factor for artificial pi_inf.
logical hyper_cleaning
Hyperbolic cleaning for MHD for divB=0.
integer adap_dt_max_iters
Maximum number of iterations.
real(wp) ic_beta
THINC Sharpness Parameter.
real(wp) bx0
Constant magnetic field in the x-direction (1D).
integer num_source
Number of acoustic sources.
real(wp), dimension(:), allocatable qvs
real(wp), dimension(:), allocatable pi_infs
logical adv_n
Solve the number density equation and compute alpha from number density.
integer num_procs
Number of processors.
character(len=path_len) case_dir
Case folder location.
real(wp) weno_eps
Binding for the WENO nonlinear weights.
integer weno_order
Order of the WENO reconstruction.
logical mhd
Magnetohydrodynamics.
real(wp), dimension(:), allocatable, target x_cc
type(acoustic_parameters), dimension(num_probes_max) acoustic
Acoustic source parameters.
real(wp) tau_star
Stress threshold for continuum damage modeling.
logical parallel_io
Format of the data files.
type(integral_parameters), dimension(num_probes_max) integral
real(wp), dimension(:), allocatable, target y_cb
real(wp) adap_dt_tol
Tolerance to control adaptive step size.
integer e_idx
Index of energy equation.
type(cell_num_bounds) cells_bounds
logical down_sample
down sample the output files
type(mpi_io_airfoil_ib_var), public mpi_io_airfoil_ib_data
logical file_per_process
shared file or not when using parallel io
real(wp) palpha_eps
trigger parameter for the p relaxation procedure, phase change model
integer num_igr_warm_start_iters
number of warm start iterations for elliptic solve
logical elasticity
elasticity modeling, true for hyper or hypo
integer nb
Number of eq. bubble sizes.
type(mpi_io_var), public mpi_io_data
impure subroutine s_initialize_parallel_io
Initializes parallel infrastructure.
logical mpp_lim
Mixture physical parameters (MPP) limits.
integer num_igr_iters
number of iterations for elliptic solve
real(wp) teno_ct
Smoothness threshold for TENO.
integer low_mach
Low Mach number fix to HLLC Riemann solver.
integer igr_order
Reconstruction order for IGR.
logical teno
TENO (Targeted ENO).
real(wp), dimension(:), allocatable, target dy
real(wp) dt
Size of the time-step.
type(subgrid_bubble_physical_parameters) bub_pp
integer time_stepper
Time-stepper algorithm.
logical relativity
Relativity (only for MHD).
real(wp), dimension(:), allocatable gammas
integer bubble_model
Gilmore or Keller–Miksis bubble model.
real(wp) finaltime
Final simulation time.
real(wp), dimension(:), allocatable, target dz
real(wp), dimension(:), allocatable, target dx
logical mapped_weno
WENO-M (WENO with mapping of nonlinear weights).
integer igr_iter_solver
IGR elliptic solver.
integer muscl_lim
MUSCL Limiter.
real(wp), dimension(:), allocatable, target x_cb
Basic floating-point utilities: approximate equality, default detection, and coordinate bounds.
elemental subroutine, public s_update_cell_bounds(bounds, m, n, p)
Updates the min and max number of cells in each set of axes.
Utility routines for bubble model setup, coordinate transforms, array sampling, and special functions...
subroutine, public s_upsample_data(q_cons_vf, q_cons_temp)
Upsamples conservative variable fields from a coarsened grid back to the original resolution using in...
impure subroutine, public s_initialize_bubbles_model()
bubbles_euler + polytropic bubbles_euler + non-polytropic bubbles_lagrange + non-polytropic
elemental subroutine, public s_int_to_str(i, res)
Converts an integer to its trimmed string representation.
Computes the left Cauchy–Green deformation tensor and hyperelastic stress source terms.
impure subroutine, public s_finalize_hyperelastic_module()
Deallocates memory for hyperelastic deformation tensor and finite-difference coefficients.
impure subroutine, public s_initialize_hyperelastic_module
The following subroutine handles the calculation of the btensor. The calculation of the btensor takes...
Computes hypoelastic stress-rate source terms and damage-state evolution.
impure subroutine, public s_initialize_hypoelastic_module
Allocates arrays and computes finite-difference coefficients for the hypoelastic stress model.
impure subroutine, public s_finalize_hypoelastic_module()
Deallocates arrays used by the hypoelastic stress module.
Ghost-node immersed boundary method: locates ghost/image points, computes interpolation coefficients,...
impure subroutine, public s_ibm_setup()
Initializes the values of various IBM variables, such as ghost points and image points.
type(integer_field), public ib_markers
impure subroutine, public s_initialize_ibm_module()
Allocates memory for the variables in the IBM module.
Iterative ghost rasterization (IGR) for sharp immersed boundary treatment.
subroutine, public s_initialize_igr_module()
Allocates and initializes arrays, coefficients, and GPU data structures for the implicit gradient rec...
integer(kind=8) j
integer(kind=8) i
integer(kind=8) l
integer(kind=8) r
integer(kind=8) k
subroutine, public s_finalize_igr_module()
Deallocates all arrays and GPU resources allocated by the IGR module.
MPI communication layer: domain decomposition, halo exchange, reductions, and parallel I/O setup.
impure subroutine s_mpi_abort(prnt, code)
The subroutine terminates the MPI execution environment.
impure subroutine s_initialize_mpi_common_module
The computation of parameters, the allocation of memory, the association of pointers and/or the execu...
impure subroutine s_mpi_barrier
Halts all processes until all have reached barrier.
impure subroutine s_mpi_initialize
The subroutine initializes the MPI execution environment and queries both the number of processors wh...
impure subroutine s_initialize_mpi_data(q_cons_vf, ib_markers, beta)
impure subroutine s_mpi_finalize
The subroutine finalizes the MPI execution environment.
subroutine s_initialize_mpi_data_ds(q_cons_vf)
impure subroutine mpi_bcast_time_step_values(proc_time, time_avg)
Gathers per-rank time step wall-clock times onto rank 0 for performance reporting.
impure subroutine s_finalize_mpi_common_module
Module deallocation and/or disassociation procedures.
subroutine s_mpi_decompose_computational_domain
The purpose of this procedure is to optimally decompose the computational domain among the available ...
MPI halo exchange, domain decomposition, and buffer packing/unpacking for the simulation solver.
subroutine s_initialize_mpi_proxy_module()
Allocates immersed boundary communication buffers for MPI halo exchanges.
subroutine s_finalize_mpi_proxy_module()
Deallocates immersed boundary MPI communication buffers.
impure subroutine s_mpi_bcast_user_inputs()
Since only the processor with rank 0 reads and verifies the consistency of user inputs,...
MUSCL reconstruction with interface sharpening for contact-preserving advection.
subroutine, public s_initialize_muscl_module()
subroutine, public s_finalize_muscl_module()
Deallocates the MUSCL direction-local work arrays.
NVIDIA NVTX profiling API bindings for GPU performance instrumentation.
Definition m_nvtx.f90:6
subroutine nvtxstartrange(name, id)
Pushes a named NVTX range for GPU profiling, optionally with a color based on the given identifier.
Definition m_nvtx.f90:63
subroutine nvtxendrange
Pops the current NVTX range to end the GPU profiling region.
Definition m_nvtx.f90:84
Phase transition relaxation solvers for liquid-vapor flows with cavitation and boiling.
impure subroutine, public s_finalize_relaxation_solver_module
This subroutine finalizes the phase change module.
subroutine, public s_infinite_relaxation_k(q_cons_vf)
This subroutine is created to activate either the pT- (N fluids) or the pTg-equilibrium (2 fluids for...
impure subroutine, public s_initialize_phasechange_module
The purpose of this subroutine is to initialize the phase change module by setting the parameters nee...
Quadrature-based moment methods (QBMM) for polydisperse bubble moment inversion and transport.
impure subroutine, public s_initialize_qbmm_module
Allocates and initializes moment coefficient arrays for the QBMM module.
Assembles the right-hand side of the governing equations using finite-volume flux differencing,...
impure subroutine, public s_initialize_rhs_module
The computation of parameters, the allocation of memory, the association of pointers and/or the execu...
impure subroutine, public s_finalize_rhs_module
Module deallocation and/or disassociation procedures.
Approximate and exact Riemann solvers (HLL, HLLC, HLLD, exact) for the multicomponent Navier–Stokes e...
impure subroutine, public s_finalize_riemann_solvers_module
Module deallocation and/or disassociation procedures.
impure subroutine, public s_initialize_riemann_solvers_module
The computation of parameters, the allocation of memory, the association of pointers and/or the execu...
Simulation helper routines for enthalpy computation, CFL calculation, and stability checks.
Reads input files, loads initial conditions and grid data, and orchestrates solver initialization and...
impure subroutine, public s_read_serial_data_files(q_cons_vf)
Reads serial initial condition and grid data files and computes cell-width distributions.
impure subroutine, public s_initialize_modules
Initializes all simulation sub-modules in the required dependency order.
impure subroutine, public s_read_data_files(q_cons_vf)
Read data files. Dispatch subroutine that replaces procedure pointer.
impure subroutine, public s_read_parallel_data_files(q_cons_vf)
Reads parallel initial condition and grid data files via MPI I/O.
subroutine, public s_initialize_internal_energy_equations(v_vf)
The purpose of this procedure is to initialize the values of the internal-energy equations of each ph...
impure subroutine, public s_save_performance_metrics(time_avg, time_final, io_time_avg, io_time_final, proc_time, io_proc_time, file_exists)
Collects per-process wall-clock times and writes aggregate performance metrics to file.
impure subroutine, public s_save_data(t_step, start, finish, io_time_avg, nt)
Saves conservative variable data to disk at the current time step.
type(scalar_field), dimension(:), allocatable q_cons_temp
subroutine, public s_initialize_gpu_vars
Transfers initial conservative variable and model parameter data to the GPU device.
impure subroutine, public s_initialize_mpi_domain
Sets up the MPI execution environment, binds GPUs, and decomposes the computational domain.
impure subroutine, public s_finalize_modules
Finalizes and deallocates all simulation sub-modules in reverse initialization order.
impure subroutine, public s_read_input_file
The purpose of this procedure is to first verify that an input file has been made available by the us...
impure subroutine, public s_check_input_file
The goal of this procedure is to verify that each of the user provided inputs is valid and that their...
impure subroutine, public s_perform_time_step(t_step, time_avg)
Advances the simulation by one time step, handling CFL-based dt and time-stepper dispatch.
Computes capillary source fluxes and color-function gradients for the diffuse-interface surface tensi...
impure subroutine, public s_initialize_surface_tension_module
impure subroutine, public s_finalize_surface_tension_module
Deallocates the color-gradient divergence and reconstructed boundary arrays for surface tension.
Total-variation-diminishing (TVD) Runge–Kutta time integrators (1st-, 2nd-, and 3rd-order SSP).
type(scalar_field) q_t_sf
Cell-average temperature variables at the current time-stage.
impure subroutine s_initialize_time_steppers_module
The computation of parameters, the allocation of memory, the association of pointers and/or the execu...
type(integer_field), dimension(:, :), allocatable bc_type
Boundary condition identifiers.
type(vector_field), dimension(:), allocatable q_cons_ts
Cell-average conservative variables at each time-stage (TS).
type(scalar_field), dimension(:), allocatable q_prim_vf
Cell-average primitive variables at the current time-stage.
impure subroutine s_finalize_time_steppers_module
Module deallocation and/or disassociation procedures.
impure subroutine s_compute_dt()
Computes the global time step size from CFL stability constraints across all cells.
impure subroutine s_tvd_rk(t_step, time_avg, nstage)
Advances the solution one full step using a TVD Runge-Kutta time integrator.
integer stor
storage index
Conservative-to-primitive variable conversion, mixture property evaluation, and pressure computation.
subroutine, public s_compute_pressure(energy, alf, dyn_p, pi_inf, gamma, rho, qv, rhoyks, pres, t, stress, mom, g, pres_mag)
This procedure conditionally calculates the appropriate pressure.
impure subroutine, public s_initialize_variables_conversion_module
The computation of parameters, the allocation of memory, the association of pointers and/or the execu...
impure subroutine s_finalize_variables_conversion_module()
Deallocates fluid property arrays and post-processing fields allocated during module initialization.
subroutine, public s_convert_to_mixture_variables(q_vf, i, j, k, rho, gamma, pi_inf, qv, re_k, g_k, g)
Dispatch to the s_convert_mixture_to_mixture_variables and s_convert_species_to_mixture_variables sub...
Computes viscous stress tensors and diffusive flux contributions for the Navier–Stokes equations.
impure subroutine, public s_initialize_viscous_module
Allocates and populates the viscous Reynolds number arrays and transfers data to the GPU.
impure subroutine, public s_finalize_viscous_module()
Deallocates the viscous Reynolds number arrays.
WENO/WENO-Z/TENO reconstruction with optional monotonicity-preserving bounds and mapped weights.
impure subroutine, public s_initialize_weno_module
The computation of parameters, the allocation of memory, the association of pointers and/or the execu...
impure subroutine, public s_finalize_weno_module()
Module deallocation and/or disassociation procedures.
Derived type annexing a scalar field (SF).