MFC
Exascale flow solver
Loading...
Searching...
No Matches
m_start_up.fpp.f90
Go to the documentation of this file.
1# 1 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
2!>
3!! @file
4!! @brief Contains module m_start_up
5
6# 1 "/home/runner/work/MFC/MFC/src/common/include/case.fpp" 1
7! This file exists so that Fypp can be run without generating case.fpp files for
8! each target. This is useful when generating documentation, for example. This
9! should also let MFC be built with CMake directly, without invoking mfc.sh.
10
11! For pre-process.
12# 9 "/home/runner/work/MFC/MFC/src/common/include/case.fpp"
13
14! For moving immersed boundaries in simulation
15# 14 "/home/runner/work/MFC/MFC/src/common/include/case.fpp"
16# 6 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp" 2
17# 1 "/home/runner/work/MFC/MFC/src/common/include/macros.fpp" 1
18# 1 "/home/runner/work/MFC/MFC/src/common/include/parallel_macros.fpp" 1
19# 1 "/home/runner/work/MFC/MFC/src/common/include/shared_parallel_macros.fpp" 1
20# 2 "/home/runner/work/MFC/MFC/src/common/include/shared_parallel_macros.fpp"
21# 3 "/home/runner/work/MFC/MFC/src/common/include/shared_parallel_macros.fpp"
22# 4 "/home/runner/work/MFC/MFC/src/common/include/shared_parallel_macros.fpp"
23# 5 "/home/runner/work/MFC/MFC/src/common/include/shared_parallel_macros.fpp"
24# 6 "/home/runner/work/MFC/MFC/src/common/include/shared_parallel_macros.fpp"
25
26# 8 "/home/runner/work/MFC/MFC/src/common/include/shared_parallel_macros.fpp"
27# 9 "/home/runner/work/MFC/MFC/src/common/include/shared_parallel_macros.fpp"
28# 10 "/home/runner/work/MFC/MFC/src/common/include/shared_parallel_macros.fpp"
29
30# 17 "/home/runner/work/MFC/MFC/src/common/include/shared_parallel_macros.fpp"
31
32# 46 "/home/runner/work/MFC/MFC/src/common/include/shared_parallel_macros.fpp"
33
34# 58 "/home/runner/work/MFC/MFC/src/common/include/shared_parallel_macros.fpp"
35
36# 68 "/home/runner/work/MFC/MFC/src/common/include/shared_parallel_macros.fpp"
37
38# 98 "/home/runner/work/MFC/MFC/src/common/include/shared_parallel_macros.fpp"
39
40# 110 "/home/runner/work/MFC/MFC/src/common/include/shared_parallel_macros.fpp"
41
42# 120 "/home/runner/work/MFC/MFC/src/common/include/shared_parallel_macros.fpp"
43! New line at end of file is required for FYPP
44# 2 "/home/runner/work/MFC/MFC/src/common/include/parallel_macros.fpp" 2
45# 1 "/home/runner/work/MFC/MFC/src/common/include/omp_macros.fpp" 1
46# 1 "/home/runner/work/MFC/MFC/src/common/include/shared_parallel_macros.fpp" 1
47# 2 "/home/runner/work/MFC/MFC/src/common/include/shared_parallel_macros.fpp"
48# 3 "/home/runner/work/MFC/MFC/src/common/include/shared_parallel_macros.fpp"
49# 4 "/home/runner/work/MFC/MFC/src/common/include/shared_parallel_macros.fpp"
50# 5 "/home/runner/work/MFC/MFC/src/common/include/shared_parallel_macros.fpp"
51# 6 "/home/runner/work/MFC/MFC/src/common/include/shared_parallel_macros.fpp"
52
53# 8 "/home/runner/work/MFC/MFC/src/common/include/shared_parallel_macros.fpp"
54# 9 "/home/runner/work/MFC/MFC/src/common/include/shared_parallel_macros.fpp"
55# 10 "/home/runner/work/MFC/MFC/src/common/include/shared_parallel_macros.fpp"
56
57# 17 "/home/runner/work/MFC/MFC/src/common/include/shared_parallel_macros.fpp"
58
59# 46 "/home/runner/work/MFC/MFC/src/common/include/shared_parallel_macros.fpp"
60
61# 58 "/home/runner/work/MFC/MFC/src/common/include/shared_parallel_macros.fpp"
62
63# 68 "/home/runner/work/MFC/MFC/src/common/include/shared_parallel_macros.fpp"
64
65# 98 "/home/runner/work/MFC/MFC/src/common/include/shared_parallel_macros.fpp"
66
67# 110 "/home/runner/work/MFC/MFC/src/common/include/shared_parallel_macros.fpp"
68
69# 120 "/home/runner/work/MFC/MFC/src/common/include/shared_parallel_macros.fpp"
70! New line at end of file is required for FYPP
71# 2 "/home/runner/work/MFC/MFC/src/common/include/omp_macros.fpp" 2
72
73# 4 "/home/runner/work/MFC/MFC/src/common/include/omp_macros.fpp"
74# 5 "/home/runner/work/MFC/MFC/src/common/include/omp_macros.fpp"
75# 6 "/home/runner/work/MFC/MFC/src/common/include/omp_macros.fpp"
76# 7 "/home/runner/work/MFC/MFC/src/common/include/omp_macros.fpp"
77# 8 "/home/runner/work/MFC/MFC/src/common/include/omp_macros.fpp"
78
79# 20 "/home/runner/work/MFC/MFC/src/common/include/omp_macros.fpp"
80
81# 43 "/home/runner/work/MFC/MFC/src/common/include/omp_macros.fpp"
82
83# 48 "/home/runner/work/MFC/MFC/src/common/include/omp_macros.fpp"
84
85# 53 "/home/runner/work/MFC/MFC/src/common/include/omp_macros.fpp"
86
87# 58 "/home/runner/work/MFC/MFC/src/common/include/omp_macros.fpp"
88
89# 63 "/home/runner/work/MFC/MFC/src/common/include/omp_macros.fpp"
90
91# 68 "/home/runner/work/MFC/MFC/src/common/include/omp_macros.fpp"
92
93# 76 "/home/runner/work/MFC/MFC/src/common/include/omp_macros.fpp"
94
95# 81 "/home/runner/work/MFC/MFC/src/common/include/omp_macros.fpp"
96
97# 86 "/home/runner/work/MFC/MFC/src/common/include/omp_macros.fpp"
98
99# 91 "/home/runner/work/MFC/MFC/src/common/include/omp_macros.fpp"
100
101# 96 "/home/runner/work/MFC/MFC/src/common/include/omp_macros.fpp"
102
103# 101 "/home/runner/work/MFC/MFC/src/common/include/omp_macros.fpp"
104
105# 106 "/home/runner/work/MFC/MFC/src/common/include/omp_macros.fpp"
106
107# 111 "/home/runner/work/MFC/MFC/src/common/include/omp_macros.fpp"
108
109# 116 "/home/runner/work/MFC/MFC/src/common/include/omp_macros.fpp"
110
111# 121 "/home/runner/work/MFC/MFC/src/common/include/omp_macros.fpp"
112
113# 151 "/home/runner/work/MFC/MFC/src/common/include/omp_macros.fpp"
114
115# 192 "/home/runner/work/MFC/MFC/src/common/include/omp_macros.fpp"
116
117# 207 "/home/runner/work/MFC/MFC/src/common/include/omp_macros.fpp"
118
119# 232 "/home/runner/work/MFC/MFC/src/common/include/omp_macros.fpp"
120
121# 243 "/home/runner/work/MFC/MFC/src/common/include/omp_macros.fpp"
122
123# 245 "/home/runner/work/MFC/MFC/src/common/include/omp_macros.fpp"
124# 255 "/home/runner/work/MFC/MFC/src/common/include/omp_macros.fpp"
125
126# 283 "/home/runner/work/MFC/MFC/src/common/include/omp_macros.fpp"
127
128# 293 "/home/runner/work/MFC/MFC/src/common/include/omp_macros.fpp"
129
130# 303 "/home/runner/work/MFC/MFC/src/common/include/omp_macros.fpp"
131
132# 312 "/home/runner/work/MFC/MFC/src/common/include/omp_macros.fpp"
133
134# 329 "/home/runner/work/MFC/MFC/src/common/include/omp_macros.fpp"
135
136# 339 "/home/runner/work/MFC/MFC/src/common/include/omp_macros.fpp"
137
138# 346 "/home/runner/work/MFC/MFC/src/common/include/omp_macros.fpp"
139
140# 352 "/home/runner/work/MFC/MFC/src/common/include/omp_macros.fpp"
141
142# 358 "/home/runner/work/MFC/MFC/src/common/include/omp_macros.fpp"
143
144# 364 "/home/runner/work/MFC/MFC/src/common/include/omp_macros.fpp"
145
146# 370 "/home/runner/work/MFC/MFC/src/common/include/omp_macros.fpp"
147
148# 376 "/home/runner/work/MFC/MFC/src/common/include/omp_macros.fpp"
149! New line at end of file is required for FYPP
150# 3 "/home/runner/work/MFC/MFC/src/common/include/parallel_macros.fpp" 2
151# 1 "/home/runner/work/MFC/MFC/src/common/include/acc_macros.fpp" 1
152# 1 "/home/runner/work/MFC/MFC/src/common/include/shared_parallel_macros.fpp" 1
153# 2 "/home/runner/work/MFC/MFC/src/common/include/shared_parallel_macros.fpp"
154# 3 "/home/runner/work/MFC/MFC/src/common/include/shared_parallel_macros.fpp"
155# 4 "/home/runner/work/MFC/MFC/src/common/include/shared_parallel_macros.fpp"
156# 5 "/home/runner/work/MFC/MFC/src/common/include/shared_parallel_macros.fpp"
157# 6 "/home/runner/work/MFC/MFC/src/common/include/shared_parallel_macros.fpp"
158
159# 8 "/home/runner/work/MFC/MFC/src/common/include/shared_parallel_macros.fpp"
160# 9 "/home/runner/work/MFC/MFC/src/common/include/shared_parallel_macros.fpp"
161# 10 "/home/runner/work/MFC/MFC/src/common/include/shared_parallel_macros.fpp"
162
163# 17 "/home/runner/work/MFC/MFC/src/common/include/shared_parallel_macros.fpp"
164
165# 46 "/home/runner/work/MFC/MFC/src/common/include/shared_parallel_macros.fpp"
166
167# 58 "/home/runner/work/MFC/MFC/src/common/include/shared_parallel_macros.fpp"
168
169# 68 "/home/runner/work/MFC/MFC/src/common/include/shared_parallel_macros.fpp"
170
171# 98 "/home/runner/work/MFC/MFC/src/common/include/shared_parallel_macros.fpp"
172
173# 110 "/home/runner/work/MFC/MFC/src/common/include/shared_parallel_macros.fpp"
174
175# 120 "/home/runner/work/MFC/MFC/src/common/include/shared_parallel_macros.fpp"
176! New line at end of file is required for FYPP
177# 2 "/home/runner/work/MFC/MFC/src/common/include/acc_macros.fpp" 2
178
179# 7 "/home/runner/work/MFC/MFC/src/common/include/acc_macros.fpp"
180
181# 17 "/home/runner/work/MFC/MFC/src/common/include/acc_macros.fpp"
182
183# 22 "/home/runner/work/MFC/MFC/src/common/include/acc_macros.fpp"
184
185# 27 "/home/runner/work/MFC/MFC/src/common/include/acc_macros.fpp"
186
187# 32 "/home/runner/work/MFC/MFC/src/common/include/acc_macros.fpp"
188
189# 37 "/home/runner/work/MFC/MFC/src/common/include/acc_macros.fpp"
190
191# 42 "/home/runner/work/MFC/MFC/src/common/include/acc_macros.fpp"
192
193# 47 "/home/runner/work/MFC/MFC/src/common/include/acc_macros.fpp"
194
195# 52 "/home/runner/work/MFC/MFC/src/common/include/acc_macros.fpp"
196
197# 57 "/home/runner/work/MFC/MFC/src/common/include/acc_macros.fpp"
198
199# 62 "/home/runner/work/MFC/MFC/src/common/include/acc_macros.fpp"
200
201# 73 "/home/runner/work/MFC/MFC/src/common/include/acc_macros.fpp"
202
203# 78 "/home/runner/work/MFC/MFC/src/common/include/acc_macros.fpp"
204
205# 83 "/home/runner/work/MFC/MFC/src/common/include/acc_macros.fpp"
206
207# 88 "/home/runner/work/MFC/MFC/src/common/include/acc_macros.fpp"
208
209# 103 "/home/runner/work/MFC/MFC/src/common/include/acc_macros.fpp"
210
211# 131 "/home/runner/work/MFC/MFC/src/common/include/acc_macros.fpp"
212
213# 160 "/home/runner/work/MFC/MFC/src/common/include/acc_macros.fpp"
214
215# 175 "/home/runner/work/MFC/MFC/src/common/include/acc_macros.fpp"
216
217# 192 "/home/runner/work/MFC/MFC/src/common/include/acc_macros.fpp"
218
219# 213 "/home/runner/work/MFC/MFC/src/common/include/acc_macros.fpp"
220
221# 241 "/home/runner/work/MFC/MFC/src/common/include/acc_macros.fpp"
222
223# 256 "/home/runner/work/MFC/MFC/src/common/include/acc_macros.fpp"
224
225# 266 "/home/runner/work/MFC/MFC/src/common/include/acc_macros.fpp"
226
227# 275 "/home/runner/work/MFC/MFC/src/common/include/acc_macros.fpp"
228
229# 291 "/home/runner/work/MFC/MFC/src/common/include/acc_macros.fpp"
230
231# 301 "/home/runner/work/MFC/MFC/src/common/include/acc_macros.fpp"
232
233# 308 "/home/runner/work/MFC/MFC/src/common/include/acc_macros.fpp"
234! New line at end of file is required for FYPP
235# 4 "/home/runner/work/MFC/MFC/src/common/include/parallel_macros.fpp" 2
236
237# 21 "/home/runner/work/MFC/MFC/src/common/include/parallel_macros.fpp"
238
239# 37 "/home/runner/work/MFC/MFC/src/common/include/parallel_macros.fpp"
240
241# 50 "/home/runner/work/MFC/MFC/src/common/include/parallel_macros.fpp"
242
243# 104 "/home/runner/work/MFC/MFC/src/common/include/parallel_macros.fpp"
244
245# 119 "/home/runner/work/MFC/MFC/src/common/include/parallel_macros.fpp"
246
247# 130 "/home/runner/work/MFC/MFC/src/common/include/parallel_macros.fpp"
248
249# 143 "/home/runner/work/MFC/MFC/src/common/include/parallel_macros.fpp"
250
251# 171 "/home/runner/work/MFC/MFC/src/common/include/parallel_macros.fpp"
252
253# 182 "/home/runner/work/MFC/MFC/src/common/include/parallel_macros.fpp"
254
255# 193 "/home/runner/work/MFC/MFC/src/common/include/parallel_macros.fpp"
256
257# 204 "/home/runner/work/MFC/MFC/src/common/include/parallel_macros.fpp"
258
259# 214 "/home/runner/work/MFC/MFC/src/common/include/parallel_macros.fpp"
260
261# 225 "/home/runner/work/MFC/MFC/src/common/include/parallel_macros.fpp"
262
263# 236 "/home/runner/work/MFC/MFC/src/common/include/parallel_macros.fpp"
264
265# 246 "/home/runner/work/MFC/MFC/src/common/include/parallel_macros.fpp"
266
267# 252 "/home/runner/work/MFC/MFC/src/common/include/parallel_macros.fpp"
268
269# 258 "/home/runner/work/MFC/MFC/src/common/include/parallel_macros.fpp"
270
271# 264 "/home/runner/work/MFC/MFC/src/common/include/parallel_macros.fpp"
272
273# 270 "/home/runner/work/MFC/MFC/src/common/include/parallel_macros.fpp"
274
275# 272 "/home/runner/work/MFC/MFC/src/common/include/parallel_macros.fpp"
276# 273 "/home/runner/work/MFC/MFC/src/common/include/parallel_macros.fpp"
277! New line at end of file is required for FYPP
278# 2 "/home/runner/work/MFC/MFC/src/common/include/macros.fpp" 2
279
280# 14 "/home/runner/work/MFC/MFC/src/common/include/macros.fpp"
281
282! Caution:
283! This macro requires the use of a binding script to set CUDA_VISIBLE_DEVICES, such that we have one GPU device per MPI rank.
284! That's because for both cudaMemAdvise (preferred location) and cudaMemPrefetchAsync we use location = device_id = 0.
285! For an example see misc/nvidia_uvm/bind.sh.
286# 63 "/home/runner/work/MFC/MFC/src/common/include/macros.fpp"
287
288# 81 "/home/runner/work/MFC/MFC/src/common/include/macros.fpp"
289
290# 88 "/home/runner/work/MFC/MFC/src/common/include/macros.fpp"
291
292# 111 "/home/runner/work/MFC/MFC/src/common/include/macros.fpp"
293
294# 127 "/home/runner/work/MFC/MFC/src/common/include/macros.fpp"
295
296# 153 "/home/runner/work/MFC/MFC/src/common/include/macros.fpp"
297
298# 159 "/home/runner/work/MFC/MFC/src/common/include/macros.fpp"
299
300# 167 "/home/runner/work/MFC/MFC/src/common/include/macros.fpp"
301! New line at end of file is required for FYPP
302# 7 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp" 2
303
304!> @brief Reads input files, loads initial conditions and grid data, and orchestrates solver initialization and finalization
306
307 use m_derived_types !< definitions of the derived types
308
309 use m_global_parameters !< definitions of the global parameters
310
311 use m_mpi_proxy !< message passing interface (mpi) module proxy
312
313 use m_mpi_common
314
315 use m_variables_conversion !< state variables type conversion procedures
316
317 use m_weno !< weighted and essentially non-oscillatory (weno)
318 !! schemes for spatial reconstruction of variables
319
320 use m_muscl !< monotonic upstream-centered (muscl)
321 !! schemes for convservation laws
322
323 use m_riemann_solvers !< exact and approximate riemann problem solvers
324
325 use m_cbc !< characteristic boundary conditions (cbc)
326
328
329 use m_acoustic_src !< acoustic source calculations
330
331 use m_rhs !< right-hane-side (rhs) evaluation procedures
332
333 use m_chemistry !< chemistry module
334
335 use m_data_output !< run-time info & solution data output procedures
336
337 use m_time_steppers !< time-stepping algorithms
338
339 use m_qbmm !< quadrature mom
340
341 use m_derived_variables !< procedures used to compute quantities derived
342 !! from the conservative and primitive variables
343 use m_hypoelastic
344
346
347 use m_phase_change !< phase-change module
348
349 use m_viscous
350
351 use m_bubbles_ee !< ensemble-averaged bubble dynamics routines
352
353 use m_bubbles_el !< lagrange bubble dynamics routines
354
355 use ieee_arithmetic
356
357 use m_helper_basic !< functions to compare floating point numbers
358
359 use m_helper
360
361
362# 65 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
363#if defined(MFC_OpenACC)
364# 65 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
365 use openacc
366# 65 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
367#elif defined(MFC_OpenMP)
368# 65 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
369 use omp_lib
370# 65 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
371#endif
372# 65 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
373
374
375 use m_nvtx
376
377 use m_ibm
378
380
382
383 use m_checker
384
386
387 use m_body_forces
388
389 use m_sim_helpers
390
391 use m_igr
392
393 implicit none
394
395 private; public :: s_read_input_file, &
405
406 type(scalar_field), allocatable, dimension(:) :: q_cons_temp
407
408 real(wp) :: dt_init
409
410contains
411
412 !> Read data files. Dispatch subroutine that replaces procedure pointer.
413 !! @param q_cons_vf Conservative variables
414 impure subroutine s_read_data_files(q_cons_vf)
415
416 type(scalar_field), &
417 dimension(sys_size), &
418 intent(inout) :: q_cons_vf
419
420 if (.not. parallel_io) then
422 else
424 end if
425
426 end subroutine s_read_data_files
427
428 !> The purpose of this procedure is to first verify that an
429 !! input file has been made available by the user. Provided
430 !! that this is so, the input file is then read in.
431 impure subroutine s_read_input_file
432
433 ! Relative path to the input file provided by the user
434 character(LEN=name_len), parameter :: file_path = './simulation.inp'
435
436 logical :: file_exist !<
437 !! Logical used to check the existence of the input file
438
439 integer :: iostatus
440 !! Integer to check iostat of file read
441
442 character(len=1000) :: line
443
444 ! Namelist of the global parameters which may be specified by user
445 namelist /user_inputs/ case_dir, run_time_info, m, n, p, dt, &
450 bc_x, bc_y, bc_z, &
451 x_a, y_a, z_a, x_b, y_b, z_b, &
454 ib, num_ibs, patch_ib, &
461# 154 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
466# 159 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
467 ca, web, re_inv, &
476 bf_x, bf_y, bf_z, &
477 k_x, k_y, k_z, w_x, w_y, w_z, p_x, p_y, p_z, &
487
488 ! Checking that an input file has been provided by the user. If it
489 ! has, then the input file is read in, otherwise, simulation exits.
490 inquire (file=trim(file_path), exist=file_exist)
491
492 if (file_exist) then
493 open (1, file=trim(file_path), &
494 form='formatted', &
495 action='read', &
496 status='old')
497 read (1, nml=user_inputs, iostat=iostatus)
498
499 if (iostatus /= 0) then
500 backspace(1)
501 read (1, fmt='(A)') line
502 print *, 'Invalid line in namelist: '//trim(line)
503 call s_mpi_abort('Invalid line in simulation.inp. It is '// &
504 'likely due to a datatype mismatch. Exiting.')
505 end if
506
507 close (1)
508
509 if ((bf_x) .or. (bf_y) .or. (bf_z)) then
510 bodyforces = .true.
511 end if
512
513 ! Store m,n,p into global m,n,p
514 m_glb = m
515 n_glb = n
516 p_glb = p
517
519
520 if (cfl_adap_dt .or. cfl_const_dt) cfl_dt = .true.
521
522 if (any((/bc_x%beg, bc_x%end, bc_y%beg, bc_y%end, bc_z%beg, bc_z%end/) == -17) .or. &
523 num_bc_patches > 0) then
524 bc_io = .true.
525 end if
526
527 else
528 call s_mpi_abort(trim(file_path)//' is missing. Exiting.')
529 end if
530
531 end subroutine s_read_input_file
532
533 !> The goal of this procedure is to verify that each of the
534 !! user provided inputs is valid and that their combination
535 !! constitutes a meaningful configuration for the simulation.
536 impure subroutine s_check_input_file
537
538 ! Relative path to the current directory file in the case directory
539 character(LEN=path_len) :: file_path
540
541 ! Logical used to check the existence of the current directory file
542 logical :: file_exist
543
544 ! Logistics
545 file_path = trim(case_dir)//'/.'
546
547 call my_inquire(file_path, file_exist)
548
549 if (file_exist .neqv. .true.) then
550 call s_mpi_abort(trim(file_path)//' is missing. Exiting.')
551 end if
552
554 call s_check_inputs()
555
556 end subroutine s_check_input_file
557
558 !> @brief Reads serial initial condition and grid data files and computes cell-width distributions.
559 !! @param q_cons_vf Cell-averaged conservative variables
560 impure subroutine s_read_serial_data_files(q_cons_vf)
561
562 type(scalar_field), dimension(sys_size), intent(INOUT) :: q_cons_vf
563
564 character(LEN=path_len + 2*name_len) :: t_step_dir !<
565 !! Relative path to the starting time-step directory
566
567 character(LEN=path_len + 3*name_len) :: file_path !<
568 !! Relative path to the grid and conservative variables data files
569
570 logical :: file_exist !<
571 ! Logical used to check the existence of the data files
572
573 integer :: i, r !< Generic loop iterator
574
575 ! Confirming that the directory from which the initial condition and
576 ! the grid data files are to be read in exists and exiting otherwise
577 if (cfl_dt) then
578 write (t_step_dir, '(A,I0,A,I0)') &
579 trim(case_dir)//'/p_all/p', proc_rank, '/', n_start
580 else
581 write (t_step_dir, '(A,I0,A,I0)') &
582 trim(case_dir)//'/p_all/p', proc_rank, '/', t_step_start
583 end if
584
585 file_path = trim(t_step_dir)//'/.'
586 call my_inquire(file_path, file_exist)
587
588 if (file_exist .neqv. .true.) then
589 call s_mpi_abort(trim(file_path)//' is missing. Exiting.')
590 end if
591
592 if (bc_io) then
594 else
596 end if
597
598 ! Cell-boundary Locations in x-direction
599 file_path = trim(t_step_dir)//'/x_cb.dat'
600
601 inquire (file=trim(file_path), exist=file_exist)
602
603 if (file_exist) then
604 open (2, file=trim(file_path), &
605 form='unformatted', &
606 action='read', &
607 status='old')
608 read (2) x_cb(-1:m); close (2)
609 else
610 call s_mpi_abort(trim(file_path)//' is missing. Exiting.')
611 end if
612
613 dx(0:m) = x_cb(0:m) - x_cb(-1:m - 1)
614 x_cc(0:m) = x_cb(-1:m - 1) + dx(0:m)/2._wp
615
616 if (ib) then
617 do i = 1, num_ibs
618 if (patch_ib(i)%c > 0) then
619 np = int((patch_ib(i)%p*patch_ib(i)%c/dx(0))*20) + int(((patch_ib(i)%c - patch_ib(i)%p*patch_ib(i)%c)/dx(0))*20) + 1
620 end if
621 end do
622 end if
623
624 ! Cell-boundary Locations in y-direction
625 if (n > 0) then
626
627 file_path = trim(t_step_dir)//'/y_cb.dat'
628
629 inquire (file=trim(file_path), exist=file_exist)
630
631 if (file_exist) then
632 open (2, file=trim(file_path), &
633 form='unformatted', &
634 action='read', &
635 status='old')
636 read (2) y_cb(-1:n); close (2)
637 else
638 call s_mpi_abort(trim(file_path)//' is missing. Exiting.')
639 end if
640
641 dy(0:n) = y_cb(0:n) - y_cb(-1:n - 1)
642 y_cc(0:n) = y_cb(-1:n - 1) + dy(0:n)/2._wp
643
644 end if
645
646 ! Cell-boundary Locations in z-direction
647 if (p > 0) then
648
649 file_path = trim(t_step_dir)//'/z_cb.dat'
650
651 inquire (file=trim(file_path), exist=file_exist)
652
653 if (file_exist) then
654 open (2, file=trim(file_path), &
655 form='unformatted', &
656 action='read', &
657 status='old')
658 read (2) z_cb(-1:p); close (2)
659 else
660 call s_mpi_abort(trim(file_path)//' is missing. Exiting.')
661 end if
662
663 dz(0:p) = z_cb(0:p) - z_cb(-1:p - 1)
664 z_cc(0:p) = z_cb(-1:p - 1) + dz(0:p)/2._wp
665
666 end if
667
668 do i = 1, sys_size
669 write (file_path, '(A,I0,A)') &
670 trim(t_step_dir)//'/q_cons_vf', i, '.dat'
671 inquire (file=trim(file_path), exist=file_exist)
672 if (file_exist) then
673 open (2, file=trim(file_path), &
674 form='unformatted', &
675 action='read', &
676 status='old')
677 read (2) q_cons_vf(i)%sf(0:m, 0:n, 0:p); close (2)
678 else
679 call s_mpi_abort(trim(file_path)//' is missing. Exiting.')
680 end if
681 end do
682
683 if (bubbles_euler .or. elasticity) then
684 ! Read pb and mv for non-polytropic qbmm
685 if (qbmm .and. .not. polytropic) then
686 do i = 1, nb
687 do r = 1, nnode
688 write (file_path, '(A,I0,A)') &
689 trim(t_step_dir)//'/pb', sys_size + (i - 1)*nnode + r, '.dat'
690 inquire (file=trim(file_path), exist=file_exist)
691 if (file_exist) then
692 open (2, file=trim(file_path), &
693 form='unformatted', &
694 action='read', &
695 status='old')
696 read (2) pb_ts(1)%sf(0:m, 0:n, 0:p, r, i); close (2)
697 else
698 call s_mpi_abort(trim(file_path)//' is missing. Exiting.')
699 end if
700 end do
701 end do
702 do i = 1, nb
703 do r = 1, nnode
704 write (file_path, '(A,I0,A)') &
705 trim(t_step_dir)//'/mv', sys_size + (i - 1)*nnode + r, '.dat'
706 inquire (file=trim(file_path), exist=file_exist)
707 if (file_exist) then
708 open (2, file=trim(file_path), &
709 form='unformatted', &
710 action='read', &
711 status='old')
712 read (2) mv_ts(1)%sf(0:m, 0:n, 0:p, r, i); close (2)
713 else
714 call s_mpi_abort(trim(file_path)//' is missing. Exiting.')
715 end if
716 end do
717 end do
718 end if
719 end if
720
721 end subroutine s_read_serial_data_files
722
723 !> @brief Reads parallel initial condition and grid data files via MPI I/O.
724 !! @param q_cons_vf Conservative variables
725 impure subroutine s_read_parallel_data_files(q_cons_vf)
726
727 type(scalar_field), &
728 dimension(sys_size), &
729 intent(INOUT) :: q_cons_vf
730
731#ifdef MFC_MPI
732
733 real(wp), allocatable, dimension(:) :: x_cb_glb, y_cb_glb, z_cb_glb
734
735 integer :: ifile, ierr, data_size
736 integer, dimension(MPI_STATUS_SIZE) :: status
737 integer(KIND=MPI_OFFSET_KIND) :: disp
738 integer(KIND=MPI_OFFSET_KIND) :: m_mok, n_mok, p_mok
739 integer(KIND=MPI_OFFSET_KIND) :: wp_mok, var_mok, str_mok
740 integer(KIND=MPI_OFFSET_KIND) :: nvars_mok
741 integer(KIND=MPI_OFFSET_KIND) :: mok
742
743 character(LEN=path_len + 2*name_len) :: file_loc
744 logical :: file_exist
745
746 character(len=10) :: t_step_start_string
747
748 integer :: i, j
749
750 ! Downsampled data variables
751 integer :: m_ds, n_ds, p_ds
752 integer :: m_glb_ds, n_glb_ds, p_glb_ds
753 integer :: m_glb_read, n_glb_read, p_glb_read ! data size of read
754
755 allocate (x_cb_glb(-1:m_glb))
756 allocate (y_cb_glb(-1:n_glb))
757 allocate (z_cb_glb(-1:p_glb))
758
759 ! Read in cell boundary locations in x-direction
760 file_loc = trim(case_dir)//'/restart_data'//trim(mpiiofs)//'x_cb.dat'
761 inquire (file=trim(file_loc), exist=file_exist)
762
763 if (down_sample) then
764 m_ds = int((m + 1)/3) - 1
765 n_ds = int((n + 1)/3) - 1
766 p_ds = int((p + 1)/3) - 1
767
768 m_glb_ds = int((m_glb + 1)/3) - 1
769 n_glb_ds = int((n_glb + 1)/3) - 1
770 p_glb_ds = int((p_glb + 1)/3) - 1
771 end if
772
773 if (file_exist) then
774 data_size = m_glb + 2
775 call mpi_file_open(mpi_comm_world, file_loc, mpi_mode_rdonly, mpi_info_int, ifile, ierr)
776 call mpi_file_read(ifile, x_cb_glb, data_size, mpi_p, status, ierr)
777 call mpi_file_close(ifile, ierr)
778 else
779 call s_mpi_abort('File '//trim(file_loc)//' is missing. Exiting.')
780 end if
781
782 ! Assigning local cell boundary locations
783 x_cb(-1:m) = x_cb_glb((start_idx(1) - 1):(start_idx(1) + m))
784 ! Computing the cell width distribution
785 dx(0:m) = x_cb(0:m) - x_cb(-1:m - 1)
786 ! Computing the cell center locations
787 x_cc(0:m) = x_cb(-1:m - 1) + dx(0:m)/2._wp
788
789 if (ib) then
790 do i = 1, num_ibs
791 if (patch_ib(i)%c > 0) then
792 np = int((patch_ib(i)%p*patch_ib(i)%c/dx(0))*20) + int(((patch_ib(i)%c - patch_ib(i)%p*patch_ib(i)%c)/dx(0))*20) + 1
793 allocate (mpi_io_airfoil_ib_data%var(1:2*np))
794 end if
795 end do
796 end if
797
798 if (n > 0) then
799 ! Read in cell boundary locations in y-direction
800 file_loc = trim(case_dir)//'/restart_data'//trim(mpiiofs)//'y_cb.dat'
801 inquire (file=trim(file_loc), exist=file_exist)
802
803 if (file_exist) then
804 data_size = n_glb + 2
805 call mpi_file_open(mpi_comm_world, file_loc, mpi_mode_rdonly, mpi_info_int, ifile, ierr)
806 call mpi_file_read(ifile, y_cb_glb, data_size, mpi_p, status, ierr)
807 call mpi_file_close(ifile, ierr)
808 else
809 call s_mpi_abort('File '//trim(file_loc)//' is missing. Exiting.')
810 end if
811
812 ! Assigning local cell boundary locations
813 y_cb(-1:n) = y_cb_glb((start_idx(2) - 1):(start_idx(2) + n))
814 ! Computing the cell width distribution
815 dy(0:n) = y_cb(0:n) - y_cb(-1:n - 1)
816 ! Computing the cell center locations
817 y_cc(0:n) = y_cb(-1:n - 1) + dy(0:n)/2._wp
818
819 if (p > 0) then
820 ! Read in cell boundary locations in z-direction
821 file_loc = trim(case_dir)//'/restart_data'//trim(mpiiofs)//'z_cb.dat'
822 inquire (file=trim(file_loc), exist=file_exist)
823
824 if (file_exist) then
825 data_size = p_glb + 2
826 call mpi_file_open(mpi_comm_world, file_loc, mpi_mode_rdonly, mpi_info_int, ifile, ierr)
827 call mpi_file_read(ifile, z_cb_glb, data_size, mpi_p, status, ierr)
828 call mpi_file_close(ifile, ierr)
829 else
830 call s_mpi_abort('File '//trim(file_loc)//'is missing. Exiting.')
831 end if
832
833 ! Assigning local cell boundary locations
834 z_cb(-1:p) = z_cb_glb((start_idx(3) - 1):(start_idx(3) + p))
835 ! Computing the cell width distribution
836 dz(0:p) = z_cb(0:p) - z_cb(-1:p - 1)
837 ! Computing the cell center locations
838 z_cc(0:p) = z_cb(-1:p - 1) + dz(0:p)/2._wp
839
840 end if
841 end if
842
843 if (file_per_process) then
844 if (cfl_dt) then
845 call s_int_to_str(n_start, t_step_start_string)
846 write (file_loc, '(I0,A1,I7.7,A)') n_start, '_', proc_rank, '.dat'
847 else
848 call s_int_to_str(t_step_start, t_step_start_string)
849 write (file_loc, '(I0,A1,I7.7,A)') t_step_start, '_', proc_rank, '.dat'
850 end if
851 file_loc = trim(case_dir)//'/restart_data/lustre_'//trim(t_step_start_string)//trim(mpiiofs)//trim(file_loc)
852 inquire (file=trim(file_loc), exist=file_exist)
853
854 if (file_exist) then
855 call mpi_file_open(mpi_comm_self, file_loc, mpi_mode_rdonly, mpi_info_int, ifile, ierr)
856
857 ! Initialize MPI data I/O
858 if (down_sample) then
860 else
861 if (ib) then
863 else
865 end if
866 end if
867
868 if (down_sample) then
869 ! Size of local arrays
870 data_size = (m_ds + 3)*(n_ds + 3)*(p_ds + 3)
871 m_glb_read = m_glb_ds + 1
872 n_glb_read = n_glb_ds + 1
873 p_glb_read = p_glb_ds + 1
874 else
875 ! Size of local arrays
876 data_size = (m + 1)*(n + 1)*(p + 1)
877 m_glb_read = m_glb + 1
878 n_glb_read = n_glb + 1
879 p_glb_read = p_glb + 1
880 end if
881
882 ! Resize some integers so MPI can read even the biggest file
883 m_mok = int(m_glb_read + 1, mpi_offset_kind)
884 n_mok = int(m_glb_read + 1, mpi_offset_kind)
885 p_mok = int(m_glb_read + 1, mpi_offset_kind)
886 wp_mok = int(4._wp, mpi_offset_kind)
887 mok = int(1._wp, mpi_offset_kind)
888 str_mok = int(name_len, mpi_offset_kind)
889 nvars_mok = int(sys_size, mpi_offset_kind)
890
891 ! Read the data for each variable
892 if (bubbles_euler .or. elasticity) then
893 do i = 1, sys_size!adv_idx%end
894 var_mok = int(i, mpi_offset_kind)
895
896 call mpi_file_read(ifile, mpi_io_data%var(i)%sf, data_size*mpi_io_type, &
897 mpi_io_p, status, ierr)
898 end do
899 !Read pb and mv for non-polytropic qbmm
900 if (qbmm .and. .not. polytropic) then
901 do i = sys_size + 1, sys_size + 2*nb*nnode
902 var_mok = int(i, mpi_offset_kind)
903
904 call mpi_file_read(ifile, mpi_io_data%var(i)%sf, data_size*mpi_io_type, &
905 mpi_io_p, status, ierr)
906 end do
907 end if
908 else
909 if (down_sample) then
910 do i = 1, sys_size
911 var_mok = int(i, mpi_offset_kind)
912
913 call mpi_file_read(ifile, q_cons_temp(i)%sf, data_size*mpi_io_type, &
914 mpi_io_p, status, ierr)
915 end do
916 else
917 do i = 1, sys_size
918 var_mok = int(i, mpi_offset_kind)
919
920 call mpi_file_read(ifile, mpi_io_data%var(i)%sf, data_size*mpi_io_type, &
921 mpi_io_p, status, ierr)
922 end do
923 end if
924 end if
925
926 call s_mpi_barrier()
927
928 call mpi_file_close(ifile, ierr)
929
930 else
931 call s_mpi_abort('File '//trim(file_loc)//' is missing. Exiting.')
932 end if
933 else
934 ! Open the file to read conservative variables
935 if (cfl_dt) then
936 write (file_loc, '(I0,A)') n_start, '.dat'
937 else
938 write (file_loc, '(I0,A)') t_step_start, '.dat'
939 end if
940 file_loc = trim(case_dir)//'/restart_data'//trim(mpiiofs)//trim(file_loc)
941 inquire (file=trim(file_loc), exist=file_exist)
942
943 if (file_exist) then
944 call mpi_file_open(mpi_comm_world, file_loc, mpi_mode_rdonly, mpi_info_int, ifile, ierr)
945
946 ! Initialize MPI data I/O
947
948 if (ib) then
950 else
951
953
954 end if
955
956 ! Size of local arrays
957 data_size = (m + 1)*(n + 1)*(p + 1)
958
959 ! Resize some integers so MPI can read even the biggest file
960 m_mok = int(m_glb + 1, mpi_offset_kind)
961 n_mok = int(n_glb + 1, mpi_offset_kind)
962 p_mok = int(p_glb + 1, mpi_offset_kind)
963 wp_mok = int(8._wp, mpi_offset_kind)
964 mok = int(1._wp, mpi_offset_kind)
965 str_mok = int(name_len, mpi_offset_kind)
966 nvars_mok = int(sys_size, mpi_offset_kind)
967
968 ! Read the data for each variable
969 if (bubbles_euler .or. elasticity) then
970 do i = 1, sys_size !adv_idx%end
971 var_mok = int(i, mpi_offset_kind)
972 ! Initial displacement to skip at beginning of file
973 disp = m_mok*max(mok, n_mok)*max(mok, p_mok)*wp_mok*(var_mok - 1)
974
975 call mpi_file_set_view(ifile, disp, mpi_io_p, mpi_io_data%view(i), &
976 'native', mpi_info_int, ierr)
977 call mpi_file_read(ifile, mpi_io_data%var(i)%sf, data_size*mpi_io_type, &
978 mpi_io_p, status, ierr)
979 end do
980 !Read pb and mv for non-polytropic qbmm
981 if (qbmm .and. .not. polytropic) then
982 do i = sys_size + 1, sys_size + 2*nb*nnode
983 var_mok = int(i, mpi_offset_kind)
984 ! Initial displacement to skip at beginning of file
985 disp = m_mok*max(mok, n_mok)*max(mok, p_mok)*wp_mok*(var_mok - 1)
986
987 call mpi_file_set_view(ifile, disp, mpi_io_p, mpi_io_data%view(i), &
988 'native', mpi_info_int, ierr)
989 call mpi_file_read(ifile, mpi_io_data%var(i)%sf, data_size*mpi_io_type, &
990 mpi_io_p, status, ierr)
991 end do
992 end if
993 else
994 do i = 1, sys_size
995 var_mok = int(i, mpi_offset_kind)
996
997 ! Initial displacement to skip at beginning of file
998 disp = m_mok*max(mok, n_mok)*max(mok, p_mok)*wp_mok*(var_mok - 1)
999
1000 call mpi_file_set_view(ifile, disp, mpi_io_p, mpi_io_data%view(i), &
1001 'native', mpi_info_int, ierr)
1002 call mpi_file_read_all(ifile, mpi_io_data%var(i)%sf, data_size*mpi_io_type, &
1003 mpi_io_p, status, ierr)
1004 end do
1005 end if
1006
1007 call s_mpi_barrier()
1008
1009 call mpi_file_close(ifile, ierr)
1010
1011 else
1012 call s_mpi_abort('File '//trim(file_loc)//' is missing. Exiting.')
1013 end if
1014
1015 end if
1016
1017 deallocate (x_cb_glb, y_cb_glb, z_cb_glb)
1018
1019 if (bc_io) then
1021 else
1023 end if
1024
1025#endif
1026
1027 end subroutine s_read_parallel_data_files
1028
1029 !> The purpose of this procedure is to initialize the
1030 !! values of the internal-energy equations of each phase
1031 !! from the mass of each phase, the mixture momentum and
1032 !! mixture-total-energy equations.
1033 !! @param v_vf conservative variables
1035
1036 type(scalar_field), dimension(sys_size), intent(inout) :: v_vf
1037
1038 real(wp) :: rho
1039 real(wp) :: dyn_pres
1040 real(wp) :: gamma
1041 real(wp) :: pi_inf
1042 real(wp) :: qv
1043 real(wp), dimension(2) :: re
1044 real(wp) :: pres, t
1045
1046 integer :: i, j, k, l, c
1047
1048 real(wp), dimension(num_species) :: rhoyks
1049
1050 real(wp) :: pres_mag
1051
1052 pres_mag = 0._wp
1053
1054 t = dflt_t_guess
1055
1056 do j = 0, m
1057 do k = 0, n
1058 do l = 0, p
1059
1060 call s_convert_to_mixture_variables(v_vf, j, k, l, rho, gamma, pi_inf, qv, re)
1061
1062 dyn_pres = 0._wp
1063 do i = mom_idx%beg, mom_idx%end
1064 dyn_pres = dyn_pres + 5.e-1_wp*v_vf(i)%sf(j, k, l)*v_vf(i)%sf(j, k, l) &
1065 /max(rho, sgm_eps)
1066 end do
1067
1068 if (chemistry) then
1069 do c = 1, num_species
1070 rhoyks(c) = v_vf(chemxb + c - 1)%sf(j, k, l)
1071 end do
1072 end if
1073
1074 if (mhd) then
1075 if (n == 0) then
1076 pres_mag = 0.5_wp*(bx0**2 + v_vf(b_idx%beg)%sf(j, k, l)**2 + v_vf(b_idx%beg + 1)%sf(j, k, l)**2)
1077 else
1078 pres_mag = 0.5_wp*(v_vf(b_idx%beg)%sf(j, k, l)**2 + v_vf(b_idx%beg + 1)%sf(j, k, l)**2 + v_vf(b_idx%beg + 2)%sf(j, k, l)**2)
1079 end if
1080 end if
1081
1082 call s_compute_pressure(v_vf(e_idx)%sf(j, k, l), 0._stp, &
1083 dyn_pres, pi_inf, gamma, rho, qv, rhoyks, pres, t, pres_mag=pres_mag)
1084
1085 do i = 1, num_fluids
1086 v_vf(i + intxb - 1)%sf(j, k, l) = v_vf(i + advxb - 1)%sf(j, k, l)*(gammas(i)*pres + pi_infs(i)) &
1087 + v_vf(i + contxb - 1)%sf(j, k, l)*qvs(i)
1088 end do
1089
1090 end do
1091 end do
1092 end do
1093
1095
1096 !> @brief Advances the simulation by one time step, handling CFL-based dt and time-stepper dispatch.
1097 impure subroutine s_perform_time_step(t_step, time_avg)
1098 integer, intent(inout) :: t_step
1099 real(wp), intent(inout) :: time_avg
1100
1101 integer :: i
1102
1103 if (cfl_dt) then
1104 if (cfl_const_dt .and. t_step == 0) call s_compute_dt()
1105
1106 if (cfl_adap_dt) call s_compute_dt()
1107
1108 if (t_step == 0) dt_init = dt
1109
1110 if (dt < 1.e-3_wp*dt_init .and. cfl_adap_dt .and. proc_rank == 0) then
1111 print *, "Delta t = ", dt
1112 call s_mpi_abort("Delta t has become too small")
1113 end if
1114 end if
1115
1116 if (cfl_dt) then
1117 if ((mytime + dt) >= t_stop) then
1118 dt = t_stop - mytime
1119
1120# 811 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1121#if defined(MFC_OpenACC)
1122# 811 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1123!$acc update device(dt)
1124# 811 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1125#elif defined(MFC_OpenMP)
1126# 811 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1127!$omp target update to(dt)
1128# 811 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1129#endif
1130 end if
1131 else
1132 if ((mytime + dt) >= finaltime) then
1133 dt = finaltime - mytime
1134
1135# 816 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1136#if defined(MFC_OpenACC)
1137# 816 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1138!$acc update device(dt)
1139# 816 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1140#elif defined(MFC_OpenMP)
1141# 816 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1142!$omp target update to(dt)
1143# 816 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1144#endif
1145 end if
1146 end if
1147
1148 if (cfl_dt) then
1149 if (proc_rank == 0 .and. mod(t_step - t_step_start, t_step_print) == 0) then
1150 print '(" [", I3, "%] Time ", ES16.6, " dt = ", ES16.6, " @ Time Step = ", I8, " Time Avg = ", ES16.6, " Time/step = ", ES12.6, "")', &
1151 int(ceiling(100._wp*(mytime/t_stop))), &
1152 mytime, &
1153 dt, &
1154 t_step, &
1155 wall_time_avg, &
1156 wall_time
1157 end if
1158 else
1159 if (proc_rank == 0 .and. mod(t_step - t_step_start, t_step_print) == 0) then
1160 print '(" [", I3, "%] Time step ", I8, " of ", I0, " @ t_step = ", I8, " Time Avg = ", ES12.6, " Time/step= ", ES12.6, "")', &
1161 int(ceiling(100._wp*(real(t_step - t_step_start)/(t_step_stop - t_step_start + 1)))), &
1162 t_step - t_step_start + 1, &
1163 t_step_stop - t_step_start + 1, &
1164 t_step, &
1165 wall_time_avg, &
1166 wall_time
1167 end if
1168 end if
1169
1170 if (probe_wrt) then
1171 do i = 1, sys_size
1172
1173# 844 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1174#if defined(MFC_OpenACC)
1175# 844 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1176!$acc update host(q_cons_ts(1)%vf(i)%sf)
1177# 844 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1178#elif defined(MFC_OpenMP)
1179# 844 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1180!$omp target update from(q_cons_ts(1)%vf(i)%sf)
1181# 844 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1182#endif
1183 end do
1184 end if
1185
1186 mytime = mytime + dt
1187
1188 ! Total-variation-diminishing (TVD) Runge-Kutta (RK) time-steppers
1189 if (any(time_stepper == (/1, 2, 3/))) then
1190 call s_tvd_rk(t_step, time_avg, time_stepper)
1191 end if
1192
1193 if (relax) call s_infinite_relaxation_k(q_cons_ts(1)%vf)
1194
1195 ! Time-stepping loop controls
1196 t_step = t_step + 1
1197
1198 end subroutine s_perform_time_step
1199
1200 !> @brief Collects per-process wall-clock times and writes aggregate performance metrics to file.
1201 impure subroutine s_save_performance_metrics(time_avg, time_final, io_time_avg, io_time_final, proc_time, io_proc_time, file_exists)
1202
1203 real(wp), intent(inout) :: time_avg, time_final
1204 real(wp), intent(inout) :: io_time_avg, io_time_final
1205 real(wp), dimension(:), intent(inout) :: proc_time
1206 real(wp), dimension(:), intent(inout) :: io_proc_time
1207 logical, intent(inout) :: file_exists
1208
1209 real(wp) :: grind_time
1210
1211 call s_mpi_barrier()
1212
1213 if (num_procs > 1) then
1214 call mpi_bcast_time_step_values(proc_time, time_avg)
1215
1216 call mpi_bcast_time_step_values(io_proc_time, io_time_avg)
1217 end if
1218
1219 if (proc_rank == 0) then
1220 time_final = 0._wp
1221 io_time_final = 0._wp
1222 if (num_procs == 1) then
1223 time_final = time_avg
1224 io_time_final = io_time_avg
1225 else
1226 time_final = maxval(proc_time)
1227 io_time_final = maxval(io_proc_time)
1228 end if
1229
1230 grind_time = time_final*1.0e9_wp/ &
1231 (real(sys_size, wp)*real(maxval((/1, m_glb/)), wp)* &
1232 real(maxval((/1, n_glb/)), wp)*real(maxval((/1, p_glb/)), wp))
1233
1234 print *, "Performance:", grind_time, "ns/gp/eq/rhs"
1235 inquire (file='time_data.dat', exist=file_exists)
1236 if (file_exists) then
1237 open (1, file='time_data.dat', position='append', status='old')
1238 else
1239 open (1, file='time_data.dat', status='new')
1240 write (1, '(A10, A15, A15)') "Ranks", "s/step", "ns/gp/eq/rhs"
1241 end if
1242
1243 write (1, '(I10, 2(F15.8))') num_procs, time_final, grind_time
1244
1245 close (1)
1246
1247 inquire (file='io_time_data.dat', exist=file_exists)
1248 if (file_exists) then
1249 open (1, file='io_time_data.dat', position='append', status='old')
1250 else
1251 open (1, file='io_time_data.dat', status='new')
1252 write (1, '(A10, A15)') "Ranks", "s/step"
1253 end if
1254
1255 write (1, '(I10, F15.8)') num_procs, io_time_final
1256 close (1)
1257
1258 end if
1259
1260 end subroutine s_save_performance_metrics
1261
1262 !> @brief Saves conservative variable data to disk at the current time step.
1263 impure subroutine s_save_data(t_step, start, finish, io_time_avg, nt)
1264 integer, intent(inout) :: t_step
1265 real(wp), intent(inout) :: start, finish, io_time_avg
1266 integer, intent(inout) :: nt
1267
1268 integer(kind=8) :: i, j, k, l
1269 integer :: stor
1270
1271 integer :: save_count
1272
1273 if (down_sample) then
1275 end if
1276
1277 stor = 1
1278
1279 if (time_stepper /= 1) then
1280
1281# 942 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1282
1283# 942 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1284#if defined(MFC_OpenACC)
1285# 942 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1286!$acc parallel loop collapse(4) gang vector default(present) copyin(idwbuff)
1287# 942 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1288#elif defined(MFC_OpenMP)
1289# 942 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1290
1291# 942 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1292
1293# 942 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1294
1295# 942 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1296!$omp target teams loop defaultmap(firstprivate:scalar) bind(teams,parallel) collapse(4) defaultmap(tofrom:aggregate) defaultmap(tofrom:allocatable) defaultmap(tofrom:pointer) map(to:idwbuff)
1297# 942 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1298#endif
1299# 942 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1300
1301 do i = 1, sys_size
1302 do l = idwbuff(3)%beg, idwbuff(3)%end
1303 do k = idwbuff(2)%beg, idwbuff(2)%end
1304 do j = idwbuff(1)%beg, idwbuff(1)%end
1305 q_cons_ts(2)%vf(i)%sf(j, k, l) = &
1306 q_cons_ts(1)%vf(i)%sf(j, k, l)
1307 end do
1308 end do
1309 end do
1310 end do
1311
1312# 953 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1313
1314# 953 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1315#if defined(MFC_OpenACC)
1316# 953 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1317!$acc end parallel loop
1318# 953 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1319#elif defined(MFC_OpenMP)
1320# 953 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1321
1322# 953 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1323
1324# 953 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1325!$omp end target teams loop
1326# 953 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1327#endif
1328# 953 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1329
1330 stor = 2
1331 end if
1332
1333 call cpu_time(start)
1334 call nvtxstartrange("SAVE-DATA")
1335 do i = 1, sys_size
1336#ifndef FRONTIER_UNIFIED
1337
1338# 961 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1339#if defined(MFC_OpenACC)
1340# 961 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1341!$acc update host(q_cons_ts(stor)%vf(i)%sf)
1342# 961 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1343#elif defined(MFC_OpenMP)
1344# 961 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1345!$omp target update from(q_cons_ts(stor)%vf(i)%sf)
1346# 961 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1347#endif
1348#endif
1349 do l = 0, p
1350 do k = 0, n
1351 do j = 0, m
1352 if (ieee_is_nan(real(q_cons_ts(stor)%vf(i)%sf(j, k, l), kind=wp))) then
1353 print *, "NaN(s) in timestep output.", j, k, l, i, proc_rank, t_step, m, n, p
1354 call s_mpi_abort("NaN(s) in timestep output.")
1355 end if
1356 end do
1357 end do
1358 end do
1359 end do
1360
1361 if (qbmm .and. .not. polytropic) then
1362
1363# 976 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1364#if defined(MFC_OpenACC)
1365# 976 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1366!$acc update host(pb_ts(1)%sf)
1367# 976 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1368#elif defined(MFC_OpenMP)
1369# 976 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1370!$omp target update from(pb_ts(1)%sf)
1371# 976 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1372#endif
1373
1374# 977 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1375#if defined(MFC_OpenACC)
1376# 977 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1377!$acc update host(mv_ts(1)%sf)
1378# 977 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1379#elif defined(MFC_OpenMP)
1380# 977 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1381!$omp target update from(mv_ts(1)%sf)
1382# 977 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1383#endif
1384 end if
1385
1386 if (cfl_dt) then
1387 save_count = int(mytime/t_save)
1388 else
1389 save_count = t_step
1390 end if
1391
1392 if (bubbles_lagrange) then
1393
1394# 987 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1395#if defined(MFC_OpenACC)
1396# 987 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1397!$acc update host(lag_id, mtn_pos, mtn_posPrev, mtn_vel, intfc_rad, intfc_vel, bub_R0, Rmax_stats, Rmin_stats, bub_dphidt, gas_p, gas_mv, gas_mg, gas_betaT, gas_betaC)
1398# 987 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1399#elif defined(MFC_OpenMP)
1400# 987 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1401!$omp target update from(lag_id, mtn_pos, mtn_posPrev, mtn_vel, intfc_rad, intfc_vel, bub_R0, Rmax_stats, Rmin_stats, bub_dphidt, gas_p, gas_mv, gas_mg, gas_betaT, gas_betaC)
1402# 987 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1403#endif
1404# 990 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1405 do i = 1, nbubs
1406 if (ieee_is_nan(intfc_rad(i, 1)) .or. intfc_rad(i, 1) <= 0._wp) then
1407 call s_mpi_abort("Bubble radius is negative or NaN, please reduce dt.")
1408 end if
1409 end do
1410
1411
1412# 996 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1413#if defined(MFC_OpenACC)
1414# 996 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1415!$acc update host(q_beta(1)%sf)
1416# 996 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1417#elif defined(MFC_OpenMP)
1418# 996 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1419!$omp target update from(q_beta(1)%sf)
1420# 996 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1421#endif
1422 call s_write_data_files(q_cons_ts(stor)%vf, q_t_sf, q_prim_vf, save_count, bc_type, q_beta(1))
1423
1424# 998 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1425#if defined(MFC_OpenACC)
1426# 998 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1427!$acc update host(Rmax_stats, Rmin_stats, gas_p, gas_mv, intfc_vel)
1428# 998 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1429#elif defined(MFC_OpenMP)
1430# 998 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1431!$omp target update from(Rmax_stats, Rmin_stats, gas_p, gas_mv, intfc_vel)
1432# 998 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1433#endif
1434 call s_write_restart_lag_bubbles(save_count) !parallel
1435 if (lag_params%write_bubbles_stats) call s_write_lag_bubble_stats()
1436 else
1437 call s_write_data_files(q_cons_ts(stor)%vf, q_t_sf, q_prim_vf, save_count, bc_type)
1438 end if
1439
1440 call nvtxendrange
1441 call cpu_time(finish)
1442 if (cfl_dt) then
1443 nt = mytime/t_save
1444 else
1445 nt = int((t_step - t_step_start)/(t_step_save))
1446 end if
1447
1448 if (nt == 1) then
1449 io_time_avg = abs(finish - start)
1450 else
1451 io_time_avg = (abs(finish - start) + io_time_avg*(nt - 1))/nt
1452 end if
1453
1454 end subroutine s_save_data
1455
1456 !> @brief Initializes all simulation sub-modules in the required dependency order.
1457 impure subroutine s_initialize_modules
1458
1459 integer :: m_ds, n_ds, p_ds
1460 integer :: i, j, k, l, x_id, y_id, z_id, ix, iy, iz
1461 real(wp) :: temp1, temp2, temp3, temp4
1462
1464# 1035 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1465 if (bubbles_euler .or. bubbles_lagrange) then
1467 end if
1471 if (grid_geometry == 3) call s_initialize_fftw_module()
1472
1474 if (ib) call s_initialize_ibm_module()
1475 if (qbmm) call s_initialize_qbmm_module()
1476
1477 if (acoustic_source) then
1479 end if
1480
1481 if (viscous .and. (.not. igr)) then
1483 end if
1484
1486
1488
1490
1494
1496
1497 if (down_sample) then
1498 m_ds = int((m + 1)/3) - 1
1499 n_ds = int((n + 1)/3) - 1
1500 p_ds = int((p + 1)/3) - 1
1501
1502 allocate (q_cons_temp(1:sys_size))
1503 do i = 1, sys_size
1504 allocate (q_cons_temp(i)%sf(-1:m_ds + 1, -1:n_ds + 1, -1:p_ds + 1))
1505 end do
1506 end if
1507
1508 ! Reading in the user provided initial condition and grid data
1509 if (down_sample) then
1512 do i = 1, sys_size
1513
1514# 1083 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1515#if defined(MFC_OpenACC)
1516# 1083 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1517!$acc update device(q_cons_ts(1)%vf(i)%sf)
1518# 1083 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1519#elif defined(MFC_OpenMP)
1520# 1083 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1521!$omp target update to(q_cons_ts(1)%vf(i)%sf)
1522# 1083 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1523#endif
1524 end do
1525 do i = 1, sys_size
1526 deallocate (q_cons_temp(i)%sf)
1527 end do
1528 deallocate (q_cons_temp)
1529 else
1530 call s_read_data_files(q_cons_ts(1)%vf)
1531 end if
1532
1533 ! Populating the buffers of the grid variables using the boundary conditions
1535
1537 if (ib) then
1538 call s_ibm_setup()
1539 call s_write_ib_data_file(0)
1540 end if
1543
1544 ! Initialize the Temperature cache.
1546
1547 ! Computation of parameters, allocation of memory, association of pointers,
1548 ! and/or execution of any other tasks that are needed to properly configure
1549 ! the modules. The preparations below DO DEPEND on the grid being complete.
1550 if (igr .or. dummy) then
1552 end if
1553 if (.not. igr .or. dummy) then
1554 if (recon_type == weno_type) then
1556 elseif (recon_type == muscl_type) then
1558 end if
1561 end if
1562
1565
1568
1569 end subroutine s_initialize_modules
1570
1571 !> @brief Sets up the MPI execution environment, binds GPUs, and decomposes the computational domain.
1572 impure subroutine s_initialize_mpi_domain
1573 integer :: ierr
1574#ifdef MFC_GPU
1575 real(wp) :: starttime, endtime
1576 integer :: num_devices, local_size, num_nodes, ppn, my_device_num
1577 integer :: dev, devnum, local_rank
1578#ifdef MFC_MPI
1579 integer :: local_comm
1580#endif
1581#if defined(MFC_OpenACC)
1582 integer(acc_device_kind) :: devtype
1583#endif
1584#endif
1585
1586 ! Initializing MPI execution environment
1587
1588 call s_mpi_initialize()
1589
1590 ! Bind GPUs if OpenACC is enabled
1591#ifdef MFC_GPU
1592#ifndef MFC_MPI
1593 local_size = 1
1594 local_rank = 0
1595#else
1596 call mpi_comm_split_type(mpi_comm_world, mpi_comm_type_shared, 0, &
1597 mpi_info_null, local_comm, ierr)
1598 call mpi_comm_size(local_comm, local_size, ierr)
1599 call mpi_comm_rank(local_comm, local_rank, ierr)
1600#endif
1601#if defined(MFC_OpenACC)
1602 devtype = acc_get_device_type()
1603 devnum = acc_get_num_devices(devtype)
1604 dev = mod(local_rank, devnum)
1605
1606 call acc_set_device_num(dev, devtype)
1607#elif defined(MFC_OpenMP)
1608 devnum = omp_get_num_devices()
1609 dev = mod(local_rank, devnum)
1610 call omp_set_default_device(dev)
1611#endif
1612#endif
1613
1614 ! The rank 0 processor assigns default values to the user inputs prior to
1615 ! reading them in from the input file. Next, the user inputs are read and
1616 ! their consistency is checked. The identification of any inconsistencies
1617 ! will result in the termination of the simulation.
1618 if (proc_rank == 0) then
1620 call s_read_input_file()
1621 call s_check_input_file()
1622
1623 print '(" Simulating a ", A, " ", I0, "x", I0, "x", I0, " case on ", I0, " rank(s) ", A, ".")', &
1624# 1185 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1625 "regular", &
1626# 1189 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1627 m, n, p, num_procs, &
1628#if defined(MFC_OpenACC)
1629 "with OpenACC offloading"
1630#elif defined(MFC_OpenMP)
1631 "with OpenMP offloading"
1632#else
1633 "on CPUs"
1634#endif
1635 end if
1636
1637 ! Broadcasting the user inputs to all of the processors and performing the
1638 ! parallel computational domain decomposition. Neither procedure has to be
1639 ! carried out if the simulation is in fact not truly executed in parallel.
1640
1642
1644
1646
1647 end subroutine s_initialize_mpi_domain
1648
1649 !> @brief Transfers initial conservative variable and model parameter data to the GPU device.
1651 integer :: i
1652 !Update GPU DATA
1653 if (.not. down_sample) then
1654 do i = 1, sys_size
1655
1656# 1217 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1657#if defined(MFC_OpenACC)
1658# 1217 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1659!$acc update device(q_cons_ts(1)%vf(i)%sf)
1660# 1217 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1661#elif defined(MFC_OpenMP)
1662# 1217 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1663!$omp target update to(q_cons_ts(1)%vf(i)%sf)
1664# 1217 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1665#endif
1666 end do
1667 end if
1668
1669 if (qbmm .and. .not. polytropic) then
1670
1671# 1222 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1672#if defined(MFC_OpenACC)
1673# 1222 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1674!$acc update device(pb_ts(1)%sf, mv_ts(1)%sf)
1675# 1222 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1676#elif defined(MFC_OpenMP)
1677# 1222 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1678!$omp target update to(pb_ts(1)%sf, mv_ts(1)%sf)
1679# 1222 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1680#endif
1681 end if
1682 if (chemistry) then
1683
1684# 1225 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1685#if defined(MFC_OpenACC)
1686# 1225 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1687!$acc update device(q_T_sf%sf)
1688# 1225 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1689#elif defined(MFC_OpenMP)
1690# 1225 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1691!$omp target update to(q_T_sf%sf)
1692# 1225 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1693#endif
1694 end if
1695
1696
1697# 1228 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1698#if defined(MFC_OpenACC)
1699# 1228 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1700!$acc update device(chem_params)
1701# 1228 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1702#elif defined(MFC_OpenMP)
1703# 1228 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1704!$omp target update to(chem_params)
1705# 1228 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1706#endif
1707
1708
1709# 1230 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1710#if defined(MFC_OpenACC)
1711# 1230 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1712!$acc update device(R0ref, p0ref, rho0ref, ss, pv, vd, mu_l, mu_v, mu_g, gam_v, gam_g, M_v, M_g, R_v, R_g, Tw, cp_v, cp_g, k_vl, k_gl, gam, gam_m, Eu, Ca, Web, Re_inv, Pe_c, phi_vg, phi_gv, omegaN, bubbles_euler, polytropic, polydisperse, qbmm, ptil, bubble_model, thermal, poly_sigma, adv_n, adap_dt, adap_dt_tol, adap_dt_max_iters, n_idx, pi_fac, low_Mach)
1713# 1230 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1714#elif defined(MFC_OpenMP)
1715# 1230 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1716!$omp target update to(R0ref, p0ref, rho0ref, ss, pv, vd, mu_l, mu_v, mu_g, gam_v, gam_g, M_v, M_g, R_v, R_g, Tw, cp_v, cp_g, k_vl, k_gl, gam, gam_m, Eu, Ca, Web, Re_inv, Pe_c, phi_vg, phi_gv, omegaN, bubbles_euler, polytropic, polydisperse, qbmm, ptil, bubble_model, thermal, poly_sigma, adv_n, adap_dt, adap_dt_tol, adap_dt_max_iters, n_idx, pi_fac, low_Mach)
1717# 1230 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1718#endif
1719# 1236 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1720
1721 if (bubbles_euler) then
1722
1723# 1238 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1724#if defined(MFC_OpenACC)
1725# 1238 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1726!$acc update device(weight, R0)
1727# 1238 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1728#elif defined(MFC_OpenMP)
1729# 1238 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1730!$omp target update to(weight, R0)
1731# 1238 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1732#endif
1733 if (.not. polytropic) then
1734
1735# 1240 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1736#if defined(MFC_OpenACC)
1737# 1240 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1738!$acc update device(pb0, Pe_T, k_g, k_v, mass_g0, mass_v0, Re_trans_T, Re_trans_c, Im_trans_T, Im_trans_c)
1739# 1240 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1740#elif defined(MFC_OpenMP)
1741# 1240 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1742!$omp target update to(pb0, Pe_T, k_g, k_v, mass_g0, mass_v0, Re_trans_T, Re_trans_c, Im_trans_T, Im_trans_c)
1743# 1240 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1744#endif
1745# 1242 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1746 else if (qbmm) then
1747
1748# 1243 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1749#if defined(MFC_OpenACC)
1750# 1243 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1751!$acc update device(pb0)
1752# 1243 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1753#elif defined(MFC_OpenMP)
1754# 1243 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1755!$omp target update to(pb0)
1756# 1243 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1757#endif
1758 end if
1759 end if
1760
1761
1762# 1247 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1763#if defined(MFC_OpenACC)
1764# 1247 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1765!$acc update device(adv_n, adap_dt, adap_dt_tol, adap_dt_max_iters, n_idx, pi_fac, low_Mach)
1766# 1247 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1767#elif defined(MFC_OpenMP)
1768# 1247 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1769!$omp target update to(adv_n, adap_dt, adap_dt_tol, adap_dt_max_iters, n_idx, pi_fac, low_Mach)
1770# 1247 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1771#endif
1772
1773
1774# 1249 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1775#if defined(MFC_OpenACC)
1776# 1249 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1777!$acc update device(acoustic_source, num_source)
1778# 1249 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1779#elif defined(MFC_OpenMP)
1780# 1249 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1781!$omp target update to(acoustic_source, num_source)
1782# 1249 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1783#endif
1784
1785# 1250 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1786#if defined(MFC_OpenACC)
1787# 1250 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1788!$acc update device(sigma, surface_tension)
1789# 1250 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1790#elif defined(MFC_OpenMP)
1791# 1250 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1792!$omp target update to(sigma, surface_tension)
1793# 1250 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1794#endif
1795
1796
1797# 1252 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1798#if defined(MFC_OpenACC)
1799# 1252 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1800!$acc update device(dx, dy, dz, x_cb, x_cc, y_cb, y_cc, z_cb, z_cc)
1801# 1252 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1802#elif defined(MFC_OpenMP)
1803# 1252 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1804!$omp target update to(dx, dy, dz, x_cb, x_cc, y_cb, y_cc, z_cb, z_cc)
1805# 1252 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1806#endif
1807
1808# 1253 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1809#if defined(MFC_OpenACC)
1810# 1253 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1811!$acc update device(bc_x%vb1, bc_x%vb2, bc_x%vb3, bc_x%ve1, bc_x%ve2, bc_x%ve3)
1812# 1253 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1813#elif defined(MFC_OpenMP)
1814# 1253 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1815!$omp target update to(bc_x%vb1, bc_x%vb2, bc_x%vb3, bc_x%ve1, bc_x%ve2, bc_x%ve3)
1816# 1253 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1817#endif
1818
1819# 1254 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1820#if defined(MFC_OpenACC)
1821# 1254 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1822!$acc update device(bc_y%vb1, bc_y%vb2, bc_y%vb3, bc_y%ve1, bc_y%ve2, bc_y%ve3)
1823# 1254 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1824#elif defined(MFC_OpenMP)
1825# 1254 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1826!$omp target update to(bc_y%vb1, bc_y%vb2, bc_y%vb3, bc_y%ve1, bc_y%ve2, bc_y%ve3)
1827# 1254 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1828#endif
1829
1830# 1255 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1831#if defined(MFC_OpenACC)
1832# 1255 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1833!$acc update device(bc_z%vb1, bc_z%vb2, bc_z%vb3, bc_z%ve1, bc_z%ve2, bc_z%ve3)
1834# 1255 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1835#elif defined(MFC_OpenMP)
1836# 1255 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1837!$omp target update to(bc_z%vb1, bc_z%vb2, bc_z%vb3, bc_z%ve1, bc_z%ve2, bc_z%ve3)
1838# 1255 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1839#endif
1840
1841
1842# 1257 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1843#if defined(MFC_OpenACC)
1844# 1257 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1845!$acc update device(bc_x%grcbc_in, bc_x%grcbc_out, bc_x%grcbc_vel_out)
1846# 1257 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1847#elif defined(MFC_OpenMP)
1848# 1257 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1849!$omp target update to(bc_x%grcbc_in, bc_x%grcbc_out, bc_x%grcbc_vel_out)
1850# 1257 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1851#endif
1852
1853# 1258 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1854#if defined(MFC_OpenACC)
1855# 1258 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1856!$acc update device(bc_y%grcbc_in, bc_y%grcbc_out, bc_y%grcbc_vel_out)
1857# 1258 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1858#elif defined(MFC_OpenMP)
1859# 1258 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1860!$omp target update to(bc_y%grcbc_in, bc_y%grcbc_out, bc_y%grcbc_vel_out)
1861# 1258 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1862#endif
1863
1864# 1259 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1865#if defined(MFC_OpenACC)
1866# 1259 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1867!$acc update device(bc_z%grcbc_in, bc_z%grcbc_out, bc_z%grcbc_vel_out)
1868# 1259 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1869#elif defined(MFC_OpenMP)
1870# 1259 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1871!$omp target update to(bc_z%grcbc_in, bc_z%grcbc_out, bc_z%grcbc_vel_out)
1872# 1259 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1873#endif
1874
1875
1876# 1261 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1877#if defined(MFC_OpenACC)
1878# 1261 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1879!$acc update device(relax, relax_model)
1880# 1261 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1881#elif defined(MFC_OpenMP)
1882# 1261 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1883!$omp target update to(relax, relax_model)
1884# 1261 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1885#endif
1886 if (relax) then
1887
1888# 1263 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1889#if defined(MFC_OpenACC)
1890# 1263 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1891!$acc update device(palpha_eps, ptgalpha_eps)
1892# 1263 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1893#elif defined(MFC_OpenMP)
1894# 1263 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1895!$omp target update to(palpha_eps, ptgalpha_eps)
1896# 1263 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1897#endif
1898 end if
1899
1900 if (ib) then
1901
1902# 1267 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1903#if defined(MFC_OpenACC)
1904# 1267 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1905!$acc update device(ib_markers%sf)
1906# 1267 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1907#elif defined(MFC_OpenMP)
1908# 1267 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1909!$omp target update to(ib_markers%sf)
1910# 1267 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1911#endif
1912 end if
1913# 1270 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1914
1915# 1270 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1916#if defined(MFC_OpenACC)
1917# 1270 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1918!$acc update device(igr, nb, igr_order)
1919# 1270 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1920#elif defined(MFC_OpenMP)
1921# 1270 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1922!$omp target update to(igr, nb, igr_order)
1923# 1270 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1924#endif
1925# 1272 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1926# 1280 "/home/runner/work/MFC/MFC/src/simulation/m_start_up.fpp"
1927
1928 end subroutine s_initialize_gpu_vars
1929
1930 !> @brief Finalizes and deallocates all simulation sub-modules in reverse initialization order.
1931 impure subroutine s_finalize_modules
1932
1939 if (igr) then
1941 else
1944 if (recon_type == weno_type) then
1946 elseif (recon_type == muscl_type) then
1948 end if
1949 end if
1951 if (grid_geometry == 3) call s_finalize_fftw_module
1957 if (viscous .and. (.not. igr)) then
1959 end if
1961
1964
1965 ! Terminating MPI execution environment
1966 call s_mpi_finalize()
1967 end subroutine s_finalize_modules
1968
1969end module m_start_up
type(scalar_field), dimension(sys_size), intent(inout) q_cons_vf
Applies acoustic pressure source terms including focused, planar, and broadband transducers.
impure subroutine, public s_precalculate_acoustic_spatial_sources
This subroutine identifies and precalculates the non-zero acoustic spatial sources before time-steppi...
impure subroutine, public s_initialize_acoustic_src
This subroutine initializes the acoustic source module.
Computes gravitational and user-defined body force source terms for the momentum equations.
impure subroutine, public s_initialize_body_forces_module
This subroutine initializes the module global array of mixture densities in each grid cell.
impure subroutine, public s_finalize_body_forces_module
Deallocates module variables used for body force computations.
Noncharacteristic and processor boundary condition application for ghost cells and buffer regions.
impure subroutine, public s_initialize_boundary_common_module()
Allocates and sets up boundary condition buffer arrays for all coordinate directions.
subroutine, public s_finalize_boundary_common_module()
Deallocates boundary condition buffer arrays allocated during module initialization.
impure subroutine, public s_populate_variables_buffers(bc_type, q_prim_vf, pb_in, mv_in)
The purpose of this procedure is to populate the buffers of the primitive variables,...
subroutine, public s_read_parallel_boundary_condition_files(bc_type)
Reads boundary condition type and buffer data from per-rank parallel files using MPI I/O.
subroutine, public s_populate_grid_variables_buffers
The purpose of this subroutine is to populate the buffers of the grid variables, which are constitute...
subroutine, public s_read_serial_boundary_condition_files(step_dirpath, bc_type)
Reads boundary condition type and buffer data from serial (unformatted) restart files.
subroutine, public s_assign_default_bc_type(bc_type)
Initializes the per-cell boundary condition type arrays with the global default BC values.
Computes ensemble-averaged (Euler–Euler) bubble source terms for radius, velocity,...
impure subroutine s_initialize_bubbles_ee_module
Allocates and initializes arrays for the Euler-Euler bubble model.
Tracks Lagrangian bubbles and couples their dynamics to the Eulerian flow via volume averaging.
impure subroutine s_write_lag_bubble_stats()
Subroutine that writes the maximum and minimum radius of each bubble.
impure subroutine s_write_restart_lag_bubbles(t_step)
Subroutine that writes the restarting files for the particles in the lagrangian solver.
integer nbubs
Number of bubbles in the local domain.
impure subroutine s_initialize_bubbles_el_module(q_cons_vf)
Initializes the lagrangian subgrid bubble solver.
real(wp), dimension(:, :), allocatable intfc_rad
Bubble radius.
type(scalar_field), dimension(:), allocatable q_beta
impure subroutine s_finalize_lagrangian_solver()
The purpose of this subroutine is to deallocate variables.
Characteristic boundary conditions (CBC) for slip walls, non-reflecting subsonic inflow/outflow,...
impure subroutine, public s_initialize_cbc_module
The computation of parameters, the allocation of memory, the association of pointers and/or the execu...
impure subroutine, public s_finalize_cbc_module
Module deallocation and/or disassociation procedures.
Shared input validation checks for grid dimensions and AMD GPU compiler limits.
impure subroutine, public s_check_inputs_common
Checks compatibility of parameters in the input file. Used by all three stages.
Validates simulation input parameters for consistency and supported configurations.
impure subroutine, public s_check_inputs
Checks compatibility of parameters in the input file. Used by the simulation stage.
Multi-species chemistry interface for thermodynamic properties, reaction rates, and transport coeffic...
subroutine s_compute_q_t_sf(q_t_sf, q_cons_vf, bounds)
Initializes the temperature field from conservative variables by inverting the energy equation.
Platform-specific file and directory operations: create, delete, inquire, getcwd, and basename.
impure subroutine my_inquire(fileloc, dircheck)
Inquires on the existence of a directory.
Writes solution data, run-time stability diagnostics (ICFL, VCFL, CCFL, Rc), and probe/center-of-mass...
impure subroutine, public s_initialize_data_output_module
The computation of parameters, the allocation of memory, the association of pointers and/or the execu...
impure subroutine, public s_finalize_data_output_module
Module deallocation and/or disassociation procedures.
impure subroutine, public s_write_data_files(q_cons_vf, q_t_sf, q_prim_vf, t_step, bc_type, beta)
Write data files. Dispatch subroutine that replaces procedure pointer.
subroutine, public s_write_ib_data_file(time_step)
Dispatches immersed boundary data output to the serial or parallel writer.
Shared derived types for field data, patch geometry, bubble dynamics, and MPI I/O structures.
Derives diagnostic flow quantities (vorticity, speed of sound, numerical Schlieren,...
impure subroutine, public s_initialize_derived_variables
Allocate and open derived variables. Computing FD coefficients.
impure subroutine, public s_initialize_derived_variables_module
Computation of parameters, allocation procedures, and/or any other tasks needed to properly setup the...
impure subroutine, public s_finalize_derived_variables_module
Deallocation procedures for the module.
Global parameters for the computational domain, fluid properties, and simulation algorithm configurat...
real(wp) mytime
Current simulation time.
logical null_weights
Null undesired WENO weights.
logical bubbles_euler
Bubbles euler on/off.
real(wp) cont_damage_s
Exponent s for continuum damage modeling.
integer wave_speeds
Wave speeds estimation method.
logical cont_damage
Continuum damage modeling.
logical igr
Use information geometric regularization.
logical hypoelasticity
hypoelasticity modeling
impure subroutine s_assign_default_values_to_user_inputs
Assigns default values to the user inputs before reading them in. This enables for an easier consiste...
impure subroutine s_finalize_global_parameters_module
Module deallocation and/or disassociation procedures.
integer thermal
Thermal behavior. 1 = adiabatic, 2 = isotherm, 3 = transfer.
integer avg_state
Average state evaluation method.
type(int_bounds_info), dimension(1:3) idwint
real(wp), dimension(:), allocatable, target z_cb
integer recon_type
Reconstruction Type.
logical igr_pres_lim
Limit to positive pressures for IGR.
logical, parameter chemistry
Chemistry modeling.
logical acoustic_source
Acoustic source switch.
type(ib_patch_parameters), dimension(num_patches_max) patch_ib
type(int_bounds_info) mom_idx
Indexes of first & last momentum eqns.
integer num_fluids
number of fluids in the simulation
logical polydisperse
Polydisperse bubbles.
type(bubbles_lagrange_parameters) lag_params
Lagrange bubbles' parameters.
integer proc_rank
Rank of the local processor.
real(wp) hyper_cleaning_tau
Hyperbolic cleaning tau.
logical mixture_err
Mixture properties correction.
logical adap_dt
Adaptive step size control.
logical weno_re_flux
WENO reconstruct velocity gradients for viscous stress tensor.
real(wp) hyper_cleaning_speed
Hyperbolic cleaning wave speed (c_h).
integer t_step_print
Number of time-steps between printouts.
logical dummy
AMDFlang workaround: keep a dummy logical to avoid a compiler case-optimization bug when a parameter+...
real(wp) poly_sigma
log normal sigma for polydisperse PDF
character(len=name_len) mpiiofs
integer, dimension(:), allocatable start_idx
Starting cell-center index of local processor in global grid.
type(vec3_dt), dimension(num_probes_max) probe
integer sys_size
Number of unknowns in system of eqns.
integer muscl_order
Order of the MUSCL reconstruction.
logical alt_soundspeed
Alternate mixture sound speed.
real(wp) ptgalpha_eps
trigger parameter for the pTg relaxation procedure, phase change model
integer relax_model
Relaxation model.
integer t_step_old
Existing IC/grid folder.
logical viscous
Viscous effects.
logical run_time_info
Run-time output flag.
type(int_bounds_info) b_idx
Indexes of first and last magnetic field eqns.
real(wp) wenoz_q
Power constant for WENO-Z.
integer riemann_solver
Riemann solver algorithm.
type(int_bounds_info), dimension(1:3) idwbuff
logical int_comp
THINC interface compression.
real(wp) re_inv
Inverse Reynolds number.
real(wp) ic_eps
THINC Epsilon to compress on surface cells.
integer model_eqns
Multicomponent flow model.
integer precision
Precision of output files.
logical hyperelasticity
hyperelasticity modeling
type(physical_parameters), dimension(num_fluids_max) fluid_pp
Database of the physical parameters of each of the fluids that is present in the flow....
real(wp) alf_factor
alpha factor for IGR
impure subroutine s_initialize_global_parameters_module
The computation of parameters, the allocation of memory, the association of pointers and/or the execu...
real(wp), dimension(:), allocatable, target y_cc
type(pres_field), dimension(:), allocatable pb_ts
type(pres_field), dimension(:), allocatable mv_ts
type(chemistry_parameters) chem_params
integer fd_order
The order of the finite-difference (fd) approximations of the first-order derivatives that need to be...
logical bubbles_lagrange
Lagrangian subgrid bubble model switch.
real(wp) ca
Cavitation number.
real(wp) alpha_bar
Damage rate factor for continuum damage modeling.
logical polytropic
Polytropic switch.
logical bf_z
body force toggle in three directions amplitude, frequency, and phase shift sinusoid in each directio...
logical mp_weno
Monotonicity preserving (MP) WENO.
real(wp), dimension(:), allocatable, target z_cc
logical relax
activate phase change
logical qbmm
Quadrature moment method.
real(wp) pi_fac
Factor for artificial pi_inf.
logical hyper_cleaning
Hyperbolic cleaning for MHD for divB=0.
integer adap_dt_max_iters
Maximum number of iterations.
real(wp) ic_beta
THINC Sharpness Parameter.
real(wp) bx0
Constant magnetic field in the x-direction (1D).
integer num_source
Number of acoustic sources.
real(wp), dimension(:), allocatable qvs
real(wp), dimension(:), allocatable pi_infs
logical adv_n
Solve the number density equation and compute alpha from number density.
integer num_procs
Number of processors.
character(len=path_len) case_dir
Case folder location.
real(wp) weno_eps
Binding for the WENO nonlinear weights.
integer weno_order
Order of the WENO reconstruction.
logical mhd
Magnetohydrodynamics.
real(wp), dimension(:), allocatable, target x_cc
type(acoustic_parameters), dimension(num_probes_max) acoustic
Acoustic source parameters.
real(wp) tau_star
Stress threshold for continuum damage modeling.
logical parallel_io
Format of the data files.
type(integral_parameters), dimension(num_probes_max) integral
real(wp), dimension(:), allocatable, target y_cb
real(wp) adap_dt_tol
Tolerance to control adaptive step size.
integer e_idx
Index of energy equation.
type(cell_num_bounds) cells_bounds
logical down_sample
down sample the output files
type(mpi_io_airfoil_ib_var), public mpi_io_airfoil_ib_data
logical file_per_process
shared file or not when using parallel io
real(wp) palpha_eps
trigger parameter for the p relaxation procedure, phase change model
integer num_igr_warm_start_iters
number of warm start iterations for elliptic solve
logical elasticity
elasticity modeling, true for hyper or hypo
integer nb
Number of eq. bubble sizes.
type(mpi_io_var), public mpi_io_data
impure subroutine s_initialize_parallel_io
Initializes parallel infrastructure.
logical mpp_lim
Mixture physical parameters (MPP) limits.
integer num_igr_iters
number of iterations for elliptic solve
real(wp) teno_ct
Smoothness threshold for TENO.
integer low_mach
Low Mach number fix to HLLC Riemann solver.
integer igr_order
Reconstruction order for IGR.
logical teno
TENO (Targeted ENO).
real(wp), dimension(:), allocatable, target dy
real(wp) dt
Size of the time-step.
type(subgrid_bubble_physical_parameters) bub_pp
integer time_stepper
Time-stepper algorithm.
logical relativity
Relativity (only for MHD).
real(wp), dimension(:), allocatable gammas
integer bubble_model
Gilmore or Keller–Miksis bubble model.
real(wp) finaltime
Final simulation time.
real(wp), dimension(:), allocatable, target dz
real(wp), dimension(:), allocatable, target dx
logical mapped_weno
WENO-M (WENO with mapping of nonlinear weights).
integer igr_iter_solver
IGR elliptic solver.
integer muscl_lim
MUSCL Limiter.
real(wp), dimension(:), allocatable, target x_cb
Basic floating-point utilities: approximate equality, default detection, and coordinate bounds.
elemental subroutine, public s_update_cell_bounds(bounds, m, n, p)
Updates the min and max number of cells in each set of axes.
Utility routines for bubble model setup, coordinate transforms, array sampling, and special functions...
subroutine, public s_upsample_data(q_cons_vf, q_cons_temp)
Upsamples conservative variable fields from a coarsened grid back to the original resolution using in...
impure subroutine, public s_initialize_bubbles_model()
bubbles_euler + polytropic bubbles_euler + non-polytropic bubbles_lagrange + non-polytropic
elemental subroutine, public s_int_to_str(i, res)
Converts an integer to its trimmed string representation.
Computes the left Cauchy–Green deformation tensor and hyperelastic stress source terms.
impure subroutine, public s_finalize_hyperelastic_module()
Deallocates memory for hyperelastic deformation tensor and finite-difference coefficients.
impure subroutine, public s_initialize_hyperelastic_module
The following subroutine handles the calculation of the btensor. The calculation of the btensor takes...
Computes hypoelastic stress-rate source terms and damage-state evolution.
impure subroutine, public s_initialize_hypoelastic_module
Allocates arrays and computes finite-difference coefficients for the hypoelastic stress model.
impure subroutine, public s_finalize_hypoelastic_module()
Deallocates arrays used by the hypoelastic stress module.
Ghost-node immersed boundary method: locates ghost/image points, computes interpolation coefficients,...
impure subroutine, public s_ibm_setup()
Initializes the values of various IBM variables, such as ghost points and image points.
type(integer_field), public ib_markers
impure subroutine, public s_initialize_ibm_module()
Allocates memory for the variables in the IBM module.
Iterative ghost rasterization (IGR) for sharp immersed boundary treatment.
subroutine, public s_initialize_igr_module()
Allocates and initializes arrays, coefficients, and GPU data structures for the implicit gradient rec...
integer(kind=8) j
integer(kind=8) i
integer(kind=8) l
integer(kind=8) r
integer(kind=8) k
subroutine, public s_finalize_igr_module()
Deallocates all arrays and GPU resources allocated by the IGR module.
MPI communication layer: domain decomposition, halo exchange, reductions, and parallel I/O setup.
impure subroutine s_mpi_abort(prnt, code)
The subroutine terminates the MPI execution environment.
impure subroutine s_initialize_mpi_common_module
The computation of parameters, the allocation of memory, the association of pointers and/or the execu...
impure subroutine s_mpi_barrier
Halts all processes until all have reached barrier.
impure subroutine s_mpi_initialize
The subroutine initializes the MPI execution environment and queries both the number of processors wh...
impure subroutine s_initialize_mpi_data(q_cons_vf, ib_markers, beta)
impure subroutine s_mpi_finalize
The subroutine finalizes the MPI execution environment.
subroutine s_initialize_mpi_data_ds(q_cons_vf)
impure subroutine mpi_bcast_time_step_values(proc_time, time_avg)
Gathers per-rank time step wall-clock times onto rank 0 for performance reporting.
impure subroutine s_finalize_mpi_common_module
Module deallocation and/or disassociation procedures.
subroutine s_mpi_decompose_computational_domain
The purpose of this procedure is to optimally decompose the computational domain among the available ...
MPI halo exchange, domain decomposition, and buffer packing/unpacking for the simulation solver.
subroutine s_initialize_mpi_proxy_module()
Allocates immersed boundary communication buffers for MPI halo exchanges.
subroutine s_finalize_mpi_proxy_module()
Deallocates immersed boundary MPI communication buffers.
impure subroutine s_mpi_bcast_user_inputs()
Since only the processor with rank 0 reads and verifies the consistency of user inputs,...
MUSCL reconstruction with interface sharpening for contact-preserving advection.
subroutine, public s_initialize_muscl_module()
subroutine, public s_finalize_muscl_module()
Deallocates the MUSCL direction-local work arrays.
NVIDIA NVTX profiling API bindings for GPU performance instrumentation.
Definition m_nvtx.f90:6
subroutine nvtxstartrange(name, id)
Pushes a named NVTX range for GPU profiling, optionally with a color based on the given identifier.
Definition m_nvtx.f90:63
subroutine nvtxendrange
Pops the current NVTX range to end the GPU profiling region.
Definition m_nvtx.f90:84
Phase transition relaxation solvers for liquid-vapor flows with cavitation and boiling.
impure subroutine, public s_finalize_relaxation_solver_module
This subroutine finalizes the phase change module.
subroutine, public s_infinite_relaxation_k(q_cons_vf)
This subroutine is created to activate either the pT- (N fluids) or the pTg-equilibrium (2 fluids for...
impure subroutine, public s_initialize_phasechange_module
The purpose of this subroutine is to initialize the phase change module by setting the parameters nee...
Quadrature-based moment methods (QBMM) for polydisperse bubble moment inversion and transport.
impure subroutine, public s_initialize_qbmm_module
Allocates and initializes moment coefficient arrays for the QBMM module.
Assembles the right-hand side of the governing equations using finite-volume flux differencing,...
impure subroutine, public s_initialize_rhs_module
The computation of parameters, the allocation of memory, the association of pointers and/or the execu...
impure subroutine, public s_finalize_rhs_module
Module deallocation and/or disassociation procedures.
Approximate and exact Riemann solvers (HLL, HLLC, HLLD, exact) for the multicomponent Navier–Stokes e...
impure subroutine, public s_finalize_riemann_solvers_module
Module deallocation and/or disassociation procedures.
impure subroutine, public s_initialize_riemann_solvers_module
The computation of parameters, the allocation of memory, the association of pointers and/or the execu...
Simulation helper routines for enthalpy computation, CFL calculation, and stability checks.
Reads input files, loads initial conditions and grid data, and orchestrates solver initialization and...
impure subroutine, public s_read_serial_data_files(q_cons_vf)
Reads serial initial condition and grid data files and computes cell-width distributions.
impure subroutine, public s_initialize_modules
Initializes all simulation sub-modules in the required dependency order.
impure subroutine, public s_read_data_files(q_cons_vf)
Read data files. Dispatch subroutine that replaces procedure pointer.
impure subroutine, public s_read_parallel_data_files(q_cons_vf)
Reads parallel initial condition and grid data files via MPI I/O.
subroutine, public s_initialize_internal_energy_equations(v_vf)
The purpose of this procedure is to initialize the values of the internal-energy equations of each ph...
impure subroutine, public s_save_performance_metrics(time_avg, time_final, io_time_avg, io_time_final, proc_time, io_proc_time, file_exists)
Collects per-process wall-clock times and writes aggregate performance metrics to file.
impure subroutine, public s_save_data(t_step, start, finish, io_time_avg, nt)
Saves conservative variable data to disk at the current time step.
type(scalar_field), dimension(:), allocatable q_cons_temp
subroutine, public s_initialize_gpu_vars
Transfers initial conservative variable and model parameter data to the GPU device.
impure subroutine, public s_initialize_mpi_domain
Sets up the MPI execution environment, binds GPUs, and decomposes the computational domain.
impure subroutine, public s_finalize_modules
Finalizes and deallocates all simulation sub-modules in reverse initialization order.
impure subroutine, public s_read_input_file
The purpose of this procedure is to first verify that an input file has been made available by the us...
impure subroutine, public s_check_input_file
The goal of this procedure is to verify that each of the user provided inputs is valid and that their...
impure subroutine, public s_perform_time_step(t_step, time_avg)
Advances the simulation by one time step, handling CFL-based dt and time-stepper dispatch.
Computes capillary source fluxes and color-function gradients for the diffuse-interface surface tensi...
impure subroutine, public s_initialize_surface_tension_module
impure subroutine, public s_finalize_surface_tension_module
Deallocates the color-gradient divergence and reconstructed boundary arrays for surface tension.
Total-variation-diminishing (TVD) Runge–Kutta time integrators (1st-, 2nd-, and 3rd-order SSP).
type(scalar_field) q_t_sf
Cell-average temperature variables at the current time-stage.
impure subroutine s_initialize_time_steppers_module
The computation of parameters, the allocation of memory, the association of pointers and/or the execu...
type(integer_field), dimension(:, :), allocatable bc_type
Boundary condition identifiers.
type(vector_field), dimension(:), allocatable q_cons_ts
Cell-average conservative variables at each time-stage (TS).
type(scalar_field), dimension(:), allocatable q_prim_vf
Cell-average primitive variables at the current time-stage.
impure subroutine s_finalize_time_steppers_module
Module deallocation and/or disassociation procedures.
impure subroutine s_compute_dt()
Computes the global time step size from CFL stability constraints across all cells.
impure subroutine s_tvd_rk(t_step, time_avg, nstage)
Advances the solution one full step using a TVD Runge-Kutta time integrator.
integer stor
storage index
Conservative-to-primitive variable conversion, mixture property evaluation, and pressure computation.
subroutine, public s_compute_pressure(energy, alf, dyn_p, pi_inf, gamma, rho, qv, rhoyks, pres, t, stress, mom, g, pres_mag)
This procedure conditionally calculates the appropriate pressure.
impure subroutine, public s_initialize_variables_conversion_module
The computation of parameters, the allocation of memory, the association of pointers and/or the execu...
impure subroutine s_finalize_variables_conversion_module()
Deallocates fluid property arrays and post-processing fields allocated during module initialization.
subroutine, public s_convert_to_mixture_variables(q_vf, i, j, k, rho, gamma, pi_inf, qv, re_k, g_k, g)
Dispatch to the s_convert_mixture_to_mixture_variables and s_convert_species_to_mixture_variables sub...
Computes viscous stress tensors and diffusive flux contributions for the Navier–Stokes equations.
impure subroutine, public s_initialize_viscous_module
Allocates and populates the viscous Reynolds number arrays and transfers data to the GPU.
impure subroutine, public s_finalize_viscous_module()
Deallocates the viscous Reynolds number arrays.
WENO/WENO-Z/TENO reconstruction with optional monotonicity-preserving bounds and mapped weights.
impure subroutine, public s_initialize_weno_module
The computation of parameters, the allocation of memory, the association of pointers and/or the execu...
impure subroutine, public s_finalize_weno_module()
Module deallocation and/or disassociation procedures.
Derived type annexing a scalar field (SF).