Merge branch 'master' into CodeTidying

Author: XZTian64

.github/workflows/frontier/submit.sh

@@ -33,7 +33,7 @@ sbatch <<EOT
 #SBATCH -A CFD154                  # charge account
 #SBATCH -N 1                       # Number of nodes required
 $sbatch_device_opts
-#SBATCH -t 02:59:00                # Duration of the job (Ex: 15 mins)
+#SBATCH -t 03:59:00                # Duration of the job (Ex: 15 mins)
 #SBATCH -o$job_slug.out            # Combined output and error messages file
 #SBATCH -p extended                # Extended partition for shorter queues
 #SBATCH -W                         # Do not exit until the submitted job terminates.

README.md

@@ -5,17 +5,17 @@
 </p>
 
 <p align="center">
-  <a href="http://dx.doi.org/10.1016/j.cpc.2020.107396" target="_blank">
-    <img src="https://zenodo.org/badge/doi/10.1016/j.cpc.2020.107396.svg" />
-  </a>
   <a href="https://github.com/MFlowCode/MFC/actions">
-    <img src="https://github.com/MFlowCode/MFC/actions/workflows/test.yml/badge.svg" />
+    <img src="https://img.shields.io/github/actions/workflow/status/mflowcode/mfc/test.yml?style=flat&label=Tests&color=slateblue%09"/>
+  </a>
+  <a href="https://github.com/MFlowCode/MFC/blob/master/.github/CONTRIBUTING.md">
+    <img src="https://img.shields.io/github/contributors-anon/mflowcode/mfc?style=flat&color=darkslategrey%09" />
   </a>
   <a href="https://join.slack.com/t/mflowcode/shared_invite/zt-y75wibvk-g~zztjknjYkK1hFgCuJxVw">
     <img src="https://img.shields.io/badge/slack-MFC-purple.svg?logo=slack" />
   </a>
   <a href="https://lbesson.mit-license.org/">
-    <img src="https://img.shields.io/badge/License-MIT-blue.svg" />
+    <img src="https://img.shields.io/badge/License-MIT-crimson.svg" />
   </a>
   <a href="https://codecov.io/github/MFlowCode/MFC" target="_blank">
     <img src="https://codecov.io/github/MFlowCode/MFC/graph/badge.svg?token=8SY043QND4">
@@ -25,12 +25,35 @@
   </a>
 </p>
 
-Welcome to the home of MFC!
-MFC simulates compressible multi-component and multi-phase flows, [amongst other things](#what-else-can-this-thing-do). 
-MFC is written in Fortran and uses metaprogramming to keep the code short (about 20K lines).
+**Welcome!**
+MFC simulates compressible multi-phase flows, [among other things](#what-else-can-this-thing-do). 
+It uses metaprogramming to stay short and portable (~20K lines).
+MFC conducted the largest known, open CFD simulation at <a href="https://arxiv.org/abs/2505.07392" target="_blank">101 trillion grid points</a> (as of July 2025).
+
+<p align="center">
+<a href="https://doi.org/10.48550/arXiv.2503.07953" target="_blank">
+    <img src="https://img.shields.io/badge/DOI-10.48550/arXiv.2503.07953-thistle.svg"/>
+</a>
+<a href="https://github.com/MFlowCode/MFC/stargazers" target="_blank">
+    <img src="https://img.shields.io/github/stars/MFlowCode/MFC?style=flat&color=maroon"/>
+</a>
+
+</br>
+Is MFC useful for you? Consider citing it or giving a star! 
+</p>
+
+```bibtex
+@article{Wilfong_2025,
+  author = {Wilfong, Benjamin and {Le Berre}, Henry and Radhakrishnan, Anand and Gupta, Ansh and Vaca-Revelo, Diego and Adam, Dimitrios and Yu, Haocheng and Lee, Hyeoksu and Chreim, Jose Rodolfo and {Carcana Barbosa}, Mirelys and Zhang, Yanjun and Cisneros-Garibay, Esteban and Gnanaskandan, Aswin and {Rodriguez Jr.}, Mauro and Budiardja, Reuben D. and Abbott, Stephen and Colonius, Tim and Bryngelson, Spencer H.},
+  title = {{MFC 5.0: A}n exascale many-physics flow solver},
+  journal = {arXiv preprint arXiv:2503.07953},
+  year = {2025},
+  doi = {10.48550/arXiv.2503.07953}
+}
+```
 
 MFC is used on the latest leadership-class supercomputers.
-It scales <b>ideally to exascale</b>; [tens of thousands of GPUs on NVIDIA- and AMD-GPU machines](#is-this-really-exascale) on Oak Ridge Summit and Frontier.
+It scales <b>ideally to exascale</b>; [tens of thousands of GPUs on NVIDIA- and AMD-GPU machines](#is-this-really-exascale) on Oak Ridge Frontier, LLNL El Capitan, CSCS Alps, among others.
 MFC is a SPEChpc benchmark candidate, part of the JSC JUPITER Early Access Program, and used OLCF Frontier and LLNL El Capitan early access systems.
 
 Get in touch with <a href="mailto:shb@gatech.edu">Spencer</a> if you have questions!

docs/documentation/gpuDebugging.md

@@ -33,17 +33,17 @@ OMP_DISPLAY_AFFINITY=TRUE
 ```bash
 CRAY_ACC_DEBUG: 0 (off), 1, 2, 3 (very noisy)
 ```
-- Dumps a time-stamped log line (`"ACC: ...`) for every allocation, data transfer, kernel launch, wait, etc. Great first stop when "nothing seems to run on the GPU.
 
-- Outputs on STDERR by default. Can be changed by setting `CRAY_ACC_DEBUG_FILE`. 
+- Dumps a time-stamped log line (`ACC: ...`) for every allocation, data transfer, kernel launch, wait, etc. Great first stop when "nothing seems to run on the GPU".
+- Outputs on STDERR by default. Can be changed by setting `CRAY_ACC_DEBUG_FILE`.
   - Recognizes `stderr`, `stdout`, and `process`.
   - `process` automatically generates a new file based on `pid` (each MPI process will have a different file)
-
 - While this environment variable specifies ACC, it can be used for both OpenACC and OpenMP
 
 ```bash
 CRAY_ACC_FORCE_EARLY_INIT=1
 ```
+
 - Force full GPU initialization at program start so you can see start-up hangs immediately
 - Default behavior without an environment variable is to defer initialization on first use
 - Device initialization includes initializing the GPU vendor’s low-level device runtime library (e.g., libcuda for NVIDIA GPUs) and establishing all necessary software contexts for interacting with the device
@@ -150,7 +150,7 @@ LIBOMPTARGET_JIT_SKIP_OPT=1
 - [Cray & OpenACC Docs](https://cpe.ext.hpe.com/docs/24.11/cce/man7/intro_openacc.7.html#environment-variables)
 - [NVHPC & OpenACC Docs](https://docs.nvidia.com/hpc-sdk/compilers/hpc-compilers-user-guide/index.html?highlight=NVCOMPILER_#environment-variables)
 - [NVHPC & OpenMP Docs](https://docs.nvidia.com/hpc-sdk/compilers/hpc-compilers-user-guide/index.html?highlight=NVCOMPILER_#id2)
-- [LLVM & OpenMP Docs] (https://openmp.llvm.org/design/Runtimes.html)
+- [LLVM & OpenMP Docs](https://openmp.llvm.org/design/Runtimes.html)
     - NVHPC is built on top of LLVM
 - [OpenMP Docs](https://www.openmp.org/spec-html/5.1/openmp.html)
 - [OpenACC Docs](https://www.openacc.org/sites/default/files/inline-files/OpenACC.2.7.pdf)