Metrics / apps to consider

[vsoch]

These are important to the labs! If you'd like to see an app, metric, or other added, please comment here.

• https://asc.llnl.gov/coral-2-benchmarks
• LAMMPS
• linktest https://www.fz-juelich.de/en/ias/jsc/services/user-support/software-tools/linktest (akin to netmark)?
• https://www2.mmm.ucar.edu/wrf/WG2/benchv3/
• autoperf https://github.com/HPCL/autoperf
• OSU benchmarks
• AMG
• Kripke
• IOR for storage
• charmpp for an app (we have hello world, I never heard back about apps, link is 404)
• qmcpack
• perfetto
• quicksilver
• pennant
• mpitrace https://github.com/IBM/mpitrace
• laghos add laghos #52
• https://github.com/LLNL/Kripke
• nekbone
• expose vmstat https://www.redhat.com/sysadmin/linux-commands-vmstat through sysstat
• https://github.com/caps-tum/qmpi but I think this needs to be linked with the library (e.g, -lpmpi) so not sure would work.
• https://hpc.llnl.gov/cts-2-benchmarks
• perf
• https://www.nersc.gov/systems/nersc-10/benchmarks'
• caliper?
• hacc (seems like it is only for powerpc)
• papi (and that is parsed with caliper maybe?) - this seems to be application specific, so would need to find something that runs and stress tests (while papi is running)
• some charm++ https://charm.readthedocs.io/en/latest/ampi/05-examples.html
• https://github.com/glennklockwood/bioinformatics-profile
• How could caliper PerfFlowAspect work here?
• https://dl.acm.org/doi/pdf/10.1145/3611007
• https://hpc.fau.de/research/tools/likwid/
• https://www.vi-hps.org/tools/tools.html
• https://open.xdmod.org/10.0/index.html
• understand how ovis works? https://github.com/ovis-hpc/ovis (note this is added as basic LDMS)
• bdas (coral-2) I got building with openmpi BDAS! 😎
• HPCToolkit (this will only work if we can create a completely isolated build we can mount to use...) this would require a spack copy view or similar. I don't know if I can muster the motivation to do this one
• chatterbug (added but hanging, needs debugging)
• hpcg
• linpack (this is HPL?) I have a container for this but no idea how to run (configure) it!
• https://proxyapps.exascaleproject.org/app/
• lulesh (with maestro)
• serac
• sundials
• hwloc (love this one)
• sysstat / pidstat (probably not fair to count these, nothing hard to build :P)

Unsure

• Dr. Memory https://drmemory.org/page_running.html

More Workflow Based

• parsl (demo for molecular design)
• merlin (demo) - too many steps / services to be considered a proxy app
• fireworks (demo)
• balsam (containers built but part of server seems buggy and/or proprietary, this is likely not going to be IT for a way to orchestrate workflows)
• [mlcommons-deepcam (also very complex to actually setup, I stopped at the base container
• nextflow ml workflow example
• snakemake bioscience example workflow
• weave (demos)

In Progress / Attempted

• perfKitBenchmark https://github.com/GoogleCloudPlatform/PerfKitBenchmarker (run locally on pod(s))? So likely we would use the launcher / worker model see here - note - having trouble.
• darshan https://github.com/darshan-hpc/darshan (WIP) likely this should be a spack view? adding darshan rse-ops/hpc-apps#24 - views are done, but segfaulted. https://github.com/converged-computing/metrics-operator/tree/add/darshan
• exaMPM based on cabana, didn't get MPI working ref

Recent readings / tools for performance

• https://www.osti.gov/servlets/purl/1649007
• https://www.intel.com/content/www/us/en/docs/vtune-profiler/user-guide/2023-0/hpc-performance-characterization-view.html
• https://sdm.lbl.gov/~sbyna/research/papers/2020/2020-HPC-workload-char-SNTA-Bang.pdf
• https://ntrs.nasa.gov/api/citations/20190000271/downloads/20190000271.pdf
• https://link.springer.com/chapter/10.1007/978-3-030-78713-4_22