An issue regarding the use of LAMMPS with the deep potential trained using MACE

[wxwth]

Dear Dr. Zeng,

I am sorry to bother you for an issue regarding the use of LAMMPS (internally installed within DeePMD-kit 3.0.0) with the deep potential trained using MACE implemented in DeePMD-kit 3.0.0.

Following the detailed guidance provided by Bohrium and the WeChat post by DeepModeling, I added the following command in my job submission script for LAMMPS:

export DP_PLUGIN_PATH=/gpfs/software/deepmd-kit-3.0.0-cuda12.6_gpu/lib/python3.12/site-packages/deepmd_gnn/lib/libdeepmd_gnn.so

The file libdeepmd_gnn.so does exist in the dictionary /gpfs/software/deepmd-kit-3.0.0-cuda12.6_gpu/lib/python3.12/site-packages/deepmd_gnn/lib/. Additionally, I successfully trained a deep potential using MACE in DeePMD-kit 3.0.0, indicating that there should not be any issues with the DeePMD-kit itself.

However, an error occurs, stating that the libdeepmd_gnn.so file cannot be found:

ERROR on proc 28: DeePMD-kit C API Error: DeePMD-kit Error: DeePMD-kit PyTorch backend error: DeePMD-kit Error: /gpfs/software/deepmd-kit-3.0.0-cuda12.6_gpu/lib/python3.12/site-packages/deepmd_gnn/lib/libdeepmd_gnn.so is not found! You can add the library directory to LD_LIBRARY_PATH (/home/conda/feedstock_root/build_artifacts/deepmd-kit_1732355244818/work/source/lmp/pair_deepmd.cpp:539)

I requested someone else to submit the job on another cluster, but they encountered the same error. My job submission script for LAMMPS is attached for reference. Could you kindly provide any expert suggestions to resolve this issue?

Thank you for your time, and I am looking forward to your reply.

Sincerely,
Xu

myjob.sh.txt

[njzjz]

Setting the environment variable LD_DEBUG=libs would print more helpful information.

[wxwth]

Dear Dr. Zeng,

Thank you for your reply. Enclosed is the output file generated with the environment variable LD_DEBUG=libs. I am sorry that I cannot find the solution to this issue. Could you kindly take a look? It should be noted that this simulation was performed on a single core to avoid redundant output. Thank you for your time!

Sincerely,
Xu

slurm-1937867.out.txt

[njzjz]

It seems that libcuda.so.1 is not found, which is the last file it tries to find before the error is thrown. The error message could be improved, though.

3463222:	find library=libcuda.so.1 [0]; searching
   3463222:	 search path=/gpfs/software/deepmd-kit-3.0.0-cuda12.6_gpu/lib/python3.12/site-packages/deepmd_gnn/lib/tls/x86_64/x86_64:/gpfs/software/deepmd-kit-3.0.0-cuda12.6_gpu/lib/python3.12/site-packages/deepmd_gnn/lib/tls/x86_64:/gpfs/software/deepmd-kit-3.0.0-cuda12.6_gpu/lib/python3.12/site-packages/deepmd_gnn/lib/tls/x86_64:/gpfs/software/deepmd-kit-3.0.0-cuda12.6_gpu/lib/python3.12/site-packages/deepmd_gnn/lib/tls:/gpfs/software/deepmd-kit-3.0.0-cuda12.6_gpu/lib/python3.12/site-packages/deepmd_gnn/lib/x86_64/x86_64:/gpfs/software/deepmd-kit-3.0.0-cuda12.6_gpu/lib/python3.12/site-packages/deepmd_gnn/lib/x86_64:/gpfs/software/deepmd-kit-3.0.0-cuda12.6_gpu/lib/python3.12/site-packages/deepmd_gnn/lib/x86_64:/gpfs/software/deepmd-kit-3.0.0-cuda12.6_gpu/lib/python3.12/site-packages/deepmd_gnn/lib		(RPATH from file /gpfs/software/deepmd-kit-3.0.0-cuda12.6_gpu/lib/python3.12/site-packages/deepmd_gnn/lib/libdeepmd_gnn.so)
   3463222:	  trying file=/gpfs/software/deepmd-kit-3.0.0-cuda12.6_gpu/lib/python3.12/site-packages/deepmd_gnn/lib/tls/x86_64/x86_64/libcuda.so.1
   3463222:	  trying file=/gpfs/software/deepmd-kit-3.0.0-cuda12.6_gpu/lib/python3.12/site-packages/deepmd_gnn/lib/tls/x86_64/libcuda.so.1
   3463222:	  trying file=/gpfs/software/deepmd-kit-3.0.0-cuda12.6_gpu/lib/python3.12/site-packages/deepmd_gnn/lib/tls/x86_64/libcuda.so.1
   3463222:	  trying file=/gpfs/software/deepmd-kit-3.0.0-cuda12.6_gpu/lib/python3.12/site-packages/deepmd_gnn/lib/tls/libcuda.so.1
   3463222:	  trying file=/gpfs/software/deepmd-kit-3.0.0-cuda12.6_gpu/lib/python3.12/site-packages/deepmd_gnn/lib/x86_64/x86_64/libcuda.so.1
   3463222:	  trying file=/gpfs/software/deepmd-kit-3.0.0-cuda12.6_gpu/lib/python3.12/site-packages/deepmd_gnn/lib/x86_64/libcuda.so.1
   3463222:	  trying file=/gpfs/software/deepmd-kit-3.0.0-cuda12.6_gpu/lib/python3.12/site-packages/deepmd_gnn/lib/x86_64/libcuda.so.1
   3463222:	  trying file=/gpfs/software/deepmd-kit-3.0.0-cuda12.6_gpu/lib/python3.12/site-packages/deepmd_gnn/lib/libcuda.so.1
   3463222:	 search path=/gpfs/software/deepmd-kit-3.0.0-cuda12.6_gpu/bin/../lib		(RPATH from file lmp)
   3463222:	  trying file=/gpfs/software/deepmd-kit-3.0.0-cuda12.6_gpu/bin/../lib/libcuda.so.1
   3463222:	 search path=/gpfs/softs/intel/oneapi/mpi/2021.2.0/libfabric/lib:/gpfs/softs/intel/oneapi/mpi/2021.2.0/lib/release:/gpfs/softs/intel/oneapi/mpi/2021.2.0/lib:/gpfs/softs/intel/oneapi/mkl/2021.2.0/lib/intel64:/gpfs/softs/intel/oneapi/tbb/2021.2.0/lib/intel64/gcc4.8:/gpfs/softs/intel/oneapi/compiler/2021.2.0/linux/lib:/gpfs/softs/intel/oneapi/compiler/2021.2.0/linux/lib/x64:/gpfs/softs/intel/oneapi/compiler/2021.2.0/linux/lib/emu:/gpfs/softs/intel/oneapi/compiler/2021.2.0/linux/compiler/lib/intel64_lin:/gpfs/softs/intel/oneapi/compiler/2021.2.0/linux/compiler/lib/intel64:/gpfs/softs/intel/oneapi/debugger/10.1.1/dep/lib:/gpfs/softs/intel/oneapi/debugger/10.1.1/gdb/intel64/lib		(LD_LIBRARY_PATH)
   3463222:	  trying file=/gpfs/softs/intel/oneapi/mpi/2021.2.0/libfabric/lib/libcuda.so.1
   3463222:	  trying file=/gpfs/softs/intel/oneapi/mpi/2021.2.0/lib/release/libcuda.so.1
   3463222:	  trying file=/gpfs/softs/intel/oneapi/mpi/2021.2.0/lib/libcuda.so.1
   3463222:	  trying file=/gpfs/softs/intel/oneapi/mkl/2021.2.0/lib/intel64/libcuda.so.1
   3463222:	  trying file=/gpfs/softs/intel/oneapi/tbb/2021.2.0/lib/intel64/gcc4.8/libcuda.so.1
   3463222:	  trying file=/gpfs/softs/intel/oneapi/compiler/2021.2.0/linux/lib/libcuda.so.1
   3463222:	  trying file=/gpfs/softs/intel/oneapi/compiler/2021.2.0/linux/lib/x64/libcuda.so.1
   3463222:	  trying file=/gpfs/softs/intel/oneapi/compiler/2021.2.0/linux/lib/emu/libcuda.so.1
   3463222:	  trying file=/gpfs/softs/intel/oneapi/compiler/2021.2.0/linux/compiler/lib/intel64_lin/libcuda.so.1
   3463222:	  trying file=/gpfs/softs/intel/oneapi/compiler/2021.2.0/linux/compiler/lib/intel64/libcuda.so.1
   3463222:	  trying file=/gpfs/softs/intel/oneapi/debugger/10.1.1/dep/lib/libcuda.so.1
   3463222:	  trying file=/gpfs/softs/intel/oneapi/debugger/10.1.1/gdb/intel64/lib/libcuda.so.1
   3463222:	 search cache=/etc/ld.so.cache
   3463222:	 search path=/lib64/tls:/lib64:/usr/lib64/tls:/usr/lib64		(system search path)
   3463222:	  trying file=/lib64/tls/libcuda.so.1
   3463222:	  trying file=/lib64/libcuda.so.1
   3463222:	  trying file=/usr/lib64/tls/libcuda.so.1
   3463222:	  trying file=/usr/lib64/libcuda.so.1
   3463222:

[wxwth]

Dear Dr. Zeng,

Thank you for your reply. I am confused about the requirement for the libcuda.so.1 file, as my job submission script indicates that I am running LAMMPS simulations using only the CPU, not the GPU.

Besides, I can only find the libcuda.so file in the /gpfs/softs/cuda/12.6.2/targets/x86_64-linux/lib/stubs/ dictionary of my cluster, but the libcuda.so.1 file is not present in the corresponding CUDA-12.6.2 directory.

Could you kindly clarify if it is still possible to perform the LAMMPS simulations (with the deep potential trained using MACE implemented in DeePMD-kit 3.0.0) under these conditions? Thank you for your time!

Sincerely,
Xu

[njzjz]

Does your pytorch link to libcuda.so.1? libdeepmd_gnn.so does not explicitly link to CUDA.