`Profile` test seems to sporadically hang on Window x86-64

[topolarity]

• https://buildkite.com/julialang/julia-master/builds/50792#01997b26-4933-4593-b892-8bd6ecd97dd2/833-1300
• https://buildkite.com/julialang/julia-master/builds/50782#0199782e-b1fc-4d94-9eb6-7aeada8cd8fa/843-1310
• https://buildkite.com/julialang/julia-master/builds/50729#01996e1c-c4c3-48be-9210-3a6ec56b1410/843-1305
• https://buildkite.com/julialang/julia-master/builds/50567#01994ed2-a772-46b8-ab03-8402834aafa8/830-1290

Not sure when this started happening, but I suspect this might be why the Windows runners are having trouble keeping up with the CI queue lately.

[topolarity]

Does not reproduce for me locally after repeated testing on Cygwin

[Liozou]

I can reproduce locally, with the following versioninfo:

Julia Version 1.13.0-DEV.1184
Commit d7034e51* (2025-09-23 13:04 UTC)
Platform Info:
  OS: Windows (x86_64-w64-mingw32)
  CPU: 48 × Intel(R) Xeon(R) Silver 4214R CPU @ 2.40GHz
  WORD_SIZE: 64
  LLVM: libLLVM-20.1.8 (ORCJIT, cascadelake)
  GC: Built with stock GC
Threads: 1 default, 1 interactive, 1 GC (on 48 virtual cores)

(the commit is from a branch with unrelated changes, the closest commit from master is a4e02ca)

The call that triggers the hang is: @profile InteractiveUtils.peakflops() which I minimized to the following program:

import Profile
function foo(n=1)
   a = ones(n, n)
   @elapsed a*a
end
Profile.start_timer()
foo() # hangs

I'm not sure I can help more than that by myself, but I can run stuff on my machine if you want.

[topolarity]

Interesting.. How reliably does it hang for you?

Do you also get a hang if you run the MWE on juliaup nightly (commit 874c330db7)?

[Liozou]

The hang is almost systematic, I had 2 runs without a hang on more than 30 tries.
It also hangs on julia nightly, with versioninfo:

Julia Version 1.13.0-DEV.1194
Commit 874c330db7 (2025-09-24 09:54 UTC)
Build Info:
  Official https://julialang.org release
Platform Info:
  OS: Windows (x86_64-w64-mingw32)
  CPU: 48 × Intel(R) Xeon(R) Silver 4214R CPU @ 2.40GHz
  WORD_SIZE: 64
  LLVM: libLLVM-20.1.8 (ORCJIT, cascadelake)
  GC: Built with stock GC
Threads: 1 default, 1 interactive, 1 GC (on 48 virtual cores)

[topolarity]

Great, thanks. Looks like that reproduces for me as long as I do:

PS C:\Users\topolarity> cat mwe.jl
import Profile
function foo(n=1)
   a = ones(n, n)
   @elapsed a*a
end
Profile.start_timer()
foo()

PS C:\Users\topolarity> julia +nightly mwe.jl
# hangs

It does not reproduce if I paste the same code into the REPL, but running as a script it's quite reliable.

Thanks for the MWE!

[Liozou]

I can reproduce the hang on v1.12.0-rc2, although it takes many more tries (about once every 20 times). I can't reproduce on v1.11.7 (no hang on 50 tries).
On the master branch, I can reproduce it quasi-systematically as far back as ab49eb1, I haven't tried earlier commits.

[topolarity]

Looks to be a deadlock between the profiling thread:

Thread 10 (Thread 27044.0x46b4):
#0  0x00007ffce45e6e84 in ntdll!ZwWaitForAlertByThreadId () from /cygdrive/c/windows/SYSTEM32/ntdll.dll
#1  0x00007ffce448639f in ntdll!RtlInitializeResource () from /cygdrive/c/windows/SYSTEM32/ntdll.dll
#2  0x00007ffce44fc87f in ntdll!RtlRaiseStatus () from /cygdrive/c/windows/SYSTEM32/ntdll.dll
#3  0x00007ffce44fbff2 in ntdll!RtlEnterCriticalSection () from /cygdrive/c/windows/SYSTEM32/ntdll.dll
#4  0x00007ffce44c68ab in ntdll!TpAllocTimer () from /cygdrive/c/windows/SYSTEM32/ntdll.dll
#5  0x00007ffce44ea48e in ntdll!RtlxUnicodeStringToOemSize () from /cygdrive/c/windows/SYSTEM32/ntdll.dll
#6  0x00007ffce451a05a in ntdll!RtlQueryCriticalSectionOwner () from /cygdrive/c/windows/SYSTEM32/ntdll.dll
#7  0x00007ffce451915c in ntdll!RtlQueryProcessDebugInformation () from /cygdrive/c/windows/SYSTEM32/ntdll.dll
#8  0x00007ffcdee4e3af in dbghelp!EnumerateLoadedModulesW64 () from /cygdrive/c/windows/SYSTEM32/dbghelp.dll
#9  0x00007ffcdefa1713 in dbghelp!SymRefreshModuleList () from /cygdrive/c/windows/SYSTEM32/dbghelp.dll
#10 0x00007ffc272783a7 in jl_refresh_dbg_module_list () at C:/workdir/src/stackwalk.c:466
#11 jl_refresh_dbg_module_list () at C:/workdir/src/stackwalk.c:463
#12 jl_unw_init (cursor=0x7f775ff0e0, Context=0x7f775ff620) at C:/workdir/src/stackwalk.c:474
#13 rec_backtrace_ctx (bt_data=0x2385902be80, maxsize=9999031, context=context@entry=0x7f775ff620, pgcstack=<optimized out>, pgcstack@entry=0x0) at C:/wo
#14 0x00007ffc2728289c in profile_bt (lparam=<optimized out>) at C:/workdir/src/signals-win.c:489
#15 0x00007ffce336e8d7 in KERNEL32!BaseThreadInitThunk () from /cygdrive/c/windows/System32/KERNEL32.DLL
#16 0x00007ffce4488d9c in ntdll!RtlUserThreadStart () from /cygdrive/c/windows/SYSTEM32/ntdll.dll
#17 0x0000000000000000 in ?? ()

and some GetProcAddress queries in libblastrampoline's DllMain:

Thread 1 (Thread 27044.0x5144):
#0  0x00007ffce4493d40 in ntdll!RtlReleaseSRWLockExclusive () from /cygdrive/c/windows/SYSTEM32/ntdll.dll
#1  0x00007ffce4487fc7 in ntdll!LdrGetProcedureAddressForCaller () from /cygdrive/c/windows/SYSTEM32/ntdll.dll
#2  0x00007ffce18381bc in KERNELBASE!GetProcAddressForCaller () from /cygdrive/c/windows/System32/KERNELBASE.dll
#3  0x0000000061942128 in DllMain (hModule=<optimized out>, code=<optimized out>, reserved=<optimized out>) at libblastrampoline.c:494
#4  0x00000000619412ab in __DllMainCRTStartup (hDllHandle=0x61940000, dwReason=1, lpreserved=0x0) at /workspace/srcdir/mingw-w64-v11.0.1/mingw-w64-crt/cr
#5  0x00007ffce45e0ede in ntdll!RtlEncodeRemotePointer () from /cygdrive/c/windows/SYSTEM32/ntdll.dll
#6  0x00007ffce44f483e in ntdll!RtlDeactivateActivationContext () from /cygdrive/c/windows/SYSTEM32/ntdll.dll
#7  0x00007ffce44f37ac in ntdll!RtlDeleteElementGenericTableAvlEx () from /cygdrive/c/windows/SYSTEM32/ntdll.dll
#8  0x00007ffce44886fa in ntdll!LdrGetProcedureAddressForCaller () from /cygdrive/c/windows/SYSTEM32/ntdll.dll
#9  0x00007ffce4487883 in ntdll!LdrGetProcedureAddress () from /cygdrive/c/windows/SYSTEM32/ntdll.dll
#10 0x00007ffce44ca584 in ntdll!LdrGetDllHandleEx () from /cygdrive/c/windows/SYSTEM32/ntdll.dll
#11 0x00007ffce44ca190 in ntdll!LdrGetDllHandleEx () from /cygdrive/c/windows/SYSTEM32/ntdll.dll
#12 0x00007ffce4494fb0 in ntdll!LdrLoadDll () from /cygdrive/c/windows/SYSTEM32/ntdll.dll
#13 0x00007ffce179dd0f in LoadLibraryExW () from /cygdrive/c/windows/System32/KERNELBASE.dll
#14 0x00007ffc272387e6 in ijl_dlopen (filename=0x7f735fba39 "C:\\Users\\cody\\.julia\\juliaup\\julia-nightly\\bin\\libblastrampoline-5.dll", flags=flags@.c:164
#15 0x00007ffc272389cb in ijl_load_dynamic_library (modname=0x2385346ae38 "C:\\Users\\cody\\.julia\\juliaup\\julia-nightly\\bin\\libblastrampoline-5.dll"=1) at C:/workdir/src/dlload.c:383
#16 0x00007ffc0aef0053 in #dlopen#3 () at libdl.jl:120
#17 julia_dlopen_49466 () at libdl.jl:119
#18 0x00007ffc0aef010b in jfptr_dlopen_49467 () from /cygdrive/c/Users/cody/.julia/juliaup/julia-nightly/lib/julia/sys.dll
#19 0x00007ffc0b07a706 in macro expansion () at libdl.jl:436
#20 macro expansion () at lock.jl:376
#21 julia_#dlopen#10_29254 () at libdl.jl:427
#22 0x00007ffc0b07960f in julia_dlopen_29251 () at libdl.jl:424
#23 0x00007ffc0b079c13 in julia_dlopen_49488 () at libdl.jl:424
#24 0x00007ffc0b079c41 in jfptr_dlopen_49489 () from /cygdrive/c/Users/cody/.julia/juliaup/julia-nightly/lib/julia/sys.dll
#25 0x00007ffc272bba7a in ijl_lazy_load_and_lookup (lib_val=<optimized out>, f_name=0x7ffc1461521e <_j_str_dgemm_64_YY.820> "dgemm_64_") at C:/workdir/sr
#26 0x00007ffc0b58e8d8 in julia_gemm!_39818 () at C:\workdir\usr\share\julia\stdlib\v1.13\LinearAlgebra\src/blas.jl:1652
#27 0x00007ffc09b9cb75 in gemm_wrapper! () at C:\workdir\usr\share\julia\stdlib\v1.13\LinearAlgebra\src/matmul.jl:905
#28 _syrk_herk_gemm_wrapper! () at C:\workdir\usr\share\julia\stdlib\v1.13\LinearAlgebra\src/matmul.jl:531
#29 julia_generic_matmatmul_wrapper!_39781 () at C:\workdir\usr\share\julia\stdlib\v1.13\LinearAlgebra\src/matmul.jl:511
#30 0x00000238583142f2 in _mul! () at C:\workdir\usr\share\julia\stdlib\v1.13\LinearAlgebra\src/matmul.jl:333
#31 mul! () at C:\workdir\usr\share\julia\stdlib\v1.13\LinearAlgebra\src/matmul.jl:302
#32 mul! () at C:\workdir\usr\share\julia\stdlib\v1.13\LinearAlgebra\src/matmul.jl:270
#33 * () at C:\workdir\usr\share\julia\stdlib\v1.13\LinearAlgebra\src/matmul.jl:141
#34 macro expansion () at timing.jl:461
#35 julia_foo_211 () at C:\Users\cody/mwe.jl:4
#36 0x0000023858314554 in julia_foo_207 () at C:\Users\cody/mwe.jl:3
#37 0x000002385831458b in jfptr_foo_208 ()
...

@vtjnash says doing GetProcAddress in DllMain is (lamentably) not legal on Windows, so libblastrampoline likely needs changing.

Likely, we've been hitting this more often because LBT is now lazily loaded (on ≤1.12, it is loaded on start-up).

[gbaraldi]

@Liozou Could you attach a debugger to the hang and get a backtrace similar to what Cody did on the comment above?
On 1.12 specifically

[Liozou]

I've been trying but I can't build the debug version of v1.12.0-rc2: I'm hiting lld: error: undefined symbol: __stack_chk_guard at the precompilation stage.

Click for the full stacktrace

$ ./julia-1.12.0-rc2/usr/bin/julia-debug mwe.jl
Info Given Profile was explicitly requested, output will be shown live
lld: error: undefined symbol: __stack_chk_guard
>>> referenced by jl_B97B.tmp(text#0.o):(.refptr.__stack_chk_guard)
>>> referenced by jl_B97B.tmp(text#1.o)
>>> referenced by jl_B97B.tmp(text#2.o)
ERROR: LoadError: failed process: Process(setenv(`'C:\cygwin64\home\liozou\julia-1.12.0-rc2\usr\tools\lld.exe' -flavor gnu -m i386pep -Bdynamic --enable-auto-image-base --allow-multiple-definition --disable-auto-import --disable-runtime-pseudo-reloc '' -shared -o 'C:\Users\lione\.julia\compiled\v1.13\StyledStrings\jl_B97C.tmp' --whole-archive 'C:\Users\lione\.julia\compiled\v1.13\StyledStrings\jl_B97B.tmp' --no-whole-archive '-LC:\cygwin64\home\liozou\julia-1.12.0-rc2\usr\lib\julia' '-LC:\cygwin64\home\liozou\julia-1.12.0-rc2\usr\bin' -ljulia-debug -ljulia-internal-debug -lopenlibm -lgcc_s -lgcc -lmsvcrt -lssp`,["WINDIR=C:\\WINDOWS", "PS1=\\[\e]633;A\a\\]\\[\\e]0;\\w\\a\\]\\n\\[\\e[32m\\]\\u@\\h \\[\\e[33m\\]\\w\\[\\e[0m\\]\\n\\\$ \\[\e]633;B\a\\]", "POWERSHELL_DISTRIBUTION_CHANNEL=MSI:Windows 10 Pro for Workstations", "FPS_BROWSER_APP_PROFILE_STRING=Internet Explorer", "CHROME_CRASHPAD_PIPE_NAME=\\\\.\\pipe\\crashpad_27344_VQVCKZVXBAFLVGMF", "ORIGINAL_PATH=/cygdrive/c/Program Files/Microsoft MPI/Bin:/cygdrive/c/WINDOWS/system32:/cygdrive/c/WINDOWS:/cygdrive/c/WINDOWS/System32/Wbem:/cygdrive/c/WINDOWS/System32/WindowsPowerShell/v1.0:/cygdrive/c/WINDOWS/System32/OpenSSH:/cygdrive/c/Program Files/PuTTY:/cygdrive/c/Program Files/dotnet:/cygdrive/c/Program Files/HDF_Group/HDF5/1.14.5/bin:/cygdrive/c/ProgramData/anaconda3:/cygdrive/c/Users/lione/AppData/Roaming/Python/Python312/Scripts:/cygdrive/c/Program Files (x86)/Intel/Intel(R) Management Engine Components/DAL:/cygdrive/c/Program Files/Intel/Intel(R) Management Engine Components/DAL:/cygdrive/c/Program Files/bin:/cygdrive/c/Program Files/Git/cmd:/cygdrive/c/Program Files/MATLAB/MATLAB Runtime/R2023a/runtime/win64:/cygdrive/c/Program Files/PowerShell/7:/cygdrive/c/Users/lione/AppData/Local/Microsoft/WindowsApps:/cygdrive/c/Users/lione/AppData/Local/Programs/Microsoft VS Code/bin:/cygdrive/c/Users/lione/AppData/Local/LAMMPS PLUMED Plugin 19Nov2024-MSMPI/bin:/cygdrive/c/Program Files/Avogadro2/bin:/cygdrive/c/texlive/2025/bin/windows:/cygdrive/c/Users/lione/AppData/Roaming/Code/User/globalStorage/github.copilot-chat/debugCommand", "TERM=xterm-256color", "OPENBLAS_NUM_THREADS=1", "VTUNE_PROFILER_2025_DIR=C:\\Program Files (x86)\\Intel\\oneAPI\\vtune\\2025.0", "ORIGINAL_XDG_CURRENT_DESKTOP=undefined"  …  "JULIA_NUM_THREADS=1", "USERDOMAIN=CDF-FCZQ6L3", "EFC_11088_1592913036=1", "MSMPI_BIN=C:\\Program Files\\Microsoft MPI\\Bin\\", "_=./usr/bin/julia-debug", "EXECIGNORE=*.dll", "COMPUTERNAME=CDF-FCZQ6L3", "APPDATA=C:\\Users\\lione\\AppData\\Roaming", "LANG=en_US.UTF-8", "ONEDRIVE=C:\\Users\\lione\\OneDrive"]), ProcessExited(1)) [1]

Stacktrace:                                                                                                                                                                                      
   [1] pipeline_error                                                                                                                                                                            
     @ .\process.jl:611 [inlined]
   [2] run(::Cmd, ::Base.DevNull, ::Vararg{Any}; wait::Bool)
     @ Base .\process.jl:526
   [3] run
     @ .\process.jl:523 [inlined]
 ┌ [4] link_image
 │   @ .\linking.jl:159 [inlined]
 ╰──── repeated 2 times
   [6] compilecache(pkg::Base.PkgId, path::String, internal_stderr::IO, internal_stdout::IO, keep_loaded_modules::Bool; flags::Cmd, cacheflags::Base.CacheFlags, reasons::Dict{String, Int64}, loadable_exts::Nothing)
     @ Base .\loading.jl:3281
   [7] kwcall(::@NamedTuple{reasons::Dict{String, Int64}, loadable_exts::Nothing}, ::typeof(Base.compilecache), pkg::Base.PkgId, path::String, internal_stderr::IO, internal_stdout::IO, keep_loaded_modules::Bool)
     @ Base .\loading.jl:3234
   [8] (::Base.var"#__require_prelocked##0#__require_prelocked##1"{Base.PkgId, String, Dict{String, Int64}})()
     @ Base .\loading.jl:2698
   [9] mkpidlock(f::Base.var"#__require_prelocked##0#__require_prelocked##1"{Base.PkgId, String, Dict{String, Int64}}, at::String, pid::Int32; kwopts::@Kwargs{stale_age::Int64, wait::Bool})    
     @ FileWatching.Pidfile C:\cygwin64\home\liozou\julia-1.12.0-rc2\usr\share\julia\stdlib\v1.13\FileWatching\src\pidfile.jl:94
  [10] #mkpidlock#7
     @ C:\cygwin64\home\liozou\julia-1.12.0-rc2\usr\share\julia\stdlib\v1.13\FileWatching\src\pidfile.jl:89 [inlined]
  [11] trymkpidlock(::Function, ::Vararg{Any}; kwargs::@Kwargs{stale_age::Int64})
     @ FileWatching.Pidfile C:\cygwin64\home\liozou\julia-1.12.0-rc2\usr\share\julia\stdlib\v1.13\FileWatching\src\pidfile.jl:115
  [12] #invokelatest_gr#234
     @ .\reflection.jl:1335 [inlined]
  [13] invokelatest_gr
     @ .\reflection.jl:1327 [inlined]
  [14] maybe_cachefile_lock(f::Base.var"#__require_prelocked##0#__require_prelocked##1"{Base.PkgId, String, Dict{String, Int64}}, pkg::Base.PkgId, srcpath::String; stale_age::Int64)
     @ Base .\loading.jl:3927
  [15] maybe_cachefile_lock
     @ .\loading.jl:3924 [inlined]
  [16] __require_prelocked(pkg::Base.PkgId, env::String)
     @ Base .\loading.jl:2684
  [17] _require_prelocked(uuidkey::Base.PkgId, env::String)
     @ Base .\loading.jl:2512
  [18] macro expansion
     @ .\loading.jl:2440 [inlined]
  [19] macro expansion
     @ .\lock.jl:376 [inlined]
  [20] __require(into::Module, mod::Symbol)
     @ Base .\loading.jl:2404
  [21] require
     @ .\loading.jl:2380 [inlined]
  [22] eval_import_path
     @ .\module.jl:36 [inlined]
  [23] eval_import_path_all(at::Module, path::Expr, keyword::String)
     @ Base .\module.jl:60
  [24] _eval_import(::Bool, ::Module, ::Expr, ::Expr, ::Vararg{Expr})
     @ Base .\module.jl:101
  [25] top-level scope
     @ C:\cygwin64\home\liozou\julia-1.12.0-rc2\usr\share\julia\stdlib\v1.13\Profile\src\Profile.jl:42
  [26] include(mod::Module, _path::String)
     @ Base .\Base.jl:309
  [27] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt128}}, source::Nothing)
     @ Base .\loading.jl:3069
  [28] top-level scope
     @ stdin:5
  [29] eval(m::Module, e::Any)
     @ Core .\boot.jl:489
  [30] include_string(mapexpr::typeof(identity), mod::Module, code::String, filename::String)
     @ Base .\loading.jl:2915
  [31] include_string
     @ .\loading.jl:2925 [inlined]
  [32] exec_options(opts::Base.JLOptions)
     @ Base .\client.jl:328
  [33] _start()
     @ Base .\client.jl:563
in expression starting at C:\cygwin64\home\liozou\julia-1.12.0-rc2\usr\share\julia\stdlib\v1.13\Profile\src\Profile.jl:3                                                              
in expression starting at stdin:5                                                                                                                                                                
  ✗ Profile (serial)
Precompiling Profile finished.
  0 dependencies successfully precompiled in 34 seconds

ERROR: LoadError: The following 1 direct dependency failed to precompile:

Profile

Failed to precompile Profile [9abbd945-dff8-562f-b5e8-e1ebf5ef1b79] to "C:\\Users\\lione\\.julia\\compiled\\v1.13\\Profile\\jl_B719.tmp" (ProcessExited(1)).
lld: error: undefined symbol: __stack_chk_guard
>>> referenced by jl_B97B.tmp(text#0.o):(.refptr.__stack_chk_guard)
>>> referenced by jl_B97B.tmp(text#1.o)
>>> referenced by jl_B97B.tmp(text#2.o)
ERROR: LoadError: failed process: Process(setenv(`'C:\cygwin64\home\liozou\julia-1.12.0-rc2\usr\tools\lld.exe' -flavor gnu -m i386pep -Bdynamic --enable-auto-image-base --allow-multiple-definition --disable-auto-import --disable-runtime-pseudo-reloc '' -shared -o 'C:\Users\lione\.julia\compiled\v1.13\StyledStrings\jl_B97C.tmp' --whole-archive 'C:\Users\lione\.julia\compiled\v1.13\StyledStrings\jl_B97B.tmp' --no-whole-archive '-LC:\cygwin64\home\liozou\julia-1.12.0-rc2\usr\lib\julia' '-LC:\cygwin64\home\liozou\julia-1.12.0-rc2\usr\bin' -ljulia-debug -ljulia-internal-debug -lopenlibm -lgcc_s -lgcc -lmsvcrt -lssp`,["WINDIR=C:\\WINDOWS", "PS1=\\[\e]633;A\a\\]\\[\\e]0;\\w\\a\\]\\n\\[\\e[32m\\]\\u@\\h \\[\\e[33m\\]\\w\\[\\e[0m\\]\\n\\\$ \\[\e]633;B\a\\]", "POWERSHELL_DISTRIBUTION_CHANNEL=MSI:Windows 10 Pro for Workstations", "FPS_BROWSER_APP_PROFILE_STRING=Internet Explorer", "CHROME_CRASHPAD_PIPE_NAME=\\\\.\\pipe\\crashpad_27344_VQVCKZVXBAFLVGMF", "ORIGINAL_PATH=/cygdrive/c/Program Files/Microsoft MPI/Bin:/cygdrive/c/WINDOWS/system32:/cygdrive/c/WINDOWS:/cygdrive/c/WINDOWS/System32/Wbem:/cygdrive/c/WINDOWS/System32/WindowsPowerShell/v1.0:/cygdrive/c/WINDOWS/System32/OpenSSH:/cygdrive/c/Program Files/PuTTY:/cygdrive/c/Program Files/dotnet:/cygdrive/c/Program Files/HDF_Group/HDF5/1.14.5/bin:/cygdrive/c/ProgramData/anaconda3:/cygdrive/c/Users/lione/AppData/Roaming/Python/Python312/Scripts:/cygdrive/c/Program Files (x86)/Intel/Intel(R) Management Engine Components/DAL:/cygdrive/c/Program Files/Intel/Intel(R) Management Engine Components/DAL:/cygdrive/c/Program Files/bin:/cygdrive/c/Program Files/Git/cmd:/cygdrive/c/Program Files/MATLAB/MATLAB Runtime/R2023a/runtime/win64:/cygdrive/c/Program Files/PowerShell/7:/cygdrive/c/Users/lione/AppData/Local/Microsoft/WindowsApps:/cygdrive/c/Users/lione/AppData/Local/Programs/Microsoft VS Code/bin:/cygdrive/c/Users/lione/AppData/Local/LAMMPS PLUMED Plugin 19Nov2024-MSMPI/bin:/cygdrive/c/Program Files/Avogadro2/bin:/cygdrive/c/texlive/2025/bin/windows:/cygdrive/c/Users/lione/AppData/Roaming/Code/User/globalStorage/github.copilot-chat/debugCommand", "TERM=xterm-256color", "OPENBLAS_NUM_THREADS=1", "VTUNE_PROFILER_2025_DIR=C:\\Program Files (x86)\\Intel\\oneAPI\\vtune\\2025.0", "ORIGINAL_XDG_CURRENT_DESKTOP=undefined"  …  "JULIA_NUM_THREADS=1", "USERDOMAIN=CDF-FCZQ6L3", "EFC_11088_1592913036=1", "MSMPI_BIN=C:\\Program Files\\Microsoft MPI\\Bin\\", "_=./usr/bin/julia-debug", "EXECIGNORE=*.dll", "COMPUTERNAME=CDF-FCZQ6L3", "APPDATA=C:\\Users\\lione\\AppData\\Roaming", "LANG=en_US.UTF-8", "ONEDRIVE=C:\\Users\\lione\\OneDrive"]), ProcessExited(1)) [1]

Stacktrace:
   [1] pipeline_error
     @ .\process.jl:611 [inlined]
   [2] run(::Cmd, ::Base.DevNull, ::Vararg{Any}; wait::Bool)
     @ Base .\process.jl:526
   [3] run
     @ .\process.jl:523 [inlined]
 ┌ [4] link_image
 │   @ .\linking.jl:159 [inlined]
 ╰──── repeated 2 times
   [6] compilecache(pkg::Base.PkgId, path::String, internal_stderr::IO, internal_stdout::IO, keep_loaded_modules::Bool; flags::Cmd, cacheflags::Base.CacheFlags, reasons::Dict{String, Int64}, loadable_exts::Nothing)
     @ Base .\loading.jl:3281
   [7] kwcall(::@NamedTuple{reasons::Dict{String, Int64}, loadable_exts::Nothing}, ::typeof(Base.compilecache), pkg::Base.PkgId, path::String, internal_stderr::IO, internal_stdout::IO, keep_loaded_modules::Bool)
     @ Base .\loading.jl:3234
   [8] (::Base.var"#__require_prelocked##0#__require_prelocked##1"{Base.PkgId, String, Dict{String, Int64}})()
     @ Base .\loading.jl:2698
   [9] mkpidlock(f::Base.var"#__require_prelocked##0#__require_prelocked##1"{Base.PkgId, String, Dict{String, Int64}}, at::String, pid::Int32; kwopts::@Kwargs{stale_age::Int64, wait::Bool})    
     @ FileWatching.Pidfile C:\cygwin64\home\liozou\julia-1.12.0-rc2\usr\share\julia\stdlib\v1.13\FileWatching\src\pidfile.jl:94
  [10] #mkpidlock#7
     @ C:\cygwin64\home\liozou\julia-1.12.0-rc2\usr\share\julia\stdlib\v1.13\FileWatching\src\pidfile.jl:89 [inlined]
  [11] trymkpidlock(::Function, ::Vararg{Any}; kwargs::@Kwargs{stale_age::Int64})
     @ FileWatching.Pidfile C:\cygwin64\home\liozou\julia-1.12.0-rc2\usr\share\julia\stdlib\v1.13\FileWatching\src\pidfile.jl:115
  [12] #invokelatest_gr#234
     @ .\reflection.jl:1335 [inlined]
  [13] invokelatest_gr
     @ .\reflection.jl:1327 [inlined]
  [14] maybe_cachefile_lock(f::Base.var"#__require_prelocked##0#__require_prelocked##1"{Base.PkgId, String, Dict{String, Int64}}, pkg::Base.PkgId, srcpath::String; stale_age::Int64)
     @ Base .\loading.jl:3927
  [15] maybe_cachefile_lock
     @ .\loading.jl:3924 [inlined]
  [16] __require_prelocked(pkg::Base.PkgId, env::String)
     @ Base .\loading.jl:2684
  [17] _require_prelocked(uuidkey::Base.PkgId, env::String)
     @ Base .\loading.jl:2512
  [18] macro expansion
     @ .\loading.jl:2440 [inlined]
  [19] macro expansion
     @ .\lock.jl:376 [inlined]
  [20] __require(into::Module, mod::Symbol)
     @ Base .\loading.jl:2404
  [21] require
     @ .\loading.jl:2380 [inlined]
  [22] eval_import_path
     @ .\module.jl:36 [inlined]
  [23] eval_import_path_all(at::Module, path::Expr, keyword::String)
     @ Base .\module.jl:60
  [24] _eval_import(::Bool, ::Module, ::Expr, ::Expr, ::Vararg{Expr})
     @ Base .\module.jl:101
  [25] top-level scope
     @ C:\cygwin64\home\liozou\julia-1.12.0-rc2\usr\share\julia\stdlib\v1.13\Profile\src\Profile.jl:42
  [26] include(mod::Module, _path::String)
     @ Base .\Base.jl:309
  [27] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt128}}, source::Nothing)
     @ Base .\loading.jl:3069
  [28] top-level scope
     @ stdin:5
  [29] eval(m::Module, e::Any)
     @ Core .\boot.jl:489
  [30] include_string(mapexpr::typeof(identity), mod::Module, code::String, filename::String)
     @ Base .\loading.jl:2915
  [31] include_string
     @ .\loading.jl:2925 [inlined]
  [32] exec_options(opts::Base.JLOptions)
     @ Base .\client.jl:328
  [33] _start()
     @ Base .\client.jl:563
in expression starting at C:\cygwin64\home\liozou\julia-1.12.0-rc2\usr\share\julia\stdlib\v1.13\Profile\src\Profile.jl:3
in expression starting at stdin:
in expression starting at C:\cygwin64\home\liozou\mwe.jl:1

Without a julia-debug, what I tried is to make the hang appear by executing from bash

for i in {1..1000}; do echo "i = $i"; /cygdrive/c/Users/lione/.julia/juliaup/julia-1.12.0-rc2+0.x64.w64.mingw32/bin/julia mwe.jl; done

and once it got stuck, I attached a gdb with gdb -p $STUCK_JULIA_PID. The gdb session is:

GNU gdb (GDB) (Cygwin 14.2-1) 14.2

[License information...]

Attaching to process 16788
[New Thread 16788.0x578c]

warning: dll path for "C:\WINDOWS\SYSTEM32\ntd1l.dll" can not be evaluated

warning: dll path for "C:\WINDOWS\System32\kern3l32.dll" can not be evaluated
Reading symbols from /cygdrive/c/Users/lione/.julia/juliaup/julia-1.12.0-rc2+0.x64.w64.mingw32/bin/julia.exe...

warning: Could not load shared library symbols for 2 libraries, e.g. C:\WINDOWS\SYSTEM32\ntd1l.dll.
Use the "info sharedlibrary" command to see the complete listing.
Do you need "set solib-search-path" or "set sysroot"?
(gdb) info threads
  Id   Target Id           Frame 
  1    Thread 16788.0x4ae0 0x00007ffd24b437b4 in ntdll!ZwTerminateProcess () from /cygdrive/c/WINDOWS/SYSTEM32/ntdll.dll
* 2    Thread 16788.0x578c 0x00007ffd24b025c1 in ntdll!DbgBreakPoint () from /cygdrive/c/WINDOWS/SYSTEM32/ntdll.dll
(gdb) thread 1
[Switching to thread 1 (Thread 16788.0x4ae0)]
#0  0x00007ffd24b437b4 in ntdll!ZwTerminateProcess () from /cygdrive/c/WINDOWS/SYSTEM32/ntdll.dll
(gdb) backtrace
#0  0x00007ffd24b437b4 in ntdll!ZwTerminateProcess () from /cygdrive/c/WINDOWS/SYSTEM32/ntdll.dll
#1  0x00007ffd216f5160 in ?? () from /cygdrive/c/Program Files/SentinelOne/Sentinel Agent 22.1.4.10010/InProcessClient64.dll
#2  0x00007ffd216f51b8 in ?? () from /cygdrive/c/Program Files/SentinelOne/Sentinel Agent 22.1.4.10010/InProcessClient64.dll
#3  0x00007ffd216f508a in ?? () from /cygdrive/c/Program Files/SentinelOne/Sentinel Agent 22.1.4.10010/InProcessClient64.dll
#4  0x00007ffd249e8e87 in ntdll!RtlExitUserProcess () from /cygdrive/c/WINDOWS/SYSTEM32/ntdll.dll
#5  0x00007ffd23e418ab in KERNEL32!FatalExit () from /cygdrive/c/WINDOWS/System32/KERNEL32.DLL
#6  0x00007ff6e8c21c7a in exit (code=44) at C:/workdir/cli/loader_win_utils.c:114
#7  mainCRTStartup () at C:/workdir/cli/loader_exe.c:61
(gdb) thread 2
[Switching to thread 2 (Thread 16788.0x578c)]
#0  0x00007ffd24b025c1 in ntdll!DbgBreakPoint () from /cygdrive/c/WINDOWS/SYSTEM32/ntdll.dll
(gdb) backtrace
#0  0x00007ffd24b025c1 in ntdll!DbgBreakPoint () from /cygdrive/c/WINDOWS/SYSTEM32/ntdll.dll
#1  0x00007ffd24b146ee in ntdll!DbgUiRemoteBreakin () from /cygdrive/c/WINDOWS/SYSTEM32/ntdll.dll
#2  0x00007ffd23e2e8d7 in KERNEL32!BaseThreadInitThunk () from /cygdrive/c/WINDOWS/System32/KERNEL32.DLL
#3  0x00007ffd249e8d9c in ntdll!RtlUserThreadStart () from /cygdrive/c/WINDOWS/SYSTEM32/ntdll.dll
#4  0x0000000000000000 in ?? ()

If I can do anything else to help, don't hesitate to let me know!

[gbaraldi]

Ok, this is a separate issue that should also be fixed.

[IanButterworth]

The earliest I've seen the Profile timeout happen on is 8cdab5b but that's not an exhaustive search (it'd be nice if buildkite could show more clearly when timeouts occurred on runs in the summaries)