Using sshfs as a URL string to open remote dataset over SSH

[scottstanie]

Hi, I've just discovered that you can use https://github.com/fsspec/sshfs as follows to open a remote dataset over SSH:

from sshfs import SSHFileSystem
import xarray as xr

fs = SSHFileSystem("my-remote-machine")
f = fs.open('/path/to/data.nc')
ds = xr.open_dataset(f)

My questions is: are the URL strings, like "gcs://<bucket-name>/path.zarr", supported for the sshfs protocol? I can't seem to make it work with different attempts like

ds = xr.open_dataset('sshfs://my-remote-machine:/path/to/data.nc')

[nmcdtu]

Dear @scottstanie, I am facing the same issue.
Have you found the solution for it?
Thank you.

[adriaat]

I think the following can work based on your example:

xr.open_dataset('ssh://my-remote-machine/path/to/data.nc')

I also think it should work if using sftp instead of ssh. However, you're probably using credentials. If using a password, you can specify it as [user[:password]@]my-remote-machine[:port] (the brackets indicate optional parameters). If you don't want to expose such details, or using an ssh key, you can also use

xr.open_dataset(fs.get_mapper('/path/to/data.nc'))

Note that if instead of opening a netCDF file you want to open a Zarr file, then the following applies.
If using Zarr 2,

xr.open_zarr(fs.get_mapper('/path/to/data.zarr')

However, for Zarr 3:

1. Wrap fs in an asynchronous context (needed for step 2)
2. Define a zarr store with this virtual asynchronous filesystem

from fsspec.implementations.asyn_wrapper import AsyncFileSystemWrapper
from zarr.storage import FsspecStore

afs = AsyncFileSystemWrapper(fs)
# FsspecStore expects an AsyncFileSystem
ds = xr.open_zarr(FsspecStore(afs, path='/path/to/data.zarr'))