Add a way serialize arrays to string/bytes

[ev-br]

In data-apis/array-api-compat#345, a user wanted to json-serialize an Array API object. At the moment, this seems to require manually converting to a list of python int/float/bool objects.

For converting an array into a binary blob, we can export a DLPack capsule, but it does not seem to be serializable further:

In [28]: bytes(np.asarray([1, 2, 3]).__dlpack__())
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Cell In[28], line 1
----> 1 bytes(np.asarray([1, 2, 3]).__dlpack__())

TypeError: cannot convert 'PyCapsule' object to bytes

Serialization was previously discussed in #266; the difference here is scope: I'm only proposing to add a way to byte-export a dlpack capsule.

[rgommers]

From https://data-apis.org/array-api/latest/purpose_and_scope.html: The following topics are out of scope: I/O, ...

I'm perfectly fine with that still. There are lots of relevant serialization formats that users may want, and there is little to gain here by trying to standardize some methods. For JSON, it's pretty trivial to do efficiently with a library-specific method like json.dumps(np.from_dlpack(x).tolist()).

[ev-br]

There are lots of relevant serialization formats that users may want

Yes, and raw bytes is pretty special among them. Which is precisely why I'm only suggesting to allow exporting raw bytes from a dlpack capsule. Fancier formats are indeed out of scope and may need library-specific methods.

[rgommers]

Maybe easier as a small standalone package? https://github.com/pearu/pydlpack provides a lot of the plumbing already.

[rgommers]

We discussed this in the community meeting yesterday, and the prevailing opinion was that serialization should remain out of scope. A key argument not given here yet which was brought up in that discussion is that it's almost always the end user who is doing the serializing, and they don't interact with the array API standard but with a concrete implementation of it - hence it's perfectly okay for that user to use a primitive like .tolist that isn't in the standard. Another one was that "raw bytes" is not useful without the accompanying metadata (dtype, strides, etc.).

I'll close this issue as not planned.