Turn all `cdef class` objects in the Python layer to `cdef public class`

[leofang]

This allows us to share the class declarations with C/C++: https://cython.readthedocs.io/en/latest/src/userguide/external_C_code.html#using-cython-declarations-from-c which is needed after PR #463 set up a path for Python-C/C++/Cython interoperability, but without adding the public qualifier the interoperability was limited to Cython only.

[leofang]

@vzhurba01 I hope this is a simple one-liner change to the codegen and the remaining task is on verifying the generated headers. Could you take a look?

[leofang]

It is unclear to me if we still need this done if we want to go by the route of #564. @oleksandr-pavlyk @shwina WDYT?

[shwina]

Hmm, if my understanding is correct, the two seem a bit orthogonal:

1. cdef public would allow e.g., third-party C/C++ code to use types defined in Cython (not sure if we know of use cases for this)
2. get_native_hande() would allow us to grab a pointer to the underlying native CUDA object wrapped by a Cython type

Am I missing something or can (1) be used in place of (2) somehow?

[oleksandr-pavlyk]

@leofang, I think it should be cdef api instead of cdef public.

[leofang]

Sorry I somehow dropped the ball and Sasha reminded me about this with a use case in mind (accessing cuda.bindings types in pybind11).

@shwina You're absolutely right. And it turns out we need both:

// pseudo code
#include <cuda_bindings_runtime.h>

cudaStream_t c_stream = <cudaStream_t>get_native_cuda_handle<cuda_bindings_runtime_cudaStream_t>(py_stream);

We need to expose the C definitions for all Python classes (via cdef public) and then use it with get_native_cuda_handle to unbox. @oleksandr-pavlyk pointed out that test code we had with get_native_cuda_handle got around with the first requirement because we tested it in Cython and we got the definition for free.

I think it should be cdef api instead of cdef public.

Yes if we need more than the type information from cuda.bindings.

Assigning to Sasha for now as he'd have a use case study. @oleksandr-pavlyk all the needed changes to cuda.bindings will need to be done in the cython-gen, so just report back what's needed to happen and we can find someone else to work on the next step.