pycuda icon indicating copy to clipboard operation
pycuda copied to clipboard

Including cuSPARSE raises C linkage errors

Open robertmaxton42 opened this issue 5 years ago • 7 comments

I'd like to be able to reference code defined in cuSPARSE in my code, but adding the line #include <cusparse.h> throws twenty-four iterations of

/opt/ssoft/apps/linux-centos7-sandybridge/gcc-4.9.4/cuda-10.1.243-prcsthu5do5pks2wkmliqdviisr6hw5m/bin/../targets/x86_64-linux/include/cuda_fp16.hpp(230): error: more than one instance of overloaded function "operator++" has "C" linkage

with different operators. (Oddly, the exact same code compiled just fine a month ago, so I suspect that my environment has changed in some way that I haven't noticed; but I have no idea how to begin hunting down the error.)

I can work around the above by turning on no_extern_c, but then I have no idea what the mangled name of my function is.

Any advice would be appreciated.

robertmaxton42 avatar Jan 25 '21 02:01 robertmaxton42

Workaround -- by rewriting my code so that it reads

}
#include <cusparse.h>
extern "C" {
....

I can avoid including the #include in the extern "C" without giving up on it entirely. However, this is less than ideal in a number of ways (the biggest one, of course, being that it's rather unstable), so it'd be cool if we could get an option to define a header before extern "C" and any semantically-similar blocks-openers that may be introduced in the future.

robertmaxton42 avatar Jan 25 '21 02:01 robertmaxton42

Could you explain how this relates to pycuda? Sorry if I'm being slow here. FWIW, if you're including <cusparse.h> in kernel code, I don't think that works. AFAIK, it needs to be called from the host.

inducer avatar Jan 25 '21 21:01 inducer

Basically, PyCUDA provides no way to #include additional header files outside extern "C, except by turning off extern "C" entirely. There's also (AFAIK) no way to get a list of all functions exported by a module, so turning off extern "C" usually requires that you turn on keep and then hunt down your mangled function name there.

robertmaxton42 avatar Jan 25 '21 23:01 robertmaxton42

I see. What do you propose? Wrapping the kernel in extern "C" by default was done to make symbol lookup more straightforward, and as a matter of compatibility, that's not an easy thing that we could change.

inducer avatar Jan 26 '21 00:01 inducer

Well, perhaps we could have a header option in the constructor, that pasted its contents in at the top of the file, before extern "C" and any similar later additions? Basically a second code block field.

robertmaxton42 avatar Jan 26 '21 00:01 robertmaxton42

I'd view that as about as complicated as managing extern "C" manually. What's the advantage vs. that in your view?

inducer avatar Jan 26 '21 00:01 inducer

Mmm... basically, "do what I mean"-ness?

Or, well, I think it would be rather confusing not to handle it automatically, since I can see the CUDA-C++ code I've written and there's no name-mangling there. Even if I know about name-mangling (not a guarantee, especially for users with a primarily Pythonic background), it's still a rather confusing moment of dissonance to have the compiler tell me that the function I just wrote doesn't exist. As it is, handling extern "C" automagically hides that extra bit of complexity in a useful way.

Besides which, simply by the fact that it's been six or seven years at this point and I don't believe there's been a previous issue on this topic, clearly most people using PyCUDA never run into this problem, so it doesn't make much sense to remove a convenience from everyone else just to make my problem a little easier to solve... :p

robertmaxton42 avatar Jan 26 '21 00:01 robertmaxton42