vlang
`-use-coroutines` fails if wget isn't installed
Describe the bug
I tried to use coroutines instead of native threads on Linux, but my computer doesn't have wget installed (I generally substitute it for curl).
On compilation with the -use-coroutines flag, an error occurs because it tries to use wget to download photonwrapper.so.
Reproduction Steps
- Create a simple hello world in a
main.vfile - Compile it with
v -use-coroutines main.v
Expected Behavior
The program is compiled successfully.
Current Behavior
The compiler doesn't detect photonwrapper.so, so it tries to download it using wget, which fails as wget isn't found.
$ v -use-coroutines main.v
coroutines .so not found, downloading...
failed cmd: wget -O "[...]/vlang/v/thirdparty/photon/photonwrapper.so" "https://github.com/vlang/photonbin/raw/master/photonwrapper_linux_amd64.so"
failed code: 127
V panic: sh: line 1: wget: command not found
v hash: 4145669
/tmp/v_1000/v2.01HPMZBJW4B1YDHVR7G8XK741E.tmp.c:21529: at _v_panic: Backtrace
/tmp/v_1000/v2.01HPMZBJW4B1YDHVR7G8XK741E.tmp.c:35533: by os__execute_or_panic
/tmp/v_1000/v2.01HPMZBJW4B1YDHVR7G8XK741E.tmp.c:41956: by v__pref__parse_args_and_show_errors
/tmp/v_1000/v2.01HPMZBJW4B1YDHVR7G8XK741E.tmp.c:53823: by main__main
/tmp/v_1000/v2.01HPMZBJW4B1YDHVR7G8XK741E.tmp.c:54612: by main
Possible Solution
The error comes from vlib/v/pref/pref.c.v, where wget is used directly from an os.execute command.
I can see that the previous line tries to use http.download_file, I'm not sure why it's commented out but maybe it is an option. Otherwise, there needs to be a check on whether wget exists, potentially use curl as a fallback, and if neither are present, instructions to manually download and install the required file.
Additional Information/Context
No response
V version
0.4.4 4145669
Environment details (OS name and version, etc.)
V full version: V 0.4.4 a374d25.4145669 OS: linux, "Arch Linux" Processor: 8 cpus, 64bit, little endian, Intel(R) Core(TM) i7-6700K CPU @ 4.00GHz
[!NOTE] You can use the 👍 reaction to increase the issue's priority for developers.
Please note that only the 👍 reaction to the issue itself counts as a vote. Other reactions and those to comments will not be taken into account.
We should really use something other than wget for this, it was just used as a quick solution. We will come up with a better solution.
Perhaps just use https://modules.vlang.io/net.http.html#download_file instead of shelling out to wget?
Perhaps just use https://modules.vlang.io/net.http.html#download_file instead of shelling out to wget?
definitely seems like the way to go
I think the reason wget was used was due to some net error on arm. well that is what I gathered from the comments :)
Even though the pipelines fail in the PR, I tried to apply it locally, and it is able to fetch the .so file. So hopefully, the pipeline issues can be fixed soon.
I'm still having an issue with trying to use coroutines though:
$ v -cg -use-coroutines 1.v
coroutines .so not found, downloading...
done!
/tmp/v_1000/1.01HPQE12QDDJN7JCMS25QB8YAF.tmp.c:24040: warning: implicit declaration of function 'photon_thread_create_and_migrate_to_work_pool'
/tmp/v_1000/1.01HPQE12QDDJN7JCMS25QB8YAF.tmp.c:24168: warning: implicit declaration of function 'delete_photon_work_pool'
tcc: error: undefined symbol 'photon_thread_create_and_migrate_to_work_pool'
tcc: error: undefined symbol 'delete_photon_work_pool'
builder error:
==================
C error. This should never happen.
I'm not sure if I should open an issue on this repo or the v/photonbin repo, since this seems to be a photonbin issue but I don't know how it is linked in V.
Even though the pipelines fail in the PR, I tried to apply it locally, and it is able to fetch the .so file. So hopefully, the pipeline issues can be fixed soon.
I'm still having an issue with trying to use coroutines though:
$ v -cg -use-coroutines 1.v coroutines .so not found, downloading... done! /tmp/v_1000/1.01HPQE12QDDJN7JCMS25QB8YAF.tmp.c:24040: warning: implicit declaration of function 'photon_thread_create_and_migrate_to_work_pool' /tmp/v_1000/1.01HPQE12QDDJN7JCMS25QB8YAF.tmp.c:24168: warning: implicit declaration of function 'delete_photon_work_pool' tcc: error: undefined symbol 'photon_thread_create_and_migrate_to_work_pool' tcc: error: undefined symbol 'delete_photon_work_pool' builder error: ================== C error. This should never happen.I'm not sure if I should open an issue on this repo or the v/photonbin repo, since this seems to be a photonbin issue but I don't know how it is linked in V.
Hi @Varpie,
Could you please check that your v is up to date. (edit: i saw your on #4145669 which should be fine)
Could you also please check to make sure thirdparty/photonwrapper.h matches https://github.com/vlang/v/blob/master/thirdparty/photon/photonwrapper.h
You are linux x64, correct?
Yes, I've run v up and also tried to recompile v after the .so file was added just in case, but it didn't change anything.
thirdparty/photon/photonwrapper.h has (omitting the non-relevant lines)
void delete_photon_work_pool();
void photon_thread_create_and_migrate_to_work_pool(void* (* f)(void*), void* arg);
which corresponds to https://github.com/vlang/v/blob/master/thirdparty/photon/photonwrapper.h and https://github.com/vlang/photonbin/blob/master/photonwrapper.h
The only thing I can think of is that the photonwrapper_linux_amd64.so doesn't have the same signatures, as I see that it was updated 5 days ago maybe something went wrong then.
I just tested on my linux and it works fine. The only thing is I built it against fairly new libs, so if anything I would expect it to complain about finding some other libraries.
I'm not sure why it is complaining about those functions. hrmm
the symbols are in there, which can be verified with:
nm thirdparty/photon/photonwrapper.so | grep -E 'photon_thread_create_and_migrate_to_work_pool|delete_photon_work_pool'
outputs:
0000000000048d02 T delete_photon_work_pool
0000000000048e1c T photon_thread_create_and_migrate_to_work_pool
Unless photonwrapper.so did not download correctly using http.download_file and it's corrupted, in which case I would expect another error though.
hrmm, I think I will have to try rebuild it on another system
I tried to clean up my /tmp, which didn't fix it, then had a look at the .tmp.c that V created. I can see that both methods are called in it, without being defined.
The symbols are indeed defined in photonwrapper.so, at the same addresses as you show.
I've also tried to download the file directly, to make sure it wasn't an error with http.download_file or that it didn't fetch the wrong one.
One interesting thing to note is that in the .tmp.c file, I can see the methods being called but not defined, and going through the includes, I also cannot see any reference to photonwrapper.h (or anything related to photon, other than the failing method calls).
Also, the photon_thread_create_and _migrate_to_work_pool call seems to be related to go instructions in the V program, if I only use a simple hello world program as an input, it fails on delete_photon_work_pool only, that is called by void _vcleanup(void) (which I assume is called when the program terminates).
ohh shit, I know what it is :D
can you try add import coroutines 😆
You can try this example:
v -cg -use-coroutines examples/coroutines/simple-coroutines.v
To get it to run currently you will need to set LD_LIBRARY_PATH (on some os/distros you can copy the photonwrapper.so to the same folder as the executable)
so run it like this:
LD_LIBRARY_PATH=~/dev/src/v/v/thirdparty/photon ./examples/coroutines/simple_coroutines
We will come up with a solution for this in the future.
If you get another error now it will be due to incompatible library versions I linked against. Hopefully it works, but I will get the wrapper rebuilt soon with more standard library versions.
Using import coroutines actually increased the number of undefined symbols...
I tried to compile the simple_coroutines.v example, and here is the output:
$ v -cg -use-coroutines simple_coroutines.v
In file included from /tmp/v_1000/simple_coroutines.01HPT460AN3VAWWJGQSFJ7H2C9.tmp.c:1881:
/[...]/v/vlib/coroutines/sp_corrector.c:24: warning: implicit declaration of function 'pthread_getattr_np'
tcc: error: undefined symbol 'photon_recv'
tcc: error: undefined symbol 'photon_send'
tcc: error: undefined symbol 'photon_accept'
tcc: error: undefined symbol 'photon_socket'
tcc: error: undefined symbol 'photon_connect'
tcc: error: undefined symbol 'photon_sleep_ms'
tcc: error: undefined symbol 'GC_set_sp_corrector'
tcc: error: undefined symbol 'GC_get_sp_corrector'
tcc: error: undefined symbol 'set_photon_thread_stack_allocator'
tcc: error: undefined symbol 'photon_init_default'
tcc: error: undefined symbol 'init_photon_work_pool'
tcc: error: undefined symbol 'photon_thread_create_and_migrate_to_work_pool'
tcc: error: undefined symbol 'delete_photon_work_pool'
builder error:
==================
C error. This should never happen.
Searching a little bit, I found that pthread_getattr_np is part of pthread.h, so I tried to add a #import <pthread.h> in sp_corrector.c, but that didn't change anything.
I guess this is related to the libraries you linked against, as mentionned, since pthread_getattr_np is a non portable (hence the np) function. Or I am missing some C development dependencies...
hi @Varpie, it still very strange, I would imagine other errors due to the new versions of software I linked against.. not those.
Could you try building using -cc gcc, or -cc clang out of interest?
In the mean time I will rebuild photonwrapper.so with some more reasonable software versions
@joe-conigliaro I've created #20857 to follow up on the build errors, since we are getting away from the original issue which was related to wget (for which I've created a PR). I've also included some details on compilation with gcc and clang, still doesn't work for me.
The photonwrapper.so can now be downloaded with curl as a fallback (if installed), and will give instructions for manual download if both wget and curl fail, so I'm closing this issue.