andywer
Ability to specify worker_threads/tinyworker
Hello, I hope you are having a nice day (Stay safe!)
Would it be possible to make a pre-launch option where you could specify the method of spawning (but only for certain workers/pools ?) Apparently, some modules like the canvas module are not context aware and therefore its impossible for me to use worker_threads... But then again I don't want to use child_processes for all pools/workers. Can spawning a certain pool using tinyworker without touching the other pools be possible without major changes?
PS: I'm on node12 but I had to make tweaks to force threads.js to use tiny-worker module but then again, all my stuff are now using child processes and consuming A LOT of resources.
If you have another advice I should use to combat the "non context aware" issue, I would love to hear it. Thanks a lot once again :D
Hey Kate! Hmm, can you elaborate a bit more on what you mean by "not context aware"?
In theory we could just add another option to the new Worker() constructor to explicitly state that you particularly want to use either tiny-worker or worker_threads by all means.
I am just not sure I understand why your canvas package would work in tiny-worker, but not in worker_threads. Maybe you can shine a light on that 😉
Stay safe as well!
@andywer Sorry about the late reply
Some modules are not context aware as in, they can't be loaded more than once by several threads. This should shed more information about it: https://nodejs.org/api/addons.html#addons_context_aware_addons
The reason why the canvas package worked with tiny-worker is because it actually spawns child processes and not worker_threads, so each process has its own independent cache and its own canvas instance. And that's why it works just fine. Regular worker_threads will just lead to one thread loading canvas and all the others erroring "Module did not self-register".
Yeah an option would be absolutely awesome, especially when you can still use worker_threads with others things and just use tiny-worker for a certain thing.
Thank you!
How about this API? Would open a PR later.
new Worker("./path/to/worker", { _implementation: "tiny-worker" })
The leading underscore of the _implementation option is not overly pretty, but it's consistent with other custom threads.js options we allow you to pass (_baseURL).
I was getting started as it just occured to me that it won't work the way I suggested… At least not easily / eleganty.
The issue is that we cannot pass the implementation selector to the new Worker() instantiation, as the Worker constructor is already part of the implementation you wish to select…
I see two alternative options here:
-
import { TinyWorker, WorkerThreadsWorker } from "threads"- Would be unusable for users who need to rely onimport "threads/register", though - Wrap the
Workerin yet another constructor which only abstracts the implementation selection - that new abstractedWorkercould then not beinstanceof NativeWorker, though
Need to think about it a little more…
Oh wow, that does seem like a lot of work and sacrifices made. If it's too much, you should honestly not do it :D
Yeah, I could not come up with an easier solution so far… Turning down the feature request for now. Sorry.
If anyone else needs this, too, feel free to leave a comment here and we might put more effort into it 😉
This causes problems with some other packages which use native addons (same issue), so an option to specify what to use would be great for me too.
Sorry, @Ugzuzg, but threads 0.x has reached its end of life. Version 0.12 is so old by now… May I ask what's stopping you from upgrading to a recent version of threads.js?
Yes, I understand, that's why I'm saying that having the option to choose the implementation would be useful for me. I was hoping to upgrade to the latest version and use worker threads, but because of the issue described in the very first comment, I am forced to use the old version.
Do you guys need to run the code in node.js only or does it also need to work in browsers?
So for creating the webworker directly, you can do this:
import * as BrowserImplementation from "threads/dist/master/implementation.browser"
const implementation = BrowserImplementation
getWorkerImplementation = implementation.getWorkerImplementation
const Worker = getWorkerImplementation().default
This just needs to be refactored in a function to ease calling it.
This is the same thing but for Nodejs (worker_thread or tinyworker)
import * as NodeImplementation from "threads/dist/master/implementation.node"
const implementation = NodeImplementation
getWorkerImplementation = implementation.getWorkerImplementation
const Worker = getWorkerImplementation().default
We need a way to choose between worker_thread or tinyworker. I think just a function will do the work. (createWorker)
Helo, how is the current state of this issue ?
Currently I am receiving this error when passing Worker from worker_thread:
Argument of type 'import("worker_threads").Worker'
is not assignable to parameter of type 'import("threads").Worker'.
Type 'Worker' is missing the following properties
from type 'Worker': addEventListener, dispatchEvent, removeEventListener
@rizkyramadhan Still an open issue. A workaround was posted before, though: https://github.com/andywer/threads.js/issues/229#issuecomment-667574249
Not pretty, but might just do for now.
This issue was done in #290. However, it was never merged to master. If you need it, I can register my fork so you can use it.
Thank you guys for the fast response, but my project needs a little bit customized solution, so I think I will manage the worker_thread manually without using threads.js.