python-irodsclient icon indicating copy to clipboard operation
python-irodsclient copied to clipboard

What is the purpose of session.iRODSSession.numThreads?

Open Xophmeister opened this issue 5 years ago • 3 comments

The session.iRODSSession class has a numThreads attribute, which is set to zero. This attribute is referred to sporadically elsewhere in the code, but doesn't seem to provide any functionality. The only reference to Python's threading module I could find was a reentrant lock used in the pool module.

Is numThreads for future use?

Xophmeister avatar Apr 29 '20 16:04 Xophmeister

The iRODS multithread protocol isn't implemented in python-irodsclient yet. I have proposed PR #195 as a possible implementation, but it is blocked with no feedback.

pigay avatar Apr 30 '20 08:04 pigay

#235 provided parallel transfers in 0.9.0. @d-w-moore can you answer this question more thoroughly?

trel avatar Oct 16 '21 16:10 trel

Since numThreads seems to be used only in messages sent to the server. I've noticed this setting being a part of a number of iRODS server data structures, so I'd assume it has more to do with server internals - and possibly to do with the old, high ports implementation of parallel data movement, but none of this impacts multithreading within the client.

The new client driven parallel transfer logic that @trel mentioned (aka multi-1247) also uses threads but via the concurrent.futures module. The threading RLock object I believe is there in the Pool logic to prevent multiple threads from grabbing the same server connection. This makes the session object more thread-safe, not that there are any guarantees on the PRC being thread-safe (although it could be cajoled in that direction, if care were taken to keep the threads' iRODS concerns sufficiently separated.)

d-w-moore avatar Oct 17 '21 01:10 d-w-moore