does it make sense to use thread and asyncio together and thread safety

[efg001]

hi, two questions 1. https://github.com/tylerebowers/Schwabdev/blob/54a6bd52739e1f2979146f3357ee1bf23e0d32a4/schwabdev/stream.py#L133-L150 it seems like you are mixing thread and asyncio. I think you just need multithreading here, I believe having a second thread running a coroutine does not help the main thread because the control will be yielded back to the event loop of the second thread without affecting the main thread

self._websocket can be accessed from multiple threads but it is not thread safe?

[tylerebowers]

1. The idea here is that the stream will be started seperately so it can recieve data continously through the inner-most while loop which is blocking (103). It might be possible to use multiprocessing (not multithreading) since threads is actually 1-core behind the scenes.
2. self._websocket is a private variable (the pythonic type) and the idea is that subscription requests can be made at the same time as it is receiving data. If there is a better way to do this while maintaining the ideas above let me know. I came up with the core idea of the streamer many years ago and have never really been too happy with the threading/asyncio mix but it does work pretty reliably.

[efg001]

Thanks for responding : )

1. I agree that because the callback is not a coroutine function, if you dont start a new thread here, the streaming client can/could block(i.e you have to use multithreading here) but my point is that I believe the mix of multithreading and asyncio here is unnecessary: maybe can you explain why do you have the new thread start the streaming client with _start_async as supposed to starting a basic streaming client?(i.e use websockets.sync.client)

2. Yes I see that but inside the streaming client class, like you pointed out, you can make request at the same time as it is receiving data and even though self._websocket is private, the class itself(which have access to private variables) has the ability to access the socket in multiple threads and I dont think it is thread safe

3. one option is what I have above -- remove all use of asyncio: you mentioned about GIL and I would guess you might counter this option with what you have above "since threads is actually 1-core behind the scenes." but asyncio is the same thing: the only difference is that asyncio uses event loop to do task scheduling and threading uses interpreter’s internal thread scheduler. I believe the use of asyncio and multithreading should be mutually exclusive most of the time. or if you insist using asyncio, you can make your interface async just like async web socket and ask user to use it as async method

        task = asyncio.create_task(ws_client.start(process_socket_data))
        request = ws_client.basic_request(service="LEVELONE_EQUITIES", command="SUBS", parameters={"keys": "XXX,XXX", "fields": "0"})  
        await ws_client.send(request)# need to modify send to send after connection is made or make socket client check for and send queued request on every iteration
        await loop()#keep program alive

or just make it blocking and ask the client to spawn a new thread to manage it

[mst409]

@efg001 I am intrested in rewriting the project in an asynchronous manner. Would you be intrested in working on it together? It seems like you have much more experience in concurrent programing then me and I would really appreciate your help. Thank you.

[tylerebowers]

Closing due to inactivity.