SQLite objects created in a thread can only be used in that same thread. The object was created in thread

[eunbee31]

Hello,

I encountered this new error. Do you have any idea to resolve it? FYI, I don't use any multithreading process neither sqlite in my code.

"Traceback (most recent call last):
  File "/layers/google.python.pip/pip/lib/python3.7/site-packages/epo_ops/middlewares/throttle/storages/sqlite.py", line 76, in prune
    self.db.execute(sql)
sqlite3.ProgrammingError: SQLite objects created in a thread can only be used in that same thread. The object was created in thread id 69052472574080 and this is thread id 69052152153856.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/layers/google.python.pip/pip/lib/python3.7/site-packages/epo_ops/api.py", line 97, in published_data_search
    range,
  File "/layers/google.python.pip/pip/lib/python3.7/site-packages/epo_ops/api.py", line 223, in _search_request
    url, {"q": cql}, {range["key"]: "{begin}-{end}".format(**range)}
  File "/layers/google.python.pip/pip/lib/python3.7/site-packages/epo_ops/api.py", line 172, in _make_request
    response = request_method(url, data=data, headers=headers, params=params)
  File "/layers/google.python.pip/pip/lib/python3.7/site-packages/epo_ops/models.py", line 86, in post
    return self._request(_post_callback, url, data, **kwargs)
  File "/layers/google.python.pip/pip/lib/python3.7/site-packages/epo_ops/models.py", line 95, in _request
    url, data, kwargs = mw.process_request(self.env, url, data, **kwargs)
  File "/layers/google.python.pip/pip/lib/python3.7/site-packages/epo_ops/middlewares/throttle/throttler.py", line 20, in process_request
    time.sleep(self.history.delay_for(service))
  File "/layers/google.python.pip/pip/lib/python3.7/site-packages/epo_ops/middlewares/throttle/storages/sqlite.py", line 110, in delay_for
    self.prune()
  File "/layers/google.python.pip/pip/lib/python3.7/site-packages/epo_ops/middlewares/throttle/storages/sqlite.py", line 76, in prune
    self.db.execute(sql)
sqlite3.ProgrammingError: SQLite objects created in a thread can only be used in that same thread. The object was created in thread id 69052472574080 and this is thread id 69052152153856."

[Abeldewit]

Hi!

I'm experiencing the same issue with trying to multithread my application. Did you find any solution to this? I've found that Sqlite3 does not like threading very much (https://stackoverflow.com/questions/524797/python-sqlite-and-threading). But I have no clue how to write my own throttler.

[gsong]

Please see https://ricardoanderegg.com/posts/python-sqlite-thread-safety/ for more info. If you want to experiment, fork and edit https://github.com/gsong/python-epo-ops-client/blob/develop/epo_ops/middlewares/throttle/storages/sqlite.py#L35 to see if it solves your problem.

If you're successful, please open a PR.

[amotl]

Hi there,

apologies for the late reply on this topic, and thanks for sharing that document, @gsong. At Conclusion, there are clear instructions about how to overcome this problem, right? So, it will be all about submitting that suggestion as a patch here?

With kind regards, Andreas.