api-python icon indicating copy to clipboard operation
api-python copied to clipboard

add chunked query to dc python api

Open clincoln8 opened this issue 5 years ago • 0 comments

adding conduct_chunked_query and its helper function recursive_query to be able to conduct large queries without overloading Data Commons server.

I had tried to add in the tests in chunked_query_test.py to tests/query_test.py, but was having a lot of trouble doing so. Should the tests be put in once conduct_chunked_query can be reached using just the api (as in dc.conduct_chunked_query() )?

Additionally, the test test_large_query() in chunked_query_test.py occasionally fails when running over command line, but I have run the same test many, many times using a colab notebook, and it never fails there. Any insight to why that may be happening would be appreciated :) I don't think it has to do with the code in the two functions since it works fine in colab. Maybe either the testing environment or the server, since it works fine in colab? I am not sure.

Thank you for the help!

clincoln8 avatar Sep 26 '20 01:09 clincoln8