databricks
[ISSUE] Incremental File Listing not working as expected
Description When using the notebooks_modified_after parameter with the wobj.workspace.list() method to filter notebooks based on modification time, it returns only 1 notebook, whereas manually filtering the notebooks using the x.modified_at property gives the expected 10 notebooks. Both methods should ideally return the same number of notebooks.
Reproduction python Copy
Method 1: Using notebooks_modified_after
for x in wobj.workspace.list("/Workspace/Users/abc", notebooks_modified_after=1719072458045, recursive=True): print(x)
Method 2: Manually checking modified_at
for x in wobj.workspace.list("/Workspace/Users/abc", recursive=True): if x.modified_at > 1719072458045: print(x) Expected Behavior Both methods should return the same number of notebooks, ideally 10 notebooks. However, Method 1 only returns 1 notebook, while Method 2 returns 10 notebooks.
Is It a Regression? This behavior has not worked as expected in the current version. The issue has been observed in the current implementation of the SDK, and the number of notebooks returned by both methods should match.
Debug Logs Debugging logs were enabled using:
python Copy import logging logging.basicConfig(level=logging.DEBUG) However, the logs don’t indicate any clear differences between the two methods.
Environment Information OS: [e.g., macOS] SDK Version: [e.g., 0.1.0] Python Version: [e.g., 3.8] Additional Context Timestamp used: 1719072458045 (This value is the same for both methods). The issue seems to be related to how the notebooks_modified_after parameter is processed in Method 1.
