sys:1: RuntimeWarning: coroutine 'GCSFileSystem._info' was never awaited

[darkclouder]

What happened: Exception:

Traceback (most recent call last):
  File "googleload.py", line 12, in <module>
    asyncio.get_event_loop().run_until_complete(main())
  File "/path/to/python/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "googleload.py", line 9, in main
    async with await fs.open("/path/to/file.txt") as fp:
  File "/path/to/python/lib/python3.8/site-packages/fsspec/spec.py", line 838, in open
    f = self._open(
  File "/path/to/python/lib/python3.8/site-packages/gcsfs/core.py", line 1141, in _open
    return GCSFile(
  File "/path/to/python/lib/python3.8/site-packages/gcsfs/core.py", line 1265, in __init__
    super().__init__(
  File "/path/to/python/lib/python3.8/site-packages/fsspec/spec.py", line 1135, in __init__
    self.size = self.details["size"]
TypeError: 'coroutine' object is not subscriptable
ERROR:asyncio:Unclosed client session
client_session: <aiohttp.client.ClientSession object at 0x10a6ad6a0>
sys:1: RuntimeWarning: coroutine 'GCSFileSystem._info' was never awaited

What you expected to happen: I'm not 100% sure how to use the new async part, as it is quite poorly documented so far. I don't even know which part of open/read/with needs to be awaited/asynced. But no matter what I choose there, it fails because the exception already fails in fs.open.

I suspect that self.details is set to the return value of some function, and in the original sync code the return value was just the details. As you can see in line 1134 of fsspec's specy.py:

self.details = fs.info(path)

I'm guessing that fs.info is a coroutine in the async implementation, therefore assigns an Awaitable to self.details.

Minimal Complete Verifiable Example:

import asyncio
import gcsfs

async def main():
    loop = asyncio.get_event_loop()
    fs = gcsfs.GCSFileSystem(project="my-project", asynchronous=True, loop=loop)
    await fs.set_session()

    async with await fs.open("/path/to/file.txt") as fp:
        print(await fp.read())

asyncio.get_event_loop().run_until_complete(main())

What I would like to know: Could you also provide me with the way how the opened file-pointer is supposed to be used afterwards? Unfortunately, you don't have any documentation beyond that I need to await the session in the beginning.

Do I await on fs.open, or is fs.open itself sync? Do I await fp.read or is everything buffered on open already anyway and the actual read can be sync because it just reads from memory?

Environment:

• gcsfs version: 0.7.0
• Python version: 3.8.5
• Operating System: macOS 10.15.4
• Install method (conda, pip, source): miniconda env, installed via pip

[martindurant]

You are quite right in your assessment: open calls info which, in an asynchronous context, is a coroutine. The file-like interface itself is not asynchronous, since there is state (the current buffers and file positions). However, the remote calls should still be able to work asynchronously - but this has not been tested or documented, as you have found.

Your workaround for now would be: use an async filesystem instance for general file operations, and a non-async one for working with file-like instances. If you have more async experience than me, then perhaps you can help to add async throughout the code, so that the type of use you want is supported too.