cannot perform operation: another operation is in progress when running from pytest

[ivictbor]

• GINO version: 1.0.1
• Python version: 3.6.9
• asyncpg version: 0.20.1
• aiocontextvars version: 0.2.2
• PostgreSQL version: 10

Description

Hi Guys, thanks for hardwork, we were dreaming about async ORM for a while, were using sqla TPEs and other workaround, and now it is here! But from first day we faced with issue which seams to be very unclear.

I have 1 REST endpoint on aiohttp

from gino.ext.aiohttp import Gino

db = Gino()

app = web.Application(middlewares=[db])

async def register(request):
    existing_user = await User.query.where(User.email == '1').gino.first()
    return 1

app.add_routes([
  web.post(f'{API_BASE}/users/', user.register),
])

asyncio.get_event_loop().run_until_complete(db.set_bind(conf.DB))

if __name__ == '__main__':
  web.run_app(app)

If I do

asyncio.get_event_loop().run_until_complete(user.register({}))

In side pytest file on root level it is ok. But once I call it from pytest function I got:

async def test_hello(aiohttp_client, loop):
    await user.register({})


pipenv run py.test test/run.py
Loading .env environment variables…
============================================= test session starts ==============================================platform linux -- Python 3.6.9, pytest-6.0.1, py-1.9.0, pluggy-0.13.1
rootdir: /mnt/c/xxxxxxxxxxxxxxxxxx/platform/server-node
plugins: aiohttp-0.3.0
collected 1 item                                                                                               

test/run.py F                                                                                            [100%]

=================================================== FAILURES ===================================================______________________________________________ test_hello[pyloop] ______________________________________________
aiohttp_client = <function aiohttp_client.<locals>.go at 0x7f6a037358c8>
loop = <_UnixSelectorEventLoop running=False closed=False debug=False>

    async def test_hello(aiohttp_client, loop):
>       await user.register({})

test/run.py:36:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
modules/user.py:17: in register
    existing_user = await User.query.where(User.email == '1').gino.first()
/home/ivan/.local/share/virtualenvs/server-node-LnTlQqzf/lib/python3.6/site-packages/gino/api.py:137: in first
    return await self._query.bind.first(self._query, *multiparams, **params)
/home/ivan/.local/share/virtualenvs/server-node-LnTlQqzf/lib/python3.6/site-packages/gino/engine.py:748: in first
    return await conn.first(clause, *multiparams, **params)
/home/ivan/.local/share/virtualenvs/server-node-LnTlQqzf/lib/python3.6/site-packages/gino/engine.py:147: in __aexit__
    await conn.release()
/home/ivan/.local/share/virtualenvs/server-node-LnTlQqzf/lib/python3.6/site-packages/gino/engine.py:279: in release
    await dbapi_conn.release(True)
/home/ivan/.local/share/virtualenvs/server-node-LnTlQqzf/lib/python3.6/site-packages/gino/engine.py:47: in release
    return await self._release()
/home/ivan/.local/share/virtualenvs/server-node-LnTlQqzf/lib/python3.6/site-packages/gino/engine.py:83: in _release
    await self._pool.release(conn)
/home/ivan/.local/share/virtualenvs/server-node-LnTlQqzf/lib/python3.6/site-packages/gino/dialects/asyncpg.py:232: in release
    await self._pool.release(conn)
/home/ivan/.local/share/virtualenvs/server-node-LnTlQqzf/lib/python3.6/site-packages/asyncpg/pool.py:654: in release
    return await asyncio.shield(ch.release(timeout))
/home/ivan/.local/share/virtualenvs/server-node-LnTlQqzf/lib/python3.6/site-packages/asyncpg/pool.py:216: in release
    raise ex
/home/ivan/.local/share/virtualenvs/server-node-LnTlQqzf/lib/python3.6/site-packages/asyncpg/pool.py:206: in release
    await self._con.reset(timeout=budget)
/home/ivan/.local/share/virtualenvs/server-node-LnTlQqzf/lib/python3.6/site-packages/asyncpg/connection.py:1114: in reset
    await self.execute(reset_query, timeout=timeout)
/home/ivan/.local/share/virtualenvs/server-node-LnTlQqzf/lib/python3.6/site-packages/asyncpg/connection.py:272:
in execute
    return await self._protocol.query(query, timeout)
asyncpg/protocol/protocol.pyx:301: in query
    ???
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

>   ???
E   asyncpg.exceptions._base.InterfaceError: cannot perform operation: another operation is in progress

asyncpg/protocol/protocol.pyx:664: InterfaceError
=========================================== short test summary info ============================================FAILED test/run.py::test_hello[pyloop] - asyncpg.exceptions._base.InterfaceError: cannot perform operation: a...============================================== 1 failed in 0.62s ===============================================

I am even not trying to perfrom another operation. If we add print to register method - it called once

[ivictbor]

Seams like applying this fixes problem https://github.com/python-gino/gino/issues/539#issuecomment-584426048 and main issue is in using separate event loops, but message is very strange

Also still have a feeling that doing something wrong here if I need such workarounds... Seams to be pretty common task to wrap Gino calls in unittests, is there any other points in documentation about it?

[fantix]

Hey thanks for trying GINO out and sorry for the late reply! Let me take a look at this one.

[fantix]

Here's a modified self-contained version of your example that works with pytest-aiohttp:

import pytest
from aiohttp import web

from gino.ext.aiohttp import Gino

DSN = "postgresql:///t713"
db = Gino()
app = web.Application(middlewares=[db])
db.init_app(app, dict(dsn=DSN))


class User(db.Model):
    __tablename__ = "users"

    email = db.Column(db.String)


async def register(request):
    existing_user = await User.query.where(User.email == "1").gino.first()
    return web.Response(text="Hello, world")


app.add_routes([web.get("/users/", register)])

if __name__ == "__main__":
    import asyncio

    asyncio.get_event_loop().run_until_complete(db.set_bind(DSN))
    asyncio.get_event_loop().run_until_complete(db.gino.create_all())
    asyncio.get_event_loop().run_until_complete(db.pop_bind().close())
    web.run_app(app)


@pytest.fixture
def cli(loop, aiohttp_client):
    return loop.run_until_complete(aiohttp_client(app))


async def test_hello(cli):
    resp = await cli.get("/users/")
    assert resp.status == 200
    text = await resp.text()
    assert "Hello, world" in text

And I think the tests in gino-aiohttp examples are also working.