async-fastapi-sqlalchemy icon indicating copy to clipboard operation
async-fastapi-sqlalchemy copied to clipboard
verified publisher icon rhoboro

Test setup flow - why such complex db initialization?

Open guyarad opened this issue 2 years ago • 2 comments

This is more of a question/discussion - I'm trying to understand if there's a reason for the rather complex "session" fixture. If you have an actual PG database already setup, all tables were recreated by setup_test_db (that said, this is happening per session and not per test), why do you need a special flow to create the session and override the fast api dependency?

Some observations I made -

  1. The engine is returned before the transaction is completed in setup_test_db - is that how you are making sure the db remains clean?
  2. That said, you begin a new transaction in session fixture. At the end, you rollback the connection (but assuming some "commits" might have happened).
  3. end_savepoint - what is the point of it? if I understand correctly, it will automatically re-start a nested transaction when one is over in a sync session. But why?

Appreciate your clarifications!

guyarad avatar Dec 20 '23 10:12 guyarad

@guyarad Thank you for the question!! But I'm sorry. I can't remember right away, except that I referred to this issue.

For now, I logged it out. I will check the details below when I have time.

# async_engine = create_async_engine(f"{settings.DB_URI}/test", echo=True)
$ pytest app/tests/api/notes/test_views.py::test_notes_update --capture=no
========================================================================================================================= test session starts =========================================================================================================================
platform darwin -- Python 3.12.0, pytest-7.4.3, pluggy-1.3.0 -- /Users/rhoboro/go/src/github.com/rhoboro/async-fastapi-sqlalchemy/venv/bin/python3.12
cachedir: .pytest_cache
Using --randomly-seed=3328913590
rootdir: /Users/rhoboro/go/src/github.com/rhoboro/async-fastapi-sqlalchemy
configfile: pyproject.toml
plugins: env-1.1.3, randomly-3.15.0, cov-4.1.0, mock-3.12.0, anyio-3.7.1
collected 1 item                                                                                                                                                                                                                                                      

app/tests/api/notes/test_views.py::test_notes_update 2023-12-24 09:59:24,289 INFO sqlalchemy.engine.Engine select pg_catalog.version()
2023-12-24 09:59:24,289 INFO sqlalchemy.engine.Engine [raw sql] ()
2023-12-24 09:59:24,292 INFO sqlalchemy.engine.Engine select current_schema()
2023-12-24 09:59:24,293 INFO sqlalchemy.engine.Engine [raw sql] ()
2023-12-24 09:59:24,295 INFO sqlalchemy.engine.Engine show standard_conforming_strings
2023-12-24 09:59:24,295 INFO sqlalchemy.engine.Engine [raw sql] ()
2023-12-24 09:59:24,296 INFO sqlalchemy.engine.Engine BEGIN (implicit)
2023-12-24 09:59:24,296 INFO sqlalchemy.engine.Engine SAVEPOINT sa_savepoint_1
2023-12-24 09:59:24,296 INFO sqlalchemy.engine.Engine [no key 0.00009s] ()
2023-12-24 09:59:24,301 INFO sqlalchemy.engine.Engine SAVEPOINT sa_savepoint_2
2023-12-24 09:59:24,302 INFO sqlalchemy.engine.Engine [no key 0.00009s] ()
2023-12-24 09:59:24,304 INFO sqlalchemy.engine.Engine INSERT INTO notebooks (title) SELECT p0::VARCHAR FROM (VALUES ($1::VARCHAR, 0), ($2::VARCHAR, 1)) AS imp_sen(p0, sen_counter) ORDER BY sen_counter RETURNING notebooks.id, notebooks.id AS id__1
2023-12-24 09:59:24,304 INFO sqlalchemy.engine.Engine [generated in 0.00006s (insertmanyvalues) 1/1 (ordered)] ('Notebook 1', 'Notebook 2')
2023-12-24 09:59:24,308 INFO sqlalchemy.engine.Engine INSERT INTO notes (title, content, notebook_id) SELECT p0::VARCHAR, p1::VARCHAR, p2::INTEGER FROM (VALUES ($1::VARCHAR, $2::VARCHAR, $3::INTEGER, 0), ($4::VARCHAR, $5::VARCHAR, $6::INTEGER, 1), ($7::VARCHAR, $8::VARCHAR, $9::INTEGER, 2)) AS imp_sen(p0, p1, p2, sen_counter) ORDER BY sen_counter RETURNING notes.id, notes.id AS id__1
2023-12-24 09:59:24,308 INFO sqlalchemy.engine.Engine [generated in 0.00006s (insertmanyvalues) 1/1 (ordered)] ('Note 1', 'Content 1', 1, 'Note 2', 'Content 2', 1, 'Note 3', 'Content 3', 2)
2023-12-24 09:59:24,310 INFO sqlalchemy.engine.Engine RELEASE SAVEPOINT sa_savepoint_2
2023-12-24 09:59:24,310 INFO sqlalchemy.engine.Engine [no key 0.00008s] ()
2023-12-24 09:59:24,311 INFO sqlalchemy.engine.Engine SAVEPOINT sa_savepoint_3
2023-12-24 09:59:24,311 INFO sqlalchemy.engine.Engine [no key 0.00008s] ()
2023-12-24 09:59:24,313 INFO sqlalchemy.engine.Engine SELECT notebooks.id, notebooks.title 
FROM notebooks ORDER BY notebooks.id
2023-12-24 09:59:24,313 INFO sqlalchemy.engine.Engine [generated in 0.00010s] ()
2023-12-24 09:59:24,316 INFO sqlalchemy.engine.Engine SELECT notes.notebook_id AS notes_notebook_id, notes.id AS notes_id, notes.title AS notes_title, notes.content AS notes_content 
FROM notes 
WHERE notes.notebook_id IN ($1::INTEGER, $2::INTEGER) ORDER BY notes.id
2023-12-24 09:59:24,316 INFO sqlalchemy.engine.Engine [generated in 0.00018s] (1, 2)
2023-12-24 09:59:24,320 INFO sqlalchemy.engine.Engine SAVEPOINT sa_savepoint_4
2023-12-24 09:59:24,320 INFO sqlalchemy.engine.Engine [no key 0.00009s] ()
2023-12-24 09:59:24,322 INFO sqlalchemy.engine.Engine SELECT notes.id, notes.title, notes.content, notes.notebook_id, notebooks_1.id AS id_1, notebooks_1.title AS title_1 
FROM notes LEFT OUTER JOIN notebooks AS notebooks_1 ON notebooks_1.id = notes.notebook_id 
WHERE notes.id = $1::INTEGER ORDER BY notes.id
2023-12-24 09:59:24,322 INFO sqlalchemy.engine.Engine [generated in 0.00011s] (1,)
2023-12-24 09:59:24,325 INFO sqlalchemy.engine.Engine UPDATE notes SET title=$1::VARCHAR, content=$2::VARCHAR WHERE notes.id = $3::INTEGER
2023-12-24 09:59:24,325 INFO sqlalchemy.engine.Engine [generated in 0.00010s] ('Test Note', 'Test Content', 1)
2023-12-24 09:59:24,327 INFO sqlalchemy.engine.Engine SELECT notes.id, notes.title, notes.content, notes.notebook_id, notebooks_1.id AS id_1, notebooks_1.title AS title_1 
FROM notes LEFT OUTER JOIN notebooks AS notebooks_1 ON notebooks_1.id = notes.notebook_id 
WHERE notes.id = $1::INTEGER
2023-12-24 09:59:24,327 INFO sqlalchemy.engine.Engine [generated in 0.00010s] (1,)
2023-12-24 09:59:24,329 INFO sqlalchemy.engine.Engine RELEASE SAVEPOINT sa_savepoint_4
2023-12-24 09:59:24,329 INFO sqlalchemy.engine.Engine [no key 0.00007s] ()
b'{"id":1,"title":"Test Note","content":"Test Content","notebook_id":1,"notebook_title":"Notebook 1"}'
2023-12-24 09:59:24,332 INFO sqlalchemy.engine.Engine SELECT notes.id, notes.title, notes.content, notes.notebook_id 
FROM notes 
WHERE notes.id = $1::INTEGER
2023-12-24 09:59:24,332 INFO sqlalchemy.engine.Engine [generated in 0.00009s] (1,)
PASSED2023-12-24 09:59:24,334 INFO sqlalchemy.engine.Engine ROLLBACK TO SAVEPOINT sa_savepoint_3
2023-12-24 09:59:24,334 INFO sqlalchemy.engine.Engine [no key 0.00008s] ()
2023-12-24 09:59:24,335 INFO sqlalchemy.engine.Engine ROLLBACK


---------- coverage: platform darwin, python 3.12.0-final-0 ----------
Name                                    Stmts   Miss  Cover
-----------------------------------------------------------
app/__init__.py                             0      0   100%
app/api/__init__.py                         0      0   100%
app/api/main.py                             6      0   100%
app/api/notebooks/__init__.py               0      0   100%
app/api/notebooks/schema.py                16      0   100%
app/api/notebooks/use_cases.py             53     34    36%
app/api/notebooks/views.py                 20      5    75%
app/api/notes/__init__.py                   0      0   100%
app/api/notes/schema.py                    18      0   100%
app/api/notes/use_cases.py                 56     34    39%
app/api/notes/views.py                     20      4    80%
app/db.py                                  15      4    73%
app/main.py                                11      3    73%
app/models/__init__.py                      4      0   100%
app/models/base.py                          9      2    78%
app/models/notebooks.py                    44     17    61%
app/models/notes.py                        53     18    66%
app/models/schema.py                       13      0   100%
app/settings.py                             8      0   100%
app/tests/__init__.py                       0      0   100%
app/tests/api/__init__.py                   0      0   100%
app/tests/api/notebooks/__init__.py         0      0   100%
app/tests/api/notebooks/test_views.py      72     72     0%
app/tests/api/notes/__init__.py             0      0   100%
app/tests/api/notes/test_views.py          76     36    53%
app/tests/conftest.py                      76      7    91%
app/tests/test_main.py                      6      6     0%
app/tests/utils.py                          2      0   100%
-----------------------------------------------------------
TOTAL                                     578    242    58%


========================================================================================================================== 1 passed in 0.26s ===================================================================

rhoboro avatar Dec 24 '23 01:12 rhoboro

I have been since figured out what might led this flow. It seems like the fact FastAPI app is a single instance throughout the test session, and creating its ows engine, but then the tests are also doing some engine stuff make things a little difficult. I will have to go through the referenced issue to understand better. Thanks for getting back to me.

guyarad avatar Dec 24 '23 09:12 guyarad