Caching bug with `--incremental --cache-dir` combination causes `AssertionError: Cannot find module`

[benbuckman]

Crash Report

There appears to be a caching bug in mypy 1.2.0 (compiled: yes).

This started occurring recently after we upgraded from 1.0.0 to 1.2.0.

I am running mypy with these parameters:

mypy --config ./mypy.ini --incremental --cache-dir ./.local/mypy_cache/app --no-install-types --follow-imports=skip ./app

and very frequently getting the error:

AssertionError: Cannot find module for google

(There is no package in our application called simply google; but there are a bunch that start with that name, and have import paths starting with google.; see below.)

If I delete the contents of .local/mypy_cache, then the error goes away momentarily, but quickly comes back on the next run.

For example:

1. I'm on branch_a, I run the mypy invocation above, and get that error.
2. rm -rf .local/mypy_cache
3. Run mypy again, it passes without error (Success: no issues found in N source files).
4. Switch to branch_b, run mypy again – errors again.
5. Repeat steps 2 and 3 – no errors.
6. Switch back to branch_a, errors again, etc.

Traceback (file paths user /Users/* modified from original)

Traceback (most recent call last):
  File "/Users/myname/.pyenv/versions/my-venv-path/bin/mypy", line 8, in <module>
    sys.exit(console_entry())
  File "/Users/myname/.pyenv/versions/3.10.11/envs/my-venv-path/lib/python3.10/site-packages/mypy/__main__.py", line 15, in console_entry
    main()
  File "mypy/main.py", line 95, in main
  File "mypy/main.py", line 174, in run_build
  File "mypy/build.py", line 194, in build
  File "mypy/build.py", line 277, in _build
  File "mypy/build.py", line 2930, in dispatch
  File "mypy/build.py", line 3321, in process_graph
  File "mypy/build.py", line 3402, in process_fresh_modules
  File "mypy/build.py", line 2121, in fix_cross_refs
  File "mypy/fixup.py", line 53, in fixup_module
  File "mypy/fixup.py", line 122, in visit_symbol_table
  File "mypy/lookup.py", line 31, in lookup_fully_qualified
AssertionError: Cannot find module for google

Your Environment

• Python 3.10.11
• mypy 1.2.0 (compiled: yes)
• OS: MacOS Ventura 13.5.2

(see mypy parameters used above)

Given the error about google, this is the output of pip freeze | grep google:

apache-airflow-providers-google==10.7.0
google-ads==21.3.0
google-api-core==2.11.1
google-api-python-client==2.97.0
google-auth==2.22.0
google-auth-httplib2==0.1.0
google-auth-oauthlib==1.0.0
google-cloud-aiplatform==1.31.1
google-cloud-appengine-logging==1.3.1
google-cloud-audit-log==0.2.5
google-cloud-automl==2.11.2
google-cloud-batch==0.16.0
google-cloud-bigquery==3.11.4
google-cloud-bigquery-datatransfer==3.12.0
google-cloud-bigquery-storage==2.22.0
google-cloud-bigtable==2.21.0
google-cloud-build==3.20.0
google-cloud-compute==1.14.0
google-cloud-container==2.30.0
google-cloud-core==2.3.3
google-cloud-datacatalog==3.15.0
google-cloud-dataflow-client==0.8.4
google-cloud-dataform==0.5.2
google-cloud-dataplex==1.6.2
google-cloud-dataproc==5.4.3
google-cloud-dataproc-metastore==1.12.0
google-cloud-dlp==3.12.2
google-cloud-kms==2.19.1
google-cloud-language==2.11.0
google-cloud-logging==3.6.0
google-cloud-memcache==1.7.2
google-cloud-monitoring==2.15.1
google-cloud-orchestration-airflow==1.9.1
google-cloud-os-login==2.10.0
google-cloud-pubsub==2.18.3
google-cloud-redis==2.13.1
google-cloud-resource-manager==1.10.3
google-cloud-run==0.9.1
google-cloud-secret-manager==2.16.3
google-cloud-spanner==3.40.1
google-cloud-speech==2.21.0
google-cloud-storage==2.10.0
google-cloud-storage-transfer==1.9.1
google-cloud-tasks==2.14.1
google-cloud-texttospeech==2.14.1
google-cloud-translate==3.12.0
google-cloud-videointelligence==2.11.3
google-cloud-vision==3.4.4
google-cloud-workflows==1.11.0
google-crc32c==1.5.0
google-re2==1.1
google-resumable-media==2.5.0
googleapis-common-protos==1.60.0
grpc-google-iam-v1==0.12.6
pydata-google-auth==1.8.2

Thank you!

[AlexWaygood]

This started occurring recently after we upgraded from 1.0.0 to 1.2.0.

Would you be able to check to see if it reproduces with the latest version of mypy (1.5.1)? We fix bugs in every release of mypy, and don't backport bugfixes, so there's a good chance this has already been fixed

[aman-wisdom]

I am running in the same error. I am on version 1.4.0 (compiled: yes). Did you find a solution for this?

[hauntsaninja]

Note that a top-level --follow-imports=skip is usually a mistake and will result in mypy performing only extremely weak type checking. Do not use it unless you really know what you're doing.

That said, if someone here is able to provide a repro, it will increase the odds of this getting fixed.