LibCST
LibCST copied to clipboard
Instagram
Test errors on openSUSE
I have a few errors building on openSUSE Tumbleweed (py38 currently) Packaging in progress at https://build.opensuse.org/package/show/home:jayvdb:py-new/python-libcst
There are tests in the sdist, but that failed due to missing test data.
The first of the remaining ones below is easy, related to using sys.executable instead of literal python. The others are more of a mystery at first glance.
[ 118s] ======================================================================
[ 118s] ERROR: test_codemod_formatter_error_input (libcst.codemod.tests.test_codemod_cli.TestCodemodCLI)
[ 118s] ----------------------------------------------------------------------
[ 118s] Traceback (most recent call last):
[ 118s] File "/home/abuild/rpmbuild/BUILD/LibCST-0.3.7/libcst/codemod/tests/test_codemod_cli.py", line 16, in test_codemod_formatter_error_input
[ 118s] rlt = subprocess.run(
[ 118s] File "/usr/lib64/python3.8/subprocess.py", line 489, in run
[ 118s] with Popen(*popenargs, **kwargs) as process:
[ 118s] File "/usr/lib64/python3.8/subprocess.py", line 854, in __init__
[ 118s] self._execute_child(args, executable, preexec_fn, close_fds,
[ 118s] File "/usr/lib64/python3.8/subprocess.py", line 1702, in _execute_child
[ 118s] raise child_exception_type(errno_num, err_msg, err_filename)
[ 118s] FileNotFoundError: [Errno 2] No such file or directory: 'python'
[ 118s]
[ 118s] ======================================================================
[ 118s] FAIL: test_codegen_clean_matcher_classes (libcst.codegen.tests.test_codegen_clean.TestCodegenClean)
[ 118s] Verifies that codegen of matcher classes would not result in a
[ 118s] ----------------------------------------------------------------------
[ 118s] Traceback (most recent call last):
[ 118s] File "/home/abuild/rpmbuild/BUILD/LibCST-0.3.7/libcst/codegen/tests/test_codegen_clean.py", line 81, in test_codegen_clean_matcher_classes
[ 118s] self.assertTrue(
[ 118s] AssertionError: False is not true : libcst.matchers.__init__ needs new codegen!
[ 118s]
[ 118s] ======================================================================
[ 118s] FAIL: test_codegen_clean_return_types (libcst.codegen.tests.test_codegen_clean.TestCodegenClean)
[ 118s] Verifies that codegen of return types would not result in a
[ 118s] ----------------------------------------------------------------------
[ 118s] Traceback (most recent call last):
[ 118s] File "/home/abuild/rpmbuild/BUILD/LibCST-0.3.7/libcst/codegen/tests/test_codegen_clean.py", line 116, in test_codegen_clean_return_types
[ 118s] self.assertTrue(
[ 118s] AssertionError: False is not true : libcst.matchers._return_types needs new codegen!
[ 118s]
[ 118s] ======================================================================
[ 118s] FAIL: test_codegen_clean_visitor_functions (libcst.codegen.tests.test_codegen_clean.TestCodegenClean)
[ 118s] Verifies that codegen of visitor functions would not result in a
[ 118s] ----------------------------------------------------------------------
[ 118s] Traceback (most recent call last):
[ 118s] File "/home/abuild/rpmbuild/BUILD/LibCST-0.3.7/libcst/codegen/tests/test_codegen_clean.py", line 47, in test_codegen_clean_visitor_functions
[ 118s] self.assertTrue(
[ 118s] AssertionError: False is not true : libcst._typed_visitor needs new codegen!
[ 118s]
[ 118s] ----------------------------------------------------------------------
[ 118s] Ran 1726 tests in 89.909s
As the error message suggested, it detected the generated code changes from codegen and you can run tox -e codegen to generate the new changes.
https://github.com/Instagram/LibCST/blob/master/tox.ini#L78
@jimmylai , I used the tarball created by this project. Why does it have this problem?
The issue is likely only show up in your build environment. To know more about the issue, you can run tox -e codegen to find out what's changed.
@jimmylai here is one of the underlying errors I am seeing
isort: error: unrecognized arguments: -y
Seems they had a arg shakeup over there at https://github.com/timothycrosley/isort/blob/master/CHANGELOG.md#500-penny---july-4-2020
To run the tests, it requires installing the dependencies (tox/isort/...). You can also run the codegen commands manually.
https://github.com/Instagram/LibCST/blob/master/tox.ini#L78-L85
The cogen runs isort -y ... to format the code and LibCST didn't use the latest version of isort.
To ensure the tests can run normally, use tox to build the environment and run tests is most convenient. In that case, you should build it from the source code, not the sdist.
If it makes sense to include the tox.ini in sdist, feel free to propose a change to include it.
tox.ini is not needed. Even with generating the data manually, the tests fail, because of missing pyre-check (no package available for openSUSE, and packaging that does not look straightforward). The error message could be more precise, though.
[ 3s] + PYTHONPATH=/home/abuild/rpmbuild/BUILD/libcst-0.3.8
[ 3s] + python3 -m libcst.codegen.generate visitors
[ 14s] Successfully generated a new /home/abuild/rpmbuild/BUILD/libcst-0.3.8/libcst/_typed_visitor.py file.
[ 15s] + python3 -m libcst.codegen.generate return_types
[ 16s] Successfully generated a new /home/abuild/rpmbuild/BUILD/libcst-0.3.8/libcst/matchers/_return_types.py file.
[ 16s] + python3 -m libcst.codegen.generate matchers
[ 40s] Successfully generated a new /home/abuild/rpmbuild/BUILD/libcst-0.3.8/libcst/matchers/__init__.py file.
...
[ 40s] copying libcst/_typed_visitor.py -> build/lib/libcst
...
[ 40s] copying libcst/matchers/_return_types.py -> build/lib/libcst/matchers
...
[ 41s] copying build/lib/libcst/matchers/_return_types.py -> /home/abuild/rpmbuild/BUILDROOT/python-libcst-0.3.8-0.x86_64/usr/lib/python3.8/site-packages/libcst/matchers
...
[ 100s] ======================================================================
[ 100s] ERROR: libcst.metadata.tests.test_full_repo_manager (unittest.loader._FailedTest)
[ 100s] ----------------------------------------------------------------------
[ 100s] ImportError: Failed to import test module: libcst.metadata.tests.test_full_repo_manager
[ 100s] Traceback (most recent call last):
[ 100s] File "/usr/lib64/python3.8/unittest/loader.py", line 436, in _find_test_path
[ 100s] module = self._get_module_from_name(name)
[ 100s] File "/usr/lib64/python3.8/unittest/loader.py", line 377, in _get_module_from_name
[ 100s] __import__(name)
[ 100s] File "/home/abuild/rpmbuild/BUILD/libcst-0.3.8/libcst/metadata/tests/test_full_repo_manager.py", line 11, in <module>
[ 100s] from libcst.metadata.tests.test_type_inference_provider import _test_simple_class_helper
[ 100s] File "/home/abuild/rpmbuild/BUILD/libcst-0.3.8/libcst/metadata/tests/test_type_inference_provider.py", line 14, in <module>
[ 100s] from libcst.tests.test_pyre_integration import TEST_SUITE_PATH
[ 100s] File "/home/abuild/rpmbuild/BUILD/libcst-0.3.8/libcst/tests/test_pyre_integration.py", line 95, in <module>
[ 100s] class PyreIntegrationTest(UnitTest):
[ 100s] File "/home/abuild/rpmbuild/BUILD/libcst-0.3.8/libcst/testing/utils.py", line 170, in __new__
[ 100s] populate_data_provider_tests(dct)
[ 100s] File "/home/abuild/rpmbuild/BUILD/libcst-0.3.8/libcst/testing/utils.py", line 96, in populate_data_provider_tests
[ 100s] raise ValueError(
[ 100s] ValueError: No data_provider tests were created for test_type_availability! Please double check your data.
[ 100s]
[ 100s]
[ 100s] ======================================================================
[ 100s] ERROR: libcst.metadata.tests.test_type_inference_provider (unittest.loader._FailedTest)
[ 100s] ----------------------------------------------------------------------
[ 100s] ImportError: Failed to import test module: libcst.metadata.tests.test_type_inference_provider
[ 100s] Traceback (most recent call last):
[ 100s] File "/usr/lib64/python3.8/unittest/loader.py", line 436, in _find_test_path
[ 100s] module = self._get_module_from_name(name)
[ 100s] File "/usr/lib64/python3.8/unittest/loader.py", line 377, in _get_module_from_name
[ 100s] __import__(name)
[ 100s] File "/home/abuild/rpmbuild/BUILD/libcst-0.3.8/libcst/metadata/tests/test_type_inference_provider.py", line 14, in <module>
[ 100s] from libcst.tests.test_pyre_integration import TEST_SUITE_PATH
[ 100s] File "/home/abuild/rpmbuild/BUILD/libcst-0.3.8/libcst/tests/test_pyre_integration.py", line 95, in <module>
[ 100s] class PyreIntegrationTest(UnitTest):
[ 100s] File "/home/abuild/rpmbuild/BUILD/libcst-0.3.8/libcst/testing/utils.py", line 170, in __new__
[ 100s] populate_data_provider_tests(dct)
[ 100s] File "/home/abuild/rpmbuild/BUILD/libcst-0.3.8/libcst/testing/utils.py", line 96, in populate_data_provider_tests
[ 100s] raise ValueError(
[ 100s] ValueError: No data_provider tests were created for test_type_availability! Please double check your data.
[ 100s]
[ 100s]
[ 100s] ======================================================================
[ 100s] ERROR: libcst.tests.test_pyre_integration (unittest.loader._FailedTest)
[ 100s] ----------------------------------------------------------------------
[ 100s] ImportError: Failed to import test module: libcst.tests.test_pyre_integration
[ 100s] Traceback (most recent call last):
[ 100s] File "/usr/lib64/python3.8/unittest/loader.py", line 436, in _find_test_path
[ 100s] module = self._get_module_from_name(name)
[ 100s] File "/usr/lib64/python3.8/unittest/loader.py", line 377, in _get_module_from_name
[ 100s] __import__(name)
[ 100s] File "/home/abuild/rpmbuild/BUILD/libcst-0.3.8/libcst/tests/test_pyre_integration.py", line 95, in <module>
[ 100s] class PyreIntegrationTest(UnitTest):
[ 100s] File "/home/abuild/rpmbuild/BUILD/libcst-0.3.8/libcst/testing/utils.py", line 170, in __new__
[ 100s] populate_data_provider_tests(dct)
[ 100s] File "/home/abuild/rpmbuild/BUILD/libcst-0.3.8/libcst/testing/utils.py", line 96, in populate_data_provider_tests
[ 100s] raise ValueError(
[ 100s] ValueError: No data_provider tests were created for test_type_availability! Please double check your data.
[ 100s]
[ 100s]
[ 100s] ======================================================================
[ 100s] FAIL: test_codemod_formatter_error_input (libcst.codemod.tests.test_codemod_cli.TestCodemodCLI)
[ 100s] ----------------------------------------------------------------------
[ 100s] Traceback (most recent call last):
[ 100s] File "/home/abuild/rpmbuild/BUILD/libcst-0.3.8/libcst/codemod/tests/test_codemod_cli.py", line 34, in test_codemod_formatter_error_input
[ 100s] self.assertIn(
[ 100s] AssertionError: 'error: cannot format -: Cannot parse: 13:10: async with AsyncExitStack() as stack:' not found in 'Finished codemodding 0 files!\n - Transformed 0 files successfully.\n - Skipped 0 files.\n - Failed to codemod 0 files.\n - 0 warnings were generated.\n'
[ 100s]
[ 100s] ----------------------------------------------------------------------
[ 100s] Ran 1733 tests in 52.760s
[ 100s]
[ 100s] FAILED (failures=1, errors=3, skipped=3, expected failures=2)
[ 100s] ======================================================================
[ 100s] ERROR: libcst.metadata.tests.test_full_repo_manager (unittest.loader._FailedTest)
[ 100s] ----------------------------------------------------------------------
[ 100s] ImportError: Failed to import test module: libcst.metadata.tests.test_full_repo_manager
[ 100s] Traceback (most recent call last):
[ 100s] File "/usr/lib64/python3.8/unittest/loader.py", line 436, in _find_test_path
[ 100s] module = self._get_module_from_name(name)
[ 100s] File "/usr/lib64/python3.8/unittest/loader.py", line 377, in _get_module_from_name
[ 100s] __import__(name)
[ 100s] File "/home/abuild/rpmbuild/BUILD/libcst-0.3.8/libcst/metadata/tests/test_full_repo_manager.py", line 11, in <module>
[ 100s] from libcst.metadata.tests.test_type_inference_provider import _test_simple_class_helper
[ 100s] File "/home/abuild/rpmbuild/BUILD/libcst-0.3.8/libcst/metadata/tests/test_type_inference_provider.py", line 14, in <module>
[ 100s] from libcst.tests.test_pyre_integration import TEST_SUITE_PATH
[ 100s] File "/home/abuild/rpmbuild/BUILD/libcst-0.3.8/libcst/tests/test_pyre_integration.py", line 95, in <module>
[ 100s] class PyreIntegrationTest(UnitTest):
[ 100s] File "/home/abuild/rpmbuild/BUILD/libcst-0.3.8/libcst/testing/utils.py", line 170, in __new__
[ 100s] populate_data_provider_tests(dct)
[ 100s] File "/home/abuild/rpmbuild/BUILD/libcst-0.3.8/libcst/testing/utils.py", line 96, in populate_data_provider_tests
[ 100s] raise ValueError(
[ 100s] ValueError: No data_provider tests were created for test_type_availability! Please double check your data.
[ 100s]
[ 100s]
[ 100s] ======================================================================
[ 100s] ERROR: libcst.metadata.tests.test_type_inference_provider (unittest.loader._FailedTest)
[ 100s] ----------------------------------------------------------------------
[ 100s] ImportError: Failed to import test module: libcst.metadata.tests.test_type_inference_provider
[ 100s] Traceback (most recent call last):
[ 100s] File "/usr/lib64/python3.8/unittest/loader.py", line 436, in _find_test_path
[ 100s] module = self._get_module_from_name(name)
[ 100s] File "/usr/lib64/python3.8/unittest/loader.py", line 377, in _get_module_from_name
[ 100s] __import__(name)
[ 100s] File "/home/abuild/rpmbuild/BUILD/libcst-0.3.8/libcst/metadata/tests/test_type_inference_provider.py", line 14, in <module>
[ 100s] from libcst.tests.test_pyre_integration import TEST_SUITE_PATH
[ 100s] File "/home/abuild/rpmbuild/BUILD/libcst-0.3.8/libcst/tests/test_pyre_integration.py", line 95, in <module>
[ 100s] class PyreIntegrationTest(UnitTest):
[ 100s] File "/home/abuild/rpmbuild/BUILD/libcst-0.3.8/libcst/testing/utils.py", line 170, in __new__
[ 100s] populate_data_provider_tests(dct)
[ 100s] File "/home/abuild/rpmbuild/BUILD/libcst-0.3.8/libcst/testing/utils.py", line 96, in populate_data_provider_tests
[ 100s] raise ValueError(
[ 100s] ValueError: No data_provider tests were created for test_type_availability! Please double check your data.
[ 100s]
[ 100s]
[ 100s] ======================================================================
[ 100s] ERROR: libcst.tests.test_pyre_integration (unittest.loader._FailedTest)
[ 100s] ----------------------------------------------------------------------
[ 100s] ImportError: Failed to import test module: libcst.tests.test_pyre_integration
[ 100s] Traceback (most recent call last):
[ 100s] File "/usr/lib64/python3.8/unittest/loader.py", line 436, in _find_test_path
[ 100s] module = self._get_module_from_name(name)
[ 100s] File "/usr/lib64/python3.8/unittest/loader.py", line 377, in _get_module_from_name
[ 100s] __import__(name)
[ 100s] File "/home/abuild/rpmbuild/BUILD/libcst-0.3.8/libcst/tests/test_pyre_integration.py", line 95, in <module>
[ 100s] class PyreIntegrationTest(UnitTest):
[ 100s] File "/home/abuild/rpmbuild/BUILD/libcst-0.3.8/libcst/testing/utils.py", line 170, in __new__
[ 100s] populate_data_provider_tests(dct)
[ 100s] File "/home/abuild/rpmbuild/BUILD/libcst-0.3.8/libcst/testing/utils.py", line 96, in populate_data_provider_tests
[ 100s] raise ValueError(
[ 100s] ValueError: No data_provider tests were created for test_type_availability! Please double check your data.
[ 100s]
[ 100s]
[ 100s] ======================================================================
[ 100s] FAIL: test_codemod_formatter_error_input (libcst.codemod.tests.test_codemod_cli.TestCodemodCLI)
[ 100s] ----------------------------------------------------------------------
[ 100s] Traceback (most recent call last):
[ 100s] File "/home/abuild/rpmbuild/BUILD/libcst-0.3.8/libcst/codemod/tests/test_codemod_cli.py", line 34, in test_codemod_formatter_error_input
[ 100s] self.assertIn(
[ 100s] AssertionError: 'error: cannot format -: Cannot parse: 13:10: async with AsyncExitStack() as stack:' not found in 'Finished codemodding 0 files!\n - Transformed 0 files successfully.\n - Skipped 0 files.\n - Failed to codemod 0 files.\n - 0 warnings were generated.\n'
[ 100s]
[ 100s] ----------------------------------------------------------------------
[ 100s] Ran 1733 tests in 52.760s
[ 100s]
[ 100s] FAILED (failures=1, errors=3, skipped=3, expected failures=2)
Test data files are now included and will be available in the next release.
0.3.10 still needs a manual codegen. Otherwise these tests fail:
[ 64s] FAIL: test_codegen_clean_return_types (libcst.codegen.tests.test_codegen_clean.TestCodegenClean)
[ 64s] Verifies that codegen of return types would not result in a
[ 64s] ----------------------------------------------------------------------
[ 64s] Traceback (most recent call last):
[ 64s] File "/home/abuild/rpmbuild/BUILD/libcst-0.3.10/libcst/codegen/tests/test_codegen_clean.py", line 116, in test_codegen_clean_return_types
[ 64s] self.assertTrue(
[ 64s] AssertionError: False is not true : libcst.matchers._return_types needs new codegen!
[ 64s]
[ 64s] ======================================================================
[ 64s] FAIL: test_codegen_clean_visitor_functions (libcst.codegen.tests.test_codegen_clean.TestCodegenClean)
[ 64s] Verifies that codegen of visitor functions would not result in a
[ 64s] ----------------------------------------------------------------------
[ 64s] Traceback (most recent call last):
[ 64s] File "/home/abuild/rpmbuild/BUILD/libcst-0.3.10/libcst/codegen/tests/test_codegen_clean.py", line 47, in test_codegen_clean_visitor_functions
[ 64s] self.assertTrue(
[ 64s] AssertionError: False is not true : libcst._typed_visitor needs new codegen!
The missing codegen is fixed in the most recent release 0.3.11, but the test/pyre directory is missing from the PyPI archive.
With 0.3.17, the shipped generated code is not identical to what is generated (apart from #467):
[ 74s] ======================================================================
[ 74s] FAIL: test_codegen_clean_return_types (libcst.codegen.tests.test_codegen_clean.TestCodegenClean)
[ 74s] ----------------------------------------------------------------------
[ 74s] Traceback (most recent call last):
[ 74s] File "/home/abuild/rpmbuild/BUILD/libcst-0.3.17/libcst/codegen/tests/test_codegen_clean.py", line 119, in test_codegen_clean_return_types
[ 74s] old_code == new_code, "libcst.matchers._return_types needs new codegen!"
[ 74s] AssertionError: False is not true : libcst.matchers._return_types needs new codegen!
[ 74s]
[ 74s] ======================================================================
[ 74s] FAIL: test_codegen_clean_visitor_functions (libcst.codegen.tests.test_codegen_clean.TestCodegenClean)
[ 74s] ----------------------------------------------------------------------
[ 74s] Traceback (most recent call last):
[ 74s] File "/home/abuild/rpmbuild/BUILD/libcst-0.3.17/libcst/codegen/tests/test_codegen_clean.py", line 50, in test_codegen_clean_visitor_functions
[ 74s] old_code == new_code, "libcst._typed_visitor needs new codegen!"
[ 74s] AssertionError: False is not true : libcst._typed_visitor needs new codegen!
[ 74s]
abuild@skylab:~/rpmbuild/BUILD/libcst-0.3.17> cp libcst/matchers/_return_types.py{,_}
abuild@skylab:~/rpmbuild/BUILD/libcst-0.3.17> python3.8 -m libcst.codegen.generate return_types
Successfully generated a new /home/abuild/rpmbuild/BUILD/libcst-0.3.17/libcst/matchers/_return_types.py file.
abuild@skylab:~/rpmbuild/BUILD/libcst-0.3.17> diff -u libcst/matchers/_return_types.py{_,}
--- libcst/matchers/_return_types.py_ 2021-03-21 17:08:31.546609249 +0000
+++ libcst/matchers/_return_types.py 2021-03-21 17:08:52.591035090 +0000
@@ -5,7 +5,8 @@
# This file was generated by libcst.codegen.gen_type_mapping
-from typing import Dict as TypingDict, Type, Union
+from typing import Dict as TypingDict
+from typing import Type, Union
from libcst._maybe_sentinel import MaybeSentinel
from libcst._nodes.base import CSTNode
@@ -176,7 +177,6 @@
)
from libcst._removal_sentinel import RemovalSentinel
-
TYPED_FUNCTION_RETURN_MAPPING: TypingDict[Type[CSTNode], object] = {
Add: BaseBinaryOp,
AddAssign: BaseAugOp,
abuild@skylab:~/rpmbuild/BUILD/libcst-0.3.17> cp libcst/_typed_visitor.py{,_}
abuild@skylab:~/rpmbuild/BUILD/libcst-0.3.17> python3.8 -m libcst.codegen.generate visitors
Successfully generated a new /home/abuild/rpmbuild/BUILD/libcst-0.3.17/libcst/_typed_visitor.py file.
abuild@skylab:~/rpmbuild/BUILD/libcst-0.3.17> diff -u libcst/_typed_visitor.py{_,}
--- libcst/_typed_visitor.py_ 2021-03-21 17:02:49.683691532 +0000
+++ libcst/_typed_visitor.py 2021-03-21 17:05:33.691010299 +0000
@@ -11,11 +11,10 @@
from libcst._removal_sentinel import RemovalSentinel
from libcst._typed_visitor_base import mark_no_op
-
if TYPE_CHECKING:
- from libcst._nodes.expression import ( # noqa: F401
+ from libcst._nodes.expression import (
Annotation,
- Arg,
+ Arg, # noqa: F401
Asynchronous,
Attribute,
Await,
@@ -75,10 +74,10 @@
Yield,
)
from libcst._nodes.module import Module # noqa: F401
- from libcst._nodes.op import ( # noqa: F401
+ from libcst._nodes.op import (
Add,
AddAssign,
- And,
+ And, # noqa: F401
AssignEqual,
BaseAugOp,
BaseBinaryOp,
@@ -130,9 +129,9 @@
Subtract,
SubtractAssign,
)
- from libcst._nodes.statement import ( # noqa: F401
+ from libcst._nodes.statement import (
AnnAssign,
- AsName,
+ AsName, # noqa: F401
Assert,
Assign,
AssignTarget,