OSError: Input needs to be a file. Path: <path>/app.py

[cancan101]

I am running python -m pyt -r compiq_api/ and I get:

Traceback (most recent call last):
  File "/Users/alex/.pyenv/versions/3.7.0/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/Users/alex/.pyenv/versions/3.7.0/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/Users/alex/.pyenv/versions/compiq-api2/lib/python3.7/site-packages/pyt/__main__.py", line 156, in <module>
    main()
  File "/Users/alex/.pyenv/versions/compiq-api2/lib/python3.7/site-packages/pyt/__main__.py", line 106, in main
    allow_local_directory_imports=args.allow_local_imports
  File "/Users/alex/.pyenv/versions/compiq-api2/lib/python3.7/site-packages/pyt/cfg/make_cfg.py", line 42, in make_cfg
    allow_local_directory_imports
  File "/Users/alex/.pyenv/versions/compiq-api2/lib/python3.7/site-packages/pyt/cfg/expr_visitor.py", line 69, in __init__
    self.init_cfg(node)
  File "/Users/alex/.pyenv/versions/compiq-api2/lib/python3.7/site-packages/pyt/cfg/expr_visitor.py", line 76, in init_cfg
    module_statements = self.visit(node)
  File "/Users/alex/.pyenv/versions/3.7.0/lib/python3.7/ast.py", line 262, in visit
    return visitor(node)
  File "/Users/alex/.pyenv/versions/compiq-api2/lib/python3.7/site-packages/pyt/cfg/stmt_visitor.py", line 67, in visit_Module
    return self.stmt_star_handler(node.body)
  File "/Users/alex/.pyenv/versions/compiq-api2/lib/python3.7/site-packages/pyt/cfg/stmt_visitor.py", line 88, in stmt_star_handler
    node = self.visit(stmt)
  File "/Users/alex/.pyenv/versions/3.7.0/lib/python3.7/ast.py", line 262, in visit
    return visitor(node)
  File "/Users/alex/.pyenv/versions/compiq-api2/lib/python3.7/site-packages/pyt/cfg/stmt_visitor.py", line 1042, in visit_ImportFrom
    return self.handle_relative_import(node)
  File "/Users/alex/.pyenv/versions/compiq-api2/lib/python3.7/site-packages/pyt/cfg/stmt_visitor.py", line 998, in handle_relative_import
    skip_init=skip_init
  File "/Users/alex/.pyenv/versions/compiq-api2/lib/python3.7/site-packages/pyt/cfg/stmt_visitor.py", line 931, in from_directory_import
    from_from=True
  File "/Users/alex/.pyenv/versions/compiq-api2/lib/python3.7/site-packages/pyt/cfg/stmt_visitor.py", line 807, in add_module
    self.visit(tree)
  File "/Users/alex/.pyenv/versions/3.7.0/lib/python3.7/ast.py", line 262, in visit
    return visitor(node)
  File "/Users/alex/.pyenv/versions/compiq-api2/lib/python3.7/site-packages/pyt/cfg/stmt_visitor.py", line 67, in visit_Module
    return self.stmt_star_handler(node.body)
  File "/Users/alex/.pyenv/versions/compiq-api2/lib/python3.7/site-packages/pyt/cfg/stmt_visitor.py", line 88, in stmt_star_handler
    node = self.visit(stmt)
  File "/Users/alex/.pyenv/versions/3.7.0/lib/python3.7/ast.py", line 262, in visit
    return visitor(node)
  File "/Users/alex/.pyenv/versions/compiq-api2/lib/python3.7/site-packages/pyt/cfg/stmt_visitor.py", line 1074, in visit_ImportFrom
    from_from=True
  File "/Users/alex/.pyenv/versions/compiq-api2/lib/python3.7/site-packages/pyt/cfg/stmt_visitor.py", line 807, in add_module
    self.visit(tree)
  File "/Users/alex/.pyenv/versions/3.7.0/lib/python3.7/ast.py", line 262, in visit
    return visitor(node)
  File "/Users/alex/.pyenv/versions/compiq-api2/lib/python3.7/site-packages/pyt/cfg/stmt_visitor.py", line 67, in visit_Module
    return self.stmt_star_handler(node.body)
  File "/Users/alex/.pyenv/versions/compiq-api2/lib/python3.7/site-packages/pyt/cfg/stmt_visitor.py", line 88, in stmt_star_handler
    node = self.visit(stmt)
  File "/Users/alex/.pyenv/versions/3.7.0/lib/python3.7/ast.py", line 262, in visit
    return visitor(node)
  File "/Users/alex/.pyenv/versions/compiq-api2/lib/python3.7/site-packages/pyt/cfg/stmt_visitor.py", line 1042, in visit_ImportFrom
    return self.handle_relative_import(node)
  File "/Users/alex/.pyenv/versions/compiq-api2/lib/python3.7/site-packages/pyt/cfg/stmt_visitor.py", line 998, in handle_relative_import
    skip_init=skip_init
  File "/Users/alex/.pyenv/versions/compiq-api2/lib/python3.7/site-packages/pyt/cfg/stmt_visitor.py", line 931, in from_directory_import
    from_from=True
  File "/Users/alex/.pyenv/versions/compiq-api2/lib/python3.7/site-packages/pyt/cfg/stmt_visitor.py", line 803, in add_module
    tree = generate_ast(module_path)
  File "/Users/alex/.pyenv/versions/compiq-api2/lib/python3.7/site-packages/pyt/core/ast_helper.py", line 49, in generate_ast
    raise IOError('Input needs to be a file. Path: ' + path)
OSError: Input needs to be a file. Path: /Users/alex/git/compiq-api2/compiq_api/app.py

my cwd is: /Users/alex/git/compiq-api2

[bcaller]

This will probably be awkward to debug. Anyway...

Does the file it's looking for (app.py) exist? Does one of your files try to import it e.g. from compiq_api import app?

Also you can try adding more logging with the -vvv flag. This might help you narrow down which file it's getting confused about.

[cancan101]

I created a stripped down example of the issue.

# test/__init__.py
from flask import Flask

app = Flask(__name__)

# -----------------------------------------------------------------------------

from . import (  # noqa # isort:skip
    models,
)

# test/models.py
from . import app

yields (cwd: /Users/alex/git/compiq-api2/test):

$ python -m pyt -r test/ -vvv
[DEBUG] __main__: Discovered file: test/models.py
[DEBUG] __main__: Discovered file: test/__init__.py
[INFO] __main__: Processing test/__init__.py
Traceback (most recent call last):
  File "/Users/alex/.pyenv/versions/3.7.0/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/Users/alex/.pyenv/versions/3.7.0/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/Users/alex/.pyenv/versions/compiq-api2/lib/python3.7/site-packages/pyt/__main__.py", line 156, in <module>
    main()
  File "/Users/alex/.pyenv/versions/compiq-api2/lib/python3.7/site-packages/pyt/__main__.py", line 106, in main
    allow_local_directory_imports=args.allow_local_imports
  File "/Users/alex/.pyenv/versions/compiq-api2/lib/python3.7/site-packages/pyt/cfg/make_cfg.py", line 42, in make_cfg
    allow_local_directory_imports
  File "/Users/alex/.pyenv/versions/compiq-api2/lib/python3.7/site-packages/pyt/cfg/expr_visitor.py", line 69, in __init__
    self.init_cfg(node)
  File "/Users/alex/.pyenv/versions/compiq-api2/lib/python3.7/site-packages/pyt/cfg/expr_visitor.py", line 76, in init_cfg
    module_statements = self.visit(node)
  File "/Users/alex/.pyenv/versions/3.7.0/lib/python3.7/ast.py", line 262, in visit
    return visitor(node)
  File "/Users/alex/.pyenv/versions/compiq-api2/lib/python3.7/site-packages/pyt/cfg/stmt_visitor.py", line 67, in visit_Module
    return self.stmt_star_handler(node.body)
  File "/Users/alex/.pyenv/versions/compiq-api2/lib/python3.7/site-packages/pyt/cfg/stmt_visitor.py", line 88, in stmt_star_handler
    node = self.visit(stmt)
  File "/Users/alex/.pyenv/versions/3.7.0/lib/python3.7/ast.py", line 262, in visit
    return visitor(node)
  File "/Users/alex/.pyenv/versions/compiq-api2/lib/python3.7/site-packages/pyt/cfg/stmt_visitor.py", line 1042, in visit_ImportFrom
    return self.handle_relative_import(node)
  File "/Users/alex/.pyenv/versions/compiq-api2/lib/python3.7/site-packages/pyt/cfg/stmt_visitor.py", line 998, in handle_relative_import
    skip_init=skip_init
  File "/Users/alex/.pyenv/versions/compiq-api2/lib/python3.7/site-packages/pyt/cfg/stmt_visitor.py", line 931, in from_directory_import
    from_from=True
  File "/Users/alex/.pyenv/versions/compiq-api2/lib/python3.7/site-packages/pyt/cfg/stmt_visitor.py", line 807, in add_module
    self.visit(tree)
  File "/Users/alex/.pyenv/versions/3.7.0/lib/python3.7/ast.py", line 262, in visit
    return visitor(node)
  File "/Users/alex/.pyenv/versions/compiq-api2/lib/python3.7/site-packages/pyt/cfg/stmt_visitor.py", line 67, in visit_Module
    return self.stmt_star_handler(node.body)
  File "/Users/alex/.pyenv/versions/compiq-api2/lib/python3.7/site-packages/pyt/cfg/stmt_visitor.py", line 88, in stmt_star_handler
    node = self.visit(stmt)
  File "/Users/alex/.pyenv/versions/3.7.0/lib/python3.7/ast.py", line 262, in visit
    return visitor(node)
  File "/Users/alex/.pyenv/versions/compiq-api2/lib/python3.7/site-packages/pyt/cfg/stmt_visitor.py", line 1042, in visit_ImportFrom
    return self.handle_relative_import(node)
  File "/Users/alex/.pyenv/versions/compiq-api2/lib/python3.7/site-packages/pyt/cfg/stmt_visitor.py", line 998, in handle_relative_import
    skip_init=skip_init
  File "/Users/alex/.pyenv/versions/compiq-api2/lib/python3.7/site-packages/pyt/cfg/stmt_visitor.py", line 931, in from_directory_import
    from_from=True
  File "/Users/alex/.pyenv/versions/compiq-api2/lib/python3.7/site-packages/pyt/cfg/stmt_visitor.py", line 803, in add_module
    tree = generate_ast(module_path)
  File "/Users/alex/.pyenv/versions/compiq-api2/lib/python3.7/site-packages/pyt/core/ast_helper.py", line 49, in generate_ast
    raise IOError('Input needs to be a file. Path: ' + path)
OSError: Input needs to be a file. Path: /Users/alex/git/compiq-api2/test/test/app.py

[bcaller]

Thanks for the example.

Issues I see immediately (there may be more):

1. pyt is handling from . import app as import .app rather than first trying from .__init__ import app. https://github.com/python-security/pyt/blob/master/pyt/cfg/stmt_visitor.py#L986-L990

            # e.g. from . import X
            else:
                name_with_dir = no_file
                # We do not want to analyse the init file of the current directory
                skip_init = True

2. When that is solved, there will be a RecursionError due to the files' mutual dependency which pyt doesn't cope nicely with. You managed to avoid the circular dependency when running the code by sandwiching app between import lines.