dash icon indicating copy to clipboard operation
dash copied to clipboard

Version 3 adds breaking type annotations that do not comply with mypy

Open gothicVI opened this issue 10 months ago • 33 comments

With the release of dash 3.0 our CI/CD fails for stuff that used to work. Here's a minimum working example:

from dash import Dash, dcc, html
from dash.dependencies import Input, Output
from typing import Callable

app = Dash(__name__)

def create_layout() -> html.Div:
    return html.Div([
        dcc.Input(id='input-text', type='text', value='', placeholder='Enter text'),
        html.Div(id='output-text')
    ])  

app.layout = create_layout

@app.callback(Output('output-text', 'children'), Input('input-text', 'value'))
def update_output(value: str) -> str:
    return f'You entered: {value}'

if __name__ == '__main__':
    app.run(debug=True, port=9000)

running mypy on this file results in:

$ mypy t.py --strict
t.py:1: error: Skipping analyzing "dash": module is installed, but missing library stubs or py.typed marker  [import-untyped]
t.py:2: error: Skipping analyzing "dash.dependencies": module is installed, but missing library stubs or py.typed marker  [import-untyped]
t.py:2: note: See https://mypy.readthedocs.io/en/stable/running_mypy.html#missing-imports
t.py:15: error: Untyped decorator makes function "update_output" untyped  [misc]
Found 3 errors in 1 file (checked 1 source file)

which we used to solve by adding # type: ignore[misc] to every callback call and by adding

[[tool.mypy.overrides]]
module = [
  "dash.*",
  "dash_ag_grid",
  "dash_bootstrap_components.*",
  "plotly.*",
]
ignore_missing_imports = true

to our pyproject.toml.

However, when updating to version 3, we get:

$ mypy --strict t.py 
t.py:8: error: Returning Any from function declared to return "Div"  [no-any-return]
t.py:13: error: Property "layout" defined in "Dash" is read-only  [misc]
t.py:15: error: Call to untyped function "Input" in typed context  [no-untyped-call]
t.py:15: error: Call to untyped function "Output" in typed context  [no-untyped-call]
t.py:15: error: Call to untyped function "callback" in typed context  [no-untyped-call]
t.py:15: note: Error code "no-untyped-call" not covered by "type: ignore" comment
Found 5 errors in 1 file (checked 1 source file)

without changing anything else.

This can't be intended behavior, right? How to fix that?

community post: https://community.plotly.com/t/dash-3-0-fails-mypy/91308

gothicVI avatar Mar 18 '25 10:03 gothicVI

I only had trouble with mypy, the typing generation tests were originally designed with mypy but at some point it updated and couldn't find types anymore. There is a py.typed file in dash package.

I suggest switching over to pyright, that is what is used in vscode and 10x better imo.

T4rk1n avatar Mar 18 '25 12:03 T4rk1n

I'll give that a look but we can not simply dump mypy from our code base. This needs to be fixed not on a linter level because there is something wrong in the code.

E.g. looking at https://github.com/plotly/dash/blob/dev/dash/dependencies.py#L29 and following it is clear, that there are no type annotations for Input, Output, State, and their superclass DashDependency. So there's no point in having py.typed in the code base if the code is not typed.

gothicVI avatar Mar 18 '25 19:03 gothicVI

There are several issues with the claimed "typedness" for dash. In a fresh virtual environment I get:

$ mypy .
dash-core-components is not a valid Python package name
$ cd dash
mypy: "types.py" shadows library module "types"
note: A user-defined top-level module with name "types" is not supported
$ cd ..
$ pyright 
# see https://gist.github.com/gothicVI/34eb60a2a2f61b02d6a2c50027355a11 for very long output
$ cd dash
$ pyright 
# see https://gist.github.com/gothicVI/b2f30b6cbfbc648909eb3f82c4560c2c for lengthy output

which tells me that the packages violates python conventions and is not typed. @T4rk1n how does that work for you?

gothicVI avatar Mar 20 '25 05:03 gothicVI

Hi @gothicVI - thanks for filing this issue. We're aware that there's still work to do on Dash typing, and given resources at our end we have to take it in stages. If you would like to post a PR with a handful of the changes you would like to see, we can review that right away, and it will help us figure out how much effort it will be to solve the whole problem. Thanks - @gvwilson

gvwilson avatar Mar 20 '25 15:03 gvwilson

Hi @gvwilson thanks for taking a look. To be completely honest I do not know how to start contributing in that regard because I do not fully understand the internal workings yet. I might give it a look but can't prommis any PR.

I think the most needed changes would be to figure out what exactly breaks mypy when changing to version 3. Don't get me wrong, mypy did complain earlier due to there not being any annotations or stub files so we commented that out as mentioned above. But the release of version 3 did break that which it shouldn't. If needed revert the changes for the moment!

After that I'd suggest sticking to the general python conventions like no - in module names and not shadowing of built in libraries like types. I'd also suggest switching from setup.py to pyproject.toml and to include mypy and potentially other linters or analyzers into the regular process. Set up pre-commit and a CI pipeline for, e.g., ruff, mypy, and pyright and have every pushed commit be checked.

Please do not get me wrong. I love this project and this should not be understood as a personal attack against anyone who contributed typing code! Yet I think it is quite dangerous to push a major version upgrade without thorough testing - especially in such a large project that so many other projects rely on.

gothicVI avatar Mar 20 '25 16:03 gothicVI

HI @gothicVI - thanks very much for your quick feedback. Reverting these changes isn't an option - there are a great many features that our users have wanted for a long time - so I'd like to focus on ways to move forward. If you can start a PR to convert from setup.py to pyproject.toml (which ought to be more straightforward than adding typing) that would be a great help. Thanks - @gvwilson

gvwilson avatar Mar 20 '25 16:03 gvwilson

@gvwilson I didn't mean to revert the release of version 3 but just the commits that break the typing system.

Either way, can you or anyone else confirm that the new version breaks mypy at all?

Independently I've looked into converting the setup.py approach to a pyproject.toml approach. However, I can't even build the current project state due to dash/labextension/dist/dash-jupyterlab.tgz being required but missing. What's happening here? How does the build process even work?

My attempt at conversion can be found here: https://github.com/gothicVI/dash/commit/ed59db7ff1fc2cefe72e51b4f82ceed7367294fb I'd start a PR once I'm sure the project can be built at all.

gothicVI avatar Mar 25 '25 21:03 gothicVI

@gvwilson any news?

gothicVI avatar Mar 31 '25 20:03 gothicVI

Nothing yet, but I'll be sure to post when there is.

gvwilson avatar Mar 31 '25 21:03 gvwilson

@gothicVI can you please tell us more about the build issue you're having? when I go through the steps in CONTRIBUTING.md on MacOS everything seems to build correctly (though there are currently some failing tests - I'm looking into those). What platform are you on and are any steps breaking before you hit the jupyterlab issue? Separately, can you please create a PR with your pyproject.toml file that doesn't delete the existing requirements/* files? We can work on getting that in place in parallel with typing changes. thanks - @gvwilson

gvwilson avatar Apr 01 '25 13:04 gvwilson

@gvwilson I honestly can no longer reproduce the issue I had but it would fail due to dash/labextension/dist/dash-jupyterlab.tgz not existing. Following the same steps though I'm running into issues because orjson does not build with python 3.13 even though that should be supported as per the project. The problem here might be that dash does not support 3.13 itself.

I'll try again with 3.11.

Irrespectively, can you reproduce the initial issue though?

gothicVI avatar Apr 01 '25 16:04 gothicVI

Investigated the type checkers, I found that mypy doesn't even resolve the html/dcc types that were added while it works fine with pyright/vscode/pycharm.

This error results from that:

t.py:8: error: Returning Any from function declared to return "Div"  [no-any-return]

It happens that mypy can't resolve the @_explicitize_args we put in front of the __init__ of the components and ends up with Any instead of the component.

Adding type vars can resolve the return type, but break the init types. Can probably refactor the decorator behavior inside the __new__ of ComponentMeta.

error: Property "layout" defined in "Dash" is read-only  [misc]

Has been fixed in 3.0.2 by community PR: #3249

t.py:15: error: Call to untyped function "Input" in typed context  [no-untyped-call]
t.py:15: error: Call to untyped function "Output" in typed context  [no-untyped-call]

The dash dependencies will be typed in next dash release by #3259

t.py:15: error: Call to untyped function "callback" in typed context  [no-untyped-call]

This can be fixed/silenced with def callback(...) -> Callable[..., Any], also added in #3259, the global @callback contains more types for the arguments that should be added in the app one. Maybe a type var can be used, but seems to work well when the function is typed.

T4rk1n avatar Apr 04 '25 17:04 T4rk1n

@T4rk1n that looks great. Any change to add mypy to the standard CI suite as well? I use pyright as well but mypy is the de-facto standard as it's the official type checker.

gothicVI avatar Apr 04 '25 19:04 gothicVI

@T4rk1n that looks great. Any change to add mypy to the standard CI suite as well? I use pyright as well but mypy is the de-facto standard as it's the official type checker.

I'll probably add a basic test for mypy to not throw error with a basic app and make sure it resolve the types of the components. But for the rest of the codebase it should run pyright as mypy is very slow in comparison for the same kind of linting.

#3259 Also fix the component types, that may reveal actual typing errors in layout when using mypy.

T4rk1n avatar Apr 04 '25 20:04 T4rk1n

@gothicVI please have a look at #3254 and let me know what you think?

gvwilson avatar Apr 08 '25 14:04 gvwilson

Some things seem to be fixed but other not so much:

/tmp $ git clone [email protected]:plotly/dash.git
/tmp $ python3 -m venv venv
/tmp $ source venv/bin/activate
(venv) /tmp $ pip install mypy
(venv) /tmp $ cd dash
(venv) /tmp/dash $ git switch pyright-typing-fixes
(venv) /tmp/dash $ pip install -e .
(venv) /tmp/dash $ pip list
Package            Version   Editable project location
------------------ --------- -------------------------
blinker            1.9.0
certifi            2025.1.31
charset-normalizer 3.4.1
click              8.1.8
dash               3.0.2     /tmp/dash
Flask              3.0.3
idna               3.10
importlib_metadata 8.6.1
itsdangerous       2.2.0
Jinja2             3.1.6
MarkupSafe         3.0.2
mypy               1.15.0
mypy-extensions    1.0.0
narwhals           1.34.1
nest-asyncio       1.6.0
packaging          24.2
pip                24.3.1
plotly             6.0.1
requests           2.32.3
retrying           1.3.4
setuptools         78.1.0
six                1.17.0
typing_extensions  4.13.1
urllib3            2.3.0
Werkzeug           3.0.6
zipp               3.21.0
(venv) /tmp/dash $ cd ..
(venv) /tmp $ cat t.py
from dash import Dash, dcc, html
from dash.dependencies import Input, Output
from typing import Callable

app = Dash(__name__)

def create_layout() -> html.Div:
    return html.Div([
        dcc.Input(id='input-text', type='text', value='', placeholder='Enter text'),
        html.Div(id='output-text')
    ])  

app.layout = create_layout

@app.callback(Output('output-text', 'children'), Input('input-text', 'value'))
def update_output(value: str) -> str:
    return f'You entered: {value}'

if __name__ == '__main__':
    app.run(debug=True, port=9000)
(venv) /tmp $ mypy --strict t.py
t.py:1: error: Module "dash" has no attribute "Dash"  [attr-defined]
t.py:1: error: Module "dash" has no attribute "dcc"  [attr-defined]
t.py:1: error: Module "dash" has no attribute "html"  [attr-defined]
t.py:2: error: Cannot find implementation or library stub for module named "dash.dependencies"  [import-not-found]
t.py:2: note: See https://mypy.readthedocs.io/en/stable/running_mypy.html#missing-imports
t.py:15: error: Untyped decorator makes function "update_output" untyped  [misc]
Found 5 errors in 1 file (checked 1 source file)

gothicVI avatar Apr 10 '25 06:04 gothicVI

You need to build the components and run it from where it can find dash if running locally.

I get no more error when running this example.

T4rk1n avatar Apr 10 '25 13:04 T4rk1n

@T4rk1n damn good point. I have 0 experience with JS/TS though and am somewhat stuck with

$ npm run build  

> build
> run-s private::build.*

sh: line 1: run-s: command not found

while

$ node -v
v23.9.0
$ npm -v
11.2.0

gothicVI avatar Apr 10 '25 13:04 gothicVI

@gothicVI what OS are you on?

gvwilson avatar Apr 10 '25 15:04 gvwilson

@gvwilson Manjaro Linux

gothicVI avatar Apr 10 '25 15:04 gothicVI

OK, I don't understand why that wouldn't work. Can you please check out a fresh copy of the dash repo, go into it, and then run the following and send me all the output? I'm [email protected].

$ npx lerna clean $ rm -rf venv node_modules $ python -m venv venv $ . venv/bin/activate $ pip install -e '.[dev,testing,ci]' # note the quotes $ nvm use lts/iron $ npm i $ npm run build $ npm run setup-tests.py

gvwilson avatar Apr 10 '25 15:04 gvwilson

@gvwilson here you go https://gist.github.com/gothicVI/8a8cb53ba0be2f060d30ebf004edbaeb This fails at installing parts of the ci dependency due to python 3.13 not being supported...

I'll try and give it a shot with a machine running python 3.11 tomorrow.

gothicVI avatar Apr 10 '25 16:04 gothicVI

@gvwilson might have fixed it with a small diff:

$ git diff
diff --git a/requirements/ci.txt b/requirements/ci.txt
index 96495aa4..43553cca 100644
--- a/requirements/ci.txt
+++ b/requirements/ci.txt
@@ -7,7 +7,7 @@ ipython<9.0.0
 mimesis<=11.1.0
 mock==4.0.3
 numpy<=1.26.3
-orjson==3.10.3
+orjson==3.10.16
 openpyxl
 pandas>=1.4.0
 pyarrow

This allowed me to build the dependencies. For the full output see https://gist.github.com/gothicVI/5410fa76c5b177b048f8809a28b4773c

Finally, leaving the cloned dash directory to check t.py still gets me:

$ mypy --strict t.py
t.py:8: error: Returning Any from function declared to return "Div"  [no-any-return]
t.py:15: error: Call to untyped function "callback" in typed context  [no-untyped-call]
t.py:15: error: Untyped decorator makes function "update_output" untyped  [misc]
t.py:15: error: Call to untyped function "Output" in typed context  [no-untyped-call]
t.py:15: error: Call to untyped function "Input" in typed context  [no-untyped-call]
Found 5 errors in 1 file (checked 1 source file)

gothicVI avatar Apr 10 '25 16:04 gothicVI

thanks @gothicVI - as noted earlier, we're focusing on pyright for type checking. If there are changes you can PR that will satisfy mypy without breaking pyright, I'd be happy to prioritize review.

gvwilson avatar Apr 11 '25 14:04 gvwilson

hi @gvwilson the initial issue is not solved as you can see. All I solved was building dash using python 3.13. Sure I can write a PR for that little change but you mentioned that the problem was solved on your side with the PR you wrote. I can not reproduce that. Please help me out and let me know what you did to make the suggested changes be compliant with my MWE. Thanks!

gothicVI avatar Apr 12 '25 10:04 gothicVI

@gvwilson you had mentioned that with your patch you didn't see the issues. What exactly did you do?

gothicVI avatar Apr 17 '25 15:04 gothicVI

hi @gothicVI - can you please re-try with the latest version of Dash?

gvwilson avatar Apr 29 '25 13:04 gvwilson

Hi @gvwilson

$ docker run -it --rm python:3.11-slim bash
root@c8109b30021c:/# python3 -m pip install dash mypy
root@c8109b30021c:/# python3 -m pip freeze
blinker==1.9.0
certifi==2025.4.26
charset-normalizer==3.4.2
click==8.1.8
dash==3.0.4
Flask==3.0.3
idna==3.10
importlib_metadata==8.7.0
itsdangerous==2.2.0
Jinja2==3.1.6
MarkupSafe==3.0.2
mypy==1.15.0
mypy_extensions==1.1.0
narwhals==1.38.0
nest-asyncio==1.6.0
packaging==25.0
plotly==6.0.1
requests==2.32.3
retrying==1.3.4
six==1.17.0
typing_extensions==4.13.2
urllib3==2.4.0
Werkzeug==3.0.6
zipp==3.21.0
root@c8109b30021c:/# cat t.py 
from dash import Dash, dcc, html
from dash.dependencies import Input, Output
from typing import Callable

app = Dash(__name__)

def create_layout() -> html.Div:
    return html.Div([
        dcc.Input(id='input-text', type='text', value='', placeholder='Enter text'),
        html.Div(id='output-text')
    ])  

app.layout = create_layout

@app.callback(Output('output-text', 'children'), Input('input-text', 'value'))
def update_output(value: str) -> str:
    return f'You entered: {value}'

if __name__ == '__main__':
    app.run(debug=True, port=9000)
root@c8109b30021c:/# mypy t.py 
Success: no issues found in 1 source file

this does look very promising! I'll give the changes in the last version a thorough look and will try to port our production code soon!

gothicVI avatar May 07 '25 16:05 gothicVI

Hi @gvwilson

I've finally found the time to take a look with our production code. Overall it looks good but there are some things I found:

    # data is some pandas.DataFrame
    table_from_data: dash_table.DataTable = dash_table.DataTable(
        style_cell={
            "textAlign": "left",
        },
        style_data={
            "whiteSpace": "normal",
            "height": "auto",
        },
        style_data_conditional=[
            {
                "if": {"row_index": "even"},
                "backgroundColor": "rgb(220, 220, 220)",
            },
        ],
        style_header={
            "whiteSpace": "normal",
            "height": "auto",
            "border": "1px solid black",
        },
        data=data.to_dict("records"),
        columns=[{"name": i, "id": i} for i in data.columns],
    )

This worked with <3.0.0 and continues to work with 3.0.4, however, we now get the complaints

"backgroundColor": "rgb(220, 220, 220)"  # Argument of type "list[dict[str, StyleDataConditionalIf | str]]" cannot be assigned to parameter "style_data_conditional" of type "Sequence[StyleDataConditional] | None" in function "__init__"
  "backgroundColor" is an undefined item in type "StyleDataConditional"

data=data.to_dict("records")  # Argument of type "list[dict[Hashable, Any]]" cannot be assigned to parameter "data" of type "Sequence[Dict[str | float | int, NumberType | str | bool]] | None" in function "__init__"
  Type "list[dict[Hashable, Any]]" is not assignable to type "Sequence[Dict[str | float | int, NumberType | str | bool]] | None"
    "list[dict[Hashable, Any]]" is not assignable to "Sequence[Dict[str | float | int, NumberType | str | bool]]"
      Type parameter "_T_co@Sequence" is covariant, but "dict[Hashable, Any]" is not a subtype of "Dict[str | float | int, NumberType | str | bool]"
        "dict[Hashable, Any]" is not assignable to "Dict[str | float | int, NumberType | str | bool]"
          Type parameter "_KT@dict" is invariant, but "Hashable" is not the same as "str | float | int"
    "list[dict[Hashable, Any]]" is not assignable to "None"

It seems that the submodule does not define DataTable

error: Name "dash_table.DataTable" is not defined  [name-defined]
error: Module has no attribute "DataTable"  [attr-defined]

We have dcc.RadioItems like

UPDATE_INTERVAL: int = 300

dcc.RadioItems(
    id=ids.DASHBOARD_INFORMATION_RADIO,
    options=[
        {"label": " 5m", "value": UPDATE_INTERVAL},
        {"label": " 10m", "value": 2 * UPDATE_INTERVAL},
        {"label": " 15m", "value": 3 * UPDATE_INTERVAL},
         {"label": " 30m", "value": 6 * UPDATE_INTERVAL},
         {"label": " 1h", "value": 12 * UPDATE_INTERVAL},
         {"label": " never", "value": 0},
     ],
     value=0,
     inline=True,
     labelStyle={"padding-right": "6px"},
),

where we get the complaint

error: Argument "options" to "RadioItems" has incompatible type "list[dict[str, object]]"; expected "Sequence[str | SupportsFloat | SupportsInt | SupportsComplex | bool] | dict[Any, Any] | Sequence[Options] | None"  [arg-type]

Also, the dash.register_page function seems untyped.

Finally, we're also getting Module "dash.dcc" does not explicitly export attribute "send_data_frame" [attr-defined]

gothicVI avatar May 19 '25 12:05 gothicVI

I gave it another run with dash==3.2.0 and dash-bootstrap-components==2.0.4 and we're getting:

error: Argument "options" to "RadioItems" has incompatible type "list[dict[str, object]]"; expected "Sequence[str | SupportsFloat | SupportsInt | SupportsComplex | bool] | dict[Any, Any] | Sequence[Options] | None"  [arg-type]
error: Call to untyped function "register_page" in typed context  [no-untyped-call]
error: Call to untyped function "send_data_frame" in typed context  [no-untyped-call]
error: Module "dash._callback" does not explicitly export attribute "NoUpdate"  [attr-defined]
error: Module "dash.dcc" does not explicitly export attribute "send_data_frame"  [attr-defined]
error: Module has no attribute "DataTable"  [attr-defined]
error: Name "dash_table.DataTable" is not defined  [name-defined]

gothicVI avatar Aug 26 '25 16:08 gothicVI