FigureWidget(Resampler) does not integrate well in google colab

[jvdd]

In google colab plotly FigureWidget & plotly-resampler FigureWidgetResampler:

🎉 work perfectly when creating an empty figure and displaying it while adding traces in another cell

Example with plotly FigureWidget:

Example with plotly-resampler FigureWidgetResampler:

😿 do not work when creating the figure and adding the data + displaying it in the same cell

Example for plotly FigureWidget & plotly-resampler FigureWidgetResampler

-> This relates to this issue; https://github.com/googlecolab/colabtools/issues/2871 (however for FigureWidgetResampler it does not work for integer datatypes, whereas for FigureWidget this does work)

What is the impact of this problem?

Registing plotly-resampler in google colab will result in figures that are not displayed;

---

FYi: to use plotly FigureWidgets in google colab you should first execute the following code (see __init__.py);

from google.colab import output
output.enable_custom_widget_manager()

[LeonieFreisinger]

@jvdd Unfortunately the plotly resampler is not working for me in colab for all of the use cases I am using it for. Is there anything one needs to be careful with to have the resampler FigureWidgets display correctly? I have created a custom plot function, that wraps around the plotly plot methods. Is this a case that you observed to not work for the resampler? Thanks for your support!

[jonasvdd]

Hi @LeonieFreisinger,

• Which version of plotly-resampler are you using?
• Could you provide a minimal (non-working) example? this way we can help you better! 😄
• Does your plotting function work when using a local jupyter server?

Kind regards, Jonas

[LeonieFreisinger]

@jvdd @jonasvdd Thanks a lot for the fast answer. In general I think the functionality provided by the plotly-resampler is great. However, I face issues with getting to work in colab.

Feel free to execute this notebook in colab: https://github.com/ourownstory/neural_prophet/blob/main/tutorials/feature-use/autoregression_yosemite_temps.ipynb

Instructions:

1. add a cell at the top cointaining:

from google.colab import output
output.enable_custom_widget_manager()

2. edit cell [4] with to the following:

forecast = m.predict(df)
fig = m.plot(forecast, plotting_backend='plotly')

3. run the newly added cell as well as cell 1-4

What you will see is that the figure is not displayed. Important note: This happens only when using the custom wrapper function m.plot(). However, wenn plotting a dataframe with the standard .plot() function and plotly backend, then the figure will be displayed. E.g.:

import plotly.graph_objects as go
data = go.Scatter(x =forecast['ds'], y=forecast['y'] )
fig = go.FigureWidget(data=data)
fig

One assumption for the reason of the problem is that there could be a dependencie/ package mismatch. Thanks a lot for having a look at it.

[jvdd]

Hey @LeonieFreisinger,

Thanks for sharing the notebook & the detailed explanation! I was able to reproduce your issue.

I (quickly) succeeded to get a functional interactive plot by using the FigureResampler (which uses a Dash application instead of an IPython FigureWidget to provide the resampling) instead of the FigureWidgetResampler. However, note that the rendering of the plot takes some additional time (as Google has to sort out the portforwarding of the underlying (Jupyter)Dash app) - locally (or on your own server) the plot should be rendered nearly instantaneous.

You can use the FigureResampler decorator (under the hood) through calling register_plotly_resampler with the "figure" mode.

register_plotly_resampler(mode="figure")

See working example below :arrow_down:

:exclamation: Currently, plotly-resampler does not properly suppoty rangeslider https://github.com/predict-idlab/plotly-resampler/issues/156 (however, @jonasvdd is currently looking into this)

I'll share my further findings in this Issue (I'll investigate FigureWidgetResampler now).

Cheers, Jeroen

[jvdd]

I further looked into using FigureWidget / FigureWidgetResampler in Google Colab - and circled back to the issue above (for which I created this issue on the Google Colab GitHub repo https://github.com/googlecolab/colabtools/issues/2871)

I don't think the issue above will get fixed any time soon. It has been open for quite some time (7 months). Thus using FigureWidget / FigureWidgetResampler in Google Colab will keep resulting in problems :neutral_face:

Basically I see two options here;

1. use FigureResampler: you will have all the interactivity (i.e., resampling), but will suffer from a longer loading time. Note that the output will get cleared from the notebook (as the Dash app stops, unless you use inline_persistent).
2. use FigureWidgetResampler and call .show(): this will return a static figure on which the dynamic resampling cannot be performed (when zooming). However this figure will persist in the cells output.

---

On another note, are you intending to integrate plotly-resampler in an existing toolkit / code base? If so, @jonasvdd & me will gladly assist / give feedback w.r.t. this :)

[LeonieFreisinger]

@jvdd @jonasvdd Thanks for the quick feedback! I will have a look at it and get back to you shortly.

[LeonieFreisinger]

@jvdd @jonasvdd I had a closer look into both options. regarding 1: Unfortunately, the dash app takes too long to load for a sufficient experience. regarding 2: Returning a static version of the figure might as well not be feasible in our case. I am afraid it is rather confusing when zooming in and having the impression that the plot is unaccurate.

Another question though: Have you been testing the FigureWidgetResampler in a pycharm environment? Unfortunately, this does not work for me either. I hget a error message 'This notebook contains widgets which is not supported by Intellij. See javaScript logs for more info.'

Thank you!

Yes, we would like to use the plotly-resampler package in an exsisting toolkit, that we are continuously developing. Would be happy to exchange on how to integrate the plotly-resampler in the best way. Maybe it's better to excahnge via mail. Feel free to contact me via [email protected]

[jonasvdd]

Hi @LeonieFreisinger;

I quickly verified whether plotly's default go.FigureWidget is supported in PyCharm; and it appears no tot be so. As such, this is an issue that should be handled by plotly.py/PyCharm.

Regarding the slow dash loading; this is unexpected behavior for me. since you are using PyCharm, I suppose you are working locally (and not remotely via port forwarding)? As a result, I think that network latency will not make your dash code slow. Could you provide me a concrete example of the slow-loading dash behavior?

Kind regards, Jonas

[LeonieFreisinger]

Hi @jonasvdd ,

thanks for your help. Ah ok, I see. You recommend using the FigureResampler as well for local (PyCharm) use cases. That works. In this case, the loading time is not a problem and the user experience is sufficient. Another question: How do I display a plotly-resampler figure without using .show() in a non-notebook environment? Simple example: I implement a very simple go.FigureWidget in a .py file and register the resampler with register_plotly_resampler('figure') or register_plotly_resampler('auto'). By calling fig, the figure, however, will not be displayed. It only will be displayed by calling fig.show(). Can you recommend a best practice for that use case? Thanks!

[LeonieFreisinger]

@jonasvdd @jvdd Would be amazing to quickly hear back from you :)

[jvdd]

Hi @LeonieFreisinger,

To serve an interactive plotly-resampler visualization from a Python file you should use the FigureResampler (as a plotly FigureWidget only works interactively within a Jupyter environment).

The FigureResampler uses (under the hood) a Dash application to provide the resampling functionality to the figure. You can configure & start this Dash application from a Python file via the .show_dash method.
=> Thus when displaying a figure from a Python file you should call .show_dash(mode="external") -> mode external signals to start the dash app on a certain port (the port and other arguments can also be passed to this method).

Minimal example :arrow_down:

import numpy as np
import plotly.graph_objects as go
from plotly_resampler import register_plotly_resampler

# Instead of registering you can also decorate the figure with FigureResampler
register_plotly_resampler(mode="auto")

fig = go.Figure()
fig.add_trace(go.Scatter(y=np.random.randn(50_000), showlegend=True))

fig.show_dash(mode="external")

By default you should be able to find the app at http://127.0.0.1:8050/

Hope this helps!

[LeonieFreisinger]

@jvdd Thanks for the super fast answer, I appreciate it. Have you thought about extending the usage of the plotly-resampler to non-Jupyter environments, such as plotting in a .py file? In my opinion, the plotly-resampler would be quite convenient in such an environment. From a user perspective, it would be best to be still able to call the standard fig.show() method and have a dynamically rendered figure. Calling fig.show_dash() is not a straight forward way, since it requires users to adapt their previous code. Is this something that you are planning to realize or that could be fastly realized?

[LeonieFreisinger]

@jvdd let me give you some more context, I hope this will help you to understand my question better. We are planning to integrate the plotly-resampler into another framework. Mainly, there are 3 different cases the resampler would need to cover: (1) plotting in a jupyter environment, (2) plotting from a .py file, and (3) plotting in a colab notebook. We can check case(2). For now, we can disregard case (3) since a non-resampled plot will be somehow fine in this case. However, it would be great to find a solution of integrating the plotly-resampler and as before calling fig.show(). In this case the user would not need to adjust to any interface changes.

[jvdd]

Have you thought about extending the usage of the plotly-resampler to non-Jupyter environments, such as plotting in a .py file? In my opinion, the plotly-resampler would be quite convenient in such an environment.

Yes. This is the main use-case of Dash applications (what is used under the hood in plotly-resampler). We have a whole set of examples showing how plotly-resampler seamlessly works in this paradigm https://github.com/predict-idlab/plotly-resampler/tree/main/examples#2-dash-apps :)

From a user perspective, it would be best to be still able to call the standard fig.show() method and have a dynamically rendered figure. Calling fig.show_dash() is not a straight forward way, since it requires users to adapt their previous code.

I beg to differ. I like that we serve the functionality to both call .show_dash and .show as they do two different things:

• .show_dash results in a FigureResampler with dynamic aggregation
• .show results in a Figure without dynamic aggregation (when adding the traces to the figure they get only aggregated once to the full overview)

IMO it only makes some sense to overwrite this behavior when the user called register_plotly_resampler. I'll give this some more thought and discuss it with @jonasvdd.

Is this something that you are planning to realize or that could be fastly realized?

I think so, you just have to write your own decorator that overwrites this behavior.

---

We are planning to integrate the plotly-resampler into another framework.

If the project is open-source, I can have a look and advise on how you best integrate plotly-resampler :)

Mainly, there are 3 different cases the resampler would need to cover: (1) plotting in a jupyter environment, (2) plotting from a .py file, and (3) plotting in a colab notebook. We can check case(2). For now, we can disregard case (3) since a non-resampled plot will be somehow fine in this case. However, it would be great to find a solution of integrating the plotly-resampler and as before calling fig.show(). In this case the user would not need to adjust to any interface changes.

Don't you mean that we can check case (1)?
IMO we perfectly cover case 1 & 2. Case 3 is just a longstanding issue with plotly & jupyterdash..

Case 2, is however somewhat more advanced use, as this should cover / be flexible to build production-ready apps (serving multiple users etc.) - which is something plotly-resampler fully supports!
You can find more details here https://github.com/predict-idlab/plotly-resampler/tree/main/examples#2-dash-apps

---

My takeaways: it makes certainly sense (to some extent) to change the .show to .show_dash(mode="external") when the user called register_plotly_resampler (with auto or figure mode). (As I am pretty sure that users who build dash apps do not use the register_plotly_resampler functionality)

-> If we agree to add this functionality to register_plotly_resampler, I would add an optional flag (that is default False) so that the user can control this behavior.

@jonasvdd what is your opinion?

[LeonieFreisinger]

@jvdd Thanks for your fast answer. I will have a more detailed look into it and get back to you asap.

[LeonieFreisinger]

@jvdd I agree, we can check case (2). Case (2) only fails in running a notebook in pycharm. However, this problem is specific to pycharm.

Let me give you some more thoughts into this. I see your core use-case are dash applications. However, if the user initially did not intend to use dash apps and is dependent on using them to have the resampling active, he might disregard from using the resampler (this holds only for use case (1), since use case (2) can be fulfilled without dash app). Would love to brainstorm if there is a way of circumventing the use of dash apps for use case (1). Would be happy to get your feedback on integrating the resampler in our framework, thanks for offering. We can also have a quick call on it.