observablehq
Datawrapper loader example
This proposal adds an example that generates and then embeds a chart created by Datawrapper.
In addition to the Python dependencies, it requires a Datawrapper API key available in the Python environment as DATAWRAPPER_ACCESS_TOKEN.
To help reviewing I have deployed the project to https://observablehq.observablehq.cloud/framework-example-loader-datawrapper/
I believe I addressed all of your requests. I'm open to whatever approach you want to take regarding the Python environment.
I'm sorry I edited my main comment just after I sent it, but you might have read it by email? https://github.com/observablehq/framework/pull/1585#pullrequestreview-2245372013
Also you have removed the "fixed" data/chart.html that I added. We need it so people can see what to expect without actually having to connect to the API (and so we can also rebuild all the examples when necessary).
I did miss that, sorry.
I'm open to another example. I was thinking, "Let's keep it super simple" and give the user a bare-bones starting point.
I'm happy to change the chart to something else. What appeals to you? A dressed-up stacked bar, a range plot, a dozen donuts?
If you ask me, I like "Germany is the third-oldest country in the world". But it's only an opinion :)
Should we make a note that the datawrapper iframe does not appear to support dark mode?
I'm a bit reluctant to publish it as is, because there are a few things that ought to be better:
dark mode
I've found a way to enable dark="auto" in the chart's preferences (editing the published chart), but I don't know how to pass this "render" option in the API call that creates the chart. Maybe it's not available in the python API?
responsive
Datawrapper has this concept of responsive charts, where the height depends on the (reactive) width and some javascript updates the iframe height. This example instead uses a fixed height, which ends up hiding some information (in our example, the footer) in a mobile view.
EDIT: I've created a helper function to solve both problems, though. See below.
tons of new charts
It's kind of a bummer that it creates a new chart each time it runs. This makes it quite damaging for the environment your account, and the Datawrapper admins might not like that; more importantly maybe, it makes a lousy user experience as it prevents you from making the chart better by editing it (post-creation) with the DataWrapper UI.
Ideally what I'd like to do is have a unique code in the script; if a chart with that code already exists, we reuse it, otherwise we populate it. If the chart depends on the data, then the code can be computed as a hash of the data state.
I have not implemented that (yet), it's more on the python side.
I've redeployed to https://observablehq.observablehq.cloud/framework-example-loader-datawrapper/
Note that this helper function should be able to accommodate several charts on the page — maybe that's something we could add to the example too.
what I'd like to do is have a unique code in the script; if a chart with that code already exists, we reuse it, otherwise we populate it
We could search for a chart that has that id set in a given hidden field, but I don't see if it's a possibility with the current API.
Thanks for this detailed feedback. I'm going to try to address your concerns piecemeal. The most recent commit is my first effort. I believe I have configured the Python chart configuration to activate Datawrapper's dark mode, which is off by default.
I'll loop back on your other items soon.
@Fil, here's a thought on the responsive iframe issue. Would we be better off switching to the web component, as described here: https://blog.datawrapper.de/web-component-embedding/
I think you're right!
Also, I have tried pipenv, and it's great. Maybe this is an opportunity to write down how it works in this case.
For me it was:
# https://docs.pipenv.org/#install-pipenv-today
brew install pipenv
# https://docs.pipenv.org/basics/#importing-from-requirements-txt
pipenv install -r ./requirements.txt
# https://app.datawrapper.de/account/api-tokens
echo 'DATAWRAPPER_ACCESS_TOKEN="****"' > ./.env
# https://docs.pipenv.org/basics/
pipenv run yarn dev
Do you have an established pattern for templating a script tag to be injected into the page? I tried passing the chart ID out of the data loader and then printing that into the page with a ${x} style template literal, but that's not working.
I was thinking something like this, but I'm not sure how that kind of thing fits into your framework.
```js
const chartId = await FileAttachment("data/chart.txt").text()
```
<div>
<script type="text/javascript" defer
src="https://datawrapper.dwcdn.net/${chartId}/embed.js?v=14">
</script>
<noscript>
<img src="https://datawrapper.dwcdn.net/${chartId}/full.png" alt="[aria-description]" />
</noscript>
</div>
I think I got it working and solved the responsive bug with:
```js echo
const chartId = await FileAttachment("data/chart.txt").text()
const tag = display(document.createElement('script'));
tag.setAttribute('src', `https://datawrapper.dwcdn.net/${chartId}/embed.js?v=14`);
```
Let me know what you think. If we have this one, I will move onto the final issue: Endless chart creation.
Yes it looks much simpler.
One issue that I had solved with my system but has a regression now, is when the page sets a color mode that is not the same as the browser's preference.
If you set this in the front-matter
---
theme: dark
---
and visit with a browser with light mode preferences, the chart is generated in light mode, and some text becomes unreadable. Same issue if you reverse the situation (theme: light, and a dark mode browser).
I think this is an upstream bug with datawrapper, though? We correctly set :root {color-scheme: dark;} and it should respect that, if set, over the browser preference. Or maybe I'm missing something?
I'm not a dark mode expert, so I'll defer to you on this. I can contact the datawrapper support team after we finish this ticket, if you think that's the right path.
On Fri, Aug 23, 2024, at 7:06 AM, Philippe Rivière wrote:
Yes it looks much simpler.
One issue that I had solved with my system but has a regression now, is when the page sets a color mode that is not the same as the browser's preference.
If you set this in the front-matter
`--- theme: dark
`
and visit with a browser with light mode preferences, the chart is generated in light mode, and some text becomes unreadable. Same issue if you reverse the situation (theme: light, and a dark mode browser).
I think this is an upstream bug with datawrapper, though? We correctly set
:root {color-scheme: dark;}and it should respect that, if set, over the browser preference. Or maybe I'm missing something?— Reply to this email directly, view it on GitHub https://github.com/observablehq/framework/pull/1585#issuecomment-2306855981, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAACOCPLQSJUMKUS2HLHL2DZS4JRRAVCNFSM6AAAAABMVKF276VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMBWHA2TKOJYGE. You are receiving this because you authored the thread.Message ID: @.***>
Inspecting the script, I see it only reads window.matchMedia("(prefers-color-scheme: dark)").matches. Probably something to report upstream.
Good catch. I'll send an email to the Datawrapper support team reporting the bug.
Separately, I've patched the loader to "get or create" the chart, rather than create a new one each time.
Are you good with that edit? If so, anything else you'd like to see before merging?
I think we only need to add the built chart.txt (so that in the future we can rebuild the project without datawrapper credentials).
What do you think of calling this "datawrapper-api" rather than "loader-datawrapper"? (It's not really "loading data", even though it uses the data loader architecture.)
Also, shouldn't we present the simpler version at the beggining of the page, explaining how a user can create a chart in datawrapper, then embed it in a framework page with a simple <script>. And then start the more powerful approach by saying "but here's how you can also create the chart from your framework code…". (Most people will probably only ever need the first approach, and for the others it can be used as a gentle introduction.)
I've added the cached chart.txt file and renamed the example as "Datawrapper API."
I'm not sure I know what you mean by "the simpler version," but I took a thwack at what I think you mean. A web component that does not rely on the data loader is at the top. It is then followed by the Python code, which is then embedded with a variation on the snippet at the top. Is that what you mean?
Reading https://blog.datawrapper.de/web-component-embedding/ I found out how to solve the dark mode issue, by adding data-dark="true" or "false". I'll fix in a moment (also doing some light copy-editing).
hmmm, I was trying to embed another chart and I'm getting errors on nki3d and K2J9t (from https://blog.datawrapper.de/pigeon-internet-speed/), looks like in these instances the script tries to get the data from my own server…?
Huh. Now you're past my knowledge. The chart isn't so old that it predates web components. Maybe we found another bug?
On Fri, Aug 23, 2024, at 10:37 AM, Philippe Rivière wrote:
hmmm, I was trying to embed another chart and I'm getting errors on nki3d and K2J9t (from https://blog.datawrapper.de/pigeon-internet-speed/), looks like in these instances the script tries to get the data from my own server…?
— Reply to this email directly, view it on GitHub https://github.com/observablehq/framework/pull/1585#issuecomment-2307228960, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAACOCJSULNKQQP65NVECYTZS5CMJAVCNFSM6AAAAABMVKF276VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMBXGIZDQOJWGA. You are receiving this because you authored the thread.Message ID: @.***>
I don't know what I was doing wrong, but K2J9t and nki3d now work as expected. I've added a different chart at the top for some diversity, polished the prose a bit, and added a helper function that handles dark mode in sync with Framework (bypassing the bug we've noticed). I think it's close to done!
Redeployed at https://observablehq.observablehq.cloud/framework-example-datawrapper-api/
Looks good. I'm ready to ship it when you are. Where is that dark variable being set?
It's a reactive variable available in all pages. See https://observablehq.com/framework/lib/generators#dark (I linked to this page in the "tip")