download or generate .glb files on the fly

[oli-obk]

While we can download .osm files in various formats, we need glb/gltf files. https://www.osm2world.org/ can convert those for us, but there are various problems to think about:

• osm2world is a java application/library
  • we can use https://github.com/duchess-rs/duchess/ to link to the library from Rust, but that still needs a jre on the system where we're running
    • [ ] check if we need jdk, too, or just jre
  • OOMs if input is too large
• osm2world randomly dies on some osm elements
  • [ ] figure out if we can avoid that (graceful degragation?)
  • [ ] if it fails, can we instead show a flat osm rendering?
    • [ ] do these exist somewhere to download?
• [ ] can we RIIR osm2world in a reasonable amount of effort?
• no public repository of gltf files generated by osm2world

[oli-obk]

• https://github.com/johanhelsing/bevy_web_asset/pull/18 bevy 0.12 support is being worked on

[DerKarlos]

OSM2World is still in development.The goal is, to offer a relaiable glb server on a plain URL, free to use for anyone. No missing tiles. Covering a lot of area and a Web-API to request not existing tiles. 2D tiles are offered directly form OSM as used at osm.org for a fair amount of use. OOMs, RIIR = ? repository of gltf files ? Tiles on Github?

Anyway, we should consider alternative tile sources and even to process them ourselves. OSM will and should not offer a server. But a vector tile server to generate 3D tiles with different "map themes". A 3D tile server may speed up things, may be only for popular areas or as a cache. But I would prefer to convert vector tiles to 3D tiles in our application. There are open sources, we could use or transcode to Rust:

• OSM2World
• Street.GL
• That one, importing to Blender?
• That plugin for JSON? There are quite some games or services, converting OSM to 3D. Some may be useful, at last to get ideas and concepts
• I remember a Train Simulator somewhere
• ... todo ...
• Not F4map.com - closed source, they will hardly cooperate
• Google Earth?

[oli-obk]

Since the osm2world server only ships .glb.gz files, we need https://github.com/bevyengine/bevy/issues/10518 before we can continue here. This has been merged since, but needs a new bevy release before we can use it.

But it looks like that is only scheduled for the 0.13 release (https://github.com/bevyengine/bevy/pull/10565) so we may need to wait a bit.

I'll check if I can hardcode a .glb.gz deserializer specifically, without having general .gz support

[DerKarlos]

The osm2world server is not proper set at the moment. I will talk with Tobias about this soon. There can be different settings on server side.

• Even as there is only a glb.gz, you may request and get a glb.
• After downloading the glb.gz, it gets decompressed to glb. May be, the server decompressed, may be it tells the client to do this. May be, the above only works for browsers

If the last is true, we need that function in Bevy glb load. Or if this takes to much time, I am sure, we could do it ourselves: Download .gz and use the ZIP-crate and use the loader with the already existing data.

[oli-obk]

I am sure, we could do it ourselves: Download .gz and use the ZIP-crate and use the loader with the already existing data.

so am I 😆 after all, I implemented it in https://github.com/DerKarlos/OSMeta/pull/19

[DerKarlos]

After the server was corrected, we now can request both .glb as well as .glb.gz So our old code should still run. Tobias told me, if .glb is requested, the compact file .glb.gz is downloaded and unpacked by the "web socket". So we could use just .glb But we cache the tiles, using the old .glb.gz code would need less file memory.

I am not sure, if the actual errors are related to the changes. May be it is related to the changing file extensions. (#52)

[oli-obk]

But we cache the tiles, using the old .glb.gz code would need less file memory.

We used to cache the decompressed version of the files

[oli-obk]

On windows it keeps working, with and without cache

[DerKarlos]

Upps, ok. We could cache the .gz for less memmory. Just an option.

[oli-obk]

is it that much larger? disk space on local machines is normally not much of an issue.

I think I may have figured out the issue (I can reproduce now, turns out I failed to delete my cache correctly). Our old code stopped working on wasm after the change because the server also tries to decompress magically if we give a .glb.gz url, which then effectively tries to decompress a .glb.gz.gz file from the server. We don't see this in the logs, because the websocket of the browser does this transparently. Can you ask Tobias if this analysis is correct and whether he can only use the magic decompression on .glb urls, but not .gz URLS

[tordanik]

Can you ask Tobias if this analysis is correct and whether he can only use the magic decompression on .glb urls, but not .gz URLS

I would be happy to change the server's behavior to serve .gz files for download normally if the full URL is used, but I'm struggling to figure out the correct Apache incantations. The relevant entry in my apache2.conf looks like this at the moment:

<Directory /var/www/html/glb>
	RewriteEngine on
	RewriteRule ^(.*)\.glb $1\.glb\.gz [QSA]
	RewriteRule \.glb\.gz$ - [T=model/gltf-binary,E=no-gzip:1,E=is_gzip:1]
	Header set Content-Encoding "gzip" env=is_gzip
</Directory>

[oli-obk]

I have created a workaround: https://github.com/DerKarlos/OSMeta/commit/0d59b7e565709b35deff2d9e90f98442ddc38662?diff=unified&w=1

this works fine, as only web/wasm will follow your rules. native doesn't understand them anyway and just loads the file 😆