graphhopper
Change format of properties file
The properties file uses our own format, based on DataAccess. This is pretty annoying to work with when we open the file in a text editor. Can we use some kind of standard format for this?
We could use YAML and the default file would be a copy of the configuration files used while import (?) The only ugliness is that we would need a dependency to the YAML parser at core level.
Or we use JSON as we already have this dependency in core.
We could use YAML and the default file would be a copy of the configuration files used while import (?)
No, I think definitely not :) We want to move away from loading already imported GraphHoppers from configuration files that are meant to configure the import. OK, we could serialize the encoded values etc. to YAML, but I don't really see an advantage compared to serializing to JSON. And we already serialize (some) to JSON (see EncodedValueSerializer).
But actually I was thinking a bit less than that: Readability would improve already if we added a few newlines, but the thing that worries me the most is that our handmade binary format is not compatible with text editors. The properties file stores (only) text, but it is not a text file. In my editor it typically looks like this:
Storing the 'properties' in proper JSON format so we can actually deserialize a -gh folder without the import configuration is also desirable, but for another issue I think.
This is what I meant. So keeping this in mind we could now convert our properties into a JSON and in a later issue use/extend this file for graph deserialization.
Ok, we should just keep in mind that this is not a copy of the config file, but rather the metadata that belongs to the imported data in the gh folder :).
we could now convert our properties into a JSON
Ok a proper JSON text file? Can we do this with DataAccess or do we need to use something else for this? I guess the DataAccess header is what causes the messy symbols in text editors?!
Ok a proper JSON text file? Can we do this with DataAccess or do we need to use something else for this?
Yes, we would need a normal JSON file. DataAccess can't do this.
I guess the DataAccess header is what causes the messy symbols in text editors?!
Yes the binary header data
Yes the binary header data
And can't we write something into the header text editors understand (like an empty string) :) But also after the actual content there are lots of garbage symbols in my editor, we would probably have to fill the entire bytearray with empty strings or something.
But why? We can currently already edit the properties file and every average editor will save the binary data of the header properly. With the strings it would be IMO easier to corrupt the header (as it doesn't cry "this is something special don't edit" :)).
If we change something I would favor normal JSON or we try if setting the header length to 0 works. Still the binary data at the end might be present (-> we could somehow trim the length/segment size)
Ok, the DataAccess header always has a fixed size currently, and even if it was variable we would probably have to store the (variable) length of the header in the header. So we can probably not use StorableProperties with a zero length header? And since the header length cannot be zero I guess you are right that editing the file might corrupt the header and break the file. Then again, the file is not supposed to be edited anyway.
My issue is just that in it's current state the properties file is a pain to look at. I mean why should we store text in some kind of self-invented binary file, that can be partly read by standard text file tools, but also shows a bunch of garbage symbols, when we could just as well use a, well, text file? The properties file should just be a normal text file without using DataAccess? But maybe separating the encoded values and flag encoders by new line symbols instead of writing them all on the same line would already help.