osm2pgsql icon indicating copy to clipboard operation
osm2pgsql copied to clipboard

Split up --extra-attributes option

Open joto opened this issue 3 years ago • 2 comments

The --extra-attributes/-x option does two things. It adds OSM attributes such as version, timestamp, uid, ... to the OSM objects processed in osm2pgsql. And it also changes the behaviour of osm2pgsql with respect to OSM objects without any tags. Normally those objects will not be processed, with -x they are.

We should, for the flex output, separate those concerns and, while we are at it, allow optional processing of untagged objects to be set differently for different object types.

See also #189.

joto avatar Jun 06 '22 15:06 joto

Here is a possible design for this: Add a call in the Flex Lua config file that allows setting some processing options. Something like this:

osm2pgsql.config('process_untagged_nodes', true)
osm2pgsql.config('process_untagged_ways', true)
osm2pgsql.config('process_untagged_relations', true)

(Not sure about the interface here, but this gives us the flexibility to add more options later.)

The default should be that the process functions are only called for tagged objects. I'd rather have a few people who are doing something with attributes complain that they don't see all objects by default than more people complaining osm2pgsql is unexplicably slow when they use -x.

We can also add osm2pgsql.with_attributes as a boolean reflecting the setting of the -x/--extra-attributes comand line option. This way we can get the old behaviour back by writing something like this in Lua:

if osm2pgsql.with_attributes then
  osm2pgsql.config('process_untagged_nodes', true)
  osm2pgsql.config('process_untagged_ways', true)
  osm2pgsql.config('process_untagged_relations', true)
end

joto avatar May 26 '23 09:05 joto

There is another aspect here: -x affects the middle and the output. If -x is set,

  • the middle will store the extra attributes
  • the flex output will have the extra attributes available in the object parameter to the process functions.

What if you want one behaviour, but not the other? Storing the extra attributes needs quite a lot of disk space, so maybe you want to avoid that but still get the attributes from the changes, for instance to get the changeset id?

joto avatar Nov 01 '23 13:11 joto