Allow flex backend to specify additional indexes to create
Many transforms need additional indexes to be used. Currently this is commonly done as a .sql file distributed alongside the transforms, but this is not ideal for a few reasons
- It is an extra process that might be forgotten
- The create index statements will need to vary with the FILLFACTOR depending on if the database is updatable or not
- The naïve way of running
osm2pgsql && psql -f indexes.sqlwill lead to a period of time when only one index is being built. This is especially bad for slim imports with the way node index. - Index builds should be run in parallel, but
psql -fcan't handle this, requiring more elaborate SQL runners - VACUUM ANALYZE should be run after index building, not before
Instead, the index builds should be run, in parallel, for each table once the table has been sorted
With the flex backend it should be possible to specify multiple indexes for each table. It needs to be possible to specify the name (e.g. admin), index type (e.g. GIST), columns or expression (e.g. ST_PointOnSurface(way)), and partial index predicate (e.g. name IS NOT NULL AND boundary = 'administrative' AND admin_level IN ('0', '1', '2', '3', '4')). osm2pgsql would supply the storage_parameter based on if it is updatable and tablespace_name, if a tablespace is set.
FYI: we've been discussing this use case as part of a more generic post processing some time ago: https://github.com/openstreetmap/osm2pgsql/issues/1211#issuecomment-645287526
It is now possible to create any kind of indexes you want with the flex output. See https://osm2pgsql.org/doc/manual.html#defining-indexes for details.