localizing datasources/resources representing more than one layer

[springmeyer]

Flagging this issue as requiring more thought.

Consider these various issues:

• Zipfiles - we currently assume its a shapefile and look for the first .shp, but there could be multiple. And eventually we'll want to support other datasources being zipped - like geojson: https://github.com/mapbox/tilemill/issues/253
• KML files - can embed more than one layer - currently millstone passes layer_by_index:0 to mapnik to just take the first
• SQLite databases - would be nice to be able to introspect and find all spatial tables

With the exception of zipped resources, the generic answer to these is using gdal/ogr to introspect the data (and therefore wrapping gdal/ogr as a node c++ addon). But, currently the approach of millstone is to handle each case in a custom way - with the benefit of avoiding the extra dependency and offering advanced functionality.

[yhahn]

Addressing this will require adjusting our architecture document a bit. Currently the only "verb" in millstone is resolve -- you pass it an MML and it hands it back to you with all of the resources localized and prepped for compilation/rendering.

To handle the cases above, however, millstone (or some other API component) would need to offer datasource inspection/introspection on a per-datasource basis with some accompanying UI in TileMill, e.g.

• User selects zipfile in TileMill, uri like file:///some/path/to/zipfile.zip is handed to millstone
• Millstone tells TileMill available datasources in zipfile like [ 'foo.shp', 'bar.shp', 'baz.shp' ] (I wonder whether millstone should actually do inspection at all or whether this should be TileMill's job, e.g. treat zipfiles, sqlite dbs, etc. like folders in the library browser)
• User further selects specific datasource from zip and the actual layer uri is set to file:///some/path/to/zipfile.zip#foo.shp or using some other convention
• Millstone resolve is called, unpacking foo.shp on the basis of uri zipfile.zip#foo.shp into a dir and referencing it in the localized MML.

^ this is just one possible workflow for this user story but I think the the basic sketch shows what kind of complexity we're talking about here.