Tiger prepare interpolation error

[artemChernitsov]

Hello guys. I have an issue with preparing Interpolation Tiger Data in the North-America project.
 I already manually downloaded Tiger data for each US State and run interpolation script 'pelias prepare interpolation' to prepare tiger data, but I have an issue:

Wed Jun 24 20:46:25 UTC 2020 /data/tiger//downloads/tl_2016_47045_addrfeat.zip
FAILURE:
Unable to open datasource `/vsizip//data/tiger//downloads/tl_2016_47045_addrfeat.zip' with the following drivers.
  -> `PCIDSK'
  -> `netCDF'
  -> `JP2OpenJPEG'
  -> `PDF'
  -> `ESRI Shapefile'
  -> `MapInfo File'
  -> `UK .NTF'
  -> `OGR_SDTS'
  -> `S57'
  -> `DGN'
  -> `OGR_VRT'
  -> `REC'
  -> `Memory'
  -> `BNA'
  -> `CSV'
  -> `NAS'
  -> `GML'
  -> `GPX'
  -> `LIBKML'
  -> `KML'
  -> `GeoJSON'
  -> `Interlis 1'
  -> `Interlis 2'
  -> `OGR_GMT'
  -> `GPKG'
  -> `SQLite'
  -> `OGR_DODS'
  -> `ODBC'
  -> `WAsP'
  -> `PGeo'
  -> `MSSQLSpatial'
  -> `OGR_OGDI'
  -> `PostgreSQL'
  -> `MySQL'
  -> `OpenFileGDB'
  -> `XPlane'
  -> `DXF'
  -> `CAD'
  -> `Geoconcept'
  -> `GeoRSS'
  -> `GPSTrackMaker'
  -> `VFK'
  -> `PGDUMP'
  -> `OSM'
  -> `GPSBabel'
  -> `SUA'
  -> `OpenAir'
  -> `OGR_PDS'
  -> `WFS'
  -> `SOSI'
  -> `HTF'
  -> `AeronavFAA'
  -> `Geomedia'
  -> `EDIGEO'
  -> `GFT'
  -> `SVG'
  -> `CouchDB'
  -> `Cloudant'
  -> `Idrisi'
  -> `ARCGEN'
  -> `SEGUKOOA'
  -> `SEGY'
  -> `XLS'
  -> `ODS'
  -> `XLSX'
  -> `ElasticSearch'
  -> `Walk'
  -> `Carto'
  -> `AmigoCloud'
  -> `SXF'
  -> `Selafin'
  -> `JML'
  -> `PLSCENES'
  -> `CSW'
  -> `VDV'
  -> `GMLAS'
  -> `TIGER'
  -> `AVCBin'
  -> `AVCE00'
  -> `HTTP'

I have 2 questions:


1. What I can do with this datasource?
2. How I can continue preparation script? Before I receive crash, data import in an understandable order 
, for e.x. tl_2016_21125_addrfeat.zip then tl_2016_55011_addrfeat.zip then tl_2016_21071_addrfeat.zip , etc.


Wed Jun 24 20:43:55 UTC 2020 /data/tiger//downloads/tl_2016_21125_addrfeat.zip
Wed Jun 24 20:44:13 UTC 2020 /data/tiger//downloads/tl_2016_55011_addrfeat.zip
Wed Jun 24 20:44:29 UTC 2020 /data/tiger//downloads/tl_2016_21071_addrfeat.zip
Wed Jun 24 20:44:47 UTC 2020 /data/tiger//downloads/tl_2016_48317_addrfeat.zip
Wed Jun 24 20:44:57 UTC 2020 /data/tiger//downloads/tl_2016_41071_addrfeat.zip
Wed Jun 24 20:45:25 UTC 2020 /data/tiger//downloads/tl_2016_47121_addrfeat.zip
Wed Jun 24 20:45:35 UTC 2020 /data/tiger//downloads/tl_2016_13293_addrfeat.zip
Wed Jun 24 20:45:48 UTC 2020 /data/tiger//downloads/tl_2016_06095_addrfeat.zip


I can delete this datasource ‘tl_2016_47045_addrfeat.zip’, but I don’t know how I can finish my interpolation preparation without starting in from 1st datasource.

Thanks in advance.

[missinglink]

Is the tl_2016_47045_addrfeat.zip file valid? What is the contents of the file?

[artemChernitsov]

@missinglink I can't open this file looks like file not valid :( I redownload it right now and I can open this archive. Sorry for the stupid question. I had to check the file itself first.

Can you please help me, how I can continue prepare database, not from scratch?

[missinglink]

It's possible to just rerun the TIGER portion of the build only, as documented in https://github.com/pelias/interpolation

Unfortunately that functionality is not exposed via the pelias command through docker so you'll have to clone the interpolation repository and run the conflate_tiger.sh command manually.

I'd recommend you just ensure all the files are valid and rerun it from scratch.

Out of curiosity why did you decide to manually download the data rather than use the Pelias download script?

[artemChernitsov]

By "manual" I mean manually changing the "state_code" in pelias.json and run pelias download tiger command for each state, because from time to time I received a loading network error if I try to download all data in one time.

"interpolation": {
      "download": {
        "tiger": {
          "datapath": "/data/tiger",
           "states": [
            {
              "state_code": 47
            }
          ]
        }
      }
    }

@missinglink Also, I have a question - can I delete already processed datasets and start importing only not enough ones, it will not destroy the database?

[missinglink]

related: https://github.com/pelias/docker/issues/115 we should try to catch this error and handle it more gracefully.