geowave icon indicating copy to clipboard operation
geowave copied to clipboard

Grid coverage plugin in ingest framework triggering on shapefile

Open chrisbennight opened this issue 10 years ago • 3 comments

Probably not being selective enough in how it identifies files/formats - but it's not playing nicely

[bennight@cacti data]$ ls /data/natural_earth/
ne_10m_land.cpg  ne_10m_land.dbf  ne_10m_land.prj  ne_10m_land.README.html  ne_10m_land.shp  ne_10m_land.shx  ne_10m_land.VERSION.txt
[bennight@cacti data]$ geowave-ingest --localingest -b /data/natural_earth/ -i $INSTANCE -n geowave -p $GEOWAVE_PASS -u $GEOWAVE_USER -z $ZOOKEEPERS
Apr 11, 2015 5:04:20 PM org.geoserver.platform.GeoServerExtensions checkContext
WARNING: Extension lookup 'GeoServerResourceLoader', but ApplicationContext is unset.
Apr 11, 2015 5:04:20 PM org.geoserver.platform.GeoServerExtensions checkContext
WARNING: Extension lookup 'GeoServerResourceLoader', but ApplicationContext is unset.
Apr 11, 2015 5:04:20 PM org.geoserver.platform.GeoServerExtensions checkContext
WARNING: Extension lookup 'GeoServerResourceLoader', but ApplicationContext is unset.
Apr 11, 2015 5:04:20 PM org.geoserver.platform.GeoServerExtensions checkContext
WARNING: Extension lookup 'GeoServerResourceLoader', but ApplicationContext is unset.
Apr 11, 2015 5:04:21 PM org.geotools.gce.imagemosaic.Utils loadMosaicProperties
INFO: properties file doesn't exist
Apr 11, 2015 5:04:21 PM org.geotools.gce.imagemosaic.Utils loadMosaicProperties
INFO: properties file doesn't exist
Exception in thread "main" java.lang.UnsupportedOperationException: Trying to get a reader from an unknown format.
        at org.geotools.coverage.grid.io.UnknownFormat.getReader(UnknownFormat.java:62)
        at mil.nga.giat.geowave.types.geotools.raster.GeoToolsRasterDataStoreIngestPlugin.toGeoWaveData(GeoToolsRasterDataStoreIngestPlugin.java:75)
        at mil.nga.giat.geowave.types.geotools.raster.GeoToolsRasterDataStoreIngestPlugin.toGeoWaveData(GeoToolsRasterDataStoreIngestPlugin.java:35)
        at mil.nga.giat.geowave.ingest.local.LocalFileIngestDriver.processFile(LocalFileIngestDriver.java:146)
        at mil.nga.giat.geowave.ingest.local.LocalFileIngestDriver.processFile(LocalFileIngestDriver.java:33)
        at mil.nga.giat.geowave.ingest.local.LocalPluginFileVisitor.visitFile(LocalPluginFileVisitor.java:126)
        at mil.nga.giat.geowave.ingest.local.LocalPluginFileVisitor.visitFile(LocalPluginFileVisitor.java:30)
        at java.nio.file.Files.walkFileTree(Files.java:2670)
        at java.nio.file.Files.walkFileTree(Files.java:2742)
        at mil.nga.giat.geowave.ingest.local.AbstractLocalFileDriver.processInput(AbstractLocalFileDriver.java:60)
        at mil.nga.giat.geowave.ingest.local.LocalFileIngestDriver.runInternal(LocalFileIngestDriver.java:114)
        at mil.nga.giat.geowave.ingest.AbstractCommandLineDriver.run(AbstractCommandLineDriver.java:72)
        at mil.nga.giat.geowave.ingest.IngestMain.main(IngestMain.java:51)
[bennight@cacti data]$ 

chrisbennight avatar Apr 11 '15 21:04 chrisbennight

Is this an outstanding issue with the localingest example that's in the documentation? I'm running into these errors but it freezes at "WARNING: Extension lookup 'GeoServerResourceLoader', but ApplicationContext is unset". I did ^C and now Accumulo Master doesn't stay running.

The command I ran looks like this: geowave -localingest -b ./ingest -i accumulo -n geowave.ne_50m_admin_0_countries -f geotools-vector -u geowave -p hadoop -z localhost:2181

Full history is here: https://gist.github.com/ScottEvil/538cbcddd46acc88304b

ScottEvil avatar Jun 02 '15 17:06 ScottEvil

Did you bump up the heap space configurations for the Accumulo Master and Tablet Server processes? Most of the distros set these too low and the process dies trying to load the GeoWave iterator. http://ngageoint.github.io/geowave/documentation.html#accumulo-configuration

spohnan avatar Jun 02 '15 18:06 spohnan

@spohnan That was exactly the issue. I bumped up the heap sizes and it completed. Thank you!

ScottEvil avatar Jun 02 '15 18:06 ScottEvil