Filtered Mapbox Clusters Appear in Incorrect Locations When Using Gzipped GeoJSON

[olowe]

We have a large dataset (~800k items) that we are rendering with Mapbox clusters. We have also precomputed our assets and stored them in cloud storage - as gzipped geojson files.

To allow users "Apply Filters" to the data, we make a request to our backend endpoint which then computes the relevant items that match the filters.

Since this is a lighter operation, our backend endpoint sends the results to the frontend as a gzip file. And this is where our issue comes in. Even though we use the same methods for both operations, when we use our backend to compute and return the gzip file, some points and clusters are rendered at very odd locations on the map - sometimes in the ocean.

While the data is accurate (as expected), the points are simply rendered at wrong locations on the map. Pls have a look at the provided screenshots.

How it works: To illustrate this issue, I'm providing sample datasets and screenshots.

Dataset 1: Large dataset. Renders properly as expected.

Dataset 2: Filtered dataset. Does not render/cluster as expected. Has a couple of odd points and clusters.

This is a link to a problematic dataset: https://drive.google.com/file/d/1jOxxtnBTbKeUjuNtj5CWS_PTYLBjlYFG/view?usp=sharing

Here's an excerpt of the React code that renders the map:

const mapSrcName = options.isFiltered ? "filtered-properties" : "properties";
const mapSrcData = options.isFiltered ? insightsData.filters.items : (process.env.NEXT_PUBLIC_MAP_SRC_URL as string);

const mapClusterOptions = {
  cluster: true,
  clusterMaxZoom: 16,
  clusterRadius: 50,
};

newMap.addSource(mapSrcName, {
  type: "geojson",
  data: mapSrcData,
  generateId: true,
  ...mapClusterOptions,
});

<div ref={mapContainerRef} id="map" />

Here's how we compute and return data via our backend endpoint:

async filterInsights(filterOptions: any) {
  const parsedFilterOptions = JSON.parse(filterOptions);

  if (parsedFilterOptions.location) {
    const response = await fetch(`https://${CLOUD_URL}/precomputed-MN.geojson.gz`,
    );

    const boroughFeatureCollection = await response.json();
    const boroughFeaturesOfInterest = (
      boroughFeatureCollection.features ?? []
    ).filter((f) => {
      // our custom filter checker
      checkFeatureMatchesFilters(f, parsedFilterOptions),
    });

    const newBoroughFC = {
      ...boroughFeatureCollection,
      features: boroughFeaturesOfInterest,
    };

    // here we create our feature collections object
    const clusteredFC = getInsightClusterFeatureCollection(newBoroughFC.features);

    // here we create a stream which we then pass to the client as gzip
    const collectionStream = createFeatureCollectionStream(clusteredFC);

    return collectionStream.pipe(zlib.createGzip({ level: 9 }));
  }

  return null;
}

Here is the method that creates the stream that's used in applyFilters method above:

createFeatureCollectionStream(featureCollection: any) {
  return new Stream.Readable({
    read() {
      // Start JSON
      this.push('{"type": "FeatureCollection", "features": [');
      
      featureCollection.features.forEach((feature, index) => {
        this.push(
          JSON.stringify(feature) +
            (index < featureCollection.features.length - 1 ? ',' : ''),
        );
      });
      this.push(']}'); // End JSON
      this.push(null); // End Stream
    },
  });
}

Any insights into this issue would be appreciated.

mapbox-gl-js version: 2.15.0

browser: Firefox, Chrome

[ibesora]

Hi @olowe , can you provide a JSBin or similar that reproduces the issue? Something that fetches that dataset and renders it in the map should be enough to reproduce the problem. Thanks!

[olowe]

Thanks for reaching out @ibesora. I am providing two JSBins that reproduce the issue.

Bin 1: This works as expected. The clusters, points and details are displayed as expected.

Link (output only): https://jsbin.com/zunigofiyu Link (with editor - you will need to set your mapbox access token): https://jsbin.com/licojobico/edit?html,output

For context: the dataset used here is essentially a collection of some properties in New York Asset link: https://sky-ny-properties.s3.eu-north-1.amazonaws.com/precomputed-clusters.geojson.gz

Bin 2: This does not work as expected. Here, you will observe that mapbox does not render any clusters or points despite the fact that when you access the gzip file link directly, you will be presented with a valid GeoJSON feature collection.

Link (output only): https://jsbin.com/kucequtuwu Link (with editor - you will need to set your mapbox access token): https://jsbin.com/qecahiyavu/edit?html,output

For context: the dataset used here is a collection of properties in the Manhattan region (filtered from the first dataset). Asset link: https://sky-ny-properties.s3.eu-north-1.amazonaws.com/precomputed-MN.geojson.gz

In the original issue description (which focuses on our React app), we are able to render some clusters and points albeit with a few odd points. However, with the JSBin, mapbox does not even render the filtered dataset. This also seems odd to me but I believe it still presents the underlying issue.

Thanks again and if you need any further information, I'm happy to provide it.

[olowe]

Hi @ibesora , any chance you've had a look at this?

[ibesora]

Not really, hopefully I can during the next week. Sorry for the delay

[olowe]

Hi @ibesora, any news?