fulltextsearch docstore does not clear when clear is called on a preloaded collection

[gotenxds]

[X] Bug report

Current behavior

When using autoload to load the DB and then calling clear on a loaded collection the fulltext docstore of that collection is not cleared.

I checked this for a newly loaded collection (that is loaded from the server) calling clear clears the fulltext docstore but If I refresh (allowing the autoload feature to work) and then call clear, when I try to insert new data into the collection I get the error Field already added. in the insert method of the fullTextSearch, when I examine the docstore in that error I can see that it is still full

Edit: After debugging the code a bit more, I see that loki does call clear on the fulltext search but the issue is that with autoLoad for some reason the _docs for the inverted_index is empty even but the actual inverted_index is not, not sure why this is happening

Expected behavior

calling collection.clear should clear the fulltext search for that collection as well. no matter how the collection was created.

[gotenxds]

After further investigation, it seems this issue actually stems from the database loading fts incorrectly,

When loki loads FTS data it desirialises the inverted index docstore but not the fts docstore this happens in fromJSONObject of the FTS

@Viatorus the fact that the deserializer only loads the invertedIndex and not the _docs property on the FTS itself causes and issue where when we call clear () on the FTS, it does not clear the inverted index as :

  public clear(): void {
    for (let id of this._docs) {
      this.removeDocument(null, id);
    }
  }

this._docs is empty

In addition, this causes an expectation when trying to add a "new" item that existed in the past, it does not exist in the FTS _docs but it does on the inverted index causing the original error I got

I tried adding this line of code to the fromJSONObject method and it seems to fix it

  fts._docs = new Set(serialized.ii[fieldNames[0]].docStore.flatMap(arr => arr[0]))

This just takes the first indexed property and extracts the id's from the pairs, as all indexed properties should have the same id's this should work fine. @Viatorus I don't know if this is the best way to handle this 🤷‍♂️