Not possible to replace/update: "Document ids should be integer"

[Dattaya]

Hi, I tried to replace a document with an id that searchApi.search returned - '3316996940775817220' (auto-generated on insert) which is a string, but the client throws this error Document ids should be integer, but I can't convert it to a number, because Manticore's ids are unsigned 64 bit integers and they're incompatible with 64 bit floating point JS numbers.

Manticore 6.2.12 with MCL=1 running in a docker container manticoresearch: 3.3.1 Ubuntu 23.04

Edit: to expand a bit on 'but I can't convert it to a number', I can convert the id to JS's BigInt, but then JSON.stringify wouldn't know what to do with it: 'TypeError: Do not know how to serialize a BigInt'

[sanikolaev]

Hi

id that searchApi.search returned - '3316996940775817220' (auto-generated on insert)

Manticore shouldn't auto-generate negative IDs. How can I reproduce this?

[Dattaya]

Sorry if I wasn't clear enough, but it's a hyphen, not a negative number.

[Dattaya]

Wrt to implementation, one way to do it would be to return BigInts from queries (they’re safe to use, because 64 bit integers are supported since Node 10.5) instead of strings and then serialize those fields with a custom JSON stringifier like json-bigint.

[duk-37]

Certain IDs being unusable in the insertion APIs bit me as well recently, since it essentially breaks any kind of search + modify behavior you might want. The workaround I ended up using was to just "serialize" the JSON and call the underlying API myself:

const index = "test"
const id = "9223372036854775807"
const data = { "test": "test" }
const res = await fetch(`http://127.0.0.1:9308/insert`, {
  method: "POST",
  headers: {
    'Content-Type': 'application/json'
  },
  body: `{ "index": "${index}", "id": ${id}, "doc": ${JSON.stringify(data)} }`
});

The TypeScript API has the same issue, of course.

[Nick-S-2018]

Blocked by https://github.com/manticoresoftware/manticoresearch/issues/2107

[sanikolaev]

Once this issue is done, let's do this as well https://github.com/manticoresoftware/manticoresearch-typescript/issues/9

[Nick-S-2018]

Done in https://github.com/manticoresoftware/manticoresearch-javascript/commit/974d1ddd4ca0f35ab053b5bece9951e35d261f83