Yu Shengnan
Yu Shengnan
same problem here
同问这个问题有解决么？
It is due to oom errors in querynodes, but I dont understand why the whole service is stucked. I restarted querycoord pod and then the service don't hang but oom...
[2022/11/28 07:17:10.960 +00:00] [ERROR] [querynode/segment_loader.go:127] ["load failed, OOM if loaded"] [collectionID=437544712534425849] [segmentType=Sealed] ["loadSegmentRequest msgID"=246] [error="load segment failed, OOM if load, collectionID = 437544712534425849, maxSegmentSize = 401 MB, concurrency = 1,...
And indexnode reported this error, All collections actually hava an index but it says "there is no index on collection" [2022/11/28 07:44:15.193 +00:00] [ERROR] [indexcoord/index_coord.go:506] ["IndexCoord get index state fail"]...
Sorry we have already manually restarted the pods and the scripts cant export those logs.
In that case we loaded a 16m collections with IVF_PQ index which caused oom. However now actutally the most significant problem is that the querynode behaved weird after oom. We...
They are almost the same
which pods, how long duration, and what log levels do you need?
[milvus-log.tar.gz](https://github.com/milvus-io/milvus/files/10112083/milvus-log.tar.gz)