matrixone
matrixone copied to clipboard
[BugFix]: Create `CROSSJOIN_L2` operator
What type of PR is this?
- [ ] API-change
- [x] BUG
- [ ] Improvement
- [ ] Documentation
- [ ] Feature
- [ ] Test and CI
- [ ] Code Refactoring
Which issue(s) this PR fixes:
issue # https://github.com/matrixorigin/matrixone/issues/16001
What this PR does / why we need it:
- Implement
CROSS_JOIN_L2for mappingtbl embeddingstocentroids.
select
centroids.`__mo_index_centroid_version`,
centroids.`__mo_index_centroid_id`,
tbl.pk,
tbl.embedding
from tbl cross_l2 join centroids using(centroids.`__mo_index_centroid`, tbl.embedding);
-
Optimize memory allocation in NormalizeL2 by using
sync.Pool -
Modify
CREATE INDEXlogic to improve the index creation time. -
Modify
INSERTlogic to make use of the newCROSS JOIN L2operator.
Benchmark
- CREATE INDEX
- VECF32(128) : reduced from
2minsto29seconds-- good - VECF32(960): reduced from
12minsto3mins 7secs. -- good
- QPS
- VECF32(128) : 55 -- good
- VECF32(960): 3 -- bad