Corrupt split after merge

Open shrijeet opened this issue 4 months ago • 4 comments

I am running git sha 2025-09-21T06:01:37.990Z INFO quickwit_cli::service: quickwit version: 0.8.0 (x86_64-unknown-linux-gnu 2025-09-12T14:57:53Z ef6b319)

and keep running into data corruption

Error: { "message": "internal error: `search failed for the following splits: 01K5NF8E7YF6C5SM8HT3BQZ9BY. For instance, split 01K5NF8E7YF6C5SM8HT3BQZ9BY failed with the following error message: internal error: `spawned task in root join failed: task 2455 panicked with message \"assertion failed: end <= orig_range.end\"``" }

this happens after a split is merged. The indexer and metastore believe the split was fine and if I query the split via describe util, I dont see a problem but search fails. This is happening no matter if I reset the index.

Sep 21 '25 07:09 shrijeet

I cant tell what was fixed in this change: https://github.com/quickwit-oss/quickwit/pull/5908, was it related?

Sep 21 '25 07:09 shrijeet

logcollector-quickwit-searcher-2 quickwit [2m2025-09-21T06:36:26.221Z[0m [32m INFO[0m [1mleaf_search[0m[2m:[0m[1mmulti_index_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m[1msingle_doc_mapping_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m [2mquickwit_search::leaf[0m[2m:[0m [3mnum_docs[0m[2m=[0m3588691 [3mnum_splits[0m[2m=[0m1 [3msplit_offsets[0m[2m=[0m[SplitIdAndFooterOffsets { split_id: "01K5NF8E7YF6C5SM8HT3BQZ9BY", split_footer_start: 166991465, split_footer_end: 167008362, timestamp_start: Some(1758434986), timestamp_end: Some(1758436011), num_docs: 3588691 }]
logcollector-quickwit-searcher-2 quickwit [2m2025-09-21T06:36:26.221Z[0m [32m INFO[0m [1mleaf_search[0m[2m:[0m[1mmulti_index_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m[1msingle_doc_mapping_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m [2mquickwit_search::leaf[0m[2m:[0m [3mnum_docs[0m[2m=[0m3588691 [3mnum_splits[0m[2m=[0m1 [3msplit_offsets[0m[2m=[0m[SplitIdAndFooterOffsets { split_id: "01K5NF8E7YF6C5SM8HT3BQZ9BY", split_footer_start: 166991465, split_footer_end: 167008362, timestamp_start: Some(1758434986), timestamp_end: Some(1758436011), num_docs: 3588691 }]
logcollector-quickwit-searcher-2 quickwit [2m2025-09-21T06:36:58.932Z[0m [32m INFO[0m [1mleaf_search[0m[2m:[0m[1mmulti_index_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m[1msingle_doc_mapping_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m [2mquickwit_search::leaf[0m[2m:[0m [3mnum_docs[0m[2m=[0m3588691 [3mnum_splits[0m[2m=[0m1 [3msplit_offsets[0m[2m=[0m[SplitIdAndFooterOffsets { split_id: "01K5NF8E7YF6C5SM8HT3BQZ9BY", split_footer_start: 166991465, split_footer_end: 167008362, timestamp_start: Some(1758434986), timestamp_end: Some(1758436011), num_docs: 3588691 }]
logcollector-quickwit-searcher-2 quickwit [2m2025-09-21T06:36:58.934Z[0m [32m INFO[0m [1mleaf_search[0m[2m:[0m[1mmulti_index_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m[1msingle_doc_mapping_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m [2mquickwit_search::leaf[0m[2m:[0m [3mnum_docs[0m[2m=[0m3588691 [3mnum_splits[0m[2m=[0m1 [3msplit_offsets[0m[2m=[0m[SplitIdAndFooterOffsets { split_id: "01K5NF8E7YF6C5SM8HT3BQZ9BY", split_footer_start: 166991465, split_footer_end: 167008362, timestamp_start: Some(1758434986), timestamp_end: Some(1758436011), num_docs: 3588691 }]
logcollector-quickwit-searcher-2 quickwit [2m2025-09-21T06:37:31.885Z[0m [32m INFO[0m [1mleaf_search[0m[2m:[0m[1mmulti_index_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m[1msingle_doc_mapping_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m [2mquickwit_search::leaf[0m[2m:[0m [3mnum_docs[0m[2m=[0m3588691 [3mnum_splits[0m[2m=[0m1 [3msplit_offsets[0m[2m=[0m[SplitIdAndFooterOffsets { split_id: "01K5NF8E7YF6C5SM8HT3BQZ9BY", split_footer_start: 166991465, split_footer_end: 167008362, timestamp_start: Some(1758434986), timestamp_end: Some(1758436011), num_docs: 3588691 }]
logcollector-quickwit-searcher-2 quickwit [2m2025-09-21T06:37:32.003Z[0m [32m INFO[0m [1mleaf_search[0m[2m:[0m[1mmulti_index_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m[1msingle_doc_mapping_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m [2mquickwit_search::leaf[0m[2m:[0m [3mnum_docs[0m[2m=[0m3588691 [3mnum_splits[0m[2m=[0m1 [3msplit_offsets[0m[2m=[0m[SplitIdAndFooterOffsets { split_id: "01K5NF8E7YF6C5SM8HT3BQZ9BY", split_footer_start: 166991465, split_footer_end: 167008362, timestamp_start: Some(1758434986), timestamp_end: Some(1758436011), num_docs: 3588691 }]
logcollector-quickwit-searcher-2 quickwit [2m2025-09-21T06:38:04.895Z[0m [32m INFO[0m [1mleaf_search[0m[2m:[0m[1mmulti_index_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m[1msingle_doc_mapping_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m [2mquickwit_search::leaf[0m[2m:[0m [3mnum_docs[0m[2m=[0m3588691 [3mnum_splits[0m[2m=[0m1 [3msplit_offsets[0m[2m=[0m[SplitIdAndFooterOffsets { split_id: "01K5NF8E7YF6C5SM8HT3BQZ9BY", split_footer_start: 166991465, split_footer_end: 167008362, timestamp_start: Some(1758434986), timestamp_end: Some(1758436011), num_docs: 3588691 }]
logcollector-quickwit-searcher-2 quickwit [2m2025-09-21T06:38:04.895Z[0m [32m INFO[0m [1mleaf_search[0m[2m:[0m[1mmulti_index_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m[1msingle_doc_mapping_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m [2mquickwit_search::leaf[0m[2m:[0m [3mnum_docs[0m[2m=[0m3588691 [3mnum_splits[0m[2m=[0m1 [3msplit_offsets[0m[2m=[0m[SplitIdAndFooterOffsets { split_id: "01K5NF8E7YF6C5SM8HT3BQZ9BY", split_footer_start: 166991465, split_footer_end: 167008362, timestamp_start: Some(1758434986), timestamp_end: Some(1758436011), num_docs: 3588691 }]
logcollector-quickwit-searcher-2 quickwit [2m2025-09-21T06:38:37.875Z[0m [32m INFO[0m [1mleaf_search[0m[2m:[0m[1mmulti_index_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m[1msingle_doc_mapping_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m [2mquickwit_search::leaf[0m[2m:[0m [3mnum_docs[0m[2m=[0m3588691 [3mnum_splits[0m[2m=[0m1 [3msplit_offsets[0m[2m=[0m[SplitIdAndFooterOffsets { split_id: "01K5NF8E7YF6C5SM8HT3BQZ9BY", split_footer_start: 166991465, split_footer_end: 167008362, timestamp_start: Some(1758434986), timestamp_end: Some(1758436011), num_docs: 3588691 }]
logcollector-quickwit-searcher-2 quickwit [2m2025-09-21T06:38:37.890Z[0m [32m INFO[0m [1mleaf_search[0m[2m:[0m[1mmulti_index_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m[1msingle_doc_mapping_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m [2mquickwit_search::leaf[0m[2m:[0m [3mnum_docs[0m[2m=[0m3588691 [3mnum_splits[0m[2m=[0m1 [3msplit_offsets[0m[2m=[0m[SplitIdAndFooterOffsets { split_id: "01K5NF8E7YF6C5SM8HT3BQZ9BY", split_footer_start: 166991465, split_footer_end: 167008362, timestamp_start: Some(1758434986), timestamp_end: Some(1758436011), num_docs: 3588691 }]
logcollector-quickwit-searcher-2 quickwit [2m2025-09-21T06:39:10.911Z[0m [32m INFO[0m [1mleaf_search[0m[2m:[0m[1mmulti_index_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m[1msingle_doc_mapping_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m [2mquickwit_search::leaf[0m[2m:[0m [3mnum_docs[0m[2m=[0m3588691 [3mnum_splits[0m[2m=[0m1 [3msplit_offsets[0m[2m=[0m[SplitIdAndFooterOffsets { split_id: "01K5NF8E7YF6C5SM8HT3BQZ9BY", split_footer_start: 166991465, split_footer_end: 167008362, timestamp_start: Some(1758434986), timestamp_end: Some(1758436011), num_docs: 3588691 }]
logcollector-quickwit-searcher-2 quickwit [2m2025-09-21T06:39:10.921Z[0m [32m INFO[0m [1mleaf_search[0m[2m:[0m[1mmulti_index_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m[1msingle_doc_mapping_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m [2mquickwit_search::leaf[0m[2m:[0m [3mnum_docs[0m[2m=[0m3588691 [3mnum_splits[0m[2m=[0m1 [3msplit_offsets[0m[2m=[0m[SplitIdAndFooterOffsets { split_id: "01K5NF8E7YF6C5SM8HT3BQZ9BY", split_footer_start: 166991465, split_footer_end: 167008362, timestamp_start: Some(1758434986), timestamp_end: Some(1758436011), num_docs: 3588691 }]
logcollector-quickwit-searcher-2 quickwit [2m2025-09-21T06:39:43.894Z[0m [32m INFO[0m [1mleaf_search[0m[2m:[0m[1mmulti_index_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m[1msingle_doc_mapping_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m [2mquickwit_search::leaf[0m[2m:[0m [3mnum_docs[0m[2m=[0m3588691 [3mnum_splits[0m[2m=[0m1 [3msplit_offsets[0m[2m=[0m[SplitIdAndFooterOffsets { split_id: "01K5NF8E7YF6C5SM8HT3BQZ9BY", split_footer_start: 166991465, split_footer_end: 167008362, timestamp_start: Some(1758434986), timestamp_end: Some(1758436011), num_docs: 3588691 }]
logcollector-quickwit-searcher-2 quickwit [2m2025-09-21T06:39:43.897Z[0m [32m INFO[0m [1mleaf_search[0m[2m:[0m[1mmulti_index_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m[1msingle_doc_mapping_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m [2mquickwit_search::leaf[0m[2m:[0m [3mnum_docs[0m[2m=[0m3588691 [3mnum_splits[0m[2m=[0m1 [3msplit_offsets[0m[2m=[0m[SplitIdAndFooterOffsets { split_id: "01K5NF8E7YF6C5SM8HT3BQZ9BY", split_footer_start: 166991465, split_footer_end: 167008362, timestamp_start: Some(1758434986), timestamp_end: Some(1758436011), num_docs: 3588691 }]
logcollector-quickwit-searcher-2 quickwit [2m2025-09-21T06:40:16.928Z[0m [32m INFO[0m [1mleaf_search[0m[2m:[0m[1mmulti_index_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m[1msingle_doc_mapping_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m [2mquickwit_search::leaf[0m[2m:[0m [3mnum_docs[0m[2m=[0m3588691 [3mnum_splits[0m[2m=[0m1 [3msplit_offsets[0m[2m=[0m[SplitIdAndFooterOffsets { split_id: "01K5NF8E7YF6C5SM8HT3BQZ9BY", split_footer_start: 166991465, split_footer_end: 167008362, timestamp_start: Some(1758434986), timestamp_end: Some(1758436011), num_docs: 3588691 }]
logcollector-quickwit-searcher-2 quickwit [2m2025-09-21T06:40:16.939Z[0m [32m INFO[0m [1mleaf_search[0m[2m:[0m[1mmulti_index_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m[1msingle_doc_mapping_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m [2mquickwit_search::leaf[0m[2m:[0m [3mnum_docs[0m[2m=[0m3588691 [3mnum_splits[0m[2m=[0m1 [3msplit_offsets[0m[2m=[0m[SplitIdAndFooterOffsets { split_id: "01K5NF8E7YF6C5SM8HT3BQZ9BY", split_footer_start: 166991465, split_footer_end: 167008362, timestamp_start: Some(1758434986), timestamp_end: Some(1758436011), num_docs: 3588691 }]
logcollector-quickwit-searcher-2 quickwit [2m2025-09-21T06:40:50.011Z[0m [32m INFO[0m [1mleaf_search[0m[2m:[0m[1mmulti_index_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m[1msingle_doc_mapping_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m [2mquickwit_search::leaf[0m[2m:[0m [3mnum_docs[0m[2m=[0m3588691 [3mnum_splits[0m[2m=[0m1 [3msplit_offsets[0m[2m=[0m[SplitIdAndFooterOffsets { split_id: "01K5NF8E7YF6C5SM8HT3BQZ9BY", split_footer_start: 166991465, split_footer_end: 167008362, timestamp_start: Some(1758434986), timestamp_end: Some(1758436011), num_docs: 3588691 }]
logcollector-quickwit-searcher-2 quickwit [2m2025-09-21T06:40:50.139Z[0m [32m INFO[0m [1mleaf_search[0m[2m:[0m[1mmulti_index_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m[1msingle_doc_mapping_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m [2mquickwit_search::leaf[0m[2m:[0m [3mnum_docs[0m[2m=[0m3588691 [3mnum_splits[0m[2m=[0m1 [3msplit_offsets[0m[2m=[0m[SplitIdAndFooterOffsets { split_id: "01K5NF8E7YF6C5SM8HT3BQZ9BY", split_footer_start: 166991465, split_footer_end: 167008362, timestamp_start: Some(1758434986), timestamp_end: Some(1758436011), num_docs: 3588691 }]
logcollector-quickwit-searcher-2 quickwit [2m2025-09-21T06:41:21.844Z[0m [32m INFO[0m [1mleaf_search[0m[2m:[0m[1mmulti_index_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m[1msingle_doc_mapping_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m [2mquickwit_search::leaf[0m[2m:[0m [3mnum_docs[0m[2m=[0m3588691 [3mnum_splits[0m[2m=[0m1 [3msplit_offsets[0m[2m=[0m[SplitIdAndFooterOffsets { split_id: "01K5NF8E7YF6C5SM8HT3BQZ9BY", split_footer_start: 166991465, split_footer_end: 167008362, timestamp_start: Some(1758434986), timestamp_end: Some(1758436011), num_docs: 3588691 }]
logcollector-quickwit-searcher-2 quickwit [2m2025-09-21T06:41:21.867Z[0m [32m INFO[0m [1mleaf_search[0m[2m:[0m[1mmulti_index_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m[1msingle_doc_mapping_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m [2mquickwit_search::leaf[0m[2m:[0m [3mnum_docs[0m[2m=[0m3588691 [3mnum_splits[0m[2m=[0m1 [3msplit_offsets[0m[2m=[0m[SplitIdAndFooterOffsets { split_id: "01K5NF8E7YF6C5SM8HT3BQZ9BY", split_footer_start: 166991465, split_footer_end: 167008362, timestamp_start: Some(1758434986), timestamp_end: Some(1758436011), num_docs: 3588691 }]
logcollector-quickwit-searcher-2 quickwit [2m2025-09-21T06:41:54.904Z[0m [32m INFO[0m [1mleaf_search[0m[2m:[0m[1mmulti_index_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m[1msingle_doc_mapping_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m [2mquickwit_search::leaf[0m[2m:[0m [3mnum_docs[0m[2m=[0m3588691 [3mnum_splits[0m[2m=[0m1 [3msplit_offsets[0m[2m=[0m[SplitIdAndFooterOffsets { split_id: "01K5NF8E7YF6C5SM8HT3BQZ9BY", split_footer_start: 166991465, split_footer_end: 167008362, timestamp_start: Some(1758434986), timestamp_end: Some(1758436011), num_docs: 3588691 }]
logcollector-quickwit-searcher-2 quickwit [2m2025-09-21T06:41:54.904Z[0m [32m INFO[0m [1mleaf_search[0m[2m:[0m[1mmulti_index_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m[1msingle_doc_mapping_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m [2mquickwit_search::leaf[0m[2m:[0m [3mnum_docs[0m[2m=[0m3588691 [3mnum_splits[0m[2m=[0m1 [3msplit_offsets[0m[2m=[0m[SplitIdAndFooterOffsets { split_id: "01K5NF8E7YF6C5SM8HT3BQZ9BY", split_footer_start: 166991465, split_footer_end: 167008362, timestamp_start: Some(1758434986), timestamp_end: Some(1758436011), num_docs: 3588691 }]
logcollector-quickwit-searcher-2 quickwit [2m2025-09-21T06:42:27.944Z[0m [32m INFO[0m [1mleaf_search[0m[2m:[0m[1mmulti_index_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m[1msingle_doc_mapping_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m [2mquickwit_search::leaf[0m[2m:[0m [3mnum_docs[0m[2m=[0m3588691 [3mnum_splits[0m[2m=[0m1 [3msplit_offsets[0m[2m=[0m[SplitIdAndFooterOffsets { split_id: "01K5NF8E7YF6C5SM8HT3BQZ9BY", split_footer_start: 166991465, split_footer_end: 167008362, timestamp_start: Some(1758434986), timestamp_end: Some(1758436011), num_docs: 3588691 }]
logcollector-quickwit-searcher-2 quickwit [2m2025-09-21T06:42:27.953Z[0m [32m INFO[0m [1mleaf_search[0m[2m:[0m[1mmulti_index_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m[1msingle_doc_mapping_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m [2mquickwit_search::leaf[0m[2m:[0m [3mnum_docs[0m[2m=[0m3588691 [3mnum_splits[0m[2m=[0m1 [3msplit_offsets[0m[2m=[0m[SplitIdAndFooterOffsets { split_id: "01K5NF8E7YF6C5SM8HT3BQZ9BY", split_footer_start: 166991465, split_footer_end: 167008362, timestamp_start: Some(1758434986), timestamp_end: Some(1758436011), num_docs: 3588691 }]
logcollector-quickwit-searcher-2 quickwit [2m2025-09-21T06:43:00.886Z[0m [32m INFO[0m [1mleaf_search[0m[2m:[0m[1mmulti_index_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m[1msingle_doc_mapping_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m [2mquickwit_search::leaf[0m[2m:[0m [3mnum_docs[0m[2m=[0m3588691 [3mnum_splits[0m[2m=[0m1 [3msplit_offsets[0m[2m=[0m[SplitIdAndFooterOffsets { split_id: "01K5NF8E7YF6C5SM8HT3BQZ9BY", split_footer_start: 166991465, split_footer_end: 167008362, timestamp_start: Some(1758434986), timestamp_end: Some(1758436011), num_docs: 3588691 }]
logcollector-quickwit-searcher-2 quickwit [2m2025-09-21T06:43:00.886Z[0m [32m INFO[0m [1mleaf_search[0m[2m:[0m[1mmulti_index_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m[1msingle_doc_mapping_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m [2mquickwit_search::leaf[0m[2m:[0m [3mnum_docs[0m[2m=[0m3588691 [3mnum_splits[0m[2m=[0m1 [3msplit_offsets[0m[2m=[0m[SplitIdAndFooterOffsets { split_id: "01K5NF8E7YF6C5SM8HT3BQZ9BY", split_footer_start: 166991465, split_footer_end: 167008362, timestamp_start: Some(1758434986), timestamp_end: Some(1758436011), num_docs: 3588691 }]
logcollector-quickwit-searcher-2 quickwit [2m2025-09-21T06:43:34.066Z[0m [32m INFO[0m [1mleaf_search[0m[2m:[0m[1mmulti_index_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m[1msingle_doc_mapping_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m [2mquickwit_search::leaf[0m[2m:[0m [3mnum_docs[0m[2m=[0m3588691 [3mnum_splits[0m[2m=[0m1 [3msplit_offsets[0m[2m=[0m[SplitIdAndFooterOffsets { split_id: "01K5NF8E7YF6C5SM8HT3BQZ9BY", split_footer_start: 166991465, split_footer_end: 167008362, timestamp_start: Some(1758434986), timestamp_end: Some(1758436011), num_docs: 3588691 }]
logcollector-quickwit-searcher-2 quickwit [2m2025-09-21T06:43:34.067Z[0m [32m INFO[0m [1mleaf_search[0m[2m:[0m[1mmulti_index_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m[1msingle_doc_mapping_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m [2mquickwit_search::leaf[0m[2m:[0m [3mnum_docs[0m[2m=[0m3588691 [3mnum_splits[0m[2m=[0m1 [3msplit_offsets[0m[2m=[0m[SplitIdAndFooterOffsets { split_id: "01K5NF8E7YF6C5SM8HT3BQZ9BY", split_footer_start: 166991465, split_footer_end: 167008362, timestamp_start: Some(1758434986), timestamp_end: Some(1758436011), num_docs: 3588691 }]
logcollector-quickwit-searcher-2 quickwit [2m2025-09-21T06:44:06.883Z[0m [32m INFO[0m [1mleaf_search[0m[2m:[0m[1mmulti_index_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m[1msingle_doc_mapping_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m [2mquickwit_search::leaf[0m[2m:[0m [3mnum_docs[0m[2m=[0m3588691 [3mnum_splits[0m[2m=[0m1 [3msplit_offsets[0m[2m=[0m[SplitIdAndFooterOffsets { split_id: "01K5NF8E7YF6C5SM8HT3BQZ9BY", split_footer_start: 166991465, split_footer_end: 167008362, timestamp_start: Some(1758434986), timestamp_end: Some(1758436011), num_docs: 3588691 }]
logcollector-quickwit-searcher-2 quickwit [2m2025-09-21T06:44:06.888Z[0m [32m INFO[0m [1mleaf_search[0m[2m:[0m[1mmulti_index_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m[1msingle_doc_mapping_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m [2mquickwit_search::leaf[0m[2m:[0m [3mnum_docs[0m[2m=[0m3588691 [3mnum_splits[0m[2m=[0m1 [3msplit_offsets[0m[2m=[0m[SplitIdAndFooterOffsets { split_id: "01K5NF8E7YF6C5SM8HT3BQZ9BY", split_footer_start: 166991465, split_footer_end: 167008362, timestamp_start: Some(1758434986), timestamp_end: Some(1758436011), num_docs: 3588691 }]
logcollector-quickwit-metastore-69549884b4-q6bq5 quickwit [2m2025-09-21T06:26:57.069Z[0m [32m INFO[0m [1mstage_splits[0m[1m{[0m[3msplit_ids[0m[2m=[0m"[\"01K5NF8E7YF6C5SM8HT3BQZ9BY\"]"[1m}[0m[2m:[0m [2mquickwit_metastore::metastore::postgres::metastore[0m[2m:[0m staged `1` splits successfully [3mindex_uid[0m[2m=[0mray-logs:01K5MSMHH3G5P5E3GYZXNKCAE6
logcollector-quickwit-metastore-69549884b4-q6bq5 quickwit [2m2025-09-21T06:26:59.663Z[0m [32m INFO[0m [1mpublish_splits[0m[1m{[0m[3mrequest[0m[2m=[0mPublishSplitsRequest { index_uid: Some(IndexUid { index_id: "ray-logs", incarnation_id: Ulid(2125791290993029397149779255855819206) }), staged_split_ids: ["01K5NF8E7YF6C5SM8HT3BQZ9BY"], replaced_split_ids: ["01K5NEJYBSJCZHAWKR1JGE3774", "01K5NED3352S9GWXXZYTS55AVK", "01K5NEP07H06P8S59G9D7Z00CV", "01K5NEZ7BNRB3TD1AF5XK9KRXN", "01K5NF8DJ2W8XQBT4QZ9NS5MHM", "01K5NEFTNR440J0T213QRYMRTS", "01K5NF5BPVCNCZ2CSAN1TG4V5W", "01K5NF290N9EFFYHEYA7H2WSC2", "01K5NES20TZBEM00M9SCZAAR5E", "01K5NEW4KBT4AWZ6AVXYQPHZYY"], index_checkpoint_delta_json_opt: None, publish_token_opt: None }[1m}[0m[2m:[0m [2mquickwit_metastore::metastore::postgres::metastore[0m[2m:[0m published 1 splits and marked 10 for deletion successfully [3mindex_uid[0m[2m=[0mray-logs:01K5MSMHH3G5P5E3GYZXNKCAE6
logcollector-quickwit-searcher-4 quickwit [2m2025-09-21T06:35:53.494Z[0m [32m INFO[0m [1mleaf_search[0m[2m:[0m[1mmulti_index_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m[1msingle_doc_mapping_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m [2mquickwit_search::leaf[0m[2m:[0m [3mnum_docs[0m[2m=[0m3588691 [3mnum_splits[0m[2m=[0m1 [3msplit_offsets[0m[2m=[0m[SplitIdAndFooterOffsets { split_id: "01K5NF8E7YF6C5SM8HT3BQZ9BY", split_footer_start: 166991465, split_footer_end: 167008362, timestamp_start: Some(1758434986), timestamp_end: Some(1758436011), num_docs: 3588691 }]
logcollector-quickwit-searcher-2 quickwit [2m2025-09-21T06:44:39.846Z[0m [32m INFO[0m [1mleaf_search[0m[2m:[0m[1mmulti_index_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m[1msingle_doc_mapping_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m [2mquickwit_search::leaf[0m[2m:[0m [3mnum_docs[0m[2m=[0m3588691 [3mnum_splits[0m[2m=[0m1 [3msplit_offsets[0m[2m=[0m[SplitIdAndFooterOffsets { split_id: "01K5NF8E7YF6C5SM8HT3BQZ9BY", split_footer_start: 166991465, split_footer_end: 167008362, timestamp_start: Some(1758434986), timestamp_end: Some(1758436011), num_docs: 3588691 }]
logcollector-quickwit-searcher-2 quickwit [2m2025-09-21T06:44:39.868Z[0m [32m INFO[0m [1mleaf_search[0m[2m:[0m[1mmulti_index_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m[1msingle_doc_mapping_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m [2mquickwit_search::leaf[0m[2m:[0m [3mnum_docs[0m[2m=[0m3588691 [3mnum_splits[0m[2m=[0m1 [3msplit_offsets[0m[2m=[0m[SplitIdAndFooterOffsets { split_id: "01K5NF8E7YF6C5SM8HT3BQZ9BY", split_footer_start: 166991465, split_footer_end: 167008362, timestamp_start: Some(1758434986), timestamp_end: Some(1758436011), num_docs: 3588691 }]
logcollector-quickwit-searcher-2 quickwit [2m2025-09-21T06:45:05.448Z[0m [32m INFO[0m [1mleaf_search[0m[2m:[0m[1mmulti_index_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m[1msingle_doc_mapping_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m [2mquickwit_search::leaf[0m[2m:[0m [3mnum_docs[0m[2m=[0m3588691 [3mnum_splits[0m[2m=[0m1 [3msplit_offsets[0m[2m=[0m[SplitIdAndFooterOffsets { split_id: "01K5NF8E7YF6C5SM8HT3BQZ9BY", split_footer_start: 166991465, split_footer_end: 167008362, timestamp_start: Some(1758434986), timestamp_end: Some(1758436011), num_docs: 3588691 }]
logcollector-quickwit-searcher-2 quickwit [2m2025-09-21T06:45:05.500Z[0m [32m INFO[0m [1mleaf_search[0m[2m:[0m[1mmulti_index_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m[1msingle_doc_mapping_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m [2mquickwit_search::leaf[0m[2m:[0m [3mnum_docs[0m[2m=[0m3588691 [3mnum_splits[0m[2m=[0m1 [3msplit_offsets[0m[2m=[0m[SplitIdAndFooterOffsets { split_id: "01K5NF8E7YF6C5SM8HT3BQZ9BY", split_footer_start: 166991465, split_footer_end: 167008362, timestamp_start: Some(1758434986), timestamp_end: Some(1758436011), num_docs: 3588691 }]
logcollector-quickwit-searcher-2 quickwit [2m2025-09-21T06:45:37.893Z[0m [32m INFO[0m [1mleaf_search[0m[2m:[0m[1mmulti_index_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m[1msingle_doc_mapping_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m [2mquickwit_search::leaf[0m[2m:[0m [3mnum_docs[0m[2m=[0m3588691 [3mnum_splits[0m[2m=[0m1 [3msplit_offsets[0m[2m=[0m[SplitIdAndFooterOffsets { split_id: "01K5NF8E7YF6C5SM8HT3BQZ9BY", split_footer_start: 166991465, split_footer_end: 167008362, timestamp_start: Some(1758434986), timestamp_end: Some(1758436011), num_docs: 3588691 }]
logcollector-quickwit-searcher-2 quickwit [2m2025-09-21T06:45:37.903Z[0m [32m INFO[0m [1mleaf_search[0m[2m:[0m[1mmulti_index_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m[1msingle_doc_mapping_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m [2mquickwit_search::leaf[0m[2m:[0m [3mnum_docs[0m[2m=[0m3588691 [3mnum_splits[0m[2m=[0m1 [3msplit_offsets[0m[2m=[0m[SplitIdAndFooterOffsets { split_id: "01K5NF8E7YF6C5SM8HT3BQZ9BY", split_footer_start: 166991465, split_footer_end: 167008362, timestamp_start: Some(1758434986), timestamp_end: Some(1758436011), num_docs: 3588691 }]
logcollector-quickwit-searcher-4 quickwit [2m2025-09-21T06:35:58.416Z[0m [32m INFO[0m [1mleaf_search[0m[2m:[0m[1mmulti_index_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m[1msingle_doc_mapping_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m [2mquickwit_search::leaf[0m[2m:[0m [3mnum_docs[0m[2m=[0m3588691 [3mnum_splits[0m[2m=[0m1 [3msplit_offsets[0m[2m=[0m[SplitIdAndFooterOffsets { split_id: "01K5NF8E7YF6C5SM8HT3BQZ9BY", split_footer_start: 166991465, split_footer_end: 167008362, timestamp_start: Some(1758434986), timestamp_end: Some(1758436011), num_docs: 3588691 }]
logcollector-quickwit-searcher-2 quickwit [2m2025-09-21T06:46:10.419Z[0m [32m INFO[0m [1mleaf_search[0m[2m:[0m[1mmulti_index_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m[1msingle_doc_mapping_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m [2mquickwit_search::leaf[0m[2m:[0m [3mnum_docs[0m[2m=[0m3588691 [3mnum_splits[0m[2m=[0m1 [3msplit_offsets[0m[2m=[0m[SplitIdAndFooterOffsets { split_id: "01K5NF8E7YF6C5SM8HT3BQZ9BY", split_footer_start: 166991465, split_footer_end: 167008362, timestamp_start: Some(1758434986), timestamp_end: Some(1758436011), num_docs: 3588691 }]
logcollector-quickwit-searcher-2 quickwit [2m2025-09-21T06:46:10.544Z[0m [32m INFO[0m [1mleaf_search[0m[2m:[0m[1mmulti_index_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m[1msingle_doc_mapping_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m [2mquickwit_search::leaf[0m[2m:[0m [3mnum_docs[0m[2m=[0m3588691 [3mnum_splits[0m[2m=[0m1 [3msplit_offsets[0m[2m=[0m[SplitIdAndFooterOffsets { split_id: "01K5NF8E7YF6C5SM8HT3BQZ9BY", split_footer_start: 166991465, split_footer_end: 167008362, timestamp_start: Some(1758434986), timestamp_end: Some(1758436011), num_docs: 3588691 }]
logcollector-quickwit-searcher-2 quickwit [2m2025-09-21T06:46:42.925Z[0m [32m INFO[0m [1mleaf_search[0m[2m:[0m[1mmulti_index_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m[1msingle_doc_mapping_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m [2mquickwit_search::leaf[0m[2m:[0m [3mnum_docs[0m[2m=[0m3588691 [3mnum_splits[0m[2m=[0m1 [3msplit_offsets[0m[2m=[0m[SplitIdAndFooterOffsets { split_id: "01K5NF8E7YF6C5SM8HT3BQZ9BY", split_footer_start: 166991465, split_footer_end: 167008362, timestamp_start: Some(1758434986), timestamp_end: Some(1758436011), num_docs: 3588691 }]
logcollector-quickwit-searcher-2 quickwit [2m2025-09-21T06:46:42.930Z[0m [32m INFO[0m [1mleaf_search[0m[2m:[0m[1mmulti_index_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m[1msingle_doc_mapping_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m [2mquickwit_search::leaf[0m[2m:[0m [3mnum_docs[0m[2m=[0m3588691 [3mnum_splits[0m[2m=[0m1 [3msplit_offsets[0m[2m=[0m[SplitIdAndFooterOffsets { split_id: "01K5NF8E7YF6C5SM8HT3BQZ9BY", split_footer_start: 166991465, split_footer_end: 167008362, timestamp_start: Some(1758434986), timestamp_end: Some(1758436011), num_docs: 3588691 }]
logcollector-quickwit-searcher-2 quickwit [2m2025-09-21T06:47:15.883Z[0m [32m INFO[0m [1mleaf_search[0m[2m:[0m[1mmulti_index_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m[1msingle_doc_mapping_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m [2mquickwit_search::leaf[0m[2m:[0m [3mnum_docs[0m[2m=[0m3588691 [3mnum_splits[0m[2m=[0m1 [3msplit_offsets[0m[2m=[0m[SplitIdAndFooterOffsets { split_id: "01K5NF8E7YF6C5SM8HT3BQZ9BY", split_footer_start: 166991465, split_footer_end: 167008362, timestamp_start: Some(1758434986), timestamp_end: Some(1758436011), num_docs: 3588691 }]
logcollector-quickwit-searcher-2 quickwit [2m2025-09-21T06:47:15.884Z[0m [32m INFO[0m [1mleaf_search[0m[2m:[0m[1mmulti_index_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m[1msingle_doc_mapping_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m [2mquickwit_search::leaf[0m[2m:[0m [3mnum_docs[0m[2m=[0m3588691 [3mnum_splits[0m[2m=[0m1 [3msplit_offsets[0m[2m=[0m[SplitIdAndFooterOffsets { split_id: "01K5NF8E7YF6C5SM8HT3BQZ9BY", split_footer_start: 166991465, split_footer_end: 167008362, timestamp_start: Some(1758434986), timestamp_end: Some(1758436011), num_docs: 3588691 }]
logcollector-quickwit-searcher-2 quickwit [2m2025-09-21T06:47:48.907Z[0m [32m INFO[0m [1mleaf_search[0m[2m:[0m[1mmulti_index_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m[1msingle_doc_mapping_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m [2mquickwit_search::leaf[0m[2m:[0m [3mnum_docs[0m[2m=[0m3588691 [3mnum_splits[0m[2m=[0m1 [3msplit_offsets[0m[2m=[0m[SplitIdAndFooterOffsets { split_id: "01K5NF8E7YF6C5SM8HT3BQZ9BY", split_footer_start: 166991465, split_footer_end: 167008362, timestamp_start: Some(1758434986), timestamp_end: Some(1758436011), num_docs: 3588691 }]
logcollector-quickwit-searcher-2 quickwit [2m2025-09-21T06:47:48.918Z[0m [32m INFO[0m [1mleaf_search[0m[2m:[0m[1mmulti_index_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m[1msingle_doc_mapping_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m [2mquickwit_search::leaf[0m[2m:[0m [3mnum_docs[0m[2m=[0m3588691 [3mnum_splits[0m[2m=[0m1 [3msplit_offsets[0m[2m=[0m[SplitIdAndFooterOffsets { split_id: "01K5NF8E7YF6C5SM8HT3BQZ9BY", split_footer_start: 166991465, split_footer_end: 167008362, timestamp_start: Some(1758434986), timestamp_end: Some(1758436011), num_docs: 3588691 }]
logcollector-quickwit-searcher-4 quickwit [2m2025-09-21T06:36:01.633Z[0m [32m INFO[0m [1mleaf_search[0m[2m:[0m[1mmulti_index_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m[1msingle_doc_mapping_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m [2mquickwit_search::leaf[0m[2m:[0m [3mnum_docs[0m[2m=[0m3588691 [3mnum_splits[0m[2m=[0m1 [3msplit_offsets[0m[2m=[0m[SplitIdAndFooterOffsets { split_id: "01K5NF8E7YF6C5SM8HT3BQZ9BY", split_footer_start: 166991465, split_footer_end: 167008362, timestamp_start: Some(1758434986), timestamp_end: Some(1758436011), num_docs: 3588691 }]
logcollector-quickwit-searcher-2 quickwit [2m2025-09-21T06:48:21.873Z[0m [32m INFO[0m [1mleaf_search[0m[2m:[0m[1mmulti_index_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m[1msingle_doc_mapping_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m [2mquickwit_search::leaf[0m[2m:[0m [3mnum_docs[0m[2m=[0m3588691 [3mnum_splits[0m[2m=[0m1 [3msplit_offsets[0m[2m=[0m[SplitIdAndFooterOffsets { split_id: "01K5NF8E7YF6C5SM8HT3BQZ9BY", split_footer_start: 166991465, split_footer_end: 167008362, timestamp_start: Some(1758434986), timestamp_end: Some(1758436011), num_docs: 3588691 }]
logcollector-quickwit-searcher-2 quickwit [2m2025-09-21T06:48:21.887Z[0m [32m INFO[0m [1mleaf_search[0m[2m:[0m[1mmulti_index_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m[1msingle_doc_mapping_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m [2mquickwit_search::leaf[0m[2m:[0m [3mnum_docs[0m[2m=[0m3588691 [3mnum_splits[0m[2m=[0m1 [3msplit_offsets[0m[2m=[0m[SplitIdAndFooterOffsets { split_id: "01K5NF8E7YF6C5SM8HT3BQZ9BY", split_footer_start: 166991465, split_footer_end: 167008362, timestamp_start: Some(1758434986), timestamp_end: Some(1758436011), num_docs: 3588691 }]
logcollector-quickwit-searcher-2 quickwit [2m2025-09-21T06:48:54.865Z[0m [32m INFO[0m [1mleaf_search[0m[2m:[0m[1mmulti_index_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m[1msingle_doc_mapping_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m [2mquickwit_search::leaf[0m[2m:[0m [3mnum_docs[0m[2m=[0m3588691 [3mnum_splits[0m[2m=[0m1 [3msplit_offsets[0m[2m=[0m[SplitIdAndFooterOffsets { split_id: "01K5NF8E7YF6C5SM8HT3BQZ9BY", split_footer_start: 166991465, split_footer_end: 167008362, timestamp_start: Some(1758434986), timestamp_end: Some(1758436011), num_docs: 3588691 }]
logcollector-quickwit-searcher-2 quickwit [2m2025-09-21T06:48:54.880Z[0m [32m INFO[0m [1mleaf_search[0m[2m:[0m[1mmulti_index_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m[1msingle_doc_mapping_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m [2mquickwit_search::leaf[0m[2m:[0m [3mnum_docs[0m[2m=[0m3588691 [3mnum_splits[0m[2m=[0m1 [3msplit_offsets[0m[2m=[0m[SplitIdAndFooterOffsets { split_id: "01K5NF8E7YF6C5SM8HT3BQZ9BY", split_footer_start: 166991465, split_footer_end: 167008362, timestamp_start: Some(1758434986), timestamp_end: Some(1758436011), num_docs: 3588691 }]
logcollector-quickwit-searcher-2 quickwit [2m2025-09-21T06:49:27.837Z[0m [32m INFO[0m [1mleaf_search[0m[2m:[0m[1mmulti_index_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m[1msingle_doc_mapping_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m [2mquickwit_search::leaf[0m[2m:[0m [3mnum_docs[0m[2m=[0m3588691 [3mnum_splits[0m[2m=[0m1 [3msplit_offsets[0m[2m=[0m[SplitIdAndFooterOffsets { split_id: "01K5NF8E7YF6C5SM8HT3BQZ9BY", split_footer_start: 166991465, split_footer_end: 167008362, timestamp_start: Some(1758434986), timestamp_end: Some(1758436011), num_docs: 3588691 }]
logcollector-quickwit-searcher-2 quickwit [2m2025-09-21T06:49:27.841Z[0m [32m INFO[0m [1mleaf_search[0m[2m:[0m[1mmulti_index_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m[1msingle_doc_mapping_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m [2mquickwit_search::leaf[0m[2m:[0m [3mnum_docs[0m[2m=[0m3588691 [3mnum_splits[0m[2m=[0m1 [3msplit_offsets[0m[2m=[0m[SplitIdAndFooterOffsets { split_id: "01K5NF8E7YF6C5SM8HT3BQZ9BY", split_footer_start: 166991465, split_footer_end: 167008362, timestamp_start: Some(1758434986), timestamp_end: Some(1758436011), num_docs: 3588691 }]
logcollector-quickwit-searcher-4 quickwit [2m2025-09-21T06:36:02.136Z[0m [32m INFO[0m [1mleaf_search[0m[2m:[0m[1mmulti_index_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m[1msingle_doc_mapping_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m [2mquickwit_search::leaf[0m[2m:[0m [3mnum_docs[0m[2m=[0m3588691 [3mnum_splits[0m[2m=[0m1 [3msplit_offsets[0m[2m=[0m[SplitIdAndFooterOffsets { split_id: "01K5NF8E7YF6C5SM8HT3BQZ9BY", split_footer_start: 166991465, split_footer_end: 167008362, timestamp_start: Some(1758434986), timestamp_end: Some(1758436011), num_docs: 3588691 }]
logcollector-quickwit-searcher-2 quickwit [2m2025-09-21T06:50:00.932Z[0m [32m INFO[0m [1mleaf_search[0m[2m:[0m[1mmulti_index_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m[1msingle_doc_mapping_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m [2mquickwit_search::leaf[0m[2m:[0m [3mnum_docs[0m[2m=[0m3588691 [3mnum_splits[0m[2m=[0m1 [3msplit_offsets[0m[2m=[0m[SplitIdAndFooterOffsets { split_id: "01K5NF8E7YF6C5SM8HT3BQZ9BY", split_footer_start: 166991465, split_footer_end: 167008362, timestamp_start: Some(1758434986), timestamp_end: Some(1758436011), num_docs: 3588691 }]
logcollector-quickwit-searcher-2 quickwit [2m2025-09-21T06:50:00.935Z[0m [32m INFO[0m [1mleaf_search[0m[2m:[0m[1mmulti_index_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m[1msingle_doc_mapping_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m [2mquickwit_search::leaf[0m[2m:[0m [3mnum_docs[0m[2m=[0m3588691 [3mnum_splits[0m[2m=[0m1 [3msplit_offsets[0m[2m=[0m[SplitIdAndFooterOffsets { split_id: "01K5NF8E7YF6C5SM8HT3BQZ9BY", split_footer_start: 166991465, split_footer_end: 167008362, timestamp_start: Some(1758434986), timestamp_end: Some(1758436011), num_docs: 3588691 }]
logcollector-quickwit-searcher-2 quickwit [2m2025-09-21T06:50:33.756Z[0m [32m INFO[0m [1mleaf_search[0m[2m:[0m[1mmulti_index_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m[1msingle_doc_mapping_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m [2mquickwit_search::leaf[0m[2m:[0m [3mnum_docs[0m[2m=[0m3588691 [3mnum_splits[0m[2m=[0m1 [3msplit_offsets[0m[2m=[0m[SplitIdAndFooterOffsets { split_id: "01K5NF8E7YF6C5SM8HT3BQZ9BY", split_footer_start: 166991465, split_footer_end: 167008362, timestamp_start: Some(1758434986), timestamp_end: Some(1758436011), num_docs: 3588691 }]
logcollector-quickwit-searcher-2 quickwit [2m2025-09-21T06:50:33.782Z[0m [32m INFO[0m [1mleaf_search[0m[2m:[0m[1mmulti_index_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m[1msingle_doc_mapping_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m [2mquickwit_search::leaf[0m[2m:[0m [3mnum_docs[0m[2m=[0m3588691 [3mnum_splits[0m[2m=[0m1 [3msplit_offsets[0m[2m=[0m[SplitIdAndFooterOffsets { split_id: "01K5NF8E7YF6C5SM8HT3BQZ9BY", split_footer_start: 166991465, split_footer_end: 167008362, timestamp_start: Some(1758434986), timestamp_end: Some(1758436011), num_docs: 3588691 }]
logcollector-quickwit-searcher-2 quickwit [2m2025-09-21T06:57:14.508Z[0m [32m INFO[0m [1mleaf_search[0m[2m:[0m[1mmulti_index_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m[1msingle_doc_mapping_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m [2mquickwit_search::leaf[0m[2m:[0m [3mnum_docs[0m[2m=[0m3588691 [3mnum_splits[0m[2m=[0m1 [3msplit_offsets[0m[2m=[0m[SplitIdAndFooterOffsets { split_id: "01K5NF8E7YF6C5SM8HT3BQZ9BY", split_footer_start: 166991465, split_footer_end: 167008362, timestamp_start: Some(1758434986), timestamp_end: Some(1758436011), num_docs: 3588691 }]
logcollector-quickwit-searcher-2 quickwit [2m2025-09-21T06:57:14.726Z[0m [32m INFO[0m [1mleaf_search[0m[2m:[0m[1mmulti_index_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m[1msingle_doc_mapping_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m [2mquickwit_search::leaf[0m[2m:[0m [3mnum_docs[0m[2m=[0m3588691 [3mnum_splits[0m[2m=[0m1 [3msplit_offsets[0m[2m=[0m[SplitIdAndFooterOffsets { split_id: "01K5NF8E7YF6C5SM8HT3BQZ9BY", split_footer_start: 166991465, split_footer_end: 167008362, timestamp_start: Some(1758434986), timestamp_end: Some(1758436011), num_docs: 3588691 }]
logcollector-quickwit-searcher-4 quickwit [2m2025-09-21T06:36:16.030Z[0m [32m INFO[0m [1mleaf_search[0m[2m:[0m[1mmulti_index_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m[1msingle_doc_mapping_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m [2mquickwit_search::leaf[0m[2m:[0m [3mnum_docs[0m[2m=[0m3588691 [3mnum_splits[0m[2m=[0m1 [3msplit_offsets[0m[2m=[0m[SplitIdAndFooterOffsets { split_id: "01K5NF8E7YF6C5SM8HT3BQZ9BY", split_footer_start: 166991465, split_footer_end: 167008362, timestamp_start: Some(1758434986), timestamp_end: Some(1758436011), num_docs: 3588691 }]
logcollector-quickwit-searcher-4 quickwit [2m2025-09-21T06:36:25.799Z[0m [32m INFO[0m [1mleaf_search[0m[2m:[0m[1mmulti_index_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m[1msingle_doc_mapping_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m [2mquickwit_search::leaf[0m[2m:[0m [3mnum_docs[0m[2m=[0m3588691 [3mnum_splits[0m[2m=[0m1 [3msplit_offsets[0m[2m=[0m[SplitIdAndFooterOffsets { split_id: "01K5NF8E7YF6C5SM8HT3BQZ9BY", split_footer_start: 166991465, split_footer_end: 167008362, timestamp_start: Some(1758434986), timestamp_end: Some(1758436011), num_docs: 3588691 }]
logcollector-quickwit-searcher-4 quickwit [2m2025-09-21T06:36:25.821Z[0m [32m INFO[0m [1mleaf_search[0m[2m:[0m[1mmulti_index_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m[1msingle_doc_mapping_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m [2mquickwit_search::leaf[0m[2m:[0m [3mnum_docs[0m[2m=[0m3588691 [3mnum_splits[0m[2m=[0m1 [3msplit_offsets[0m[2m=[0m[SplitIdAndFooterOffsets { split_id: "01K5NF8E7YF6C5SM8HT3BQZ9BY", split_footer_start: 166991465, split_footer_end: 167008362, timestamp_start: Some(1758434986), timestamp_end: Some(1758436011), num_docs: 3588691 }]
logcollector-quickwit-searcher-4 quickwit [2m2025-09-21T06:36:58.647Z[0m [32m INFO[0m [1mleaf_search[0m[2m:[0m[1mmulti_index_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m[1msingle_doc_mapping_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m [2mquickwit_search::leaf[0m[2m:[0m [3mnum_docs[0m[2m=[0m3588691 [3mnum_splits[0m[2m=[0m1 [3msplit_offsets[0m[2m=[0m[SplitIdAndFooterOffsets { split_id: "01K5NF8E7YF6C5SM8HT3BQZ9BY", split_footer_start: 166991465, split_footer_end: 167008362, timestamp_start: Some(1758434986), timestamp_end: Some(1758436011), num_docs: 3588691 }]
logcollector-quickwit-searcher-4 quickwit [2m2025-09-21T06:36:58.653Z[0m [32m INFO[0m [1mleaf_search[0m[2m:[0m[1mmulti_index_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m[1msingle_doc_mapping_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m [2mquickwit_search::leaf[0m[2m:[0m [3mnum_docs[0m[2m=[0m3588691 [3mnum_splits[0m[2m=[0m1 [3msplit_offsets[0m[2m=[0m[SplitIdAndFooterOffsets { split_id: "01K5NF8E7YF6C5SM8HT3BQZ9BY", split_footer_start: 166991465, split_footer_end: 167008362, timestamp_start: Some(1758434986), timestamp_end: Some(1758436011), num_docs: 3588691 }]
logcollector-quickwit-searcher-4 quickwit [2m2025-09-21T06:37:31.311Z[0m [32m INFO[0m [1mleaf_search[0m[2m:[0m[1mmulti_index_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m[1msingle_doc_mapping_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m [2mquickwit_search::leaf[0m[2m:[0m [3mnum_docs[0m[2m=[0m3588691 [3mnum_splits[0m[2m=[0m1 [3msplit_offsets[0m[2m=[0m[SplitIdAndFooterOffsets { split_id: "01K5NF8E7YF6C5SM8HT3BQZ9BY", split_footer_start: 166991465, split_footer_end: 167008362, timestamp_start: Some(1758434986), timestamp_end: Some(1758436011), num_docs: 3588691 }]
logcollector-quickwit-searcher-4 quickwit [2m2025-09-21T06:37:31.502Z[0m [32m INFO[0m [1mleaf_search[0m[2m:[0m[1mmulti_index_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m[1msingle_doc_mapping_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m [2mquickwit_search::leaf[0m[2m:[0m [3mnum_docs[0m[2m=[0m3588691 [3mnum_splits[0m[2m=[0m1 [3msplit_offsets[0m[2m=[0m[SplitIdAndFooterOffsets { split_id: "01K5NF8E7YF6C5SM8HT3BQZ9BY", split_footer_start: 166991465, split_footer_end: 167008362, timestamp_start: Some(1758434986), timestamp_end: Some(1758436011), num_docs: 3588691 }]
logcollector-quickwit-searcher-2 quickwit [2m2025-09-21T06:57:54.044Z[0m [32m INFO[0m [1mleaf_search[0m[2m:[0m[1mmulti_index_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m[1msingle_doc_mapping_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m [2mquickwit_search::leaf[0m[2m:[0m [3mnum_docs[0m[2m=[0m3588691 [3mnum_splits[0m[2m=[0m1 [3msplit_offsets[0m[2m=[0m[SplitIdAndFooterOffsets { split_id: "01K5NF8E7YF6C5SM8HT3BQZ9BY", split_footer_start: 166991465, split_footer_end: 167008362, timestamp_start: Some(1758434986), timestamp_end: Some(1758436011), num_docs: 3588691 }]
logcollector-quickwit-searcher-2 quickwit [2m2025-09-21T06:57:54.180Z[0m [32m INFO[0m [1mleaf_search[0m[2m:[0m[1mmulti_index_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m[1msingle_doc_mapping_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m [2mquickwit_search::leaf[0m[2m:[0m [3mnum_docs[0m[2m=[0m3588691 [3mnum_splits[0m[2m=[0m1 [3msplit_offsets[0m[2m=[0m[SplitIdAndFooterOffsets { split_id: "01K5NF8E7YF6C5SM8HT3BQZ9BY", split_footer_start: 166991465, split_footer_end: 167008362, timestamp_start: Some(1758434986), timestamp_end: Some(1758436011), num_docs: 3588691 }]
logcollector-quickwit-searcher-2 quickwit [2m2025-09-21T06:58:27.165Z[0m [32m INFO[0m [1mleaf_search[0m[2m:[0m[1mmulti_index_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m[1msingle_doc_mapping_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m [2mquickwit_search::leaf[0m[2m:[0m [3mnum_docs[0m[2m=[0m3588691 [3mnum_splits[0m[2m=[0m1 [3msplit_offsets[0m[2m=[0m[SplitIdAndFooterOffsets { split_id: "01K5NF8E7YF6C5SM8HT3BQZ9BY", split_footer_start: 166991465, split_footer_end: 167008362, timestamp_start: Some(1758434986), timestamp_end: Some(1758436011), num_docs: 3588691 }]
logcollector-quickwit-searcher-2 quickwit [2m2025-09-21T06:58:27.245Z[0m [32m INFO[0m [1mleaf_search[0m[2m:[0m[1mmulti_index_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m[1msingle_doc_mapping_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m [2mquickwit_search::leaf[0m[2m:[0m [3mnum_docs[0m[2m=[0m3588691 [3mnum_splits[0m[2m=[0m1 [3msplit_offsets[0m[2m=[0m[SplitIdAndFooterOffsets { split_id: "01K5NF8E7YF6C5SM8HT3BQZ9BY", split_footer_start: 166991465, split_footer_end: 167008362, timestamp_start: Some(1758434986), timestamp_end: Some(1758436011), num_docs: 3588691 }]
logcollector-quickwit-searcher-2 quickwit [2m2025-09-21T06:59:01.151Z[0m [32m INFO[0m [1mleaf_search[0m[2m:[0m[1mmulti_index_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m[1msingle_doc_mapping_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m [2mquickwit_search::leaf[0m[2m:[0m [3mnum_docs[0m[2m=[0m3588691 [3mnum_splits[0m[2m=[0m1 [3msplit_offsets[0m[2m=[0m[SplitIdAndFooterOffsets { split_id: "01K5NF8E7YF6C5SM8HT3BQZ9BY", split_footer_start: 166991465, split_footer_end: 167008362, timestamp_start: Some(1758434986), timestamp_end: Some(1758436011), num_docs: 3588691 }]
logcollector-quickwit-searcher-2 quickwit [2m2025-09-21T06:59:01.729Z[0m [32m INFO[0m [1mleaf_search[0m[2m:[0m[1mmulti_index_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m[1msingle_doc_mapping_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m [2mquickwit_search::leaf[0m[2m:[0m [3mnum_docs[0m[2m=[0m3588691 [3mnum_splits[0m[2m=[0m1 [3msplit_offsets[0m[2m=[0m[SplitIdAndFooterOffsets { split_id: "01K5NF8E7YF6C5SM8HT3BQZ9BY", split_footer_start: 166991465, split_footer_end: 167008362, timestamp_start: Some(1758434986), timestamp_end: Some(1758436011), num_docs: 3588691 }]
logcollector-quickwit-searcher-4 quickwit [2m2025-09-21T06:38:04.586Z[0m [32m INFO[0m [1mleaf_search[0m[2m:[0m[1mmulti_index_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m[1msingle_doc_mapping_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m [2mquickwit_search::leaf[0m[2m:[0m [3mnum_docs[0m[2m=[0m3588691 [3mnum_splits[0m[2m=[0m1 [3msplit_offsets[0m[2m=[0m[SplitIdAndFooterOffsets { split_id: "01K5NF8E7YF6C5SM8HT3BQZ9BY", split_footer_start: 166991465, split_footer_end: 167008362, timestamp_start: Some(1758434986), timestamp_end: Some(1758436011), num_docs: 3588691 }]
logcollector-quickwit-searcher-4 quickwit [2m2025-09-21T06:38:04.594Z[0m [32m INFO[0m [1mleaf_search[0m[2m:[0m[1mmulti_index_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m[1msingle_doc_mapping_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m [2mquickwit_search::leaf[0m[2m:[0m [3mnum_docs[0m[2m=[0m3588691 [3mnum_splits[0m[2m=[0m1 [3msplit_offsets[0m[2m=[0m[SplitIdAndFooterOffsets { split_id: "01K5NF8E7YF6C5SM8HT3BQZ9BY", split_footer_start: 166991465, split_footer_end: 167008362, timestamp_start: Some(1758434986), timestamp_end: Some(1758436011), num_docs: 3588691 }]
logcollector-quickwit-searcher-4 quickwit [2m2025-09-21T06:38:37.642Z[0m [32m INFO[0m [1mleaf_search[0m[2m:[0m[1mmulti_index_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m[1msingle_doc_mapping_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m [2mquickwit_search::leaf[0m[2m:[0m [3mnum_docs[0m[2m=[0m3588691 [3mnum_splits[0m[2m=[0m1 [3msplit_offsets[0m[2m=[0m[SplitIdAndFooterOffsets { split_id: "01K5NF8E7YF6C5SM8HT3BQZ9BY", split_footer_start: 166991465, split_footer_end: 167008362, timestamp_start: Some(1758434986), timestamp_end: Some(1758436011), num_docs: 3588691 }]
logcollector-quickwit-searcher-4 quickwit [2m2025-09-21T06:38:37.644Z[0m [32m INFO[0m [1mleaf_search[0m[2m:[0m[1mmulti_index_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m[1msingle_doc_mapping_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m [2mquickwit_search::leaf[0m[2m:[0m [3mnum_docs[0m[2m=[0m3588691 [3mnum_splits[0m[2m=[0m1 [3msplit_offsets[0m[2m=[0m[SplitIdAndFooterOffsets { split_id: "01K5NF8E7YF6C5SM8HT3BQZ9BY", split_footer_start: 166991465, split_footer_end: 167008362, timestamp_start: Some(1758434986), timestamp_end: Some(1758436011), num_docs: 3588691 }]
logcollector-quickwit-searcher-2 quickwit [2m2025-09-21T06:59:34.845Z[0m [32m INFO[0m [1mleaf_search[0m[2m:[0m[1mmulti_index_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m[1msingle_doc_mapping_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m [2mquickwit_search::leaf[0m[2m:[0m [3mnum_docs[0m[2m=[0m3588691 [3mnum_splits[0m[2m=[0m1 [3msplit_offsets[0m[2m=[0m[SplitIdAndFooterOffsets { split_id: "01K5NF8E7YF6C5SM8HT3BQZ9BY", split_footer_start: 166991465, split_footer_end: 167008362, timestamp_start: Some(1758434986), timestamp_end: Some(1758436011), num_docs: 3588691 }]
logcollector-quickwit-searcher-2 quickwit [2m2025-09-21T06:59:34.872Z[0m [32m INFO[0m [1mleaf_search[0m[2m:[0m[1mmulti_index_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m[1msingle_doc_mapping_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m [2mquickwit_search::leaf[0m[2m:[0m [3mnum_docs[0m[2m=[0m3588691 [3mnum_splits[0m[2m=[0m1 [3msplit_offsets[0m[2m=[0m[SplitIdAndFooterOffsets { split_id: "01K5NF8E7YF6C5SM8HT3BQZ9BY", split_footer_start: 166991465, split_footer_end: 167008362, timestamp_start: Some(1758434986), timestamp_end: Some(1758436011), num_docs: 3588691 }]
logcollector-quickwit-searcher-2 quickwit [2m2025-09-21T07:00:10.242Z[0m [32m INFO[0m [1mleaf_search[0m[2m:[0m[1mmulti_index_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m[1msingle_doc_mapping_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m [2mquickwit_search::leaf[0m[2m:[0m [3mnum_docs[0m[2m=[0m3588691 [3mnum_splits[0m[2m=[0m1 [3msplit_offsets[0m[2m=[0m[SplitIdAndFooterOffsets { split_id: "01K5NF8E7YF6C5SM8HT3BQZ9BY", split_footer_start: 166991465, split_footer_end: 167008362, timestamp_start: Some(1758434986), timestamp_end: Some(1758436011), num_docs: 3588691 }]
logcollector-quickwit-searcher-2 quickwit [2m2025-09-21T07:00:10.346Z[0m [32m INFO[0m [1mleaf_search[0m[2m:[0m[1mmulti_index_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m[1msingle_doc_mapping_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m [2mquickwit_search::leaf[0m[2m:[0m [3mnum_docs[0m[2m=[0m3588691 [3mnum_splits[0m[2m=[0m1 [3msplit_offsets[0m[2m=[0m[SplitIdAndFooterOffsets { split_id: "01K5NF8E7YF6C5SM8HT3BQZ9BY", split_footer_start: 166991465, split_footer_end: 167008362, timestamp_start: Some(1758434986), timestamp_end: Some(1758436011), num_docs: 3588691 }]
logcollector-quickwit-searcher-2 quickwit [2m2025-09-21T07:00:45.429Z[0m [32m INFO[0m [1mleaf_search[0m[2m:[0m[1mmulti_index_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m[1msingle_doc_mapping_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m [2mquickwit_search::leaf[0m[2m:[0m [3mnum_docs[0m[2m=[0m3588691 [3mnum_splits[0m[2m=[0m1 [3msplit_offsets[0m[2m=[0m[SplitIdAndFooterOffsets { split_id: "01K5NF8E7YF6C5SM8HT3BQZ9BY", split_footer_start: 166991465, split_footer_end: 167008362, timestamp_start: Some(1758434986), timestamp_end: Some(1758436011), num_docs: 3588691 }]
logcollector-quickwit-searcher-2 quickwit [2m2025-09-21T07:00:45.562Z[0m [32m INFO[0m [1mleaf_search[0m[2m:[0m[1mmulti_index_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m[1msingle_doc_mapping_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m [2mquickwit_search::leaf[0m[2m:[0m [3mnum_docs[0m[2m=[0m3588691 [3mnum_splits[0m[2m=[0m1 [3msplit_offsets[0m[2m=[0m[SplitIdAndFooterOffsets { split_id: "01K5NF8E7YF6C5SM8HT3BQZ9BY", split_footer_start: 166991465, split_footer_end: 167008362, timestamp_start: Some(1758434986), timestamp_end: Some(1758436011), num_docs: 3588691 }]
logcollector-quickwit-indexer-3 quickwit [2m2025-09-21T06:26:54.334Z[0m [32m INFO[0m [2mquickwit_indexing::actors::merge_planner[0m[2m:[0m schedule merge operation [3mmerge_operation[0m[2m=[0mMerge(operation_type=Merge, merged_split_id=01K5NF8E7YF6C5SM8HT3BQZ9BY,splits=[01K5NF8DJ2W8XQBT4QZ9NS5MHM,01K5NF5BPVCNCZ2CSAN1TG4V5W,01K5NF290N9EFFYHEYA7H2WSC2,01K5NEZ7BNRB3TD1AF5XK9KRXN,01K5NEW4KBT4AWZ6AVXYQPHZYY,01K5NES20TZBEM00M9SCZAAR5E,01K5NEP07H06P8S59G9D7Z00CV,01K5NEJYBSJCZHAWKR1JGE3774,01K5NEFTNR440J0T213QRYMRTS,01K5NED3352S9GWXXZYTS55AVK,])
logcollector-quickwit-indexer-3 quickwit [2m2025-09-21T06:26:54.334Z[0m [32m INFO[0m [1mmerge[0m[1m{[0m[3mmerge_split_id[0m[2m=[0m01K5NF8E7YF6C5SM8HT3BQZ9BY [3msplit_ids[0m[2m=[0m["01K5NF8DJ2W8XQBT4QZ9NS5MHM", "01K5NF5BPVCNCZ2CSAN1TG4V5W", "01K5NF290N9EFFYHEYA7H2WSC2", "01K5NEZ7BNRB3TD1AF5XK9KRXN", "01K5NEW4KBT4AWZ6AVXYQPHZYY", "01K5NES20TZBEM00M9SCZAAR5E", "01K5NEP07H06P8S59G9D7Z00CV", "01K5NEJYBSJCZHAWKR1JGE3774", "01K5NEFTNR440J0T213QRYMRTS", "01K5NED3352S9GWXXZYTS55AVK"] [3mtyp[0m[2m=[0mMerge[1m}[0m[2m:[0m[1mmerge_split_downloader[0m[2m:[0m [2mquickwit_indexing::actors::merge_split_downloader[0m[2m:[0m download-merge-splits [3mdir[0m[2m=[0m/quickwit/qwdata/indexing/ray-logs%01K5MSMHH3G5P5E3GYZXNKCAE6%_ingest-source%01K5NDSAEKFMEMF664QC9WWS44%Lew1uR/merge%503dsv
logcollector-quickwit-indexer-3 quickwit [2m2025-09-21T06:26:57.036Z[0m [32m INFO[0m [1mmerge[0m[1m{[0m[3mmerge_split_id[0m[2m=[0m01K5NF8E7YF6C5SM8HT3BQZ9BY [3msplit_ids[0m[2m=[0m["01K5NF8DJ2W8XQBT4QZ9NS5MHM", "01K5NF5BPVCNCZ2CSAN1TG4V5W", "01K5NF290N9EFFYHEYA7H2WSC2", "01K5NEZ7BNRB3TD1AF5XK9KRXN", "01K5NEW4KBT4AWZ6AVXYQPHZYY", "01K5NES20TZBEM00M9SCZAAR5E", "01K5NEP07H06P8S59G9D7Z00CV", "01K5NEJYBSJCZHAWKR1JGE3774", "01K5NEFTNR440J0T213QRYMRTS", "01K5NED3352S9GWXXZYTS55AVK"] [3mtyp[0m[2m=[0mMerge[1m}[0m[2m:[0m[1mmerge_executor[0m[2m:[0m [2mquickwit_indexing::actors::merge_executor[0m[2m:[0m merge-operation-success [3mmerged_num_docs[0m[2m=[0m3588691 [3melapsed_secs[0m[2m=[0m2.701556 [3moperation_type[0m[2m=[0mMerge
logcollector-quickwit-indexer-3 quickwit [2m2025-09-21T06:26:59.669Z[0m [32m INFO[0m [1mmerge[0m[1m{[0m[3mmerge_split_id[0m[2m=[0m01K5NF8E7YF6C5SM8HT3BQZ9BY [3msplit_ids[0m[2m=[0m["01K5NF8DJ2W8XQBT4QZ9NS5MHM", "01K5NF5BPVCNCZ2CSAN1TG4V5W", "01K5NF290N9EFFYHEYA7H2WSC2", "01K5NEZ7BNRB3TD1AF5XK9KRXN", "01K5NEW4KBT4AWZ6AVXYQPHZYY", "01K5NES20TZBEM00M9SCZAAR5E", "01K5NEP07H06P8S59G9D7Z00CV", "01K5NEJYBSJCZHAWKR1JGE3774", "01K5NEFTNR440J0T213QRYMRTS", "01K5NED3352S9GWXXZYTS55AVK"] [3mtyp[0m[2m=[0mMerge[1m}[0m[2m:[0m[1mpublisher[0m[1m{[0m[3msplit_update[0m[2m=[0mSplitsUpdate { index_id: "ray-logs", new_splits: "01K5NF8E7YF6C5SM8HT3BQZ9BY", checkpoint_delta: None }[1m}[0m[2m:[0m [2mquickwit_indexing::actors::publisher[0m[2m:[0m publish-new-splits
logcollector-quickwit-searcher-4 quickwit [2m2025-09-21T06:39:10.628Z[0m [32m INFO[0m [1mleaf_search[0m[2m:[0m[1mmulti_index_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m[1msingle_doc_mapping_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m [2mquickwit_search::leaf[0m[2m:[0m [3mnum_docs[0m[2m=[0m3588691 [3mnum_splits[0m[2m=[0m1 [3msplit_offsets[0m[2m=[0m[SplitIdAndFooterOffsets { split_id: "01K5NF8E7YF6C5SM8HT3BQZ9BY", split_footer_start: 166991465, split_footer_end: 167008362, timestamp_start: Some(1758434986), timestamp_end: Some(1758436011), num_docs: 3588691 }]
logcollector-quickwit-searcher-4 quickwit [2m2025-09-21T06:39:10.633Z[0m [32m INFO[0m [1mleaf_search[0m[2m:[0m[1mmulti_index_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m[1msingle_doc_mapping_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m [2mquickwit_search::leaf[0m[2m:[0m [3mnum_docs[0m[2m=[0m3588691 [3mnum_splits[0m[2m=[0m1 [3msplit_offsets[0m[2m=[0m[SplitIdAndFooterOffsets { split_id: "01K5NF8E7YF6C5SM8HT3BQZ9BY", split_footer_start: 166991465, split_footer_end: 167008362, timestamp_start: Some(1758434986), timestamp_end: Some(1758436011), num_docs: 3588691 }]
logcollector-quickwit-searcher-4 quickwit [2m2025-09-21T06:39:43.629Z[0m [32m INFO[0m [1mleaf_search[0m[2m:[0m[1mmulti_index_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m[1msingle_doc_mapping_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m [2mquickwit_search::leaf[0m[2m:[0m [3mnum_docs[0m[2m=[0m3588691 [3mnum_splits[0m[2m=[0m1 [3msplit_offsets[0m[2m=[0m[SplitIdAndFooterOffsets { split_id: "01K5NF8E7YF6C5SM8HT3BQZ9BY", split_footer_start: 166991465, split_footer_end: 167008362, timestamp_start: Some(1758434986), timestamp_end: Some(1758436011), num_docs: 3588691 }]
logcollector-quickwit-searcher-4 quickwit [2m2025-09-21T06:39:43.638Z[0m [32m INFO[0m [1mleaf_search[0m[2m:[0m[1mmulti_index_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m[1msingle_doc_mapping_leaf_search[0m[1m{[0m[3mindex[0m[2m=[0m["ray-logs"][1m}[0m[2m:[0m [2mquickwit_search::leaf[0m[2m:[0m [3mnum_docs[0m[2m=[0m3588691 [3mnum_splits[0m[2m=[0m1 [3msplit_offsets[0m[2m=[0m[SplitIdAndFooterOffsets { split_id: "01K5NF8E7YF6C5SM8HT3BQZ9BY", split_footer_start: 166991465, split_footer_end: 167008362, timestamp_start: Some(1758434986), timestamp_end: Some(1758436011), num_docs: 3588691 }]
logcollector-quickwit-searcher-2 quickw

Sep 21 '25 07:09 shrijeet

Hey @shrijeet, https://github.com/quickwit-oss/quickwit/pull/5908 is unrelated. I saw your message on discord, we can continue the conversation there.

Sep 21 '25 09:09 fmassot

Fix in #5919

Sep 24 '25 19:09 shrijeet