client-java icon indicating copy to clipboard operation
client-java copied to clipboard

Region Exception occurred too many sst files are ingesting

Open tank-plus opened this issue 3 years ago • 2 comments

Bug Report

1. Describe the bug

Multiple thread ingest batch kvs into a region.

2. Minimal reproduce step (Required)

batch kvs use ImporterClient do ingest method to imort data into tikv

3. What did you see instead (Required)

2022-11-17 06:45:42,079 WARN org.tikv.common.importer.ImporterClient [] - ingest failed. org.tikv.common.exception.RegionException: Region Exception occurred too many sst files are ingesting at org.tikv.common.importer.ImporterStoreClient.multiIngest(ImporterStoreClient.java:158) ~[?:?] at org.tikv.common.importer.ImporterClient.ingestWithRetry(ImporterClient.java:266) ~[?:?] at org.tikv.common.importer.ImporterClient.ingestWithRetry(ImporterClient.java:320) ~[?:?] at org.tikv.common.importer.ImporterClient.ingestWithRetry(ImporterClient.java:320) ~[?:?] at org.tikv.common.importer.ImporterClient.ingest(ImporterClient.java:261) ~[?:?] at org.tikv.common.importer.ImporterClient.write(ImporterClient.java:136) ~[?:?] at com.fabarta.loader.tikv.TiKVClient.doIngest(TiKVClient.java:277) ~[?:?] at com.fabarta.loader.tikv.TiKVClient.groupBatchPut(TiKVClient.java:261) ~[?:?] at com.fabarta.loader.tikv.TiKVImporter.importData(TiKVImporter.java:71) ~[?:?] at com.fabarta.loader.core.WriteTiKVTask.flushBuffer(WriteTiKVTask.java:22) ~[?:?] at com.fabarta.loader.core.WriteTiKVTask.run(WriteTiKVTask.java:29) ~[?:?] at com.fabarta.loader.core.FabartaLoader.lambda$flushCache$1(FabartaLoader.java:307) ~[?:?] at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) [?:?] at java.util.concurrent.FutureTask.run(Unknown Source) [?:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) [?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) [?:?] at java.lang.Thread.run(Unknown Source) [?:?]

4. What did you expect to see? (Required)

5. What are your Java Client and TiKV versions? (Required)

3.3.0

  • Client Java:
  • TiKV:

tank-plus avatar Nov 24 '22 11:11 tank-plus

Can you make sure don't ingest too many set files and don't ingest the same region concurrently.

sunxiaoguang avatar Nov 26 '22 13:11 sunxiaoguang

Can you make sure don't ingest too many set files and don't ingest the same region concurrently.

I think that SDK should handle this issue.

liangyuanpeng avatar Jan 01 '23 02:01 liangyuanpeng