arcus-java-client icon indicating copy to clipboard operation
arcus-java-client copied to clipboard

INTERNAL: make piped insert operations process synchronously

Open oliviarla opened this issue 10 months ago โ€ข 1 comments

๐Ÿ”— Related Issue

  • https://github.com/naver/arcus-java-client/pull/795 ๋ฅผ ๋Œ€์ฒดํ•  ์ˆ˜ ์žˆ๋Š” PR์ž…๋‹ˆ๋‹ค.
  • PR ์ƒ์„ฑ ์ด์œ ๋Š” https://github.com/naver/arcus-java-client/pull/795#issuecomment-2720428907 ์ฐธ๊ณ  ๋ฐ”๋ž๋‹ˆ๋‹ค.

โŒจ๏ธ What I did

  • ArcusClient ํด๋ž˜์Šค

    • syncCollectionPipedInsert() / syncCollectionPipedUpdate() ๋ฉ”์„œ๋“œ
      • Operation ๊ฐ์ฒด๊ฐ€ 2๊ฐœ ์ด์ƒ ์ƒ์„ฑ๋˜๋Š” ๊ฒฝ์šฐ, Operation ๊ฐ์ฒด์— next Operation ๋ณ€์ˆ˜๋ฅผ ๋‹ด์•„ ๋‘๋„๋ก ํ•ฉ๋‹ˆ๋‹ค.

      • callback.complete() ์ฒ˜๋ฆฌ ๋ฐฉ๋ฒ•

        • ํ˜„์žฌ Operation์— ๋‹ด๊ธด ๋ชจ๋“  ์š”์ฒญ์ด ์„ฑ๊ณตํ•  ๊ฒฝ์šฐ, next Operation์„ ํ์™€ Future์— ์ถ”๊ฐ€ํ•ฉ๋‹ˆ๋‹ค.

        • ํ˜„์žฌ Operation์— ๋‹ด๊ธด ์š”์ฒญ์ด ์‹คํŒจํ•  ๊ฒฝ์šฐ, ๋‹ค์Œ ์ผ€์ด์Šค๋กœ ๋‚˜๋‰ฉ๋‹ˆ๋‹ค. (cancel/error ๋ฐœ์ƒ ์‹œ ์ง€๊ธˆ์€ failedResult๋ฅผ ์กฐํšŒํ•  ์ˆ˜ ์—†์œผ๋‚˜, ๋ณธ PR ๋ฐ˜์˜๋œ ํ›„ ์กฐํšŒ ๊ธฐ๋Šฅ์„ ์ถ”๊ฐ€ํ•˜๋Š” PR์„ ์˜ฌ๋ฆฌ๊ฒ ์Šต๋‹ˆ๋‹ค.)

          • cancel๋กœ ์ธํ•ด ์ผ๋ถ€ item ๊ฒฐ๊ณผ๋ฅผ ๊ธฐ๋‹ค๋ฆฌ์ง€ ์•Š์Œ -> Future#failedResult์— ๊ฒฐ๊ณผ๊ฐ€ ์˜ค์ง€ ์•Š์€ ๊ฐ€์žฅ ์ฒซ ์•„์ดํ…œ์— undefined ์ƒํƒœ๋กœ ์ถ”๊ฐ€ํ•˜๊ณ , ๋‹ค์Œ op์˜ ์ฒซ๋ฒˆ์งธ ์•„์ดํ…œ์— stopped ์ƒํƒœ ์ถ”๊ฐ€ (์ด๋ฅผ ํ†ตํ•ด undefined ~ stopped - 1 ์ธ๋ฑ์Šค๋“ค์€ ์ˆ˜ํ–‰ ์—ฌ๋ถ€๋ฅผ ์‘์šฉ์—์„œ ๋‹ค์‹œ ์ฒดํฌํ•  ์ˆ˜ ์žˆ์Œ)
          • error๋กœ ์ธํ•ด ์ผ๋ถ€ item์ด ์‹คํ–‰๋˜์ง€ ์•Š์Œ -> Future#failedResult์— ์‹คํ–‰๋˜์ง€ ์•Š์€ ์ตœ์ดˆ์˜ item์— stopped ์ƒํƒœ ์ถ”๊ฐ€
          • item์ด ๋ชจ๋‘ ์ˆ˜ํ–‰๋˜์—ˆ์ง€๋งŒ not found ๋“ฑ ์‹คํŒจ ์กด์žฌ -> Future#failedResult์— ์‹คํŒจํ•œ ์ƒํƒœ๋“ค ์ถ”๊ฐ€ํ•˜๊ณ , ๋‹ค์Œ op ์žˆ์œผ๋ฉด stopped ์ƒํƒœ ์ถ”๊ฐ€
        • cancel๋กœ ์ธํ•ด ์—ฐ์‚ฐ ์‘๋‹ต์„ ๋” ์ด์ƒ ๊ธฐ๋‹ค๋ฆฌ์ง€ ์•Š๋Š” ์ƒํƒœ์ด๋”๋ผ๋„, ๋‚˜์ค‘์— ์‘๋‹ต์„ ์ฒ˜๋ฆฌํ•  ๋•Œ ์„ฑ๊ณต ์‘๋‹ต์ด ์˜ค๋ฉด failedResult์—์„œ ๋ฐ์ดํ„ฐ๋ฅผ ์ œ๊ฑฐํ•˜๊ณ , ์‹คํŒจ ์‘๋‹ต์ด ์˜ค๋ฉด failedResult์— ์žˆ๋˜ undefined ์ƒํƒœ๋ฅผ ์‹คํŒจ ์›์ธ์œผ๋กœ ๊ต์ฒดํ•œ๋‹ค.

  • SingleKeyPipeOperationImpl ํด๋ž˜์Šค

    • pipedCancel() ๋ฉ”์„œ๋“œ: cancel ์š”์ฒญ์ด ๋“ค์–ด์™”์„ ๋•Œ ํ˜„์žฌ Operation cancel์— ์‹คํŒจํ•˜๋ฉด, next Operation cancel์„ ์‹œ๋„ํ•œ ํ›„ ๊ฒฐ๊ณผ๋ฅผ ๋ฐ˜ํ™˜ํ•ฉ๋‹ˆ๋‹ค.
  • PipedCollectionFuture ํด๋ž˜์Šค

    • removeEachResult() ๋ฉ”์„œ๋“œ: cancel๋œ ํ›„ ์„ฑ๊ณต ์‘๋‹ต์„ ๋ฐ›์•˜์„ ๋•Œ failedResult์—์„œ ๋ฐ์ดํ„ฐ๋ฅผ ์ œ๊ฑฐํ•˜๋„๋ก ํ•ฉ๋‹ˆ๋‹ค.
    • ๊ทธ์™ธ ๋ฉ”์„œ๋“œ๋“ค์„ ์ˆ˜์ •ํ–ˆ์Šต๋‹ˆ๋‹ค.

oliviarla avatar Mar 14 '25 08:03 oliviarla

@jhpark816 ๋ฆฌ๋ทฐ ๋ฐ˜์˜ํ•˜์˜€์Šต๋‹ˆ๋‹ค.

oliviarla avatar Mar 27 '25 07:03 oliviarla

@uhm0311 ๋ฆฌ๋ทฐ ๋ฐ”๋ž๋‹ˆ๋‹ค.

jhpark816 avatar Apr 02 '25 01:04 jhpark816

ํ˜„์žฌ ๋‹ค์Œ๊ณผ ๊ฐ™์ด ๋™์‹œ์„ฑ ๋ฌธ์ œ ํ•ด๊ฒฐํ•ด๋‘” ์ƒํƒœ์ž…๋‹ˆ๋‹ค.

  • receivedStatus๊ฐ€ IO/Worker ์Šค๋ ˆ๋“œ์—์„œ ์ค‘๋ณต ํ˜ธ์ถœ๋˜๋Š” ๋ฌธ์ œ

    • CANCEL/์ตœ์ข… ์‹คํŒจ ์ •๋ณด๊ฐ€ ๋“ค์–ด๊ฐ€๊ณ  ์–ด๋–ค ์ •๋ณด๊ฐ€ ๋จผ์ € ๋“ค์–ด๊ฐ€๋„, complete ์ฝœ๋ฐฑ ๋ฉ”์„œ๋“œ์—์„œ ์ฒ˜๋ฆฌ๋Š” ๋ฌธ์ œ ์—†๋‹ค. ์ฆ‰, ์š”์ฒญ์ด CANCEL ๋˜์—ˆ์–ด๋„ FAIL_END ์ƒํƒœ๊ฐ€ Future#OperationStatus ์— ์ €์žฅ๋˜๋Š” ๊ฒƒ์— ๋ฌธ์ œ ์—†๋‹ค๊ณ  ํŒ๋‹จํ•œ๋‹ค.
  • addEachResult(gotStatus ์ฝœ๋ฐฑ) ๋ฉ”์„œ๋“œ๊ฐ€ cancel๋œ ํ›„์—๋„ IO ์Šค๋ ˆ๋“œ์— ์˜ํ•ด ํ˜ธ์ถœ๋˜์–ด ๊ธฐ์กด ๊ฐ’(UNDEFINED) ๋ฎ์–ด์“ฐ๋Š” ๋ฌธ์ œ

    • handleLine์—์„œ ํ˜ธ์ถœ๋˜๋Š” gotStatus ์ฝœ๋ฐฑ ๋ฉ”์„œ๋“œ๊ฐ€ cancel ์ƒํƒœ๋ฉด ๋ถˆ๋ฆฌ์ง€ ์•Š๋„๋ก ์ˆ˜์ •ํ•˜์˜€๋‹ค.

ํ•˜์ง€๋งŒ ์—ฌ์ „ํžˆ ๋ฌธ์ œ๊ฐ€ ๋  ๋ถ€๋ถ„์ด ๋‚จ์•„์žˆ๋Š”๋ฐ์š”, ๋‹ค์Œ๊ณผ ๊ฐ™์ด cancel ๋ณ€์ˆ˜์— ๋Œ€ํ•œ ๋™์‹œ์„ฑ ์ œ์–ด๊ฐ€ ๋˜์–ด์žˆ์ง€ ์•Š๊ธฐ ๋•Œ๋ฌธ์ž…๋‹ˆ๋‹ค. ๋”ฐ๋ผ์„œ iscancelled์™€ cancel ๋ฉ”์„œ๋“œ ๊ฐ„์— ๋™์‹œ ์ ‘๊ทผ์„ ํ•˜์ง€ ๋ชปํ•˜๋„๋ก synchronized ๋ฉ”์„œ๋“œ๋กœ ๋งŒ๋“œ๋Š” ๊ฒƒ์— ๋Œ€ํ•ด ๊ณ ๋ฏผ ์ค‘์ž…๋‹ˆ๋‹ค. image

oliviarla avatar Apr 18 '25 01:04 oliviarla

@oliviarla ~op์˜ cancelled ํ”Œ๋ž˜๊ทธ๊ฐ€ true์ด๋ฉด ๊ทธ op์˜ ์ƒํƒœ๋ฅผ WRITING ์ƒํƒœ๋กœ ๋ณ€๊ฒฝํ•˜์ง€ ๋ชปํ•˜๋ฏ€๋กœ, ์œ„์˜ ๊ณ ๋ฏผ ์‚ฌํ•ญ์€ ๋ฌธ์ œ๋˜์ง€ ์•Š์„ ๊ฒƒ์ž…๋‹ˆ๋‹ค. ๊ฒ€ํ†  ๋ฐ”๋ž๋‹ˆ๋‹ค.~

jhpark816 avatar Apr 18 '25 01:04 jhpark816

@jhpark816 @uhm0311 PR ์ฒซ ์ฝ”๋ฉ˜ํŠธ๋ฅผ ํ˜„์žฌ ํ˜•ํƒœ์— ๋งž๊ฒŒ ์ˆ˜์ •ํ–ˆ์Šต๋‹ˆ๋‹ค. ๋ณต์žกํ•˜๊ฒŒ failedResult์— ์•„์ดํ…œ๋“ค์„ ์ถ”๊ฐ€ํ•˜๋Š” ๋กœ์ง์€ ๋ณธ PR๊ณผ ๋ฌด๊ด€ํ•˜๊ฒŒ ์ถ”ํ›„ ๋…ผ์˜ํ•  ์˜ˆ์ •์ž…๋‹ˆ๋‹ค. ๋ฆฌ๋ทฐ ๋ถ€ํƒ๋“œ๋ฆฝ๋‹ˆ๋‹ค.

oliviarla avatar Apr 23 '25 07:04 oliviarla

https://github.com/jam2in/arcus-works/issues/716 ์™„๋ฃŒ ํ›„ ๋‹ค์‹œ ์ง„ํ–‰ํ•˜๊ฒ ์Šต๋‹ˆ๋‹ค.

oliviarla avatar May 02 '25 00:05 oliviarla

@oliviarla ๋ฆฌ๋ทฐ ์ฝ”๋ฉ˜ํŠธ ๋‹ฌ์•˜์œผ๋‹ˆ, ํ™•์ธ ๋ฐ”๋ž๋‹ˆ๋‹ค.

jhpark816 avatar Aug 19 '25 01:08 jhpark816