zstd-ruby icon indicating copy to clipboard operation
zstd-ruby copied to clipboard

Add reusable context for performance

Open kreynolds opened this issue 4 months ago • 1 comments

Pull Request Summary: Using the Context Feature in zstd-ruby

The zstd-ruby gem provides a Zstd::Context class for efficient, reusable compression and decompression, offering 2-3x faster performance and memory savings compared to module methods. Here's how to use it:

  1. Initialize a Context:

    ctx = Zstd::Context.new(level: 6) # Optional: set compression level (default 6)
    ctx = Zstd::Context.new(level: 6, dict: dictionary) # With dictionary for better compression
    
  2. Compress and Decompress:

    compressed = ctx.compress(data) # Compress data
    original = ctx.decompress(compressed) # Decompress data
    
  3. Specialized Contexts (for memory optimization):

    • Compression-only: cctx = Zstd::CContext.new(level: 6); cctx.compress(data)
    • Decompression-only: dctx = Zstd::DContext.new; dctx.decompress(compressed)
  4. Benefits:

    • Performance: 2-3x faster for repeated operations.
    • Memory Efficiency: Reuses context, with specialized CContext/DContext saving ~50% memory.
    • Thread-Safe: Each context instance is independent.
  5. Use Case: Ideal for multiple compression/decompression tasks or dictionary-based workflows.

Sample output from benchmarks/context_reuse.rb showing 3.5x speed improvement:

Comparison:
CContext compress (reused):   719483.0 i/s
Context compress (reused):   710765.0 i/s - 1.01x  (± 0.00) slower
DContext decompress (reused):   210555.5 i/s - 3.42x  (± 0.00) slower
Context decompress (reused):   209596.5 i/s - 3.43x  (± 0.00) slower
Module decompress (new context each time):   204158.0 i/s - 3.52x  (± 0.00) slower
Module compress (new context each time):   201053.2 i/s - 3.58x  (± 0.00) slower

kreynolds avatar Aug 31 '25 17:08 kreynolds

Accidentally made this against an outdated main, will update shortly.

kreynolds avatar Aug 31 '25 17:08 kreynolds