rsync icon indicating copy to clipboard operation
rsync copied to clipboard

Add `--no-i-r-s-u`/`--no-inc-recursive-skip-unchanged` for accurate progress on resumed transfers

Open RyuX51 opened this issue 3 months ago • 0 comments

Add --no-inc-recursive-skip-unchanged for accurate progress on resumed transfers

Problem

When using --info=progress2 with --no-i-r, interrupted transfers show incorrect progress percentages after resuming. For example, if a transfer is interrupted at 20%, restarting it will show progress from 1% to 80% instead of 0% to 100%.

Root cause: stats.total_size includes already-transferred unchanged files. rsync's normal operation will skip these files (they won't actually transfer), but they're already counted in the progress denominator, making the percentage calculation wrong.

Solution

This PR introduces --no-inc-recursive-skip-unchanged (with aliases --no-i-r-skip-unchanged and --no-i-r-s-u), which pre-scans destination files during generator initialization to identify unchanged files and adjusts stats.total_size before progress reporting begins.

Implementation

Generator-side pre-scanning:

  • Runs before the main processing loop in generate_files()
  • Uses existing quick_check_ok() logic to identify unchanged files
  • Marks unchanged files inactive with clear_file() to exclude them from processing
  • Adjusts stats.total_size by subtracting unchanged file sizes

Inter-process communication:

  • Introduces MSG_FLIST_COUNT message to sync statistics between generator and sender
  • Ensures accurate progress display regardless of which process shows progress
  • Sender receives both skipped count and adjusted stats.total_size

Universal compatibility:

  • Works for all transfer types: local-to-local, local-to-remote, remote-to-local, and daemon
  • Generator always has destination access, making this approach viable in all scenarios
  • Implies --no-i-r (requires full file list upfront for pre-scan)

Benefits

Accurate progress reporting: Shows genuine 0-100% progress on resumed transfers
Correct file counts: to-chk display reflects only files needing transfer
No significant overhead: Uses same stat operations rsync would perform anyway, just earlier in the pipeline

Testing

# Initial transfer (interrupted mid-way)
rsync -av --info=progress2 --no-i-r-s-u src/ host:dest/
# Transfer shows 0-50%, then interrupts

# Resume transfer
rsync -av --info=progress2 --no-i-r-s-u src/ host:dest/
# Progress correctly shows 0-100% (only remaining files counted)

Naming Convention

Following rsync's established patterns (like --no-inc-recursive / --no-i-r):

  • Full name: --no-inc-recursive-skip-unchanged (documented in man page only)
  • Medium alias: --no-i-r-skip-unchanged (hidden from --help)
  • Short alias: --no-i-r-s-u (hidden from --help)

All variants function identically; only the full documentation appears in the man page, maintaining consistency with rsync's help output philosophy.

Files Changed

  • generator.c: Pre-scan logic in prescan_for_unchanged()
  • io.c: IPC handler for MSG_FLIST_COUNT
  • options.c: Option definitions and validation
  • progress.c: Adjusted progress calculation
  • rsync.h: New message type and stats field
  • rsync.1.md: Comprehensive documentation
  • NEWS.md: Enhancement entry

Example Use Cases

Resumed large backups with accurate progress:

rsync -av --no-i-r-s-u --info=progress2 /data/ /backup/

Remote synchronization with meaningful progress tracking:

rsync -av --no-i-r-skip-unchanged --info=progress2 host:src/ dest/

Monitoring actual transfer progress (not inflated by unchanged files):

rsync -av --no-inc-recursive-skip-unchanged /src/ /build/

This enhancement addresses a long-standing UX issue that may confuse users, particularly those relying on progress reporting for automated backup systems and large data transfers where most files are typically unchanged.

RyuX51 avatar Oct 27 '25 17:10 RyuX51