devpod icon indicating copy to clipboard operation
devpod copied to clipboard

Fix: Handle existing symlinks during tar extraction

Open haroondilshad opened this issue 5 months ago โ€ข 2 comments

๐Ÿ› Bug Description

DevPod workspace uploads fail when extracting tar archives containing symlinks that already exist in the destination directory. This causes the following error:

symlink <target> <destination>: file exists

๐Ÿ”„ Bug Reproduction

Prerequisites

  • DevPod workspace with local folder source
  • Directory containing symlinks (e.g., link.md -> target.md)
  • Existing workspace that has been uploaded before

Steps to Reproduce

  1. Create a workspace with symlinks in the local folder:

    echo "content" > target.md
    ln -s target.md link.md
    
  2. Upload workspace to DevPod:

    devpod up my-workspace
    
  3. Make any change and rebuild the workspace:

    devpod up my-workspace --recreate
    
  4. Expected: Workspace rebuilds successfully

  5. Actual: Fails with symlink target.md link.md: file exists error

Root Cause

The tar extraction code in pkg/extract/extract.go calls os.Symlink() without checking if the target file already exists. When DevPod re-uploads the workspace, it tries to create symlinks that already exist from the previous upload, causing the extraction to fail.

โœ… Solution

Changes Made

  1. Enhanced symlink extraction logic in pkg/extract/extract.go:

    • Check if file/symlink already exists at target location
    • If existing symlink points to the same target, preserve it (no-op)
    • If existing symlink has different target, remove and recreate
    • If regular file exists, remove and create symlink
    • Improve error messages with context
  2. Added comprehensive tests in pkg/extract/extract_test.go:

    • Test creating new symlinks
    • Test replacing existing symlinks with different targets
    • Test preserving existing symlinks with same targets
    • Test replacing regular files with symlinks
    • Test multiple symlinks in same archive
    • Test gzipped tar archives with symlinks

Code Changes

// Before (fails on existing files)
err := os.Symlink(header.Linkname, outFileName)
if err != nil {
    return false, err
}

// After (handles existing files intelligently)
if _, err := os.Lstat(outFileName); err == nil {
    if existingLink, err := os.Readlink(outFileName); err == nil {
        if existingLink == header.Linkname {
            return true, nil // Same symlink, no change needed
        }
    }
    // Remove existing file/symlink
    if err := os.Remove(outFileName); err != nil {
        return false, perrors.Wrapf(err, "remove existing file for symlink %s", outFileName)
    }
}

err := os.Symlink(header.Linkname, outFileName)
if err != nil {
    return false, perrors.Wrapf(err, "create symlink %s -> %s", outFileName, header.Linkname)
}

๐Ÿงช Testing

All tests pass:

$ go test ./pkg/extract -v
=== RUN   TestExtractSymlinkConflicts
=== RUN   TestExtractSymlinkConflicts/create_new_symlink
=== RUN   TestExtractSymlinkConflicts/replace_existing_symlink_different_target  
=== RUN   TestExtractSymlinkConflicts/preserve_existing_symlink_same_target
=== RUN   TestExtractSymlinkConflicts/replace_existing_regular_file
--- PASS: TestExtractSymlinkConflicts (0.00s)
=== RUN   TestExtractSymlinkMultipleConflicts
--- PASS: TestExtractSymlinkMultipleConflicts (0.00s)
=== RUN   TestExtractGzippedTarWithSymlinks  
--- PASS: TestExtractGzippedTarWithSymlinks (0.00s)
PASS

๐ŸŽฏ Impact

What This Fixes

  • โœ… DevPod workspace rebuilds no longer fail on symlink conflicts
  • โœ… No more manual intervention required to delete conflicting files
  • โœ… Symlinks are preserved when unchanged, replaced when different
  • โœ… Works with both regular and gzipped tar archives

Backward Compatibility

  • โœ… Fully backward compatible - only affects the failing case
  • โœ… No changes to existing successful extraction behavior
  • โœ… No breaking changes to API or interfaces

Performance

  • โœ… Minimal performance overhead (one extra file check per symlink)
  • โœ… No impact on extraction of regular files
  • โœ… Early return for unchanged symlinks avoids unnecessary work

๐Ÿ“ Related Issues

This fix addresses workspace upload failures experienced by users with symlinks in their projects, particularly common in documentation and configuration scenarios where multiple files reference a common target.

haroondilshad avatar Sep 06 '25 22:09 haroondilshad

@pascalbreuninger

haroondilshad avatar Sep 06 '25 22:09 haroondilshad

Hi, I'm facing exactly this problem. Can someone review this PR?

jrx-sjg avatar Oct 17 '25 13:10 jrx-sjg