Microsoft.PowerShell.Archive icon indicating copy to clipboard operation
Microsoft.PowerShell.Archive copied to clipboard

Expand-Archive performance

Open MathiasMagnus opened this issue 8 years ago • 19 comments

Allow me to copy the contents of an issue posted under PowerShell/PowerShell.

Expand-Archive performance is 6X away from 7zip. In it's current state I would say it barely passes automation perf level. In an interactive shell, one would rarely resort to it if there is an alternative that is 6X faster.

Steps to reproduce

Download some large archive (qt-everywhere-opensource-src-5.9.0-alpha.zip was my use case) and unzip.

Expected behavior

Something like this:

PS C:\Users\Matty\Downloads> Measure-Command \
>> { C:\Kellekek\7-Zip\7z.exe x .\qt-everywhere-opensource-src-5.9.0-alpha.zip }


Days              : 0
Hours             : 0
Minutes           : 37
Seconds           : 57
Milliseconds      : 938
Ticks             : 22779388628
TotalDays         : 0,0263650331342593
TotalHours        : 0,632760795222222
TotalMinutes      : 37,9656477133333
TotalSeconds      : 2277,9388628
TotalMilliseconds : 2277938,8628

Actual behavior

PS C:\Users\Matty\Downloads> Measure-Command \
>> { Expand-Archive -Path .\qt-everywhere-opensource-src-5.9.0-alpha.zip -DestinationPath C:\Kellekek\Qt }


Days              : 0
Hours             : 3
Minutes           : 44
Seconds           : 34
Milliseconds      : 218
Ticks             : 134742180246
TotalDays         : 0,155951597506944
TotalHours        : 3,74283834016667
TotalMinutes      : 224,57030041
TotalSeconds      : 13474,2180246
TotalMilliseconds : 13474218,0246

Environment data

PS C:\Users\Matty\Downloads> $PSVersionTable

Name                           Value
----                           -----
CLRVersion
WSManStackVersion              3.0
PSVersion                      6.0.0-alpha
PSEdition                      Core
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
GitCommitId                    v6.0.0-alpha.17
BuildVersion                   3.0.0.0

PS C:\Users\Matty\Downloads> C:\Kellekek\7-Zip\7z.exe

7-Zip [64] 15.14 : Copyright (c) 1999-2015 Igor Pavlov : 2015-12-31
...

PS C:\Users\Matty\Downloads> (Get-ComputerInfo).CsProcessors


Name                      : AMD FX-7500 Radeon R7, 10 Compute Cores 4C+6G
Manufacturer              : AuthenticAMD
Description               : AMD64 Family 21 Model 48 Stepping 1
Architecture              : x64
AddressWidth              : 64
DataWidth                 : 64
MaxClockSpeed             : 2100
CurrentClockSpeed         : 2100
NumberOfCores             : 4
NumberOfLogicalProcessors : 4
ProcessorID               : 178BFBFF00630F01
SocketDesignation         : Socket FS1r2
ProcessorType             : CentralProcessor
Role                      : CPU
Status                    : OK
CpuStatus                 : Enabled
Availability              : RunningOrFullPower

MathiasMagnus avatar Mar 17 '17 12:03 MathiasMagnus

I have the same problem. When it will be fixed?

FilippoBostrenghi avatar Apr 24 '18 13:04 FilippoBostrenghi

will this ever get fixed?

mkress avatar Aug 29 '19 14:08 mkress

Also interested by the resolution of this issue

GMouron avatar Sep 03 '19 14:09 GMouron

It is very slow. It needs 10 minutes to extract 200MB. Any workaround? Maybe an option that could improve the extraction performance?

kavalagios avatar Sep 09 '19 08:09 kavalagios

My intent is after PS7 GA to have resources applied to this module and convert it to C# using some OSS libs that will solve a number of outstanding issues.

SteveL-MSFT avatar Sep 09 '19 17:09 SteveL-MSFT

By the way, if anybody has ideas on a good archive engine that potentially can be used - feel free to post your suggestions here...

anmenaga avatar Sep 10 '19 17:09 anmenaga

Also have the same issue. It's unbelievably slow.

giampaolo avatar Feb 12 '20 17:02 giampaolo

You can get some performance if you do not show progress when Expanding. Setting $progressPreference = 'SilentlyContinue' before the expand

diegohi avatar Jun 11 '20 11:06 diegohi

I found 2 ways to improve unzip speeds.

First, exclude the directory from Windows Defender. It makes things really slow. I'm extracting a 50mb file and doing this changed it from taking about 4 minutes to taking 26 seconds. You must decide if you're OK with the security risk.

Add-MpPreference -ExclusionPath C:\installers

Second, use this .NET class to extract the zip. This changed it from taking 26 seconds down to 12 seconds.

Add-Type -Assembly "System.IO.Compression.Filesystem"
[System.IO.Compression.ZipFile]::ExtractToDirectory("C:\installers\app.zip", "C:\installers\app")

raymondbutcher avatar Aug 04 '20 17:08 raymondbutcher

I am having the same issues, same zip file, used to take 3-4 minutes. it is now taking over like 2 hours, randomly, after yesterday 07 Feb 2022, azure outage issues. Was working perfectly before no issues.

wesleyolis avatar Feb 08 '22 13:02 wesleyolis

I actually cannot understand why Microsoft is not fixing this?

mkarg avatar Feb 21 '22 10:02 mkarg

@mkarg This is on Microsoft's radar.

ayousuf23 avatar Jun 03 '22 16:06 ayousuf23

Love to see this issue being addressed

CEbbinghaus avatar Jul 13 '22 07:07 CEbbinghaus

Blog post on Archive module 2.0 and the Github project tracking progress, with this very issue in it.

MathiasMagnus avatar Jul 26 '22 17:07 MathiasMagnus

Compress-Archive is also very slow even with -CompressionLevel NoCompression , much slower than 7zip

throwable-one avatar Jul 25 '23 01:07 throwable-one

Add-Type -Assembly "System.IO.Compression.Filesystem"
[System.IO.Compression.ZipFile]::ExtractToDirectory("C:\installers\app.zip", "C:\installers\app")

@raymondbutcher This! For a 120MB file this improves extraction times from 1min. 20sec. to just 9 sec.

s-geiger-si avatar Sep 23 '23 17:09 s-geiger-si

That example by @s-geiger-si is sooo much faster it is just crazy...

JensNordenbro avatar Mar 04 '24 12:03 JensNordenbro

@mkarg This is on Microsoft's radar.

Nice. And when will Microsoft fix it? Two years are gone.

mkarg avatar Jun 21 '24 09:06 mkarg

Agree with @mkarg. @SydneyhSmith, @ayousuf23 any clarification on when to expext this?

JensNordenbro avatar Jun 22 '24 12:06 JensNordenbro