CortexDocs icon indicating copy to clipboard operation
CortexDocs copied to clipboard

reduce analyzer requirements installation time

Open xeaon opened this issue 5 years ago • 1 comments

While your solution to install all analyzer requirements looks clean and easy to understand, i've always been annoyed a litte about how time consuming it is to watch pip & pip3 going through all requirements.txt, especially because a lot of analyzers share the same 'base' requirements like cortexutils or requests.

So by deduplicating all requirements and combining them into one single requirements file, you can speed p the hole process. As of today, the linecount of combined_requirements.txt is 71, compared to 277 overall requirements count.

root@cortex:/opt# for I in $(find Cortex-Analyzers -name 'requirements.txt'); do cat >> combined_requirements.txt < $I; sort -u -o combined_requirements.txt combined_requirements.txt; done'
root@cortex:/opt# for I in $(find Cortex-Analyzers -name 'requirements.txt'); do cat >> all_requirements.txt < $I; sort -o all_requirements.txt all_requirements.txt; done

root@cortex:/opt# wc combined_requirements.txt 
  71   78 1142 all_requirements.txt
root@cortex:/opt# wc all_requirements.txt 
 277  290 3427 combined_requirements.tx

It might not be the prettiest solution so feel free to edit, if you're considering to accept this pull request.

xeaon avatar Aug 13 '20 09:08 xeaon

+1 agreed python2 is supposed to be dead :) https://pythonclock.org/

desnij avatar Aug 28 '20 11:08 desnij