model-transparency icon indicating copy to clipboard operation
model-transparency copied to clipboard

MacOS speed

Open laurentsimon opened this issue 2 years ago • 5 comments

Investigate the speed on OSX. The unit tests seem to be slow. It could be because the runners are not as fast

laurentsimon avatar Dec 05 '23 23:12 laurentsimon

/assign

Gregory-Pereira avatar May 16 '24 02:05 Gregory-Pereira

The issue is not speed to the best of my knowledge. After playing with this last night I realized that the test will hang and appear to be slow, but in reality it just is not running the signing. I identified this by running the benchmarking, and notice it hanging for signing and verifying YOLOP for about 90 minutes, but then was able to manually sign and verify the ~220 MB model in about 2 minutes when running manually. Additional Darwin ships with Bash version 3.X.X (or at least mine did):

/bin/bash --version
GNU bash, version 3.2.57(1)-release (arm64-apple-darwin23)

Bash v4 is required for the associative arrays used in benchmarking. I am considering a re-write to build a JSON summary file with jq instead. This does introduce another depedency, however I think it would harden our support for other platform. Thoughts?

Gregory-Pereira avatar May 16 '24 14:05 Gregory-Pereira

ah, good catch. I've assigned the issue to you.

@mihaimaruseac wrote some benchmark code in Python a while back. @mihaimaruseac would you be able to share?

laurentsimon avatar May 16 '24 15:05 laurentsimon

I will, need to clean some stuff first. It will remove the need for #182 though.

@Gregory-Pereira the issue is that the CI under .github/workflows is slow, not the so much benchmarking

mihaimaruseac avatar May 16 '24 15:05 mihaimaruseac

Sounds good, ill TAL

Gregory-Pereira avatar May 16 '24 19:05 Gregory-Pereira

The tests are fast now with refactoring and using hatch test with parallelism

mihaimaruseac avatar Aug 14 '24 19:08 mihaimaruseac