human-eval
human-eval copied to clipboard
Adds Windows support
Windows Support for Human-Eval
This PR adds comprehensive Windows support to the code evaluation pipeline. Here are the key changes:
Process Management
- Replaced Unix-specific
Processhandling with Windows-compatiblePoolimplementation - Added proper process cleanup using
pool.terminate()andpool.join()to prevent zombie processes on timeout - Switched to 'spawn' context instead of 'fork' for Windows compatibility:
multiprocessing.get_context('spawn')
Timeout Implementation
- Added Windows-specific timeout mechanism using threading since SIGALRM is not available on Windows
- Implemented thread-based timeout using
PyThreadState_SetAsyncExcto inject exceptions into running threads - Added proper thread state cleanup to prevent memory leaks and ensure thread state consistency
- Introduced
doneandraisedevents to handle race conditions in timeout scenarios
Memory Management
- Added Windows-specific memory limits using Windows Job Objects
- Integrated
pywin32package for Job Object API access - Added conditional dependency:
pywin32>=306; platform_system == "Windows" - Implemented memory limits through
JOB_OBJECT_LIMIT_PROCESS_MEMORYflag
Platform Detection
- Added
IS_WINDOWSflag usingplatform.system()for conditional code paths - Separated Unix and Windows-specific implementations into distinct functions
- Made all path handling Windows-compatible
These changes ensure that the human-eval framework runs reliably on Windows while maintaining feature parity with Unix systems. The implementation handles Windows-specific quirks around process management, timeouts, and memory limits while preserving the security and isolation guarantees of the original implementation.
Following is the sanity check using the example results done on Windows:
Closes #51 #45 CC #50