Incompatible Architecture issue when Converting Models to Core ML

[ankushagarwal]

I am on an M1 Mac and created the conda environment using

CONDA_SUBDIR=osx-arm64 conda create -n coreml_stable_diffusion python=3.8 -y

Yet when I try to generate Core ML model files by running python -m python_coreml_stable_diffusion.torch2coreml --convert-unet, I run into this issue :

(mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64'))

Here's the full error:

ImportError: dlopen(/Users/kush/opt/miniconda3/envs/coreml_stable_diffusion/lib/python3.8/site-packages/tokenizers/tokenizers.cpython-38-darwin.so, 0x0002): 
tried: '/Users/kush/opt/miniconda3/envs/coreml_stable_diffusion/lib/python3.8/site-packages/tokenizers/tokenizers.cpython-38-darwin.so' 
(mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64')),

'/System/Volumes/Preboot/Cryptexes/OS/Users/kush/opt/miniconda3/envs/coreml_stable_diffusion/lib/python3.8/site-packages/tokenizers/tokenizers.cpython-38-darwin.so' (no such file), 
'/Users/kush/opt/miniconda3/envs/coreml_stable_diffusion/lib/python3.8/site-packages/tokenizers/tokenizers.cpython-38-darwin.so' 
(mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64'))

[adammarples]

You might need to build the tokenizers package from source https://github.com/huggingface/tokenizers/issues/712

[ankushagarwal]

Sounds good. Thanks @adammarples 🙏 - will give that a shot!