gibberish output?

[husnoo]

Hi, I followed these instructions and downloaded the models, but the output looks like gibberish... (I don't have a working cuda card, so I hope it's using cpu?)

conda env create -f environment.yaml
conda activate midas-py310

python run.py --model_type dpt_beit_large_512 --input_path input --output_path output
python run.py --model_type dpt_levit_224 --input_path input --output_path output
python run.py --model_type dpt_swin2_large_384 --input_path input --output_path output
python run.py --model_type dpt_swin2_tiny_256 --input_path input --output_path output
python run.py --model_type midas_v21_384 --input_path input --output_path output
python run.py --model_type midas_v21_small_256 --input_path input --output_path output

[heyoeyo]

That's very strange! Are there any errors/warnings printed in the console when you run this? Also, it should say which device it's using in the terminal, so you can confirm that it's using cpu. It should print out something like:

Initialize
Device: cpu
... some warning about 'meshgrid' ...
Model loaded, number of parameters = <some big number>
  Processing input/<name of image> (1/1)
    Input resized to <HxW> before entering the encoder
Finished

It's worth double checking that the cpu is being used, and that the image you think you're processing is actually being loaded (and not for example, some hidden/auto-save copy of the image, which could cause problems). Also, it's worth checking if there is any message about something going wrong.

If there isn't some error message, I'd assume that the problem is the model weights, since that's the most obvious way to get nonsense results. So you may want to re-download them and try again. I know the weights on the releases page of this repo work, so if you didn't originally get them from there, that's worth a try.

[husnoo]

~~ Turned out the problem was that my images weren't the right shape? When I resized them to match the label on the model, they worked fine! e.g. 640x192px. ~~ (see below for my next message)

On Fri, 22 Dec 2023 at 15:19, heyoeyo @.***> wrote:

That's very strange! Are there any errors/warnings printed in the console when you run this? Also, it should say which device it's using in the terminal, so you can confirm that it's using cpu. It should print out something like:

Initialize Device: cpu ... some warning about 'meshgrid' ... Model loaded, number of parameters = Processing input/ (1/1) Input resized to <HxW> before entering the encoder Finished

It's worth double checking that the cpu is being used, and that the image you think you're processing is actually being loaded (and not for example, some hidden/auto-save copy of the image, which could cause problems). Also, it's worth checking if there is any message about something going wrong.

If there isn't some error message, I'd assume that the problem is the model weights, since that's the most obvious way to get nonsense results. So you may want to re-download them and try again. I know the weights on the releases page https://github.com/isl-org/MiDaS/releases/tag/v3_1 of this repo work, so if you didn't originally get them from there, that's worth a try.

— Reply to this email directly, view it on GitHub https://github.com/isl-org/MiDaS/issues/258#issuecomment-1867804879, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAWJWSWLOVZNF242UK5LFYDYKWQGNAVCNFSM6AAAAABA63J3UKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNRXHAYDIOBXHE . You are receiving this because you authored the thread.Message ID: @.***>

[heyoeyo]

That is strange... The run.py script transforms the image prior to processing it with the model, and one of the transformations is a resizing step, to make sure it's properly sized for use in the model. But if it works, it works I guess!

Since manually resizing (especially to 192px) may hurt the quality of the output, it might be worth re-saving your image without any resizing using the same tool that did the resizing and see if that works? It may just be that the original image format isn't supported properly by the midas script, but the format of the resizing tool is supported.

[husnoo]

Sorry I messed up - it was monodepth2 that worked with the right size image, I never got Midas to work! I'll have another look at your suggestions soon!