DepthViewer icon indicating copy to clipboard operation
DepthViewer copied to clipboard

Depthviewer only generates 518x518 for DAN

Open ThatDocBoi opened this issue 1 year ago • 1 comments

Hey, I am trying to convert a bunch of images, however I've noticed that if I use the depth.py it generates much better depth. Looking at the depth files it appears that using depth.py downscales the image so that 1 side is 518 and the other is bigger, but the viewer only does 518x518. Since the viewer does not see the previously generated .depthfiles, this makes it so that for each image, the depth file needs to be opened, then the image file, every time.

Would it be possible to have a way to automatically read the generated depth files or make the viewer generate it like how depth.py does please?

Thank you.

ThatDocBoi avatar Feb 26 '24 03:02 ThatDocBoi

Well it's been long since I haven't seen the code so I have to test it, but in the Unity C# script when it detects depth-anything model it sets the fallback to 518x518: #

...Which set the SentisDepthModel's WidthFallback and HeightFallback

But as "fallback" implies, this is only used when it could not (or I could not) detect the model size from the model itself:

		//Set these to (518, 518) for Depth-Anything models

		var heightSym = _model.inputs[0].shape[2];
		var widthSym = _model.inputs[0].shape[3];

		if (heightSym.isValue)
			_height  = heightSym.value;
		else {
			Debug.Log($"SentisDepthModel: Height cannot be inferred, falling back to {_heightFallback}");
			_height = _heightFallback;
		}

		if (widthSym.isValue)
			_width = widthSym.value;
		else {
			Debug.Log($"SentisDepthModel: Width cannot be inferred, falling back to {_widthFallback}");
			_width = _widthFallback;
		}

		_output = new float[_width*_height];

#

So two questions arise:

  • Is it possible to detect the output size from the model metadata?
  • Does the quality difference between the current 518x518 code and the hypothetical non-518x518 output exist? If so, is it significant?

I'll take a look into it.

(btw sorry for the long hiatus)

parkchamchi avatar Aug 01 '24 13:08 parkchamchi