UMEP icon indicating copy to clipboard operation
UMEP copied to clipboard

Length mismatch: Expected axis has 38 elements, new values have 24 elements

Open marciaamatias opened this issue 1 year ago • 47 comments

I'm running the SUEWS model within UMEP and I get this error when runngi the SUEWS simple

Length mismatch: Expected axis has 38 elements, new values have 24 elements

What length?!! Can someone help?

image

I checked the Supy.log and it doesn't present any information on the error

"2024-04-28 18:47:38,762 - SuPy - INFO - All cache cleared. 2024-04-28 18:47:39,549 - SuPy - INFO - All cache cleared."

marciaamatias avatar Apr 28 '24 17:04 marciaamatias

Hi @marciaamatias,

  • Can you zip your data and attach it here for us to have a look at?
  • please also clarify the versions of UMEP and QGIS you are using.

Thanks!

biglimp avatar Apr 30 '24 07:04 biglimp

Hello @biglimp I assume the SUEWS simple plugin would only use the default dataset shipped in UMEP, right? if so, I guest this issue is related to some upstream packages used in pre-processing the default dataset.

sunt05 avatar Apr 30 '24 08:04 sunt05

Hello @biglimp I assume the SUEWS simple plugin would only use the default dataset shipped in UMEP, right? if so, I guest this issue is related to some upstream packages used in pre-processing the default dataset.

We will have to see when the user submit the data.

biglimp avatar Apr 30 '24 08:04 biglimp

I'm sorry I am taking so long to get you the files

But first I need to complete the problems I'm having with the Tree morphology

marciaamatias avatar Apr 30 '24 11:04 marciaamatias

So this is the input data that I have. I think that is all you need. If there is anything missing I'll send it.

files_M_1.zip

I've run the model again after I've fixed the tree morphology problem with @biglimp's help but the same error appeared.

image

And this is the version of QGIS and umep I'm using

UMEP: 4.0.4 UMEP for processing: 2.0.13 QGIS: image

marciaamatias avatar May 01 '24 07:05 marciaamatias

Hello @biglimp, it seems the issue could be on the upstream side since the error message doesn't appear to be directly generated by SuPy itself.


Also, the data provided by UMEP users isn’t immediately usable for testing on the SuPy side. We need to determine whether the problem originates from SuPy or elsewhere. (This might be more of a UMEP development issue that should be branched out separately.)

sunt05 avatar May 01 '24 14:05 sunt05

Is there anything missing I should have provided for testing on the SuPy side?

marciaamatias avatar May 01 '24 14:05 marciaamatias

No, not at the moment - Fredrik and I need to figure sth else first and update this further.

sunt05 avatar May 01 '24 15:05 sunt05

Your attached data is not complete. If you want run use your own data in SUEWSSimple, you need isotrophic files from buildings, vegetation and landcover using DSM, DEM, CDSM and LandCover rasters. Please provide your isotrphic file for buildings.

biglimp avatar May 02 '24 07:05 biglimp

Sorry, here it is

buildings_IMPPoint_isotropic.txt

marciaamatias avatar May 02 '24 08:05 marciaamatias

Your data is inconsistent. In your land cover file, building fraction is 0.233 whereas in your building file you have a building fraction of 0.338.

Furthermore, when I look at your geodata, you lack a DEM and your DSM has tall vegetation included. Is should consist of only buildings and ground. Have a close look at the tutorial data for our London test case. There you can see how the data should be. https://umep-docs.readthedocs.io/projects/tutorial/en/latest/Tutorials/IntroductionToSuews.html#introductiontosuews https://umep-docs.readthedocs.io/projects/tutorial/en/latest/Tutorials/SuewsAdvanced.html#suewsadvanced

biglimp avatar May 02 '24 08:05 biglimp

So, I now have the correct DSM (only buildings)

However, the same error popped up:

image

So is the error because of the DSM or is it another thing that is making the length mistake appear?

marciaamatias avatar May 02 '24 11:05 marciaamatias

Not sure yet where your error come from but you still have large differences between your land cover building fraction and Building morphology fraction. When I test with your data I get this message since the data is inconsistent:

image

I also see that you are using the met_data from London (if the attached files are correct. That is not possible since that dataset is only valid for that location. Please clean your data and resubmit for us to test. As it is now I cannot debug your problem.

biglimp avatar May 02 '24 12:05 biglimp

@sunt05 , I was able to debug and it seem to be something wrong with the input metfile (apart from the issues with the input surface data). Do you automatically add a spinup year? I attached the data used. Can you check?

RunControl.zip

biglimp avatar May 02 '24 12:05 biglimp

Met_data.txt

This is the met data I'm using for the model. I've created this based on the data EPW data for Lisbon. That is the file I've sent you.

marciaamatias avatar May 02 '24 13:05 marciaamatias

About the difference between the DSM and the LCF

I don't understand why this is happening since the LCF for the buildings is exactly the same that is accounted in the DSM

Please see images:

  1. Buildings over the DSM image

  2. only the DSM image

  3. Building grid image

marciaamatias avatar May 02 '24 14:05 marciaamatias

What grid are you using as landcover?

biglimp avatar May 02 '24 15:05 biglimp

A raster with the classification that UMEP requires, from 1 to 5

land_cover_umep_telheiras.zip

marciaamatias avatar May 02 '24 15:05 marciaamatias

The extent of the land cover is smaller that the other data. If you use a 200 meter radius a large part of your analysis will be outside your land cover grid.

image

biglimp avatar May 02 '24 15:05 biglimp

I've moved the point and changed the search area to 130 meters, so now everything is within the area and the values do change a little bit.

However, the morphometric values for the trees I can't calculate because of this error, that I think is the same I've had before

This is the data I'm using. I'm only using the CDSM files_M_.zip

And I've checked the box for the DSM (only 3D objects) image

ERROR: Algoritmo 'Urban Morphology: Morphometric Calculator (Point)' a iniciar… Parâmetros de entrada: { 'FILE_PREFIX' : 'arvores', 'INPUT_DEM' : None, 'INPUT_DISTANCE' : 130, 'INPUT_DSM' : None, 'INPUT_DSMBUILD' : 'E:/Doutoramento/TESE DE DOUTORAMENTO/ARTIGOS/Artigo 2/QGIS/CDSM_telheiras.tif', 'INPUT_INTERVAL' : 5, 'INPUT_POINT' : None, 'INPUT_POINTLAYER' : 'E:/Doutoramento/TESE DE DOUTORAMENTO/ARTIGOS/Artigo 2/QGIS/ponto_interesse.shp', 'OUTPUT_DIR' : 'E:\Doutoramento\TESE DE DOUTORAMENTO\ARTIGOS\Artigo 2\QGIS', 'OUTPUT_POINT' : 'TEMPORARY_OUTPUT', 'OUTPUT_POLYGON' : 'TEMPORARY_OUTPUT', 'ROUGH' : 0, 'SAVE_POINT' : False, 'USE_DSM_BUILD' : True }

Point location obtained from point in vector layer x = 484855.7461486188 y = 4290302.393890262 Traceback (most recent call last): File "C:\Users/marci/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\processing_umep\preprocessor\imagemorphparmspoint_algorithm.py", line 247, in processAlgorithm immorphresult = morph.imagemorphparam_v2(dsm, dem, scale, 1, degree, feedback, 1) File "C:\Users/marci/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\processing_umep\util\imageMorphometricParms_v2.py", line 100, in imagemorphparam_v2 lineMid = a[0:imidy,imid] IndexError: only integers, slices (:), ellipsis (...), numpy.newaxis (None) and integer or boolean arrays are valid indices

Execution failed after 0.09 segundos

marciaamatias avatar May 02 '24 16:05 marciaamatias

I cannot replicate your error. Used your data and settings.

image

Try start a new project in QGIS and make sure that you use same CRS (your shapefile point has no registered CRS).

biglimp avatar May 03 '24 08:05 biglimp

New blank project

All layers with the same CRS - even the point layer but it failed

image image

marciaamatias avatar May 03 '24 08:05 marciaamatias

Sorry, cannot help since i cannot replicate your error.

biglimp avatar May 03 '24 08:05 biglimp

can you send me the output file you got on the morphometric calculator?

so I can see if with that I can run the model?

marciaamatias avatar May 03 '24 08:05 marciaamatias

Thank you.

It seems the problem remains in the difference between Land Cover and the DSM, manly on the vegetation difference

My question is: beeing a possitive differece means there is more vegetation in the DSM than in the Land Cover?

I've updated the Land Cover, to account for all the trees the morphometric calculator is assuming but the difference remains

image

marciaamatias avatar May 03 '24 13:05 marciaamatias

You have 12.2% evergreen in your landcover and 0.4% tree cover from the morphology file, i.e. no trees in your CDSM.

biglimp avatar May 07 '24 10:05 biglimp

Can you help me with this error the plugin is crashing on?

What GridLayoutMETDATA_NEW.nml file is UMEP referring to?

This is what is on the Supy.log

2024-05-13 15:13:48,999 - SuPy - INFO - All cache cleared. 2024-05-13 15:13:49,621 - SuPy - ERROR - C:\Users\marci\AppData\Roaming\QGIS\QGIS3\profiles\default\python\plugins\UMEP\suewsmodel\Input\GridLayoutMETDATA_NEW.nml does not exists! Traceback (most recent call last): File "C:\Users\marci\AppData\Roaming\Python\Python39\site-packages\supy_load.py", line 278, in load_SUEWS_nml df_nml = pd.DataFrame(parser.read(p_nml)) File "C:\Users\marci\AppData\Roaming\Python\Python39\site-packages\f90nml\parser.py", line 279, in read nml_file = open(nml_fname, 'r') if nml_is_path else nml_fname FileNotFoundError: [Errno 2] No such file or directory: 'C:\Users\marci\AppData\Roaming\QGIS\QGIS3\profiles\default\python\plugins\UMEP\suewsmodel\Input\GridLayoutMETDATA_NEW.nml'

marciaamatias avatar May 13 '24 14:05 marciaamatias

And what about this error (again) on the Land Cover Grid? I'm using new shapes, on a blank project so I make sure is not that that is making the error. I've seen that all layers have the same CRS, same size, extent and resolution. I was able to use the same grid on the Morphometric calculator and it added to the grid attribute table the values. But with the Land cover is not working.

image

image

marciaamatias avatar May 13 '24 15:05 marciaamatias

This error indicates that you have i discrapency in CRS between your land cover and vector data OR that your land cover consist of NoData values. The land cover dataset should only consist of values between 1 and 7.

biglimp avatar May 14 '24 07:05 biglimp