HDDS-11017. Migrated to ECharts, Vite and AntD v4 with eslint, prettier

[devabhishekpal]

What changes were proposed in this pull request?

• While conducting a fossa scan we found that node-notifier and jsdom is not compatible with Apache license
• This PR migrates from the react-scripts used in the current project to Vite and Vitest which resolved the licensing issue
• Apart from that this will bring additional improvements like:
  • Faster local development time
  • Faster builds
  • Migration to AntD v5 which will allow us to directly upgrade to newer versions
  • Shift from xo to eslint and prettier - this will make the development experience more smooth and consistent for contributors

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-11017

How was this patch tested?

The patch was tested manually on local build

[devabhishekpal]

Currently library selection no longer has any licensing issues.

[smitajoshi12]

@devabhishekpal Can we update node and pnpm version also?

[devabhishekpal]

Hi @smitajoshi12 , so for Node v18 - it no longer supports CentOS 7 and older linux distros. We would need to then specify it that the project cannot be built on older linux dist. Hence we are stuck with Node v16 for the time being. pnpm v9 required Node 18. So I am upgrading to the latest supported pnpm v8.15.7.

[smitajoshi12]

@devabhishekpal In Disk Usage CSS aligmnet has been changed for Show Metata Data button.

[smitajoshi12]

@devabhishekpal

Can we move pi chart alignment to left side.

[smitajoshi12]

@devabhishekpal

Css Issue in heatmap when selecting dropdown

[devabhishekpal]

Hi @smitajoshi12, for https://github.com/apache/ozone/pull/6841#issuecomment-2188075777, if we move the pie-chart to the left side it would look odd as without the metadata panel open, it would have lots of empty space on the right hand side. So we are keeping it in the center as when the metadata is viewed the user will either way not be interacting with the chart.

Please do let me know if you want to shift to the left or if you have any other alternate inputs. Thanks

[smitajoshi12]

Hi @smitajoshi12, for #6841 (comment), if we move the pie-chart to the left side it would look odd as without the metadata panel open, it would have lots of empty space on the right hand side. So we are keeping it in the center as when the metadata is viewed the user will either way not be interacting with the chart.

Please do let me know if you want to shift to the left or if you have any other alternate inputs. Thanks

@ivandika3 @dombizita Can suggest but if entity name is lengthy then meta data summary pop is overlapping or we can mimimize metadata summary pop up so we will get clear visibility of entity names.

[devabhishekpal]

@smitajoshi12 could you let me know the screen resolution you are using for https://github.com/apache/ozone/pull/6841#issuecomment-2188080629? It seems in my resolution this issue is not present.

https://github.com/apache/ozone/assets/43001336/493257b8-712a-4bc9-b1bb-0fa8fe4e40fc

It is good to have other resolutions to test as well. Thanks a lot for bringing this to my attention.

[devabhishekpal]

Hi @smitajoshi12, for #6841 (comment), if we move the pie-chart to the left side it would look odd as without the metadata panel open, it would have lots of empty space on the right hand side. So we are keeping it in the center as when the metadata is viewed the user will either way not be interacting with the chart. Please do let me know if you want to shift to the left or if you have any other alternate inputs. Thanks

@ivandika3 @dombizita Can suggest but if entity name is lengthy then meta data summary pop is overlapping or we can mimimize metadata summary pop up so we will get clear visibility of entity names.

So I changed the current legend to go on the left hand side for better visibility. After my current changes it looks like below:

[smitajoshi12]

Hi @smitajoshi12, for #6841 (comment), if we move the pie-chart to the left side it would look odd as without the metadata panel open, it would have lots of empty space on the right hand side. So we are keeping it in the center as when the metadata is viewed the user will either way not be interacting with the chart. Please do let me know if you want to shift to the left or if you have any other alternate inputs. Thanks

@ivandika3 @dombizita Can suggest but if entity name is lengthy then meta data summary pop is overlapping or we can mimimize metadata summary pop up so we will get clear visibility of entity names.

So I changed the current legend to go on the left hand side for better visibility. After my current changes it looks like below:

@devabhishekpal These disk usage changes looks good to me.

[devabhishekpal]

After adjusting for media queries in the heatmap we are having the following UI: Video resolution was reduced to adjust for Github media size

https://github.com/apache/ozone/assets/43001336/a3469127-9df4-4465-b0c8-1038548363ea

[smitajoshi12]

@devabhishekpal

while doing cluster testing volume1 was having data and 100% and Total Data Size: 63.1 KB It should be inside pie chart Size and Percentage is accurate just position is changed. How we will come to know which are zero size entities and which entities are having data.

{ "status": "OK", "path": "/", "size": 64614, "sizeWithReplica": -1, "subPathCount": 39, "subPaths": [ { "key": false, "path": "/volume1", "size": 64614, "sizeWithReplica": -1, "isKey": false }, { "key": false, "path": "/s3v", "size": 0, "sizeWithReplica": -1, "isKey": false }, { "key": false, "path": "/vol-0-79279", "size": 0, "sizeWithReplica": -1, "isKey": false }, { "key": false, "path": "/vol-1-38323", "size": 0, "sizeWithReplica": -1, "isKey": false }, { "key": false, "path": "/vol-10-54251", "size": 0, "sizeWithReplica": -1, "isKey": false }, { "key": false, "path": "/vol-11-70290", "size": 0, "sizeWithReplica": -1, "isKey": false }, { "key": false, "path": "/vol-12-44556", "size": 0, "sizeWithReplica": -1, "isKey": false }, { "key": false, "path": "/vol-13-18787", "size": 0, "sizeWithReplica": -1, "isKey": false }, { "key": false, "path": "/vol-14-46679", "size": 0, "sizeWithReplica": -1, "isKey": false }, { "key": false, "path": "/vol-15-55529", "size": 0, "sizeWithReplica": -1, "isKey": false }, { "key": false, "path": "/vol-16-51548", "size": 0, "sizeWithReplica": -1, "isKey": false }, { "key": false, "path": "/vol-17-87656", "size": 0, "sizeWithReplica": -1, "isKey": false }, { "key": false, "path": "/vol-18-09750", "size": 0, "sizeWithReplica": -1, "isKey": false }, { "key": false, "path": "/vol-19-95360", "size": 0, "sizeWithReplica": -1, "isKey": false }, { "key": false, "path": "/vol-2-75113", "size": 0, "sizeWithReplica": -1, "isKey": false }, { "key": false, "path": "/vol-20-41102", "size": 0, "sizeWithReplica": -1, "isKey": false }, { "key": false, "path": "/vol-21-75670", "size": 0, "sizeWithReplica": -1, "isKey": false }, { "key": false, "path": "/vol-22-58601", "size": 0, "sizeWithReplica": -1, "isKey": false }, { "key": false, "path": "/vol-23-97664", "size": 0, "sizeWithReplica": -1, "isKey": false }, { "key": false, "path": "/vol-24-56683", "size": 0, "sizeWithReplica": -1, "isKey": false }, { "key": false, "path": "/vol-25-42101", "size": 0, "sizeWithReplica": -1, "isKey": false }, { "key": false, "path": "/vol-26-76437", "size": 0, "sizeWithReplica": -1, "isKey": false }, { "key": false, "path": "/vol-27-82747", "size": 0, "sizeWithReplica": -1, "isKey": false }, { "key": false, "path": "/vol-28-42441", "size": 0, "sizeWithReplica": -1, "isKey": false }, { "key": false, "path": "/vol-29-31116", "size": 0, "sizeWithReplica": -1, "isKey": false }, { "key": false, "path": "/vol-3-08351", "size": 0, "sizeWithReplica": -1, "isKey": false }, { "key": false, "path": "/vol-30-12401", "size": 0, "sizeWithReplica": -1, "isKey": false }, { "key": false, "path": "/vol-31-88150", "size": 0, "sizeWithReplica": -1, "isKey": false }, { "key": false, "path": "/vol-32-90258", "size": 0, "sizeWithReplica": -1, "isKey": false }, { "key": false, "path": "/vol-33-64420", "size": 0, "sizeWithReplica": -1, "isKey": false } ], "sizeDirectKey": -1 }

[devabhishekpal]

@devabhishekpal

while doing cluster testing volume1 was having data and 100% and Total Data Size: 63.1 KB It should be inside pie chart Size and Percentage is accurate just position is changed. How we will come to know which are zero size entities and which entities are having data.

Hi @smitajoshi12, so in general when we are hovering over an area the area becomes highlighted and elevated in ECharts. Apart from that I added new changes to also display the current path element name in the tooltip in the legend color. After changes it looks like below:

[smitajoshi12]

With Same Cluster data results are different in pi chart

filecount

[ { "volume": "volume1", "bucket": "bucket1", "fileSize": 32768, "count": 5 },{ "volume": "volume1", "bucket": "bucket2", "fileSize": 32768, "count": 3 } ]

---

container count

[{"containerSize":536870912,"count":1}]

Old Library

New Library

[devabhishekpal]

Hi @smitajoshi12, thanks for pointing out this issue. Fixed it in the latest patch https://github.com/apache/ozone/pull/6841/commits/5db74c1e8ec7b29371d66936332b4b3ba2afdbc2 With your data now it is looking like below:

One change from the original is that we are referring to the range instead of the actual value as in the original it seemed like multiple entities were having the same fixed container size. Showing the range let's the user know that it is not a fixed size but a range in which the entities are spread

[smitajoshi12]

@devabhishekpal In Overview Page In Route.json entries are also missing for local run getting 404 in local.

[smitajoshi12]

@devabhishekpal

Looks like Container ID column alignment has been changed.

[smitajoshi12]

@devabhishekpal

Can we increase width and Height of Pi Chart align more to left side so can get visibility for all entities.

[devabhishekpal]

@devabhishekpal In Overview Page In Route.json entries are also missing for local run getting 404 in local.

Hi @smitajoshi12 , I checked the older routes.json as well. I mean from older branch (Ozone 1.4.0) it seems the path mapping to /triggerdbsync/om was never present, hence the 404 response. But if we test it on a cluster instead of in dev mode, the API is being called. So it is okay in built files.

[smitajoshi12]

@devabhishekpal In Overview Page In Route.json entries are also missing for local run getting 404 in local.

Hi @smitajoshi12 , I checked the older routes.json as well. I mean from older branch (Ozone 1.4.0) it seems the path mapping to /triggerdbsync/om was never present, hence the 404 response. But if we test it on a cluster instead of in dev mode, the API is being called. So it is okay in built files.

No Issues closing this comment

[devabhishekpal]

@devabhishekpal

Looks like Container ID column alignment has been changed.

It seems this is the default behaviour with expandable rows. This is also present in the existing code, I cross checked with the current code. The following is built on master

[devabhishekpal]

@devabhishekpal

Can we increase width and Height of Pi Chart align more to left side so can get visibility for all entities.

So even if we increase the pie-chart width the percentage of entity area will be the same. I added the changes to spread out the pie-chart and added padding to the Tab panels as you suggested.

However with a larger pie-chart the labels will overlap on the chart as seen below.

So sticking to current size. I believe your PR to add a normalized width to the small entities would solve this issue of small areas.

[smitajoshi12]

It looks good to me now.

[smitajoshi12]

@devabhishekpal

Response and Pi Chart are different

Response:- { "status": "OK", "path": "/vol1/bucket1/dir1/key1", "size": 3800, "sizeWithReplica": 11400, "subPathCount": 0, "subPaths": [], "sizeDirectKey": 0 }

[smitajoshi12]

@devabhishekpal

Response and Pi Chart are different

Response:- { "status": "OK", "path": "/vol1/bucket1/dir1/key1", "size": 3800, "sizeWithReplica": 11400, "subPathCount": 0, "subPaths": [], "sizeDirectKey": 0 }

It looks Good to me after your recent changes

[ArafatKhan2198]

Thanks for the patch @devabhishekpal Smitha recently merged a change in upstream, which brings normalisation to the pie charts of disk usage! Have we tested out our new UI changes to the on it?

[devabhishekpal]

Hi @ArafatKhan2198, yes - the changes have been tested with the the datasets that were used by @smitajoshi12 in the original normalization PR and it is working correctly. The commit https://github.com/apache/ozone/pull/6841/commits/d8cdd9214036467ca91341cf369eadf1c3c83e50 adds the changes for the normalization.

[ArafatKhan2198]

Thanks for the work @devabhishekpal & thanks @devmadhuu @smitajoshi12 @ivandika3 @adoroszlai for the review.