app rendering slow for wide columns

[mccanne]

I noticed the difference in time-to-data between

sample

and

sample | fuse

was quite stark with our all.pcap test file even though the output of sample is only a dozen or so records.

I compared this to zq and the time difference was negligible.

This makes me think there are very large rendering times in the app for a small number of very wide (but not hugely wide) columns. This smells fishy.

[mccanne]

James says solution is to use react window with virtualized grid.

[philrz]

Today we received another piece of test data from a zync user that performs even worse due to this problem. It's sensitive data so I'm not attaching it to this issue, but this comment is a note-to-self to go back into their support channel to dig up and re-test with the data when this issue is addressed.

[philrz]

I've found the GitHub test data downloaded in #2283 is also good for repro'ing this issue, once you preprocess it with over this.

[philrz]

This one is on track to be addressed by the changes in #2626.

The attached video shows a repro with the current tip of main, commit b0e7212. The test data is loaded via:

$ zed create prs
$ zq 'over this | fuse' https://api.github.com/repos/brimdata/zed/pulls\?state\=all\&sort\=desc\&per_page=100 | zed -use prs load -

Notice the delay as rows fill in as I scroll down and then again as I scroll back to the top.

https://user-images.githubusercontent.com/5934157/210011177-463ed21d-a3c3-498c-afea-a704c638d787.mp4

By comparison, it's super fast on commit 4014451 of the branch from #2626.

https://user-images.githubusercontent.com/5934157/210011254-d5b44f4f-631a-4e2e-af37-548c1ab5ea68.mp4

I'll do a final verification when the changes merge, but looks 👍 right now.

[philrz]

Verified in Brim commit 499a2c9.

As shown in the attached video, now after we load the test data and scroll down, the additional data is filled in instantly rather than with the long delays we saw in the baseline video back at https://github.com/brimdata/brim/issues/1881#issuecomment-1367586796.

https://user-images.githubusercontent.com/5934157/214930910-6e7cfb5d-bcfa-49e7-baa0-3974f188a1c0.mp4

Thanks @jameskerr!