Syntax highlighting performance

[johntyree]

When using the syntax highlighting from vim2hs I experience significant lag, even just moving the cursor up and down. My old highlighting file didn't suffer from this. Have you noticed any performance problems? Can you compare with this highlighting file and see if you have a noticeable difference?

https://gist.github.com/3122151

[dag]

Yes, I notice it if I scroll quickly in a larger file (edit: actually, this turns out to be mainly due to my using relativenumber). In part it might be because vim2hs simply does more, but mainly it's probably because it doesn't do syntax synchronization at all. I'll look into adding it, but syncing can cause the highlighting to be incorrect. Thinking about it now though I realize it might be less of an issue in Haskell than it is in Python which is where I grew a distaste for syncing...

[dag]

BTW:

• Do you use any of these settings: relativenumber, cursorline, cursorcolumn, colorcolumn?
• Do you use GVim or terminal Vim?
• What kind of computer are you trying this on?
• What OS?
• Any particular Haskell source file you experience this on? How many lines is it? What syntax items does it include?

[dag]

You could try:

• Disable settings that slow down drawing (see previous comment)
• Play with :syn sync yourself (it turns out to be hard for me to figure out good settings here because I don't experience much lag myself either way)
• Disable vim2hs syntax options that you don't need (see README)
• Disable all vim2hs syntax options just to see if it helps

And please report back what you learn. ☺

[osa1]

I had moved to issues section to file a new issue, but since this is exactly what my problem is, even though it's not updated for last 6 months I decided to write here.

Editing with vim2hs, even when working on small files(eg. ~200 sloc) can be pain because of slow rendering speed. I think it's not only related with the amount of lines of code that vim2hs needs to highlight. Here are some thoughts I gained after some experiments:

• Vim2hs's highlighting and auto-indentations work great, but it's highlighter is very slow compared with vim's default Haskell highlighter. If I don't add these lines to my .vimrc:

  let g:haskell_conceal       = 0
  let g:haskell_quasi         = 0
  let g:haskell_interpolation = 0
  let g:haskell_regex         = 0
  let g:haskell_jmacro        = 0
  let g:haskell_shqq          = 0
  let g:haskell_sql           = 0
  let g:haskell_json          = 0
  let g:haskell_xml           = 0
  let g:haskell_hsp           = 0
  let g:haskell_tabular       = 0
  

  It's almost unusable even with small files(~200 sloc).

• Even if I add those lines to my .vimrc, highlighting gets incredibly slower when editing some specific types of code. For instance, the code in below screenshot, when I move the cursor to that code, rendering gets slower and it's almost impossible to move cursor around that code:

  

• As a more specific example, highlighting of lists is so slow that I have to kill my gvim when I edit this code: (this code is generated by Alex and not written by hand)

  

• And lastly, disabling cursorline improves the rendering speed, but it's still not usable in the case of screenshot above until disabling syntax completely.

Thanks for your effort for this project. I hope I can help identifying and solving performance problems and make vim2hs better!

[johntyree]

I haven't had the time to do this yet because I'm working on a different project that, sadly, doesn't involve Haskell, but thanks for the profiling.

In my own setup I do use cursorline and relativenumber in general, but I saw the same problems without them. For simple files, the syntax file in the gist I pasted above is sufficient, but then again, I'm sure the standard vim2hs one is too when you've turned everything off.

For what it's worth, my experience is that vim just cannot do long lines interactively. This is true for many different filetypes. My usual solution, if I absolutely must use long lines, is to temporarily unset the filetype :set ft= and work "raw". Then reset it when I'm done. Yes, it sucks.

[osa1]

OK, I just disabled cursorline permanently and it really helps(and I realized it wasn't helping much). Now I have usable Haskell highlighting(except the case of Alex generated long lists. fortunately I'm editing that kind of code only rarely).

[johntyree]

I'm not really satisfied with that though. I have large screens and cursorline helps a lot when I'm jumping between windows.

Is it such that enabling any of those options causes it to slow down, or just when many of them are turned on together?

Also, is there a reasonable way to profile this? Usually slow syntax highlighting comes down to just one rule somewhere that has exponential time complexity.

[osa1]

Is it such that enabling any of those options causes it to slow down, or just when many of them are turned on together?

I don't know, I didn't try them one by one, when I realized rendering is really slow, I just disabled all optional features and it helped(I never needed that features anyway).

[dag]

If I set lazyredraw I can scroll much faster, so maybe experiment with that. It might cause rendering problems if you use macros though, I haven't tested that. Lowering redrawtime might also help, but might also cause some things to not highlight. Try them out and report back!

[chreekat]

I find that setting foldmethod=manual makes a big difference. Here are results of a redraw with the two methods:

foldmethod=manual

TOTAL      COUNT  MATCH   SLOWEST     AVERAGE   NAME               PATTERN
0.079719   150    2       0.003570    0.000531  hsDeriving         \%(deriving\s*(\)\@<=.*,.*)\@=
0.011225   147    99      0.000236    0.000076  hsStructure        [[:punct:]]\@<!\%(->\|→\|<-\|←\|::\|∷\)[[:punct:]]\@!
0.009377   184    36      0.000314    0.000051  hsFold             ^\%(type\|newtype\|class\|instance\|data\)\@!\k\+.*\%(\n\s.*\|\n\)*[[:punct:]]\@<!=[[:punct:]]\@!
0.006505   48     0       0.000231    0.000136  hsStructure        [[:punct:]]\@<!\%(=>\|⇒\|-<<\?\)[[:punct:]]\@!
0.002186   314    166     0.000040    0.000007  hsType             ^\(\s*\)\%(default\s\+\)\?\%(\k\+\|([^[:alnum:]]\+)\)\_s*\(::\|∷\).*\%(\n\1\s.*\)*

foldmethod=syntax

TOTAL      COUNT  MATCH   SLOWEST     AVERAGE   NAME               PATTERN
1.964496   4478   24      0.005904    0.000439  hsDeriving         \%(deriving\s*(\)\@<=.*,.*)\@=
0.100560   5237   786     0.000506    0.000019  hsFold             ^\%(type\|newtype\|class\|instance\|data\)\@!\k\+.*\%(\n\s.*\|\n\)*[[:punct:]]\@<!=[[:punct:]]\@!
0.030429   5301   850     0.001173    0.000006  hsIdentifier       ^\k\+\ze.*\%(\n\s.*\|\n\)*[[:punct:]]\@<!=[[:punct:]]\@!
0.026430   5996   1113    0.000023    0.000004  hsFold             \ze\%(\s*\n\)\+\S
0.018317   5428   974     0.000057    0.000003  hsType             ^\(\s*\)\%(default\s\+\)\?\%(\k\+\|([^[:alnum:]]\+)\)\_s*\(::\|∷\).*\%(\n\1\s.*\)*

This is just a datapoint; I haven't thought about how to proceed with this information.

[johntyree]

@chreekat how did you produce those tables?

[chreekat]

:syntime. I just found out about it myself!

[edit]: I should have also mentioned which file I was in, and what line number, and what settings I had.

All of these were off: relativenumber, cursorline, cursorcolumn, colorcolumn.

I was on line 908 of ghc/compiler/main/DynFlags.hs, a file that has 4,262 lines.

The method used was setting foldmethod, :syntime clear, , :syntime report.

[chreekat]

As a workaround for slow folds, perhaps FastFold could be of some use.