TOC generation does not work properly when document headers have parens

[fmenozzi]

Description

When calling :WikiTocGenerate on a document whose header has parens, the parens are not URL-encoded and thus conflict with the markdown link syntax []().

Minimal working example

Suppose you have a document like so:

# Title (with parens)

## Heading

## Another Heading

Calling :WikiTocGenerate will produce the following TOC:

Contents:
* [Title (with parens)](#Title (with parens))
    * [Heading](#Title (with parens)#Heading)
    * [Another Heading](#Title (with parens)#Another Heading)

# Title (with parens)

## Heading

## Another Heading

These links produced in the TOC will not work when using Enter to try and follow them.

I believe the solution would simply be to URL-encode ( and ) in header names when generating the TOC.

[lervag]

Thanks. I agree that this example should preferably "just work". Right now, you will need to customize the url handler, see e.g. :help wiki-advanced-config-2 for an example of how to do it. The example is not precisely what you want here, though.

I'll look into it and see if I can find a simple, viable solution for the default behaviour. I think I should also add a builtin transformer and resolver that is more "in line" with standard Markdown expected behaviour.

[lervag]

Sorry for the long delay here!

I believe I've implemented what you want now. :WikiTocGenerate should now apply URL encoding for parantheses and the URLs will be decoded when the are resolved.

[fmenozzi]

Hey @lervag, thanks for looking into this. Unfortunately it doesn't appear that the original issue has been resolved. If I use the same minimal working example as before, calling :WikiTocGenerate producing a table of contents whose links are still unnavigable using Enter. Specifically,

# Title (with parens)

## Heading

## Another Heading

becomes

*Contents*
* [Title (with parens)](#title-(with-parens))
    * [Heading](#title-(with-parens)#heading)
    * [Another Heading](#title-(with-parens)#another-heading)

# Title (with parens)

## Heading

## Another Heading

It does not appear that the generated ToC links were URL-encoded.

[lervag]

Can you show me your wiki.vim configuration?

[fmenozzi]

Plug 'lervag/wiki.vim'

let g:wiki_root = '~/documents/notes'

nmap <leader>wf :WikiPages<cr>
nmap <leader>wt :WikiTags<cr>
nmap <leader>wc :WikiToc<cr>

" When creating links from text under the cursor, convert to lowercase and
" replace spaces with dashes.
let g:wiki_link_creation = {
            \ 'md': {
            \   'link_type': 'md',
            \   'url_extension': '.md',
            \   'url_transform': { x ->
            \       substitute(tolower(x), '\s\+', '-', 'g') },
            \ },
            \}

" Use fzf for searching pages, tags, etc., with nice markdown previews.
let g:wiki_select_method = {
            \ 'pages': function('wiki#fzf#pages'),
            \ 'tags': function('wiki#fzf#tags'),
            \ 'toc': function('wiki#fzf#toc'),
            \ 'links': function('wiki#fzf#links'),
            \}
let g:wiki_fzf_pages_opts = '--preview "mdcat {1}" --preview-window="up"'
let g:wiki_fzf_tags_opts = '--preview "mdcat {2..}" --preview-window="up"'

" Use custom tag parser to recognize tags on the first line in the form
" tags: tag1, tag2, tag3...
let g:wiki_tag_scan_num_lines = 1
let s:tag_parser = {
      \ 're_findstart': '\v^tags:\s*(\w*,\s*)*\zs\w+$',
      \}
function! s:tag_parser.match(line) dict abort
  return a:line =~# '^tags:'
endfunction
function! s:tag_parser.parse(line) dict abort
  let l:tagstring = matchstr(a:line, '^tags:\s*\zs.*')
  return split(l:tagstring, ',\s*')
endfunction
function! s:tag_parser.make(taglist, ...) dict abort
  if empty(a:taglist)
    return ''
  endif
  return 'tags: ' . join(a:taglist, ", ")
endfunction
let g:wiki_tag_parsers = [s:tag_parser]

Looking at it now, perhaps my overwriting g:wiki_link_creation is the problem? My guess is that I need to incorporate some kind of URL-encoding in the url_transform function. I think I took this example from the docs so I forgot that I had even made this change.

[lervag]

Looking at it now, perhaps my overwriting g:wiki_link_creation is the problem? My guess is that I need to incorporate some kind of URL-encoding in the url_transform function. I think I took this example from the docs so I forgot that I had even made this change.

Yes, precisely. You can try to just remove it at first. And if you find you want more adjustment to the transforms I can help you make that.

Notice that you don't have to include the link_type and url_extension, since these are not changed from the default value.

[fmenozzi]

Sounds good. Thanks again for implementing this feature!

[artkpv]

@lervag This commit breaks my links :( I have links in Cyrillic. And I think wiki#url#utils#url_encode_specific(a:url, '()') replaces Cyrillic chars with URL entities, which is not the chars in the filesnames on my disk. Example:

- [Информатика](02 Информатика, Computer Science.md)

When I revert this commit, 197282b, i.e. I checkout the commit just before it, my links work.

My config:

  {
    'lervag/wiki.vim',
    lazy = false, -- we don't want to lazy load VimTeX
    config = function()
      vim.g.wiki_root = '/home/art/mydir/notes/'

      vim.g.wiki_select_method = {
        pages = require('wiki.telescope').pages,
        tags = require('wiki.telescope').tags,
        toc = require('wiki.telescope').toc,
        links = require('wiki.telescope').links,
      }
      vim.g.wiki_filetypes = { 'md' }

      vim.g.wiki_link_creation = {
        md = {
          link_type = 'md',
          url_extension = '.md',
        },
        _ = {
          link_type = 'md',
          url_extension = '.md',
        },
      }

      vim.g.wiki_month_names = {
        '01.Январь',
        '02.Февраль',
        '03.Март',
        '04.Апрель',
        '05.Май',
        '06.Июнь',
        '07.Июль',
        '08.Август',
        '09.Сентябрь',
        '10.Октябрь',
        '11.Ноябрь',
        '12.Декабрь',
      }
      vim.g.wiki_toc_title = 'Содержание'
      vim.g.wiki_journal = {
        -- index_use_journal_scheme = false,
        name = '2024',
        frequency = 'daily',
        date_format = {
          daily = '%Y-%m-%d',
        },
        root = '~/mydir/notes/2024',
      }

      vim.g.init_my_wiki_config = function()
        vim.opt.number = false
        vim.opt.relativenumber = false
        vim.opt.cursorline = false
        vim.opt.cursorcolumn = false
        vim.opt.foldcolumn = '0'
        vim.opt.list = false
        vim.opt.foldlevel = 1
        vim.diagnostic.enable(false)
        vim.cmd.colorscheme 'neobones'
        vim.o.laststatus = 1
        vim.o.ruler = false
        vim.cmd 'Copilot disable'
        vim.cmd 'LspStop'

        -- au! BufWritePost ~/mydir/notes/* !git add "%";git commit -m "Auto commit of %:t." "%"

        vim.api.nvim_create_autocmd('BufWritePost', {
          desc = 'Commit wiki changes',
          group = vim.api.nvim_create_augroup('wiki-commit-hook', { clear = true }),
          pattern = { '*.md' },
          callback = function()
            vim.cmd '!git add "%";git commit -m "Auto commit of %:t." "%"'
          end,
        })
      end
      vim.keymap.set('n', '<leader>wq', vim.g.init_my_wiki_config, { desc = 'Wiki [Q]uite mode on' })

      local function wiki_page_rename()
        local current_file = vim.fn.expand '%'
        local current_name = vim.fn.expand '%:t'
        local name = vim.fn.input('Enter new name (' .. current_name .. '): ')

        if name ~= '' then
          vim.fn.termopen { './ren.sh', current_file, name }
        end
      end

      vim.keymap.set('n', '<leader>wpr', wiki_page_rename, { desc = 'Rename wiki page' })
    end,
  },

[lervag]

@artkpv Could you please open a new issue for this? I won't have time to look at it immediately, so it would help a lot to have a fresh issue when I start looking - that way I don't have to spend time reading a long issue thread for context.

[artkpv]

https://github.com/lervag/wiki.vim/issues/395#issue-2757600281