Use custom search expression

[viktomas]

Problem

Currently, we use vscode.workspace.findFiles('**/*.md') to find all fiels that we are about to parse (parse.ts)

This finds files in the whole workspace which might not be desirable. My particular use case is that I've got two folders in my workspace:

• permanent notes
• reference notes

The permanent notes are about a particular topic and the reference notes are notes taken from a book or an article. permanent notes are almost always linking to reference notes (i.e. saying that I learnt that particular information in book XYZ).

The current parsing mechanism results in showing huge clusters of notes all linked by reference note (a book or an article) whilst I'm only interested in the connections between permanent notes.

Suggested solution

We could allow users to configure the findFiles expression. So instead of

parse.ts

const files = await vscode.workspace.findFiles(
    `**/*{${(getFileTypesSetting() as string[]).map((f) => `.${f}`).join(",")}}`
  );

we would use the configured value and only if that's missing, we would default to the current expression:

const searchExpression = getConfiguration("searchExpression");
const defaultExpression = `**/*{${(getFileTypesSetting() as string[]).map((f) => `.${f}`).join(",")}}`;
const files = await vscode.workspace.findFiles(searchExpression || defaultExpression);

This would be the most performant solution, but we could as well introduce some allow/deny list that would be used when we iterate through the findFiles() result:

for (const file of files) {
    const hiddenFile = path.basename(file.path).startsWith(".");
    if (!hiddenFile) {
      promises.push(fileCallback(graph, file.path));
    }
  }

How does this sound @tchayen ?

[tchayen]

Definitely having a findFiles regex is ok for me (as we have similar one for ignoring files).

It is also important to make sure that then those two won't collide. Probably it makes it sense to skip the other if one is provided.

@ingalless do you have some thoughts about it?

@viktomas BTW thanks for taking the time to make so detailed issue with code examples. I appreciate it a lot!

[raffomania]

I have a folder with old versions of my notes which is in my .gitignore - it would be nice to have an option to respect the .gitignore, as vscode does by default with e.g. the file picker. Having a custom regex would totally work as well, though!