! symbol does not work in .continueignore( Whitelist Rules )

[AfterStories]

Before submitting your bug report

• [x] I believe this is a bug. I'll try to join the Continue Discord for questions
• [ ] I'm not able to find an open issue that reports the same bug
• [ ] I've seen the troubleshooting guide on the Continue Docs

Relevant environment info

- OS:win
- Continue version:main branch

Description

using the latest main branch code for testing I think the latest code seems do something to the .continueignore file content, which makes the use of the ! symbol to implement the whitelist rule not work.

I first wrote a * in the .continueignore file to ignore all files,

# First, ignore all files
*

which means not to index any files. I think it worked. The indexing progress bar flashed and finished instantly.

Then I edited the .continueignore file as follows, hoping to specify that the gsrc/com directories should not be ignored ，I want to indexing it:

# 1. Ignore all 
*

# 2. Allow the gsrc directory itself
!gsrc/

# 3. Allow the gsrc/com directory itself
!gsrc/com/

# 4. Allow everything under gsrc/com (including subdirectories and files)
!gsrc/com/**

Then I clicked re-index

The indexing progress bar still flashed and finished instantly. I checked the index.sqlite file through the DB.Browser.for.SQLite tool and it was still empty

I think this configuration is correct, If this is incorrect please let me know ，I think it complies with the writing rules of .gitignore： first ignore everything, then use the ! symbol to allow something not to be ignore. Rules are matched line by line from top to bottom, and the later rules will overwrite the earlier ones.

And this configuration was worked when I used Continue 0.9 version before, now it does not work in the new code

To reproduce

It seems that the current use of ! symbol does not work in .continueignore

If I just write a directory normally in the .continueignore file:

gsrc/

This gsrc directory will indeed be ignored, and other files are indexed normally and saved in index.sqlite

Log output

[RomneyDa]

@AfterStories

There's a test for this in shouldIgnore.test.ts that seems to be passing but it's possible that it only works for the case of files or something. I'll take a look at this

Was implemented here https://github.com/continuedev/continue/pull/4531

[AfterStories]

@AfterStories

There's a test for this in shouldIgnore.test.ts that seems to be passing but it's possible that it only works for the case of files or something. I'll take a look at this

Was implemented here #4531

Thanks RomneyDa , looks like shouldIgnore.test.ts only test file, but not test folder In my usage scenario, I hope to use whitelist rules to set reserved 'folders', but my configuration seems to be ineffective.

Additionally, regarding this issue, I have put forward a suggestion in Discord: https://discord.com/channels/1108621136150929458/1376288829115465879

Writing complex rules to implement whitelist in ignore configuration can be cumbersome for users. I hope to add a configuration item directly in the UI that allows users to configure the whitelist for indexing, which is very useful for large projects. In my large project, there are hundreds of thousands of files, and most of them do not require embedding index

[AfterStories]

@RomneyDa Sorry I want to ask you agent about any updates on this issues curently the .continueignore file can only use ! symbol for file, but not folders This issue has troubled me for some week time

[uinstinct]

I was trying to solve this now but it seems single wildcard * character works to ignore all files and folders. Only if it is prepended with a directory, then it can un-ignore the particular files and folders. I verified this behaviour with git and the npm package as well.

[AfterStories]

Hi, @uinstinct thank you very much for your help. I'd like to reiterate that this problem is a very common scenario: for large projects with numerous folders, if the user only wants to index a few specific folders, use a whitelisting in the .continueignore file is undoubtedly the best solution. For example, in a project with 100 folders, if only folders A and B are need to be indexed, it's impossible to add all 98 folders path into the .continueignore file. Instead, the user should create a whitelisting rule like :

Rule 1: Ignore everything Rule 2: Allow A_folder Rule 3: Allow B_folder

Therefore, I considered adding a whitelisting configuration option in the index section of the UI settings page. Users could input root/A_folder, root/B_folder, and the whitelisting rules would be auto updated and written to the .continueignore file.

I've researched .gitignore file rules. (same rule with .continueignore file ) Git applies configuration from the top down, so Rule 1: Ignore everything should be at the top, followed by other rules at the bottom. My previous tests show that keeping specific files works, but not keeping folders.

[uinstinct]

Right. My mistake - I missed out the file ignoring part. When there something like the following:

*
!myfile.txt

all files will be ignored, except myfile.txt

However this will not happen if there is a folder (ex. !myfolder/). This behaviour is same with git and gitignore. If there is a single nested folder, such as src/abc/ then it can be unignored but not root levels.

does not work (in both continueignore and gitignore)

*
!abc/

this works

src/*
!src/abc/

---

I understand the use case you mentioned. Actually the gitignore behavior is followed for continueignore. Maybe we can make a feature request and can add it if more users want it.

[github-actions[bot]]

This issue hasn't been updated in 90 days and will be closed after an additional 10 days without activity. If it's still important, please leave a comment and share any new information that would help us address the issue.

[github-actions[bot]]

This issue was closed because it wasn't updated for 10 days after being marked stale. If it's still important, please reopen + comment and we'll gladly take another look!