feat: railsignore added for config loading of LLMRails / RailsConfig.

[ajanitshimanga]

PR adding support for a .railsignore to ignore .co files in LLMRails / RailsConfig.

For issue: https://github.com/NVIDIA/NeMo-Guardrails/issues/277

Tests added and passing fully.

Two follow ups after this PR are:

1. Looking into add ignoring .py files for ActionDispatcher
2. Discuss if we want .kb files to be specified as a follow up.

[Pouyanpi]

Thank you very much @ajanitshimanga for your nice PR. I've skimmed through the changes and it looks good, with minor changes we'll be good to go. I'll add comments in the review later today.

Some nice to have features:

• [ ] aggregate patterns from multiple .railsignore files.
• [ ] Other potential ignore patterns like directory specific patterns (i.e., patterns ending with /), patterns with leading slashes to denote paths relative to the root, etc. Shall we support negation?
• [ ] ignore any file that is matched with the patterns (no need to check the file extension)

[ajanitshimanga]

Thank you very much @ajanitshimanga for your nice PR. I've skimmed through the changes and it looks good, with minor changes we'll be good to go. I'll add comments in the review later today.

Some nice to have features:

• [x] aggregate patterns from multiple .railsignore files.
• [x] Other potential ignore patterns like directory specific patterns (i.e., patterns ending with /), patterns with leading slashes to denote paths relative to the root, etc. Shall we support negation?
• [X] ignore any file that is matched with the patterns (no need to check the file extension)

Thanks for the reply, when looking through the nice to have features I wanted to follow up with a few questions to gain alignment:

1. With respect to adding aggregating patterns in multiple .railsignore files, what is the direct benefit here? I may be missing the use case benefit.
2. I think this would be a good follow up to expand patterns but to leave this out of scope for now, does that sound reasonable?
3. This is a good idea, I've relaxed the file extension constraint in a new commit to achieve this.

[Pouyanpi]

1. With respect to adding aggregating patterns in multiple .railsignore files, what is the direct benefit here? I may be missing the use case benefit.

It allows the user to ignore specific patterns for a specific config directory, for example I want to exclude kb from one specific config called 'config_1' but ignore all flows.co files, so I need a .railsignore for the 'config_1' with kb included and the .railsignore at root with flows.co.

2. I think this would be a good follow up to expand patterns but to leave this out of scope for now, does that sound reasonable?

Yes, we can leave it as is, and expand it later.

3. This is a good idea, I've relaxed the file extension constraint in a new commit to achieve this.

Thanks!

[Pouyanpi]

@ajanitshimanga Thank you once again for you contribution!

I made some minor changes to your PR and will merge it latest tomorrow. It will be available with 0.11.0 release.

Please have a look and let me know if you think there is any issue.

So mainly the config.py was changed to improve the performance, it is expensive to find the ignorefile every time so it is upper in the code. Also, we start from the config_path and goes higher to the root to find the railsignore file. The rest are refactoring to respect this change.

It allows having multiple .railsignore per different config. Thanks and look forward to your future contributions

[ajanitshimanga]

Thank you as well @Pouyanpi for the changes you made with refactoring and improving performance.

The refactor looks good to me, I left some comments and I'll put out a quick commit for those minor changes so you can review and merge when you're online.

Cheers! I look forward to further contributions in the future too! :)