Feat: Add support for scoping jackett query with a `jackett_filter_index`

[disaac]

Currently nefarious performs searches with jackett using the all/results endpoint which searches all available indexers. In some cases there might be problematic indexers used for specific reasons that slow down or halt queries made by nefarious. Scoping queries to specific indexers would address this.

Issue Summary Unable to scope queries from nefarious to specific indexers in jackett

Proposed solution Jackett supports a filter-indexers endpoint allowing a search to be scoped to a set of indexers based on query parameters such as indexer tags. A setting could be added to the Jackett configuration nefarious_settings.jackett_filter_index allowing a query string to be set that overrides /all/ in jackett.py. By default if no value is set it will remain all

example before

return 'http://{}:{}/api/v2.0/indexers/all/results'.format(
        nefarious_settings.jackett_host, nefarious_settings.jackett_port)

after

return 'http://{}:{}/api/v2.0/indexers/{}/results'.format(
        nefarious_settings.jackett_host, 
        nefarious_settings.jackett_port,
        nefarious_settings.jackett_filter_index)

example nefarious_settings.jackett_filter_index value of !tag:appsonly+lang:en

The above would scope the query to only indexers that have a lang of english and no tag set as appsonly

[lardbit]

This is a good feature suggestion. I didn't realize jackett's api accepts a status:healthy parameter. I should probably make that a default.

This seems pretty straight forward. I'll tackle it when I have some free time, or welcome contributions as well.

[disaac]

This is a good feature suggestion. I didn't realize jackett's api accepts a status:healthy parameter. I should probably make that a default.

@lardbit Ah didn't notice that either.

or welcome contributions as well.

I added the index-filter in PR #293 and added it to verifications so it will fail appropriately if a bad value is used. PR #293 should maintain current behavior but good point also scoping it to status healthy would be a sane default as well. In my situation I tag my indexers since some of them are ones that aren't applicable to media such as tv and movies even though they report the categories in some cases. This should tackle that use case.

[lardbit]

Thanks for kicking this off. Are you running/testing this locally? We'll want to add to Django's migrations via python manage.py makemigrations which will create the necessary migration python files, then you'll run python manage.py migrate to actually apply the changes locally (this all happens in docker for regular users). I can help fix this in your PR, but I'm at a loss as how to contribute changes to your branch. Do you know how that works?

And lastly we'll need to update the GUI to accept the user's custom jacket filter.

[disaac]

Thanks for kicking this off. Are you running/testing this locally? We'll want to add to Django's migrations via python manage.py makemigrations which will create the necessary migration python files, then you'll run python manage.py migrate to actually apply the changes locally (this all happens in docker for regular users).

And lastly we'll need to update the GUI to accept the user's custom jacket filter.

@lardbit Ah was just building this on my fork with your action updated to test actually in my repo would def fail then. Also noticed this so entered another defect which you might run into as well. Issue #294

I can help fix this in your PR, but I'm at a loss as how to contribute changes to your branch. Do you know how that works?

To add to my changes I would just add my remote locally to your checkout then push my branch up to your repo as the same name or whatever you want. I could then just start making any contributions to that branch.

I am tied up for rest of the day but can ad the changes you mentioned later this week no rush and thanks so much for the feedback

@lardbit other option just clone my repo make your change and push back to my branch on my repo which is already in the PR so will update

git clone [email protected]:disaac/nefarious.git disaac-nefarious
cd disaac-nefarious
git fetch
git branch -v -a
git switch -c disaac/fixes-issue-292-add-jackett-filter-index-support origin/disaac/fixes-issue-292-add-jackett-filter-index-support

[lardbit]

I followed your steps to contribute to your branch but it's being rejected on the final git push stage. I cloned your repo, switched to the proper branch, made some changes, and did a git push:

ERROR: Permission to disaac/nefarious.git denied to [me]. fatal: Could not read from remote repository.

I think your repo would explicitly need to allow me to contribute.

However, I just added you as a contributor so you should be able to push your branch directly to this repo. Then, I'll be able to commit to your branch (on this repo).

[disaac]

@lardbit thanks i pushed it up sorry yeah figured the fork would automatically give you push access but nope.

[lardbit]

Great, can you update your PR to use the branch disaac/fixes-issue-292-add-jackett-filter-index-support in this repo? I added a commit to hook into django's db migration system.

Now we'll both be able to commit to the same branch/PR.

[disaac]

Great, can you update your PR to use the branch disaac/fixes-issue-292-add-jackett-filter-index-support in this repo? I added a commit to hook into django's db migration system.

Now we'll both be able to commit to the same branch/PR.

Yep will take a look now sorry wasn't able to find time last week will take a look now

[disaac]

@lardbit OK I updated the PR on this repo with some other fixes that were needed to get it to work/build added a couple new defects and documented the related chore defects in the PR as well for reference. Though This PR should as done be ready for merge tested locally as recommended in the DEVELOPMENT.md.

Some things of interest I noted is if building on an arm64 macOS system you will need to use the latest cryptography package and remove the version lock in addition to the other pip/python package related issues I created but didn't put an issue in for this since it's an edge case only related to building on macOS arm64 systems do the unavailability of the cryptography wheel when using a version as old as the one pinned.

[lardbit]

Merged with #297.

Some things of interest I noted is if building on an arm64 macOS system you will need to use the latest cryptography package and remove the version lock in addition to the other pip/python package related issues I created but didn't put an issue in for this since it's an edge case only related to building on macOS arm64 systems do the unavailability of the cryptography wheel when using a version as old as the one pinned.

Interesting. Well, if you have any ideas on how to update the build process to be compatible with macOS arm64 let me know.