Fix all tests

[litetex]

• [x] I carefully read the contribution guidelines and agree to them.
• [x] I have tested the API against NewPipe.

This PR fixes all test.

It also introduces an option to change the consent.

Supersedes #871

[AudricV]

The consent is only required for some playlists (#876) and mixes (#820) in EU.

That's why I'm strongly against allowing tracking in all YouTube requests we made. We should only do so where it is needed.

Remember that one of the main goals of the NewPipe projects is to enhance privacy.

[Stypox]

Yes. This PR does not allow any new consent, as it is false by default.

[opusforlife2]

What I see is that the consent variable has been wrapped in a setter/getter. I assume this is so that other NPE consumers can easily show a consent prompt, e.g. Piped?

[litetex]

What I see is that the consent variable has been wrapped in a setter/getter. I assume this is so that other NPE consumers can easily show a consent prompt, e.g. Piped?

1. Yes, in theory this should be possible now
2. The variable is static so it's technically no getter or setter
3. The static variable might change in the future for now it's primarily designed as a workaround so that the tests work again

[opusforlife2]

The variable is static so it's technically no getter or setter

I didn't get this bit.

[litetex]

@Stypox I'm done with my changes you can review again if you have time.

[litetex]

Update: So there have been a bunch of merge conflicts due to the recent hotfix.

These should be fixed now.

While fixing the conflicts I also fixed 2 additional problems:

1. YouTube Mixes no longer work for the specified (kurzgesagt) video
2. The suggestion/"Did you mean..." now longer works for the specified search term

[AudricV]

For https://github.com/TeamNewPipe/NewPipeExtractor/pull/882#issuecomment-1205007648, it seems you don't have to change something, as addYoutubeHeaders() is never called except by YoutubeMixPlaylistExtractor (this method should be called everywhere, don't touch anything here because in order to respect what I said, additional changes would be needed¹), so the consent cookie is only sent on mixes.

Note that the current code as it is would not fix #876, as this consent is not applied on regular playlists.

¹Here is my idea:

• Change generateConsentCookie() to generateConsentCookie(boolean), where the boolean parameter would be something like useYesValue. If useYesValue is true, return YES+; otherwise return PENDING= and the three random digits;
• Change addCookieHeader(@Nonnull final Map<String, List<String>> headers) to addCookieHeader(@Nonnull final Map<String, List<String>> headers, final boolean consentValue);
• Use the boolean value provided to call generateConsentCookie(boolean);
• Change addYoutubeHeaders(@Nonnull final Map<String, List<String>> headers) to addYoutubeHeaders(@Nonnull final Map<String, List<String>> headers, final boolean consentValue);
• Call addCookieHeader(headers, consentValue) in this method;
• Use addYoutubeHeaders(headers, YoutubeParsingHelper.isConsentAccepted()) in playlists and mixes (addYoutubeHeaders should be used anywhere, but that's not related to your PR at all);
• Note: singleton lists should not be used on Cookie header, to allow putting a reCAPTCHA one.

[litetex]

For #882 (comment), it seems you don't have to change something, as addYoutubeHeaders() is never called except by YoutubeMixPlaylistExtractor (this method should be called everywhere, don't touch anything here because in order to respect what I said, additional changes would be needed1), so the consent cookie is only sent on mixes.

Note that the current code as it is would not fix #876, as this consent is not applied on regular playlists.

1Here is my idea:

Should I react to this?

If yes: Your solution doesn't make much sense to me, you can simply call addCookieHeader or not ;) It also adds a lot of overhead for doing technically the same.

Also with foresighted to https://github.com/TeamNewPipe/NewPipeExtractor/pull/833: This would be an instance specific property and I think it can be handled in the settings of an instance with e.g. a toggle.

[litetex]

Note: I also fixed https://github.com/TeamNewPipe/NewPipeExtractor/issues/902 by

1. doing a check if the extracted JS is valid JavaScript and if not try the fallback (Regex)
2. Removing a } from the regex so that it matches correctly

[opusforlife2]

@litetex Could you update the PR description to mention all the changes this makes, such as https://github.com/TeamNewPipe/NewPipeExtractor/pull/882#issuecomment-1221596544?