Age restricted videos are broken again

[Tyrrrz]

Version

6.0.4 & master

Details

With the removal of get_video_info we currently don't have a way to resolve age-restricted videos anymore.

Steps to reproduce

// Arrange
var youtube = new YoutubeClient();

// Act
var manifest = await youtube.Videos.Streams.GetManifestAsync("SkRSXFQerZs");

// Assert
manifest.Streams.Should().NotBeEmpty();

[89z]

Request like this works as expected:

POST /youtubei/v1/player HTTP/1.1
Host: www.youtube.com
X-Goog-Api-Key: AIzaSyAO_FJ2SlqU8Q4STEHLGCilw_Y9_11qcW8

{"context":{"client":{"clientName":"ANDROID","clientScreen":"EMBED",
"clientVersion":"16.05"},"thirdParty":{"embedUrl":"https://www.youtube.com"}},
"videoId":"SkRSXFQerZs"}

[Tyrrrz]

@89z can you please make a PR?

[Yuvraj786]

Bro, how does your http requests work, I mean how do you not get banned by youtube 402 status code. I need to understand, to solve this issue. Mail me to explain - [email protected]

[Yuvraj786]

Bro, I tried using requests module in python. I even used rotating paid proxies, But I just keep getting banned status code 403. How do you resolve that, & the request is sent from which proxy or it is sent from the user's ip?

[Yuvraj786]

https://www.youtube.com/watch?v=vp_ZxQ0CTJk&list=PLG49S3nxzAnmpdmX7RoTOyuNJQAb-r-gd&index=4&ab_channel=ProfessorMesser

On Wed, 4 Aug 2021 at 17:41, Steven Penny @.***> wrote:

example video?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/Tyrrrz/YoutubeExplode/issues/581#issuecomment-892605732, or unsubscribe https://github.com/notifications/unsubscribe-auth/AO6TPMSJ4CMOYNS2VDRQ5UDT3EU5JANCNFSM5BFEVP6A . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email .

[Yuvraj786]

I didn't understand then If I try it with "requests" module in python. If I send more than 2 requests in a minute I get status code 403 . Are you changing user-agent ? or rotating proxies or using some sort of reverse engineered requests, Bcuz I tried to use this code:

for i in range 100: Response = requests.get(" https://www.youtube.com/watch?v=vp_ZxQ0CTJk&list=PLG49S3nxzAnmpdmX7RoTOyuNJQAb-r-gd&index=4&ab_channel=ProfessorMesser ") print(Response.status_code)

Just that.... Is anything wrong with my code?

On Wed, 4 Aug 2021 at 20:44, Steven Penny @.***> wrote:

Works fine:

POST /youtubei/v1/player HTTP/1.1 Host: www.youtube.com X-Goog-Api-Key: AIzaSyAO_FJ2SlqU8Q4STEHLGCilw_Y9_11qcW8

{"context":{"client":{"clientName":"ANDROID","clientVersion":"16.05"}}, "videoId":"vp_ZxQ0CTJk"}

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/Tyrrrz/YoutubeExplode/issues/581#issuecomment-892743623, or unsubscribe https://github.com/notifications/unsubscribe-auth/AO6TPMTOOBOLTCRAJJEQO73T3FKOHANCNFSM5BFEVP6A . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email .

[Tyrrrz]

Thanks @89z

[Tyrrrz]

Looks like downloads are way faster now, but age-restricted videos sometimes still fail, with both old player response (from watch page) and new (from android endpoint). It does succeed about 95% of the time though, but still.

[89z]

I may have a fix, if you can name an example that fails

[Tyrrrz]

I may have a fix, if you can name an example that fails

SkRSXFQerZs fails about once every 10-15 requests right now

[89z]

I just made this request 15 times with no issue:

POST /youtubei/v1/player HTTP/1.1
Host: www.youtube.com
X-Goog-Api-Key: AIzaSyAO_FJ2SlqU8Q4STEHLGCilw_Y9_11qcW8

{"context":{"client":{"clientName":"ANDROID","clientScreen":"EMBED",
"clientVersion":"16.43.34"},"thirdParty":{"embedUrl":"https://www.youtube.com"}},
"videoId":"SkRSXFQerZs"}

Are you just hammering requests one after another? Usually its good etiquette to sleep between requests, if you are doing a bunch.

[Tyrrrz]

I just made this request 15 times with no issue:

POST /youtubei/v1/player HTTP/1.1
Host: www.youtube.com
X-Goog-Api-Key: AIzaSyAO_FJ2SlqU8Q4STEHLGCilw_Y9_11qcW8

{"context":{"client":{"clientName":"ANDROID","clientScreen":"EMBED",
"clientVersion":"16.43.34"},"thirdParty":{"embedUrl":"https://www.youtube.com"}},
"videoId":"SkRSXFQerZs"}

Are you just hammering requests one after another? Usually its good etiquette to sleep between requests, if you are doing a bunch.

Yeah but it still responds with "This video is not appropriate for all audiences". That POST will succeed, but inside of it you will see a fail status and no video streams.

[Martymoose98]

Is this broken again? I'm running 6.2.1.

[89z]

Previous workaround are no longer valid. Instead you must authenticate:

POST /youtubei/v1/player HTTP/1.1
Host: www.youtube.com
Authorization: Bearer ya29.A0AVA9y1uNYvKYhalRrVbp2LHkyUSaW42G3qHlN_Tgti-V2gC1...

{
 "context": {
  "client": {
   "clientName": "ANDROID",
   "clientVersion": "17.32.35"
  }
 },
 "racyCheckOk": true,
 "videoId": "SkRSXFQerZs"
}

Alternatively, you can use the TVHTML5_SIMPLY_EMBEDDED_PLAYER client, but it has its own drawbacks.

[Martymoose98]

Previous workaround are no longer valid. Instead you must authenticate:

POST /youtubei/v1/player HTTP/1.1
Host: www.youtube.com
Authorization: Bearer ya29.A0AVA9y1uNYvKYhalRrVbp2LHkyUSaW42G3qHlN_Tgti-V2gC1...

{
 "context": {
  "client": {
   "clientName": "ANDROID",
   "clientVersion": "17.32.35"
  }
 },
 "racyCheckOk": true,
 "videoId": "SkRSXFQerZs"
}

Alternatively, you can use the TVHTML5_SIMPLY_EMBEDDED_PLAYER client, but it has its own drawbacks.

So there's no api for this in the lib? How would do achieve this? Pardon my ignorance, I'm not very good with networking. Ok so it looks like you need to Create the YoutubeClient with a custom HttpClient and add custom cookies to it but which cookies would that be?

[xBaank]

@Martymoose98 From what i have seen, to authenticate with cookies we need two cookies: SAPISID or __Secure-3PAPISID (These two have the same value) and __Secure-3PSID.

//Needed to login
const string origin = "https://www.youtube.com";
request.Headers.Add("Cookie", $"CONSENT=YES+cb; YSC=DwKYllHNwuw; SAPISID={Sapisid}; __Secure-3PAPISID={Sapisid}; __Secure-3PSID={Psid};");
request.Headers.Add("Authorization", $"SAPISIDHASH {GenerateSidBasedAuth(Sapisid, origin)}");
request.Headers.Add("Origin", origin);
request.Headers.Add("X-Origin", origin);
//We can use origin as the referer
request.Headers.Add("Referer", origin);

private static string GenerateSidBasedAuth(string sid, string origin)
{
    var date = DateTimeOffset.UtcNow.ToUnixTimeMilliseconds();

    var timestamp = date / 1000;
    var sidBytes = Encoding.UTF8.GetBytes($"{timestamp} {sid} {origin}");
    var sidHash = SHA1.HashData(sidBytes);
    //To hex string is only compatible with net 5 and 6, look for workaround
    var sidHashBase64 = Convert.ToHexString(sidHash).ToLower();
    return $"{timestamp}_{sidHashBase64}";
}

The problem comes when using a custom httpclient, if I set the cookie header in the httpclient defaultrequestheaders and then it is setted again in the request again, two cookie headers will be send and the request will be unauthorized.

https://github.com/Tyrrrz/YoutubeExplode/blob/477ee322ec5ea4605d0d048a5be30054ce3f5f26/YoutubeExplode/Bridge/YoutubeControllerBase.cs#L33

Should we check if httpclient already has the header? or add a way for people who wants authorized request to add those cookie values and then set them in each request.

BTW the above code was adapted from https://github.com/zerodytrash/Simple-YouTube-Age-Restriction-Bypass/blob/main/account-proxy/lib/innertubeApi.js

[89z]

I would avoid using cookie if possible. If you must use authentication, I think OAuth approach would be better, using the Authorization header, or the TVHTML5_SIMPLY_EMBEDDED_PLAYER client does not require authentication

[xBaank]

Correct me if I am wrong but the TVHTML5_SIMPLY_EMBEDDED_PLAYER drawbacks are:

• Throttling from what I have tested
• Embed & age restricted videos are not bypassed with this client (I guess there are no clients that can get these types of video)

but for now it could be an alternative of the android embed client version for those who doesn't want to use authentication.

[xBaank]

And about the OAuth approach, How do we get the token?

[Tyrrrz]

The problem comes when using a custom httpclient, if I set the cookie header in the httpclient defaultrequestheaders and then it is setted again in the request again, two cookie headers will be send and the request will be unauthorized.

In theory, it should allow setting cookies (or any other header) multiple times, so maybe there's a different way that works without resetting the value. Headers are inherently string⇿string[] pairs.

[89z]

Correct me if I am wrong but the TVHTML5_SIMPLY_EMBEDDED_PLAYER drawbacks are:

• Throttling from what I have tested

true, but this can be avoided by decrypting the n parameter, as is done with other projects, or by using concurrent download (on the order of 99 streams).

• Embed & age restricted videos are not bypassed with this client (I guess there are no clients that can get these types of video)

that's not my experience, see here:

https://github.com/zerodytrash/YouTube-Internal-Clients

And about the OAuth approach, How do we get the token?

I have an implementation here:

https://github.com/89z/mech/blob/v1.55.8/youtube/header.go

should be pretty straight forward to translate it.