python-panasonic-comfort-cloud Authentication no longer functional

Hi @lostfields

The library is no longer able to authenticate after a recent update on the Panasonic Comfort Cloud side.

Logs:

pcomfortcloud user passwd
Traceback (most recent call last):
  File "/opt/homebrew/bin/pcomfortcloud", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/pcomfortcloud/__main__.py", line 202, in main
    session.login()
  File "/opt/homebrew/lib/python3.11/site-packages/pcomfortcloud/session.py", line 99, in login
    self._create_token()
  File "/opt/homebrew/lib/python3.11/site-packages/pcomfortcloud/session.py", line 135, in _create_token
    raise ResponseError(response.status_code, response.text)
pcomfortcloud.session.ResponseError: Invalid response, status code: 403 - Data: <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<HTML><HEAD><META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=iso-8859-1">
<TITLE>ERROR: The request could not be satisfied</TITLE>
</HEAD><BODY>
<H1>403 ERROR</H1>
<H2>The request could not be satisfied.</H2>
<HR noshade size="1px">
Request blocked.
We can't connect to the server for this app or website at this time. There might be too much traffic or a configuration error. Try again later, or contact the app or website owner.
<BR clear="all">
If you provide content to customers through CloudFront, you can find steps to troubleshoot and help prevent this error by reviewing the CloudFront documentation.
<BR clear="all">
<HR noshade size="1px">
<PRE>
Generated by cloudfront (CloudFront)
Request ID: qhQd9CECTFw0OASNLkavpSlzzcevRATZ0xRdeM1sM_K-f5OzEGR1oA==
</PRE>
<ADDRESS>
</ADDRESS>
</BODY></HTML>

This has also been raised with the HA custom integration here where this library is used.

Jun 17 '24 11:06 craibo

I've seen that Panasonic are trying to migrate user over to use OAuth2 where you can link your account to Google etc. If this is going forward, I think it will be harder and harder for us to have a library like this without support for third-party developers by Panasonic.

Jun 17 '24 13:06 lostfields

It looks like the HA community have identified what has changed but would rely on coding done in this project to resolve. https://github.com/sockless-coding/panasonic_cc/issues/191#issuecomment-2174505567

Jun 18 '24 00:06 majurgens

@majurgens I'll be looking a little into the HA Integration the trickiest part might be a user friendly way of getting the oauth code, I can do it through postman or curl, an option would be doing the full login process programmatically this way we can control the flow and ignore redirects, we would only need it once though after that it's standard oauth refresh.

Jun 18 '24 09:06 nelsongraca

the accsmart api endpoint was updated to only support Bearer token it seems. the legacy token that one could fetch with the login & password is defunct.

by intercepting the traffic from the android app (for a recipie see https://github.com/sockless-coding/panasonic_cc/issues/191#issuecomment-2174505567), I found the following minimal workflows that need to be supported to get a bearer token and use it in subsequent calls. the workflows assume that you already fully configured your account in the app (f.e. enabled 2FA etc).

the app itself uses the auth0 SDK to do handle the oauth authorization. the authenticator webservice also publishes the standard oauth definitions (https://authglb.digital.panasonic.com/.well-known/openid-configuration), so maybe using a third party oauth client would be the easiest route...

Workflow: Login in the App

POST to https://authglb.digital.panasonic.com/usernamepassword/login

Notes:

not sure if the cookies are required at all - they are set in later workflows, but are now present in the Webview used by the app when the login process is repeated
the form is protected by google recaptcha
the client_id header value is a hardcoded string in the app, and possibly changes with each app version.

Request Header:

"Auth0-Client: eyJuYW1lIjoiYXV0aDAuanMtdWxwIiwidmVyc2lvbiI6IjkuMjMuMiJ9" (value is base64-encoded string of the used SDK: {"name":"auth0.js-ulp","version":"9.23.2"}
"Content-Type: application/json"

Request Cookies:

_csrf
did
did_compat
auth0
auth0_compat

Request Body:

{
  "client_id": "Xmy6xIYIitMxngjB2rHvlm6HSDNnaMJx",
  "redirect_uri": "panasonic-iot-cfc://authglb.digital.panasonic.com/android/com.panasonic.ACCsmart/callback?lang=en",
  "tenant": "pdpauthglb-a1",
  "response_type": "code",
  "scope": "openid offline_access comfortcloud.control a2w.control",
  "audience": "https://digital.panasonic.com/Xmy6xIYIitMxngjB2rHvlm6HSDNnaMJx/api/v1/",
  "_csrf": "cIf...WjI",
  "state": "hKF...KeA",
  "_intstate": "deprecated",
  "nonce": "dW3...OSg",
  "username": "<my-email>",
  "password": "<my-password>",
  "lang": "en",
  "connection": "PanasonicID-Authentication"
}

Response Body: html form with 3 hidden inputs:

wa
wresult
wctx

POST https://authglb.digital.panasonic.com/login/callback

Request Header:

"Content-Type: application/x-www-form-urlencoded"

Request Cookies:

did
did_compat
auth0
auth0_compat

Request Body: <url-encoded values of the form fields "wa", "wresult", "wctx" of the previous step>

Response Header:

Status 302
"Location: /authorize/resume?state=Bm6...T2O-"

Response Cookies:

auth0
auth0_compat

GET https://authglb.digital.panasonic.com/authorize/resume?state=Bm6...T2O-

Request Cookies:

did
did_compat
auth0
auth0_compat

Response Header:

Status 302
"Location: panasonic-iot-cfc://authglb.digital.panasonic.com/android/com.panasonic.ACCsmart/callback?code=vUc...SyQ"

Response Cookies:

auth0
auth0_compat

POST https://authglb.digital.panasonic.com/oauth/token

Request Header:

"Auth0-Client: eyJuYW1lIjoiYXV0aDAuanMtdWxwIiwidmVyc2lvbiI6IjkuMjMuMiJ9"
"Content-Type: application/json"
"User-Agent: okhttp/4.10.0"

Request Body:

{
  "scope": "openid",
  "client_id": "Xmy6xIYIitMxngjB2rHvlm6HSDNnaMJx",
  "grant_type": "authorization_code",
  "code": "vUc...o7n",
  "redirect_uri": "panasonic-iot-cfc://authglb.digital.panasonic.com/android/com.panasonic.ACCsmart/callback",
  "code_verifier": "IEg...Ug4"
}

Response Header:

Status 200

Response Cookies:

did
did_compat

Response Body:

{
  "access_token": "eyJ...PwQ",
  "refresh_token": "v1.M...yII",
  "id_token": "eyJ...fHQ",
  "scope": "openid comfortcloud.control a2w.control offline_access",
  "expires_in": 86400,
  "token_type": "Bearer"
}

Workflow: Refresh access token

POST https://authglb.digital.panasonic.com/oauth/token

Request Header:

"Auth0-Client: eyJuYW1lIjoiYXV0aDAuanMtdWxwIiwidmVyc2lvbiI6IjkuMjMuMiJ9"
"Content-Type: application/json"
"User-Agent: okhttp/4.10.0"

Request Body:

{
  "scope": "openid offline_access comfortcloud.control a2w.control",
  "client_id": "Xmy6xIYIitMxngjB2rHvlm6HSDNnaMJx",
  "refresh_token": "v1.M...OVg",
  "grant_type": "refresh_token"
}

Response Header:

Status 200

Response Cookies:

did
did_compat

Response Body:

{
  "access_token": "eyJ...TSw",
  "refresh_token": "v1.M...HYk",
  "id_token": "eyJ...8XA",
  "scope": "openid comfortcloud.control a2w.control offline_access",
  "expires_in": 86400,
  "token_type": "Bearer"
}

Workflow: Get ACCSmart Client-Id

POST https://accsmart.panasonic.com/auth/v2/login

Notes:

this request only needs to be done when the ACCSmart client id is not yet known

Request Header:

"Content-Type: application/json;charset=utf-8"
"User-Agent: G-RAC"
"X-APP-NAME: Comfort Cloud"
"X-APP-TIMESTAMP: 2024-06-18 13:27:51"
"X-APP-TYPE: 1"
"X-APP-VERSION: 1.20.0"
"X-CFC-API-KEY: ece...409"
"X-User-Authorization-V2: Bearer eyJ...PwQ"

Request Body:

{ 
    "language": 0 
}

Response Header

Status 200

Response Body:

{
  "country": "CH",
  "extUsrId": "d04...b41",
  "clientId": "048...a68",
  "language": 0
}

Workflow: Get ACCSmart Device Groups

GET https://accsmart.panasonic.com/device/group

Request Header:

"Content-Type: application/json;charset=utf-8"
"User-Agent: G-RAC"
"X-APP-NAME: Comfort Cloud"
"X-APP-TIMESTAMP: 2024-06-18 13:27:51"
"X-APP-TYPE: 1"
"X-APP-VERSION: 1.20.0"
"X-CFC-API-KEY: 9e6...d06"
"X-Client-Id: 048...a68"
"X-User-Authorization-V2: Bearer eyJ...PwQ"

Response Header:

Status 200

Response Body:

{
  "uiFlg": false,
  "groupCount": 1,
  "groupList": [
    ...
  ]
}

Jun 18 '24 12:06 heldchen

not yet clear where the Auth0-Client value comes from. could be a device-calculated unique identifier

@heldchen it might be this https://community.auth0.com/t/auth0client-parameter-in-the-authorize-request/114446

The auth0client parameter contains the telemetry information sent by the Auth0 SDK. It can be decoded with base64 to reveal the information it carries.

Jun 18 '24 12:06 danielcherubini

so maybe using a third party oauth client would be the easiest route...

https://github.com/auth0/auth0-python

That might be the best route, if the above missing value is indeed comes from the Auth0 SDK, then using the above pythonSDK should provide it..

Jun 18 '24 12:06 danielcherubini

so maybe using a third party oauth client would be the easiest route...

https://github.com/auth0/auth0-python

That might be the best route, if the above missing value is indeed comes from the Auth0 SDK, then using the above pythonSDK should provide it..

as someone who had to integrate with oauth, yes don't reinvent the wheel if a library exists use it.

Jun 18 '24 13:06 nelsongraca

@heldchen Wow! Great work!

I think that Auth0-Client is also hardcoded. Look here: https://github.com/cjaliaga/aioaquarea/blob/523aaba080e2f2e9e315de86ed728bcc270c6527/aioaquarea/core.py#L274C15-L274C32

can someone write here this value?

In last step (Get ACCSmart Device Groups) X-User-Authorization-V2 it is token and it have login creditentials included?
In one step before last (Get ACCSmart Client-Id) you get client id but you dont use it in last step?

Jun 18 '24 14:06 mkz212

@mkz212 thanks for finding a value of that header. Looks like I was right with what I found... It is just a base64 encoded of what SDK and version was used.

Decoded it is. {"name":"auth0.js-ulp","version":"9.23.2"}

Jun 18 '24 14:06 danielcherubini

I think that Auth0-Client is also hardcoded. Look here: https://github.com/cjaliaga/aioaquarea/blob/523aaba080e2f2e9e315de86ed728bcc270c6527/aioaquarea/core.py#L274C15-L274C32

great, added to the writeup

In last step (Get ACCSmart Device Groups) X-User-Authorization-V2 it is token and it have login creditentials included?

it's a bearer token, a temporary token assigned to your user account that is valid for 24h, then needs to be refreshed. your credentials are not encoded in this, it's a separate token.

In one step before last (Get ACCSmart Client-Id) you get client id but you dont use it in last step?

good catch, forgot to add it to the header list in the writeup, updated now.

Jun 18 '24 14:06 heldchen

Sitting on the sidelines here but just wanted to say thanks for working on fixing this!

Jun 18 '24 14:06 lawrencedudley

@heldchen

I already stuck at the first point. I have error 400. I did everything as you stated. I only deleted these 3 lines because I don't know where to get this data?

"_csrf": "cIf...WjI",
  "state": "hKF...KeA",
  "nonce": "dW3...OSg",

I think that these values also can be hardcoded? They are definitely not linked to the account because you are not logging in yet, only after this step are the username and password sent and you have these values before first post.

Screenshots form Postman app:

headers: Zrzut ekranu 2024-06-18 o 18 43 48

body: Zrzut ekranu 2024-06-18 o 18 44 29

Jun 18 '24 16:06 mkz212

those are values found in the requests & login form before submitting it - and from google recaptcha which potentially is an issue as those are dynamically injected using javascript and then server-side verified. I think the more promising way to try first is to use the published oauth endpoints using an oauth client instead of their legacy (it says "deprecated" in the response) login form.

pre-login the following requests are done (I'll update the writeup a bit later with all details):

the very first request when starting the app's login process is:

https://authglb.digital.panasonic.com/authorize?scope=openid%20offline_access%20comfortcloud.control%20a2w.control&audience=https%3A%2F%2Fdigital.panasonic.com%2FXmy6xIYIitMxngjB2rHvlm6HSDNnaMJx%2Fapi%2Fv1%2F&response_type=code&code_challenge=ovh...yZo&code_challenge_method=S256&auth0Client=eyJ...In0%3D&client_id=Xmy6xIYIitMxngjB2rHvlm6HSDNnaMJx&redirect_uri=panasonic-iot-cfc%3A%2F%2Fauthglb.digital.panasonic.com%2Fandroid%2Fcom.panasonic.ACCsmart%2Fcallback&state=WM_F...h9_I&nonce=ViL...MMY

(so all these values are somehow known to the app so most likely are either irrelevant or can be recreated/regenerated through code)

this then redirects to:

https://authglb.digital.panasonic.com/login?state=hKF...KeA&client=Xmy6xIYIitMxngjB2rHvlm6HSDNnaMJx&protocol=oauth2&scope=openid%20offline_access%20comfortcloud.control%20a2w.control&audience=https%3A%2F%2Fdigital.panasonic.com%2FXmy6xIYIitMxngjB2rHvlm6HSDNnaMJx%2Fapi%2Fv1%2F&response_type=code&code_challenge=ovh...yZo&code_challenge_method=S256&auth0Client=eyJ...In0%3D&redirect_uri=panasonic-iot-cfc%3A%2F%2Fauthglb.digital.panasonic.com%2Fandroid%2Fcom.panasonic.ACCsmart%2Fcallback&nonce=ViL...MMY

which returns the login html form.

Jun 18 '24 16:06 heldchen

@mkz212 those values can't be hard coded, they are specifically there to show that the flow is being done correctly

CSRF - Cross Site Request Forgery token, it's something the server gives to you to use in your requests. State - could be anything, but commonly used to pass state between each request. Nonce - a single use token

Jun 18 '24 16:06 danielcherubini

Looking from the end.. From last step..

We need X-Client-Id and X-User-Authorization-V2 (token).
Getting X-Client-Id having a token is a simple task.
To get first token, we need:

header: Auth0-Client: eyJuYW1lIjoiYXV0aDAuanMtdWxwIiwidmVyc2lvbiI6IjkuMjMuMiJ9, (hardcoded)
data: client_id": "Xmy6xIYIitMxngjB2rHvlm6HSDNnaMJx, (hardcoded, different for every App Store app version)
"code": WE NEED THIS! This code is probably different for everyone because we provide username and password in previous step and code is only value that we get from there.?
"code_verifier":WE NEED THIS?

Getting refresh token is also easy when we get first token, cause we get there"refresh_token": and "id_token":

Jun 18 '24 17:06 mkz212

Looking from the end.. From last step..

We need X-Client-Id and X-User-Authorization-V2 (token).

Getting X-Client-Id having a token is a simple task.

To get first token, we need:

header: Auth0-Client: eyJuYW1lIjoiYXV0aDAuanMtdWxwIiwidmVyc2lvbiI6IjkuMjMuMiJ9, (hardcoded)

data: client_id": "Xmy6xIYIitMxngjB2rHvlm6HSDNnaMJx, (hardcoded)

"code": WE NEED THIS!

"code_verifier":WE NEED THIS?

Getting refresh token is also easy when we get first token, cause we get there"refresh_token": and "id_token":

Code and code_verifier means it's using the PKCE flow.

https://auth0.com/docs/get-started/authentication-and-authorization-flow/authorization-code-flow-with-pkce/call-your-api-using-the-authorization-code-flow-with-pkce

Check step 1 and 2. You create those two values

Jun 18 '24 17:06 danielcherubini

As i discussed before, it seems that Panasonic is using auth0 as their authentication provider. So it might be best to just use the auth0 SDK. This will also provide the auth0-client

Jun 18 '24 17:06 danielcherubini

@danielcherubini @heldchen In my opinion, it all comes down to getting code and code_verifier values. With these values, it's probably easy to move on.

Jun 18 '24 17:06 mkz212

@danielcherubini @heldchen In my opinion, it all comes down to getting code and code_verifier values. With these values, it's probably easy to move on.

Check my reply to you. It's in there.

Jun 18 '24 17:06 danielcherubini

@mkz212 you're using postman i see, so you're going to need a pre-request script here to generate those values

code

// Dependency: Node.js crypto module
// https://nodejs.org/api/crypto.html#crypto_crypto
function base64URLEncode(str) {
    return str.toString('base64')
        .replace(/\+/g, '-')
        .replace(/\//g, '_')
        .replace(/=/g, '');
}
var code = base64URLEncode(crypto.randomBytes(32));

code_verifier

// Dependency: Node.js crypto module
// https://nodejs.org/api/crypto.html#crypto_crypto
function sha256(buffer) {
    return crypto.createHash('sha256').update(buffer).digest();
}
var code_verifier = base64URLEncode(sha256(verifier));

You're also probably going to have to have code_challenge_method and have that value be S256 because you're using Sha256 in the verifier