google-cloud-python icon indicating copy to clipboard operation
google-cloud-python copied to clipboard
verified publisher icon googleapis

Improve error information when using gRPC transport in case of invalid argument

Open Indavelopers opened this issue 10 months ago • 5 comments

Determine this is the right repository

  • [x] I determined this is the correct repository in which to report this bug.

Summary of the issue

Context I'm trying to use the Cloud Shell Python v1 library for checking my Cloud Shell environment. I'm running the provided sample code https://github.com/googleapis/google-cloud-python/blob/main/packages/google-cloud-shell/samples/generated_samples/cloudshell_v1_generated_cloud_shell_service_get_environment_sync.py

REST v1 API equivalent request answers with a correct 200 response (https://cloud.google.com/shell/docs/reference/rest/v1/users.environments/get)

  • gcloud config is correctly set, authenticated with user account and proper GCP project
  • GC application default credentials have been properly set with gcloud auth application-default login
  • Project has Cloud Shell API enabled

My code can be found in this public GH repo: https://github.com/Indavelopers/add-cloud-shell-to-oauth-authorized-domains/blob/main/cloudshell_v1_generated_cloud_shell_service_get_environment_sync.py

Expected Behavior: I expected this method https://cloud.google.com/python/docs/reference/cloudshell/latest/google.cloud.shell_v1.services.cloud_shell_service.CloudShellServiceClient#google_cloud_shell_v1_services_cloud_shell_service_CloudShellServiceClient_get_environment to be called correctly, and the generated sample code to work.

Actual Behavior: Code throws this error:

google.api_core.exceptions.InternalServerError: 500 Internal error encountered.

API client name and version

google-cloud-shell v1.12.1

Reproduction steps: code

file: main.py

from google.cloud import shell_v1


def sample_get_environment():
    # Create a client
    client = shell_v1.CloudShellServiceClient()

    # Initialize request argument(s)
    request = shell_v1.GetEnvironmentRequest(
        name="name_value",
    )

    # Make the request
    response = client.get_environment(request=request)

    # Handle the response
    print(response)


if __name__ == '__main__':
    sample_get_environment()

https://github.com/Indavelopers/add-cloud-shell-to-oauth-authorized-domains/blob/main/cloudshell_v1_generated_cloud_shell_service_get_environment_sync.py

Reproduction steps: supporting files

No response

Reproduction steps: actual results

Traceback (most recent call last):
  File "/home/marcos/add-cloud-shell-to-oauth-authorized-domains/cloudshell_v1_generated_cloud_shell_service_get_environment_sync.py", line 23, in <module>
    sample_get_environment()
    ~~~~~~~~~~~~~~~~~~~~~~^^
  File "/home/marcos/add-cloud-shell-to-oauth-authorized-domains/cloudshell_v1_generated_cloud_shell_service_get_environment_sync.py", line 16, in sample_get_environment
    response = client.get_environment(request=request)
  File "/home/marcos/add-cloud-shell-to-oauth-authorized-domains/.venv/lib/python3.13/site-packages/google/cloud/shell_v1/services/cloud_shell_service/client.py", line 829, in get_environment
    response = rpc(
        request,
    ...<2 lines>...
        metadata=metadata,
    )
  File "/home/marcos/add-cloud-shell-to-oauth-authorized-domains/.venv/lib/python3.13/site-packages/google/api_core/gapic_v1/method.py", line 131, in __call__
    return wrapped_func(*args, **kwargs)
  File "/home/marcos/add-cloud-shell-to-oauth-authorized-domains/.venv/lib/python3.13/site-packages/google/api_core/retry/retry_unary.py", line 293, in retry_wrapped_func
    return retry_target(
        target,
    ...<3 lines>...
        on_error=on_error,
    )
  File "/home/marcos/add-cloud-shell-to-oauth-authorized-domains/.venv/lib/python3.13/site-packages/google/api_core/retry/retry_unary.py", line 153, in retry_target
    _retry_error_helper(
    ~~~~~~~~~~~~~~~~~~~^
        exc,
        ^^^^
    ...<6 lines>...
        timeout,
        ^^^^^^^^
    )
    ^
  File "/home/marcos/add-cloud-shell-to-oauth-authorized-domains/.venv/lib/python3.13/site-packages/google/api_core/retry/retry_base.py", line 212, in _retry_error_helper
    raise final_exc from source_exc
  File "/home/marcos/add-cloud-shell-to-oauth-authorized-domains/.venv/lib/python3.13/site-packages/google/api_core/retry/retry_unary.py", line 144, in retry_target
    result = target()
  File "/home/marcos/add-cloud-shell-to-oauth-authorized-domains/.venv/lib/python3.13/site-packages/google/api_core/timeout.py", line 130, in func_with_timeout
    return func(*args, **kwargs)
  File "/home/marcos/add-cloud-shell-to-oauth-authorized-domains/.venv/lib/python3.13/site-packages/google/api_core/grpc_helpers.py", line 78, in error_remapped_callable
    raise exceptions.from_grpc_error(exc) from exc
google.api_core.exceptions.InternalServerError: 500 Internal error encountered.

Reproduction steps: expected results

Same as https://cloud.google.com/python/docs/reference/cloudshell/latest/google.cloud.shell_v1.services.cloud_shell_service.CloudShellServiceClient#google_cloud_shell_v1_services_cloud_shell_service_CloudShellServiceClient_get_environment

OS & version + platform

WSL2 Ubuntu-24.04, Cloud Shell

Python environment

Python 3.13.2

Python dependencies

Package Version

cachetools 5.5.2 certifi 2025.1.31 charset-normalizer 3.4.1 google-api-core 2.24.2 google-auth 2.38.0 google-cloud-shell 1.12.1 googleapis-common-protos 1.69.2 grpcio 1.71.0 grpcio-status 1.71.0 idna 3.10 pip 25.0.1 proto-plus 1.26.1 protobuf 5.29.3 pyasn1 0.6.1 pyasn1_modules 0.4.1 requests 2.32.3 rsa 4.9 urllib3 2.3.0

Additional context

No response

Indavelopers avatar Mar 20 '25 07:03 Indavelopers

Hi @Indavelopers,

Thanks for sharing a code sample for this issue. Regrettably I'm not able to reproduce the problem. It's possible that this was a transient issue however if the problem persists, please file an issue in the API-specific issue tracking page to obtain better quality answers, as this problem appears to be related to the API itself rather than the client library. For the Cloud Shell API , the public issue tracker is located at https://issuetracker.google.com/issues/new?component=187188&template=0 .

To confirm that this is an API-specific issue and not a client library issue, try the following:

  1. Send the request using curl 
curl -X GET -H "Authorization: Bearer $(gcloud auth print-access-token)" https://cloudshell.googleapis.com/v1/users/<your email>/environments/default
  1. Send the request using the Try this method interface on this page.

I was unable to reproduce the issue with the client library, curl or the Try this method interface.

I'm going to close this issue as the problem cannot be reproduced but please feel free to open a new issue if you have additional questions.

parthea avatar Mar 20 '25 15:03 parthea

Thanks for getting back to me, but I just retryed it and still the same result:

  • The API explorer answers back with a 200 status and correct payload
  • Your suggested curl command also works ok
  • But my code throws the same output error in my local env and in Cloud Shell, with auth supposedly working - at least in Cloud Shell

What could I do to further troubleshoot this? Could I get the code to work using the HTTP API instead of the gRPC one?

Thanks,

Indavelopers avatar Mar 20 '25 15:03 Indavelopers

Thanks for providing this feedback. Yes, you can set the transport argument of CloudShellServiceClient to "rest". See the code sample below.

from google.cloud import shell_v1


def sample_get_environment():
    # Create a client
    client = shell_v1.CloudShellServiceClient(transport="rest")

    # Initialize request argument(s)
    request = shell_v1.GetEnvironmentRequest(
        name="users/<your user>/environments/default",
    )

    # Make the request
    response = client.get_environment(request=request)

    # Handle the response
    print(response)


if __name__ == '__main__':
    sample_get_environment()

parthea avatar Mar 20 '25 15:03 parthea

Now we found the issue! The problem was that I forgot to modify the environment's name in the request:

request = shell_v1.GetEnvironmentRequest(
        name="users/<your user>/environments/default",
    )

I was using name="name_value".

Changed to the proper name worked in both gRPC and REST variants.

But it allowed us to found an issue:

When you use the client library using REST and you don't provide a correct environment name, the error throws a 500 Internal error, with a confusing message:

Traceback (most recent call last):
  File "/home/marcos/add-cloud-shell-to-oauth-authorized-domains/cloudshell_v1_generated_cloud_shell_service_get_environment_sync.py", line 21, in <module>
    sample_get_environment()
    ~~~~~~~~~~~~~~~~~~~~~~^^
  File "/home/marcos/add-cloud-shell-to-oauth-authorized-domains/cloudshell_v1_generated_cloud_shell_service_get_environment_sync.py", line 14, in sample_get_environment
    response = client.get_environment(request=request)
  File "/home/marcos/add-cloud-shell-to-oauth-authorized-domains/.venv/lib/python3.13/site-packages/google/cloud/shell_v1/services/cloud_shell_service/client.py", line 829, in get_environment
    response = rpc(
        request,
    ...<2 lines>...
        metadata=metadata,
    )
  File "/home/marcos/add-cloud-shell-to-oauth-authorized-domains/.venv/lib/python3.13/site-packages/google/api_core/gapic_v1/method.py", line 131, in __call__
    return wrapped_func(*args, **kwargs)
  File "/home/marcos/add-cloud-shell-to-oauth-authorized-domains/.venv/lib/python3.13/site-packages/google/api_core/retry/retry_unary.py", line 293, in retry_wrapped_func
    return retry_target(
        target,
    ...<3 lines>...
        on_error=on_error,
    )
  File "/home/marcos/add-cloud-shell-to-oauth-authorized-domains/.venv/lib/python3.13/site-packages/google/api_core/retry/retry_unary.py", line 153, in retry_target
    _retry_error_helper(
    ~~~~~~~~~~~~~~~~~~~^
        exc,
        ^^^^
    ...<6 lines>...
        timeout,
        ^^^^^^^^
    )
    ^
  File "/home/marcos/add-cloud-shell-to-oauth-authorized-domains/.venv/lib/python3.13/site-packages/google/api_core/retry/retry_base.py", line 212, in _retry_error_helper
    raise final_exc from source_exc
  File "/home/marcos/add-cloud-shell-to-oauth-authorized-domains/.venv/lib/python3.13/site-packages/google/api_core/retry/retry_unary.py", line 144, in retry_target
    result = target()
  File "/home/marcos/add-cloud-shell-to-oauth-authorized-domains/.venv/lib/python3.13/site-packages/google/api_core/timeout.py", line 130, in func_with_timeout
    return func(*args, **kwargs)
  File "/home/marcos/add-cloud-shell-to-oauth-authorized-domains/.venv/lib/python3.13/site-packages/google/api_core/grpc_helpers.py", line 78, in error_remapped_callable
    raise exceptions.from_grpc_error(exc) from exc
google.api_core.exceptions.InternalServerError: 500 Internal error encountered.

But with the same code, if you use the REST variant, the error correctly states that there's an issue with the request, and that it was expecting a proper request argument of environment name and the compliant name syntax:

Traceback (most recent call last):
  File "/home/marcos/add-cloud-shell-to-oauth-authorized-domains/cloudshell_v1_generated_cloud_shell_service_get_environment_sync.py", line 21, in <module>
    sample_get_environment()
    ~~~~~~~~~~~~~~~~~~~~~~^^
  File "/home/marcos/add-cloud-shell-to-oauth-authorized-domains/cloudshell_v1_generated_cloud_shell_service_get_environment_sync.py", line 14, in sample_get_environment
    response = client.get_environment(request=request)
  File "/home/marcos/add-cloud-shell-to-oauth-authorized-domains/.venv/lib/python3.13/site-packages/google/cloud/shell_v1/services/cloud_shell_service/client.py", line 829, in get_environment
    response = rpc(
        request,
    ...<2 lines>...
        metadata=metadata,
    )
  File "/home/marcos/add-cloud-shell-to-oauth-authorized-domains/.venv/lib/python3.13/site-packages/google/api_core/gapic_v1/method.py", line 131, in __call__
    return wrapped_func(*args, **kwargs)
  File "/home/marcos/add-cloud-shell-to-oauth-authorized-domains/.venv/lib/python3.13/site-packages/google/api_core/retry/retry_unary.py", line 293, in retry_wrapped_func
    return retry_target(
        target,
    ...<3 lines>...
        on_error=on_error,
    )
  File "/home/marcos/add-cloud-shell-to-oauth-authorized-domains/.venv/lib/python3.13/site-packages/google/api_core/retry/retry_unary.py", line 153, in retry_target
    _retry_error_helper(
    ~~~~~~~~~~~~~~~~~~~^
        exc,
        ^^^^
    ...<6 lines>...
        timeout,
        ^^^^^^^^
    )
    ^
  File "/home/marcos/add-cloud-shell-to-oauth-authorized-domains/.venv/lib/python3.13/site-packages/google/api_core/retry/retry_base.py", line 212, in _retry_error_helper
    raise final_exc from source_exc
  File "/home/marcos/add-cloud-shell-to-oauth-authorized-domains/.venv/lib/python3.13/site-packages/google/api_core/retry/retry_unary.py", line 144, in retry_target
    result = target()
  File "/home/marcos/add-cloud-shell-to-oauth-authorized-domains/.venv/lib/python3.13/site-packages/google/api_core/timeout.py", line 130, in func_with_timeout
    return func(*args, **kwargs)
  File "/home/marcos/add-cloud-shell-to-oauth-authorized-domains/.venv/lib/python3.13/site-packages/google/api_core/grpc_helpers.py", line 76, in error_remapped_callable
    return callable_(*args, **kwargs)
  File "/home/marcos/add-cloud-shell-to-oauth-authorized-domains/.venv/lib/python3.13/site-packages/google/cloud/shell_v1/services/cloud_shell_service/transports/rest.py", line 856, in __call__
    transcoded_request = _BaseCloudShellServiceRestTransport._BaseGetEnvironment._get_transcoded_request(
        http_options, request
    )
  File "/home/marcos/add-cloud-shell-to-oauth-authorized-domains/.venv/lib/python3.13/site-packages/google/cloud/shell_v1/services/cloud_shell_service/transports/rest_base.py", line 202, in _get_transcoded_request
    transcoded_request = path_template.transcode(http_options, pb_request)
  File "/home/marcos/add-cloud-shell-to-oauth-authorized-domains/.venv/lib/python3.13/site-packages/google/api_core/path_template.py", line 337, in transcode
    raise ValueError(
    ...<8 lines>...
    )
ValueError: Invalid request.
Some of the fields of the request message are either not initialized or initialized with an invalid value.
Please make sure your request matches at least one accepted HTTP binding.
To match a binding the request message must have all the required fields initialized with values matching their patterns as listed below:
        URI: "/v1/{name=users/*/environments/*}"
        Required request fields:
                field: "name", pattern: "users/*/environments/*"

Maybe that's an issue with the client library, that the gRPC error message can be improved, and specially that the error code should be a 400 instead of a 500?

Thanks,

Indavelopers avatar Mar 20 '25 18:03 Indavelopers

Thank you for the investigation @Indavelopers! You're right the server should return 400 rather than 500. I've filed http://issuetracker.google.com/issues/405153584 in the API specific issue tracker. Please subscribe to http://issuetracker.google.com/issues/405153584 for updates. I'm going to switch this to a feature request to determine if we can improve the error information that we provide in client libraries when using gRPC.

parthea avatar Mar 20 '25 18:03 parthea