googleapis
Invalid format in BigQuery destination dataset in request
Environment details
- OS type and version: Mac OS Monterry 12.4
- Python version: Python 3.9.13
- pip version: pip 22.0.4 from /Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pip (python 3.9)
-
google-cloud-assetversion:pip show google-cloud-assetName: google-cloud-asset Version: 3.9.1 Summary: Cloud Asset API API client library Home-page: https://github.com/googleapis/python-asset Author: Google LLC Author-email: [email protected] License: Apache 2.0 Location: /Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages Requires: google-api-core, google-cloud-access-context-manager, google-cloud-org-policy, google-cloud-os-config, grpc-google-iam-v1, proto-plus, protobuf Required-by:
Steps to reproduce
Trying to run the below python code snippet to export assets data to bigquery dataset
Code example
from google.cloud import asset_v1
project_id = 'my-project-id'
dataset = 'my-dataset-id'
table = 'res_table'
content_type = asset_v1.ContentType.RESOURCE
client = asset_v1.AssetServiceClient()
parent = project_id
output_config = asset_v1.OutputConfig()
output_config.bigquery_destination.dataset = dataset
output_config.bigquery_destination.table = table
output_config.bigquery_destination.force = False
response = client.export_assets(
request={
"parent": parent,
"content_type": content_type,
"output_config": output_config
}
)
print(response.result())
Stack trace
myhomedir python-code % python3 collector.py
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/google/api_core/grpc_helpers.py", line 50, in error_remapped_callable
return callable_(*args, **kwargs)
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/grpc/_channel.py", line 946, in __call__
return _end_unary_response_blocking(state, call, False, None)
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/grpc/_channel.py", line 849, in _end_unary_response_blocking
raise _InactiveRpcError(state)
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
status = StatusCode.INVALID_ARGUMENT
details = "Invalid format in BigQuery destination dataset in request."
debug_error_string = "{"created":"@1656575459.813917000","description":"Error received from peer ipv4:142.250.71.10:443","file":"src/core/lib/surface/call.cc","file_line":967,"grpc_message":"Invalid format in BigQuery destination dataset in request.","grpc_status":3}"
>
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/myhomedir/python-code/collector.py", line 14, in <module>
response = client.export_assets(
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/google/cloud/asset_v1/services/asset_service/client.py", line 617, in export_assets
response = rpc(
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/google/api_core/gapic_v1/method.py", line 154, in __call__
return wrapped_func(*args, **kwargs)
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/google/api_core/grpc_helpers.py", line 52, in error_remapped_callable
raise exceptions.from_grpc_error(exc) from exc
google.api_core.exceptions.InvalidArgument: 400 Invalid format in BigQuery destination dataset in request. [field_violations {
field: "output_config.bigquery_destination.dataset"
description: "Invalid format in BigQuery destination dataset in request."
}
]
Thanks!
Hello @Sheshgiriv, please make sure that all of the values you are sending match the expected format. Specifically I see that you are setting parent = project_id, but in the most similar code sample snippet I could find it is specified as parent = "projects/{}".format(project_id) https://github.com/googleapis/python-asset/blob/473e133e2674bf6a5ae655fe67be7d79fed2d8e9/samples/snippets/quickstart_exportassets.py#L49
Hopefully changing that line (so that parent value is prefixed with projects/ can fix your issue.
Please let us know if that fixes issue for you.
Hi @vam-google , I changed the code to below format even then the issue exists
parent = project_id to
parent = "projects/{}".format(project_id)
not able to provide the project id in the correct format gives the below exception
`myhomedir python-code % python3 collector.py Traceback (most recent call last): File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/google/api_core/grpc_helpers.py", line 50, in error_remapped_callable return callable_(*args, **kwargs) File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/grpc/_channel.py", line 946, in call return _end_unary_response_blocking(state, call, False, None) File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/grpc/_channel.py", line 849, in _end_unary_response_blocking raise _InactiveRpcError(state) grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with: status = StatusCode.PERMISSION_DENIED details = "The caller does not have permission" debug_error_string = "{"created":"@1658318128.385897000","description":"Error received from peer ipv4:142.250.196.10:443","file":"src/core/lib/surface/call.cc","file_line":967,"grpc_message":"The caller does not have permission","grpc_status":7}"
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/myhomedir/python-code/collector.py", line 14, in
I'm going to transfer this issue to the google-cloud-python repository as we are preparing to move the code for google-cloud-asset to that repository in the next 1-2 weeks.
