EOS on SearchGoogleAdsStreamRequests
Describe the bug: We experience an EOS after almost exactly 1 hour for SearchGoogleAdsStreamRequests like
SELECT ad_group.id,, ad_group_ad.ad.id, ad_group_ad.status, ad_group_ad.ad.responsive_search_ad.headlines, ad_group_ad.ad.responsive_search_ad.descriptions, ad_group_ad.ad.responsive_search_ad.path1, ad_group_ad.ad.responsive_search_ad.path2, ad_group_ad.ad.final_urls, ad_group_ad.labels, ad_group.base_ad_group FROM ad_group_ad WHERE ad_group.status != 'REMOVED' AND ad_group_ad.status != 'REMOVED' AND ad_group_ad.ad.type = 'RESPONSIVE_SEARCH_AD' AND ad_group_ad.labels CONTAINS ANY ('customers/xxx/labels/xxx')
resulting in a io.grpc.StatusRuntimeException: INTERNAL: Received unexpected EOS on empty DATA frame from server
This happens sometimes for keyword_view requests too, but mainly for the above-mentioned request.
The Google Ads API support pointed me to the GRPC issue 8856 and suggested opening a bug here.
Corresponding Google Ads API Forum thread. The remaining communication has been done privately via email.
Steps to Reproduce: Don't know. This seems to happen just for bigger accounts with many elements (ads in this case).
Expected behavior: Successful request or at least a meaningful exception to what is wrong. Do NOT wait for the entire 1-hour timeout.
Client library version and API version: Client library version: 18.0.0 Google Ads API version: 10 JDK version: 18.0.2
Request/Response Logs:
I replaced all IDs with xxx. If you need them, I may send them to you privately.
FAILURE REQUEST SUMMARY. Method: google.ads.googleads.v10.services.GoogleAdsService/SearchStream, Endpoint: googleads.googleapis.com:443, CustomerID: xxx, RequestID: null, ResponseCode: INTERNAL, Fault: RST_STREAM closed stream. HTTP/2 error code: INTERNAL_ERROR.
FAILURE REQUEST DETAIL.
Request
-------
MethodName: google.ads.googleads.v10.services.GoogleAdsService/SearchStream
Endpoint: googleads.googleapis.com:443
Headers: {developer-token=REDACTED, login-customer-id=xxx, x-goog-api-client=gl-java/18.0.2 gccl/18.0.0 gapic/18.0.0 gax/2.16.0 grpc/1.45.1}
Body: customer_id: "xxx"
query: "SELECT\nad_group.id,\nad_group_ad.ad.id,\nad_group_ad.status,\nad_group_ad.ad.responsive_search_ad.headlines,\nad_group_ad.ad.responsive_search_ad.descriptions,\nad_group_ad.ad.responsive_search_ad.path1,\nad_group_ad.ad.responsive_search_ad.path2,\nad_group_ad.ad.final_urls,\nad_group_ad.labels,\nad_group.base_ad_group\nFROM ad_group_ad\nWHERE ad_group.status != \'REMOVED\'\nAND ad_group_ad.status != \'REMOVED\'\nAND ad_group_ad.ad.type = \'RESPONSIVE_SEARCH_AD\'\nAND ad_group_ad.labels CONTAINS ANY (\'customers/xxx/labels/xxx\')\n"
Response
--------
Headers: Metadata(content-disposition=attachment,content-type=application/grpc,date=Sun, 31 Jul 2022 15:24:17 GMT)
Body: results {
ad_group {
resource_name: "customers/xxx/adGroups/xxx"
id: xxx
base_ad_group: "customers/xxx/adGroups/xxx"
}
ad_group_ad {
resource_name: "customers/xxx/adGroupAds/xxx~xxx"
status: ENABLED
ad {
responsive_search_ad {
headlines {
text: "Gute Jeans"
asset_performance_label: PENDING
policy_summary_info {
review_status: REVIEWED
approval_status: APPROVED
}
}
headlines {
text: "Gute Jeans Kollektion SS22"
asset_performance_label: PENDING
policy_summary_info {
review_status: REVIEWED
approval_status: APPROVED
}
}
headlines {
text: "Entdecke jetzt Gute Jeans"
asset_performance_label: PENDING
policy_summary_info {
review_status: REVIEWED
approval_status: APPROVED
... TRUNCATED. See README.md to configure/disable log truncation.
Failure message: null
Status: Status{code=INTERNAL, description=RST_STREAM closed stream. HTTP/2 error code: INTERNAL_ERROR, cause=null}.
Stacktrace:
com.google.api.gax.rpc.InternalException: io.grpc.StatusRuntimeException: INTERNAL: RST_STREAM closed stream. HTTP/2 error code: INTERNAL_ERROR
at com.google.api.gax.rpc.ApiExceptionFactory.createException(ApiExceptionFactory.java:110)
at com.google.api.gax.rpc.ApiExceptionFactory.createException(ApiExceptionFactory.java:41)
at com.google.api.gax.grpc.GrpcApiExceptionFactory.create(GrpcApiExceptionFactory.java:86)
at com.google.api.gax.grpc.GrpcApiExceptionFactory.create(GrpcApiExceptionFactory.java:66)
at com.google.api.gax.grpc.ExceptionResponseObserver.onErrorImpl(ExceptionResponseObserver.java:82)
at com.google.api.gax.rpc.StateCheckingResponseObserver.onError(StateCheckingResponseObserver.java:84)
at com.google.api.gax.grpc.GrpcDirectStreamController$ResponseObserverAdapter.onClose(GrpcDirectStreamController.java:149)
at io.grpc.PartialForwardingClientCallListener.onClose(PartialForwardingClientCallListener.java:39)
at io.grpc.ForwardingClientCallListener.onClose(ForwardingClientCallListener.java:23)
at io.grpc.ForwardingClientCallListener$SimpleForwardingClientCallListener.onClose(ForwardingClientCallListener.java:40)
at com.google.api.gax.grpc.ChannelPool$ReleasingClientCall$1.onClose(ChannelPool.java:535)
at io.grpc.PartialForwardingClientCallListener.onClose(PartialForwardingClientCallListener.java:39)
at io.grpc.ForwardingClientCallListener.onClose(ForwardingClientCallListener.java:23)
at io.grpc.ForwardingClientCallListener$SimpleForwardingClientCallListener.onClose(ForwardingClientCallListener.java:40)
at com.google.ads.googleads.lib.logging.LoggingInterceptor$1$1.onClose(LoggingInterceptor.java:113)
at io.grpc.internal.DelayedClientCall$DelayedListener$3.run(DelayedClientCall.java:463)
at io.grpc.internal.DelayedClientCall$DelayedListener.delayOrExecute(DelayedClientCall.java:427)
at io.grpc.internal.DelayedClientCall$DelayedListener.onClose(DelayedClientCall.java:460)
at io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:562)
at io.grpc.internal.ClientCallImpl.access$300(ClientCallImpl.java:70)
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInternal(ClientCallImpl.java:743)
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext(ClientCallImpl.java:722)
at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:133)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base/java.lang.Thread.run(Thread.java:833)
Suppressed: java.lang.RuntimeException: Asynchronous task failed
at com.google.api.gax.rpc.ServerStreamIterator.hasNext(ServerStreamIterator.java:105)
...
Caused by: io.grpc.StatusRuntimeException: INTERNAL: RST_STREAM closed stream. HTTP/2 error code: INTERNAL_ERROR
at io.grpc.Status.asRuntimeException(Status.java:535)
... 21 common frames omitted
Reduced code:
var request = SearchGoogleAdsStreamRequest.newBuilder().setCustomerId(...).setQuery(query).build();
try (GoogleAdsServiceClient service = ...) {
for (var response : service.searchStreamCallable().call(request)) {
response.getResultsList().forEach(...);
}
}
Is there any progress on this issue?
Apologies for the delayed response. I just sent you an email on the support thread and will continue to follow up there.
I'm closing this since it's not library-specific, but rest assured I'll continue to follow up with you on the support thread with updates.