ray icon indicating copy to clipboard operation
ray copied to clipboard
verified publisher icon ray-project

[core] Remove grpc ClientCallTag

Open wuxibin89 opened this issue 2 years ago • 8 comments

Why are these changes needed?

ClientCallTag is no longer need after ClientCallManager::CreateCall, so we can remove it and use ClientCall as grpc CompleteQueue tag. By removing ClientCallTag, we may expect reducing memory footprint slightly.

Related issue number

Checks

  • [x] I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR.
  • [x] I've run scripts/format.sh to lint the changes in this PR.
  • [x] I've included any doc changes needed for https://docs.ray.io/en/master/.
    • [ ] I've added any new APIs to the API Reference. For example, if I added a method in Tune, I've added it in doc/source/tune/api/ under the corresponding .rst file.
  • [x] I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
  • Testing Strategy
    • [x] Unit tests
    • [x] Release tests
    • [ ] This PR is not tested :(

wuxibin89 avatar Jul 06 '23 07:07 wuxibin89

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 14 days if no further activity occurs. Thank you for your contributions.

  • If you'd like to keep this open, just leave any comment, and the stale label will be removed.

stale[bot] avatar Aug 10 '23 02:08 stale[bot]

old...but this is interesting... @wuxibin89 do you have benchmarks/data to show how much mem improvement this would be?

anyscalesam avatar Jun 17 '24 18:06 anyscalesam

@wuxibin89 sorry for missing this PR. The PR actually looks pretty good to me. Could you rebase with master and I'll take a final look and merge it.

jjyao avatar Aug 27 '24 19:08 jjyao

Actually I resolved the conflicts :)

jjyao avatar Aug 27 '24 19:08 jjyao

There is tsan test failure that I think is related.

jjyao avatar Aug 27 '24 22:08 jjyao 


[2024-08-27T20:52:49Z] WARNING: ThreadSanitizer: data race (pid=1944)
--
  | [2024-08-27T20:52:49Z]   Write of size 1 at 0x7b0c0000e2d8 by thread T14:
  | [2024-08-27T20:52:49Z]     #0 void ray::rpc::GrpcClient<ray::rpc::InternalKVGcsService>::CallMethod<ray::rpc::InternalKVKeysRequest, ray::rpc::InternalKVKeysReply>(std::__1::unique_ptr<grpc::ClientAsyncResponseReader<ray::rpc::InternalKVKeysReply>, std::__1::default_delete<grpc::ClientAsyncResponseReader<ray::rpc::InternalKVKeysReply> > > (ray::rpc::InternalKVGcsService::Stub::*)(grpc::ClientContext*, ray::rpc::InternalKVKeysRequest const&, grpc::CompletionQueue*), ray::rpc::InternalKVKeysRequest const&, std::__1::function<void (ray::Status const&, ray::rpc::InternalKVKeysReply&&)> const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, long) /proc/self/cwd/bazel-out/k8-opt/bin/_virtual_includes/grpc_common_lib/ray/rpc/grpc_client.h:157:26 (liblibgcs_Uclient_Ulib.so+0x32a016)
  | [2024-08-27T20:52:49Z]     #1 void ray::rpc::GcsRpcClient::invoke_async_method<ray::rpc::InternalKVGcsService, ray::rpc::InternalKVKeysRequest, ray::rpc::InternalKVKeysReply, true>(std::__1::unique_ptr<grpc::ClientAsyncResponseReader<ray::rpc::InternalKVKeysReply>, std::__1::default_delete<grpc::ClientAsyncResponseReader<ray::rpc::InternalKVKeysReply> > > (ray::rpc::InternalKVGcsService::Stub::*)(grpc::ClientContext*, ray::rpc::InternalKVKeysRequest const&, grpc::CompletionQueue*), ray::rpc::GrpcClient<ray::rpc::InternalKVGcsService>&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, ray::rpc::InternalKVKeysRequest const&, std::__1::function<void (ray::Status const&, ray::rpc::InternalKVKeysReply&&)> const&, long)::'lambda'()::operator()() const /proc/self/cwd/bazel-out/k8-opt/bin/_virtual_includes/gcs_service_rpc/ray/rpc/gcs_server/gcs_rpc_client.h:283:28 (liblibgcs_Uclient_Ulib.so+0x32a016)
  | [2024-08-27T20:52:49Z]     #2 decltype(std::__1::forward<ray::rpc::InternalKVGcsService>(fp)(std::__1::forward<ray::rpc::InternalKVKeysRequest>(fp0)...)) std::__1::__invoke<void ray::rpc::GcsRpcClient::invoke_async_method<ray::rpc::InternalKVGcsService, ray::rpc::InternalKVKeysRequest, ray::rpc::InternalKVKeysReply, true>(std::__1::unique_ptr<grpc::ClientAsyncResponseReader<ray::rpc::InternalKVKeysReply>, std::__1::default_delete<grpc::ClientAsyncResponseReader<ray::rpc::InternalKVKeysReply> > > (ray::rpc::InternalKVGcsService::Stub::*)(grpc::ClientContext*, ray::rpc::InternalKVKeysRequest const&, grpc::CompletionQueue*), ray::rpc::GrpcClient<ray::rpc::InternalKVGcsService>&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, ray::rpc::InternalKVKeysRequest const&, std::__1::function<void (ray::Status const&, ray::rpc::InternalKVKeysReply&&)> const&, long)::'lambda'()&>(ray::rpc::InternalKVGcsService&&, ray::rpc::InternalKVKeysRequest&&...) /opt/llvm/bin/../include/c++/v1/type_traits:3694:1 (liblibgcs_Uclient_Ulib.so+0x329b0d)
  | [2024-08-27T20:52:49Z]     #3 void std::__1::__invoke_void_return_wrapper<void, true>::__call<void ray::rpc::GcsRpcClient::invoke_async_method<ray::rpc::InternalKVGcsService, ray::rpc::InternalKVKeysRequest, ray::rpc::InternalKVKeysReply, true>(std::__1::unique_ptr<grpc::ClientAsyncResponseReader<ray::rpc::InternalKVKeysReply>, std::__1::default_delete<grpc::ClientAsyncResponseReader<ray::rpc::InternalKVKeysReply> > > (ray::rpc::InternalKVGcsService::Stub::*)(grpc::ClientContext*, ray::rpc::InternalKVKeysRequest const&, grpc::CompletionQueue*), ray::rpc::GrpcClient<ray::rpc::InternalKVGcsService>&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, ray::rpc::InternalKVKeysRequest const&, std::__1::function<void (ray::Status const&, ray::rpc::InternalKVKeysReply&&)> const&, long)::'lambda'()&>(ray::rpc::InternalKVGcsService&&...) /opt/llvm/bin/../include/c++/v1/__functional_base:348:9 (liblibgcs_Uclient_Ulib.so+0x329b0d)
  | [2024-08-27T20:52:49Z]     #4 std::__1::__function::__alloc_func<void ray::rpc::GcsRpcClient::invoke_async_method<ray::rpc::InternalKVGcsService, ray::rpc::InternalKVKeysRequest, ray::rpc::InternalKVKeysReply, true>(std::__1::unique_ptr<grpc::ClientAsyncResponseReader<ray::rpc::InternalKVKeysReply>, std::__1::default_delete<grpc::ClientAsyncResponseReader<ray::rpc::InternalKVKeysReply> > > (ray::rpc::InternalKVGcsService::Stub::*)(grpc::ClientContext*, ray::rpc::InternalKVKeysRequest const&, grpc::CompletionQueue*), ray::rpc::GrpcClient<ray::rpc::InternalKVGcsService>&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, ray::rpc::InternalKVKeysRequest const&, std::__1::function<void (ray::Status const&, ray::rpc::InternalKVKeysReply&&)> const&, long)::'lambda'(), std::__1::allocator<void ray::rpc::GcsRpcClient::invoke_async_method<ray::rpc::InternalKVGcsService, ray::rpc::InternalKVKeysRequest, ray::rpc::InternalKVKeysReply, true>(std::__1::unique_ptr<grpc::ClientAsyncResponseReader<ray::rpc::InternalKVKeysReply>, std::__1::default_delete<grpc::ClientAsyncResponseReader<ray::rpc::InternalKVKeysReply> > > (ray::rpc::InternalKVGcsService::Stub::*)(grpc::ClientContext*, ray::rpc::InternalKVKeysRequest const&, grpc::CompletionQueue*), ray::rpc::GrpcClient<ray::rpc::InternalKVGcsService>&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, ray::rpc::InternalKVKeysRequest const&, std::__1::function<void (ray::Status const&, ray::rpc::InternalKVKeysReply&&)> const&, long)::'lambda'()>, void ()>::operator()() /opt/llvm/bin/../include/c++/v1/functional:1558:16 (liblibgcs_Uclient_Ulib.so+0x329b0d)
  | [2024-08-27T20:52:49Z]     #5 std::__1::__function::__func<void ray::rpc::GcsRpcClient::invoke_async_method<ray::rpc::InternalKVGcsService, ray::rpc::InternalKVKeysRequest, ray::rpc::InternalKVKeysReply, true>(std::__1::unique_ptr<grpc::ClientAsyncResponseReader<ray::rpc::InternalKVKeysReply>, std::__1::default_delete<grpc::ClientAsyncResponseReader<ray::rpc::InternalKVKeysReply> > > (ray::rpc::InternalKVGcsService::Stub::*)(grpc::ClientContext*, ray::rpc::InternalKVKeysRequest const&, grpc::CompletionQueue*), ray::rpc::GrpcClient<ray::rpc::InternalKVGcsService>&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, ray::rpc::InternalKVKeysRequest const&, std::__1::function<void (ray::Status const&, ray::rpc::InternalKVKeysReply&&)> const&, long)::'lambda'(), std::__1::allocator<void ray::rpc::GcsRpcClient::invoke_async_method<ray::rpc::InternalKVGcsService, ray::rpc::InternalKVKeysRequest, ray::rpc::InternalKVKeysReply, true>(std::__1::unique_ptr<grpc::ClientAsyncResponseReader<ray::rpc::InternalKVKeysReply>, std::__1::default_delete<grpc::ClientAsyncResponseReader<ray::rpc::InternalKVKeysReply> > > (ray::rpc::InternalKVGcsService::Stub::*)(grpc::ClientContext*, ray::rpc::InternalKVKeysRequest const&, grpc::CompletionQueue*), ray::rpc::GrpcClient<ray::rpc::InternalKVGcsService>&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, ray::rpc::InternalKVKeysRequest const&, std::__1::function<void (ray::Status const&, ray::rpc::InternalKVKeysReply&&)> const&, long)::'lambda'()>, void ()>::operator()() /opt/llvm/bin/../include/c++/v1/functional:1732:12 (liblibgcs_Uclient_Ulib.so+0x329b0d)
  | [2024-08-27T20:52:49Z]     #6 std::__1::__function::__value_func<void ()>::operator()() const /opt/llvm/bin/../include/c++/v1/functional:1885:16 (liblibgcs_Uclient_Ulib.so+0x293e09)
  | [2024-08-27T20:52:49Z]     #7 std::__1::function<void ()>::operator()() const /opt/llvm/bin/../include/c++/v1/functional:2560:12 (liblibgcs_Uclient_Ulib.so+0x293e09)
  | [2024-08-27T20:52:49Z]     #8 ray::rpc::Executor::Retry() /proc/self/cwd/bazel-out/k8-opt/bin/_virtual_includes/gcs_service_rpc/ray/rpc/gcs_server/gcs_rpc_client.h:50:18 (liblibgcs_Uclient_Ulib.so+0x293e09)
  | [2024-08-27T20:52:49Z]     #9 ray::rpc::GcsRpcClient::CheckChannelStatus(bool) /proc/self/cwd/bazel-out/k8-opt/bin/_virtual_includes/gcs_service_rpc/ray/rpc/gcs_server/gcs_rpc_client.h:677:50 (liblibgcs_Uclient_Ulib.so+0x293e09)
  | [2024-08-27T20:52:49Z]     #10 ray::rpc::GcsRpcClient::SetupCheckTimer()::'lambda'(boost::system::error_code)::operator()(boost::system::error_code) const /proc/self/cwd/bazel-out/k8-opt/bin/_virtual_includes/gcs_service_rpc/ray/rpc/gcs_server/gcs_rpc_client.h:614:9 (liblibgcs_Uclient_Ulib.so+0x299491)
  | [2024-08-27T20:52:49Z]     #11 boost::asio::detail::binder1<ray::rpc::GcsRpcClient::SetupCheckTimer()::'lambda'(boost::system::error_code), boost::system::error_code>::operator()() /proc/self/cwd/external/boost/boost/asio/detail/bind_handler.hpp:171:5 (liblibgcs_Uclient_Ulib.so+0x299491)
  | [2024-08-27T20:52:49Z]     #12 void boost::asio::asio_handler_invoke<boost::asio::detail::binder1<ray::rpc::GcsRpcClient::SetupCheckTimer()::'lambda'(boost::system::error_code), boost::system::error_code> >(boost::asio::detail::binder1<ray::rpc::GcsRpcClient::SetupCheckTimer()::'lambda'(boost::system::error_code), boost::system::error_code>&, ...) /proc/self/cwd/external/boost/boost/asio/handler_invoke_hook.hpp:88:3 (liblibgcs_Uclient_Ulib.so+0x299491)
  | [2024-08-27T20:52:49Z]     #13 void boost_asio_handler_invoke_helpers::invoke<boost::asio::detail::binder1<ray::rpc::GcsRpcClient::SetupCheckTimer()::'lambda'(boost::system::error_code), boost::system::error_code>, ray::rpc::GcsRpcClient::SetupCheckTimer()::'lambda'(boost::system::error_code)>(boost::asio::detail::binder1<ray::rpc::GcsRpcClient::SetupCheckTimer()::'lambda'(boost::system::error_code), boost::system::error_code>&, ray::rpc::GcsRpcClient::SetupCheckTimer()::'lambda'(boost::system::error_code)&) /proc/self/cwd/external/boost/boost/asio/detail/handler_invoke_helpers.hpp:54:3 (liblibgcs_Uclient_Ulib.so+0x299491)
  | [2024-08-27T20:52:49Z]     #14 void boost::asio::detail::handler_work<ray::rpc::GcsRpcClient::SetupCheckTimer()::'lambda'(boost::system::error_code), boost::asio::any_io_executor, void>::complete<boost::asio::detail::binder1<ray::rpc::GcsRpcClient::SetupCheckTimer()::'lambda'(boost::system::error_code), boost::system::error_code> >(boost::asio::detail::binder1<ray::rpc::GcsRpcClient::SetupCheckTimer()::'lambda'(boost::system::error_code), boost::system::error_code>&, ray::rpc::GcsRpcClient::SetupCheckTimer()::'lambda'(boost::system::error_code)&) /proc/self/cwd/external/boost/boost/asio/detail/handler_work.hpp:520:7 (liblibgcs_Uclient_Ulib.so+0x299491)
  | [2024-08-27T20:52:49Z]     #15 boost::asio::detail::wait_handler<ray::rpc::GcsRpcClient::SetupCheckTimer()::'lambda'(boost::system::error_code), boost::asio::any_io_executor>::do_complete(void*, boost::asio::detail::scheduler_operation*, boost::system::error_code const&, unsigned long) /proc/self/cwd/external/boost/boost/asio/detail/wait_handler.hpp:76:9 (liblibgcs_Uclient_Ulib.so+0x299491)
  | [2024-08-27T20:52:49Z]     #16 boost::asio::detail::scheduler_operation::complete(void*, boost::system::error_code const&, unsigned long) /proc/self/cwd/external/boost/boost/asio/detail/scheduler_operation.hpp:40:5 (libexternal_Sboost_Slibasio.so+0xa9840)
  | [2024-08-27T20:52:49Z]     #17 boost::asio::detail::scheduler::do_run_one(boost::asio::detail::conditionally_enabled_mutex::scoped_lock&, boost::asio::detail::scheduler_thread_info&, boost::system::error_code const&) /proc/self/cwd/external/boost/boost/asio/detail/impl/scheduler.ipp:492:12 (libexternal_Sboost_Slibasio.so+0xa9840)
  | [2024-08-27T20:52:49Z]     #18 boost::asio::detail::scheduler::run(boost::system::error_code&) /proc/self/cwd/external/boost/boost/asio/detail/impl/scheduler.ipp:210:10 (libexternal_Sboost_Slibasio.so+0x96501)
  | [2024-08-27T20:52:49Z]     #19 boost::asio::io_context::run() /proc/self/cwd/external/boost/boost/asio/impl/io_context.ipp:63:24 (libexternal_Sboost_Slibasio.so+0x9638e)
  | [2024-08-27T20:52:49Z]     #20 GcsClientReconnectionTest::CreateGCSClient()::'lambda'()::operator()() const /proc/self/cwd/src/ray/gcs/gcs_client/test/gcs_client_reconnection_test.cc:92:27 (gcs_client_reconnection_test+0x11408c)
  | [2024-08-27T20:52:49Z]     #21 decltype(std::__1::forward<GcsClientReconnectionTest::CreateGCSClient()::'lambda'()>(fp)()) std::__1::__invoke<GcsClientReconnectionTest::CreateGCSClient()::'lambda'()>(GcsClientReconnectionTest::CreateGCSClient()::'lambda'()&&) /opt/llvm/bin/../include/c++/v1/type_traits:3694:1 (gcs_client_reconnection_test+0x113f63)
  | [2024-08-27T20:52:49Z]     #22 void std::__1::__thread_execute<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, GcsClientReconnectionTest::CreateGCSClient()::'lambda'()>(std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, GcsClientReconnectionTest::CreateGCSClient()::'lambda'()>&, std::__1::__tuple_indices<>) /opt/llvm/bin/../include/c++/v1/thread:280:5 (gcs_client_reconnection_test+0x113f63)
  | [2024-08-27T20:52:49Z]     #23 void* std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, GcsClientReconnectionTest::CreateGCSClient()::'lambda'()> >(void*) /opt/llvm/bin/../include/c++/v1/thread:291:5 (gcs_client_reconnection_test+0x113f63)
  | [2024-08-27T20:52:49Z]
  | [2024-08-27T20:52:49Z]   Previous write of size 1 at 0x7b0c0000e2d8 by main thread:
  | [2024-08-27T20:52:49Z]     #0 void ray::rpc::GrpcClient<ray::rpc::InternalKVGcsService>::CallMethod<ray::rpc::InternalKVKeysRequest, ray::rpc::InternalKVKeysReply>(std::__1::unique_ptr<grpc::ClientAsyncResponseReader<ray::rpc::InternalKVKeysReply>, std::__1::default_delete<grpc::ClientAsyncResponseReader<ray::rpc::InternalKVKeysReply> > > (ray::rpc::InternalKVGcsService::Stub::*)(grpc::ClientContext*, ray::rpc::InternalKVKeysRequest const&, grpc::CompletionQueue*), ray::rpc::InternalKVKeysRequest const&, std::__1::function<void (ray::Status const&, ray::rpc::InternalKVKeysReply&&)> const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, long) /proc/self/cwd/bazel-out/k8-opt/bin/_virtual_includes/grpc_common_lib/ray/rpc/grpc_client.h:157:26 (liblibgcs_Uclient_Ulib.so+0x32a016)
  | [2024-08-27T20:52:49Z]     #1 void ray::rpc::GcsRpcClient::invoke_async_method<ray::rpc::InternalKVGcsService, ray::rpc::InternalKVKeysRequest, ray::rpc::InternalKVKeysReply, true>(std::__1::unique_ptr<grpc::ClientAsyncResponseReader<ray::rpc::InternalKVKeysReply>, std::__1::default_delete<grpc::ClientAsyncResponseReader<ray::rpc::InternalKVKeysReply> > > (ray::rpc::InternalKVGcsService::Stub::*)(grpc::ClientContext*, ray::rpc::InternalKVKeysRequest const&, grpc::CompletionQueue*), ray::rpc::GrpcClient<ray::rpc::InternalKVGcsService>&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, ray::rpc::InternalKVKeysRequest const&, std::__1::function<void (ray::Status const&, ray::rpc::InternalKVKeysReply&&)> const&, long)::'lambda'()::operator()() const /proc/self/cwd/bazel-out/k8-opt/bin/_virtual_includes/gcs_service_rpc/ray/rpc/gcs_server/gcs_rpc_client.h:283:28 (liblibgcs_Uclient_Ulib.so+0x32a016)
  | [2024-08-27T20:52:49Z]     #2 decltype(std::__1::forward<ray::rpc::InternalKVGcsService>(fp)(std::__1::forward<ray::rpc::InternalKVKeysRequest>(fp0)...)) std::__1::__invoke<void ray::rpc::GcsRpcClient::invoke_async_method<ray::rpc::InternalKVGcsService, ray::rpc::InternalKVKeysRequest, ray::rpc::InternalKVKeysReply, true>(std::__1::unique_ptr<grpc::ClientAsyncResponseReader<ray::rpc::InternalKVKeysReply>, std::__1::default_delete<grpc::ClientAsyncResponseReader<ray::rpc::InternalKVKeysReply> > > (ray::rpc::InternalKVGcsService::Stub::*)(grpc::ClientContext*, ray::rpc::InternalKVKeysRequest const&, grpc::CompletionQueue*), ray::rpc::GrpcClient<ray::rpc::InternalKVGcsService>&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, ray::rpc::InternalKVKeysRequest const&, std::__1::function<void (ray::Status const&, ray::rpc::InternalKVKeysReply&&)> const&, long)::'lambda'()&>(ray::rpc::InternalKVGcsService&&, ray::rpc::InternalKVKeysRequest&&...) /opt/llvm/bin/../include/c++/v1/type_traits:3694:1 (liblibgcs_Uclient_Ulib.so+0x329b0d)
  | [2024-08-27T20:52:49Z]     #3 void std::__1::__invoke_void_return_wrapper<void, true>::__call<void ray::rpc::GcsRpcClient::invoke_async_method<ray::rpc::InternalKVGcsService, ray::rpc::InternalKVKeysRequest, ray::rpc::InternalKVKeysReply, true>(std::__1::unique_ptr<grpc::ClientAsyncResponseReader<ray::rpc::InternalKVKeysReply>, std::__1::default_delete<grpc::ClientAsyncResponseReader<ray::rpc::InternalKVKeysReply> > > (ray::rpc::InternalKVGcsService::Stub::*)(grpc::ClientContext*, ray::rpc::InternalKVKeysRequest const&, grpc::CompletionQueue*), ray::rpc::GrpcClient<ray::rpc::InternalKVGcsService>&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, ray::rpc::InternalKVKeysRequest const&, std::__1::function<void (ray::Status const&, ray::rpc::InternalKVKeysReply&&)> const&, long)::'lambda'()&>(ray::rpc::InternalKVGcsService&&...) /opt/llvm/bin/../include/c++/v1/__functional_base:348:9 (liblibgcs_Uclient_Ulib.so+0x329b0d)
  | [2024-08-27T20:52:49Z]     #4 std::__1::__function::__alloc_func<void ray::rpc::GcsRpcClient::invoke_async_method<ray::rpc::InternalKVGcsService, ray::rpc::InternalKVKeysRequest, ray::rpc::InternalKVKeysReply, true>(std::__1::unique_ptr<grpc::ClientAsyncResponseReader<ray::rpc::InternalKVKeysReply>, std::__1::default_delete<grpc::ClientAsyncResponseReader<ray::rpc::InternalKVKeysReply> > > (ray::rpc::InternalKVGcsService::Stub::*)(grpc::ClientContext*, ray::rpc::InternalKVKeysRequest const&, grpc::CompletionQueue*), ray::rpc::GrpcClient<ray::rpc::InternalKVGcsService>&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, ray::rpc::InternalKVKeysRequest const&, std::__1::function<void (ray::Status const&, ray::rpc::InternalKVKeysReply&&)> const&, long)::'lambda'(), std::__1::allocator<void ray::rpc::GcsRpcClient::invoke_async_method<ray::rpc::InternalKVGcsService, ray::rpc::InternalKVKeysRequest, ray::rpc::InternalKVKeysReply, true>(std::__1::unique_ptr<grpc::ClientAsyncResponseReader<ray::rpc::InternalKVKeysReply>, std::__1::default_delete<grpc::ClientAsyncResponseReader<ray::rpc::InternalKVKeysReply> > > (ray::rpc::InternalKVGcsService::Stub::*)(grpc::ClientContext*, ray::rpc::InternalKVKeysRequest const&, grpc::CompletionQueue*), ray::rpc::GrpcClient<ray::rpc::InternalKVGcsService>&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, ray::rpc::InternalKVKeysRequest const&, std::__1::function<void (ray::Status const&, ray::rpc::InternalKVKeysReply&&)> const&, long)::'lambda'()>, void ()>::operator()() /opt/llvm/bin/../include/c++/v1/functional:1558:16 (liblibgcs_Uclient_Ulib.so+0x329b0d)
  | [2024-08-27T20:52:49Z]     #5 std::__1::__function::__func<void ray::rpc::GcsRpcClient::invoke_async_method<ray::rpc::InternalKVGcsService, ray::rpc::InternalKVKeysRequest, ray::rpc::InternalKVKeysReply, true>(std::__1::unique_ptr<grpc::ClientAsyncResponseReader<ray::rpc::InternalKVKeysReply>, std::__1::default_delete<grpc::ClientAsyncResponseReader<ray::rpc::InternalKVKeysReply> > > (ray::rpc::InternalKVGcsService::Stub::*)(grpc::ClientContext*, ray::rpc::InternalKVKeysRequest const&, grpc::CompletionQueue*), ray::rpc::GrpcClient<ray::rpc::InternalKVGcsService>&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, ray::rpc::InternalKVKeysRequest const&, std::__1::function<void (ray::Status const&, ray::rpc::InternalKVKeysReply&&)> const&, long)::'lambda'(), std::__1::allocator<void ray::rpc::GcsRpcClient::invoke_async_method<ray::rpc::InternalKVGcsService, ray::rpc::InternalKVKeysRequest, ray::rpc::InternalKVKeysReply, true>(std::__1::unique_ptr<grpc::ClientAsyncResponseReader<ray::rpc::InternalKVKeysReply>, std::__1::default_delete<grpc::ClientAsyncResponseReader<ray::rpc::InternalKVKeysReply> > > (ray::rpc::InternalKVGcsService::Stub::*)(grpc::ClientContext*, ray::rpc::InternalKVKeysRequest const&, grpc::CompletionQueue*), ray::rpc::GrpcClient<ray::rpc::InternalKVGcsService>&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, ray::rpc::InternalKVKeysRequest const&, std::__1::function<void (ray::Status const&, ray::rpc::InternalKVKeysReply&&)> const&, long)::'lambda'()>, void ()>::operator()() /opt/llvm/bin/../include/c++/v1/functional:1732:12 (liblibgcs_Uclient_Ulib.so+0x329b0d)
  | [2024-08-27T20:52:49Z]     #6 std::__1::__function::__value_func<void ()>::operator()() const /opt/llvm/bin/../include/c++/v1/functional:1885:16 (liblibgcs_Uclient_Ulib.so+0x328c0e)
  | [2024-08-27T20:52:49Z]     #7 std::__1::function<void ()>::operator()() const /opt/llvm/bin/../include/c++/v1/functional:2560:12 (liblibgcs_Uclient_Ulib.so+0x328c0e)
  | [2024-08-27T20:52:49Z]     #8 ray::rpc::Executor::Execute(std::__1::function<void ()>) /proc/self/cwd/bazel-out/k8-opt/bin/_virtual_includes/gcs_service_rpc/ray/rpc/gcs_server/gcs_rpc_client.h:46:5 (liblibgcs_Uclient_Ulib.so+0x328c0e)
  | [2024-08-27T20:52:49Z]     #9 void ray::rpc::GcsRpcClient::invoke_async_method<ray::rpc::InternalKVGcsService, ray::rpc::InternalKVKeysRequest, ray::rpc::InternalKVKeysReply, true>(std::__1::unique_ptr<grpc::ClientAsyncResponseReader<ray::rpc::InternalKVKeysReply>, std::__1::default_delete<grpc::ClientAsyncResponseReader<ray::rpc::InternalKVKeysReply> > > (ray::rpc::InternalKVGcsService::Stub::*)(grpc::ClientContext*, ray::rpc::InternalKVKeysRequest const&, grpc::CompletionQueue*), ray::rpc::GrpcClient<ray::rpc::InternalKVGcsService>&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, ray::rpc::InternalKVKeysRequest const&, std::__1::function<void (ray::Status const&, ray::rpc::InternalKVKeysReply&&)> const&, long) /proc/self/cwd/bazel-out/k8-opt/bin/_virtual_includes/gcs_service_rpc/ray/rpc/gcs_server/gcs_rpc_client.h:286:15 (liblibgcs_Uclient_Ulib.so+0x328c0e)
  | [2024-08-27T20:52:49Z]     #10 ray::rpc::GcsRpcClient::InternalKVKeys(ray::rpc::InternalKVKeysRequest const&, std::__1::function<void (ray::Status const&, ray::rpc::InternalKVKeysReply&&)> const&, long) /proc/self/cwd/bazel-out/k8-opt/bin/_virtual_includes/gcs_service_rpc/ray/rpc/gcs_server/gcs_rpc_client.h:529:3 (liblibgcs_Uclient_Ulib.so+0x28a663)
  | [2024-08-27T20:52:49Z]     #11 ray::gcs::InternalKVAccessor::AsyncInternalKVKeys(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, long, std::__1::function<void (ray::Status, std::__1::optional<std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > >&&)> const&) /proc/self/cwd/src/ray/gcs/gcs_client/accessor.cc:1230:35 (liblibgcs_Uclient_Ulib.so+0x28a663)
  | [2024-08-27T20:52:49Z]     #12 ray::gcs::InternalKVAccessor::Keys(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, long, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > >&) /proc/self/cwd/src/ray/gcs/gcs_client/accessor.cc:1268:3 (liblibgcs_Uclient_Ulib.so+0x28ae40)
  | [2024-08-27T20:52:49Z]     #13 GcsClientReconnectionTest_Timeout_Test::TestBody() /proc/self/cwd/src/ray/gcs/gcs_client/test/gcs_client_reconnection_test.cc:355:3 (gcs_client_reconnection_test+0x110635)
  | [2024-08-27T20:52:49Z]     #14 void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /proc/self/cwd/external/com_google_googletest/googletest/src/gtest.cc:2612:10 (libexternal_Scom_Ugoogle_Ugoogletest_Slibgtest.so+0xd4a1f)
  | [2024-08-27T20:52:49Z]     #15 void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /proc/self/cwd/external/com_google_googletest/googletest/src/gtest.cc:2648:14 (libexternal_Scom_Ugoogle_Ugoogletest_Slibgtest.so+0xd4a1f)
  | [2024-08-27T20:52:49Z]     #16 testing::Test::Run() /proc/self/cwd/external/com_google_googletest/googletest/src/gtest.cc:2687:5 (libexternal_Scom_Ugoogle_Ugoogletest_Slibgtest.so+0xd4901)
  | [2024-08-27T20:52:49Z]     #17 testing::TestInfo::Run() /proc/self/cwd/external/com_google_googletest/googletest/src/gtest.cc:2836:11 (libexternal_Scom_Ugoogle_Ugoogletest_Slibgtest.so+0xd63c8)
  | [2024-08-27T20:52:49Z]     #18 testing::TestSuite::Run() /proc/self/cwd/external/com_google_googletest/googletest/src/gtest.cc:3015:30 (libexternal_Scom_Ugoogle_Ugoogletest_Slibgtest.so+0xd79e4)
  | [2024-08-27T20:52:49Z]     #19 testing::internal::UnitTestImpl::RunAllTests() /proc/self/cwd/external/com_google_googletest/googletest/src/gtest.cc:5920:44 (libexternal_Scom_Ugoogle_Ugoogletest_Slibgtest.so+0xecc44)
  | [2024-08-27T20:52:49Z]     #20 bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /proc/self/cwd/external/com_google_googletest/googletest/src/gtest.cc:2612:10 (libexternal_Scom_Ugoogle_Ugoogletest_Slibgtest.so+0xec05f)
  | [2024-08-27T20:52:49Z]     #21 bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /proc/self/cwd/external/com_google_googletest/googletest/src/gtest.cc:2648:14 (libexternal_Scom_Ugoogle_Ugoogletest_Slibgtest.so+0xec05f)
  | [2024-08-27T20:52:49Z]     #22 testing::UnitTest::Run() /proc/self/cwd/external/com_google_googletest/googletest/src/gtest.cc:5484:10 (libexternal_Scom_Ugoogle_Ugoogletest_Slibgtest.so+0xebe4c)
  | [2024-08-27T20:52:49Z]     #23 RUN_ALL_TESTS() /proc/self/cwd/external/com_google_googletest/googletest/include/gtest/gtest.h:2317:73 (gcs_client_reconnection_test+0x1113a3)
  | [2024-08-27T20:52:49Z]     #24 main /proc/self/cwd/src/ray/gcs/gcs_client/test/gcs_client_reconnection_test.cc:369:10 (gcs_client_reconnection_test+0x1113a3)
  | [2024-08-27T20:52:49Z]
  | [2024-08-27T20:52:49Z]   Location is heap block of size 48 at 0x7b0c0000e2b0 allocated by main thread:
  | [2024-08-27T20:52:49Z]     #0 malloc /tmp/llvm/utils/release/final/llvm-project/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:651:5 (gcs_client_reconnection_test+0x9b62b)
  | [2024-08-27T20:52:49Z]     #1 operator new(unsigned long) <null> (liblibgcs_Userver_Ulib.so+0x7a2d24)
  | [2024-08-27T20:52:49Z]     #2 std::__1::__shared_ptr_emplace<ray::rpc::GcsRpcClient, std::__1::allocator<ray::rpc::GcsRpcClient> >::__shared_ptr_emplace<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >&, int&, ray::rpc::ClientCallManager&>(std::__1::allocator<ray::rpc::GcsRpcClient>, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >&, int&, ray::rpc::ClientCallManager&) /opt/llvm/bin/../include/c++/v1/memory:2622:37 (liblibgcs_Uclient_Ulib.so+0x36f66d)
  | [2024-08-27T20:52:49Z]     #3 std::__1::shared_ptr<ray::rpc::GcsRpcClient> std::__1::allocate_shared<ray::rpc::GcsRpcClient, std::__1::allocator<ray::rpc::GcsRpcClient>, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >&, int&, ray::rpc::ClientCallManager&, void>(std::__1::allocator<ray::rpc::GcsRpcClient> const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >&, int&, ray::rpc::ClientCallManager&) /opt/llvm/bin/../include/c++/v1/memory:3380:55 (liblibgcs_Uclient_Ulib.so+0x36f66d)
  | [2024-08-27T20:52:49Z]     #4 std::__1::shared_ptr<ray::rpc::GcsRpcClient> std::__1::make_shared<ray::rpc::GcsRpcClient, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >&, int&, ray::rpc::ClientCallManager&, void>(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >&, int&, ray::rpc::ClientCallManager&) /opt/llvm/bin/../include/c++/v1/memory:3389:12 (liblibgcs_Uclient_Ulib.so+0x36f66d)
  | [2024-08-27T20:52:49Z]     #5 ray::gcs::GcsClient::Connect(instrumented_io_context&, long) /proc/self/cwd/src/ray/gcs/gcs_client/gcs_client.cc:112:21 (liblibgcs_Uclient_Ulib.so+0x36f66d)
  | [2024-08-27T20:52:49Z]     #6 GcsClientReconnectionTest::CreateGCSClient() /proc/self/cwd/src/ray/gcs/gcs_client/test/gcs_client_reconnection_test.cc:100:5 (gcs_client_reconnection_test+0x10d0dd)
  | [2024-08-27T20:52:49Z]     #7 GcsClientReconnectionTest_Timeout_Test::TestBody() /proc/self/cwd/src/ray/gcs/gcs_client/test/gcs_client_reconnection_test.cc:342:17 (gcs_client_reconnection_test+0x11000e)
  | [2024-08-27T20:52:49Z]     #8 void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /proc/self/cwd/external/com_google_googletest/googletest/src/gtest.cc:2612:10 (libexternal_Scom_Ugoogle_Ugoogletest_Slibgtest.so+0xd4a1f)
  | [2024-08-27T20:52:49Z]     #9 void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /proc/self/cwd/external/com_google_googletest/googletest/src/gtest.cc:2648:14 (libexternal_Scom_Ugoogle_Ugoogletest_Slibgtest.so+0xd4a1f)
  | [2024-08-27T20:52:49Z]     #10 testing::Test::Run() /proc/self/cwd/external/com_google_googletest/googletest/src/gtest.cc:2687:5 (libexternal_Scom_Ugoogle_Ugoogletest_Slibgtest.so+0xd4901)
  | [2024-08-27T20:52:49Z]     #11 testing::TestInfo::Run() /proc/self/cwd/external/com_google_googletest/googletest/src/gtest.cc:2836:11 (libexternal_Scom_Ugoogle_Ugoogletest_Slibgtest.so+0xd63c8)
  | [2024-08-27T20:52:49Z]     #12 testing::TestSuite::Run() /proc/self/cwd/external/com_google_googletest/googletest/src/gtest.cc:3015:30 (libexternal_Scom_Ugoogle_Ugoogletest_Slibgtest.so+0xd79e4)
  | [2024-08-27T20:52:49Z]     #13 testing::internal::UnitTestImpl::RunAllTests() /proc/self/cwd/external/com_google_googletest/googletest/src/gtest.cc:5920:44 (libexternal_Scom_Ugoogle_Ugoogletest_Slibgtest.so+0xecc44)
  | [2024-08-27T20:52:49Z]     #14 bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /proc/self/cwd/external/com_google_googletest/googletest/src/gtest.cc:2612:10 (libexternal_Scom_Ugoogle_Ugoogletest_Slibgtest.so+0xec05f)
  | [2024-08-27T20:52:49Z]     #15 bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /proc/self/cwd/external/com_google_googletest/googletest/src/gtest.cc:2648:14 (libexternal_Scom_Ugoogle_Ugoogletest_Slibgtest.so+0xec05f)
  | [2024-08-27T20:52:49Z]     #16 testing::UnitTest::Run() /proc/self/cwd/external/com_google_googletest/googletest/src/gtest.cc:5484:10 (libexternal_Scom_Ugoogle_Ugoogletest_Slibgtest.so+0xebe4c)
  | [2024-08-27T20:52:49Z]     #17 RUN_ALL_TESTS() /proc/self/cwd/external/com_google_googletest/googletest/include/gtest/gtest.h:2317:73 (gcs_client_reconnection_test+0x1113a3)
  | [2024-08-27T20:52:49Z]     #18 main /proc/self/cwd/src/ray/gcs/gcs_client/test/gcs_client_reconnection_test.cc:369:10 (gcs_client_reconnection_test+0x1113a3)
  | [2024-08-27T20:52:49Z]
  | [2024-08-27T20:52:49Z]   Thread T14 (tid=2089, running) created by main thread at:
  | [2024-08-27T20:52:49Z]     #0 pthread_create /tmp/llvm/utils/release/final/llvm-project/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:965:3 (gcs_client_reconnection_test+0x9ce1b)
  | [2024-08-27T20:52:49Z]     #1 std::__1::__libcpp_thread_create(unsigned long*, void* (*)(void*), void*) /opt/llvm/bin/../include/c++/v1/__threading_support:509:10 (gcs_client_reconnection_test+0x113e86)
  | [2024-08-27T20:52:49Z]     #2 std::__1::thread::thread<GcsClientReconnectionTest::CreateGCSClient()::'lambda'(), void>(GcsClientReconnectionTest::CreateGCSClient()::'lambda'()&&) /opt/llvm/bin/../include/c++/v1/thread:307:16 (gcs_client_reconnection_test+0x113e86)
  | [2024-08-27T20:52:49Z]     #3 std::__1::__unique_if<std::__1::thread>::__unique_single std::__1::make_unique<std::__1::thread, GcsClientReconnectionTest::CreateGCSClient()::'lambda'()>(GcsClientReconnectionTest::CreateGCSClient()::'lambda'()&&) /opt/llvm/bin/../include/c++/v1/memory:2094:32 (gcs_client_reconnection_test+0x10ced4)
  | [2024-08-27T20:52:49Z]     #4 GcsClientReconnectionTest::CreateGCSClient() /proc/self/cwd/src/ray/gcs/gcs_client/test/gcs_client_reconnection_test.cc:89:33 (gcs_client_reconnection_test+0x10ced4)
  | [2024-08-27T20:52:49Z]     #5 GcsClientReconnectionTest_Timeout_Test::TestBody() /proc/self/cwd/src/ray/gcs/gcs_client/test/gcs_client_reconnection_test.cc:342:17 (gcs_client_reconnection_test+0x11000e)
  | [2024-08-27T20:52:49Z]     #6 void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /proc/self/cwd/external/com_google_googletest/googletest/src/gtest.cc:2612:10 (libexternal_Scom_Ugoogle_Ugoogletest_Slibgtest.so+0xd4a1f)
  | [2024-08-27T20:52:49Z]     #7 void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /proc/self/cwd/external/com_google_googletest/googletest/src/gtest.cc:2648:14 (libexternal_Scom_Ugoogle_Ugoogletest_Slibgtest.so+0xd4a1f)
  | [2024-08-27T20:52:49Z]     #8 testing::Test::Run() /proc/self/cwd/external/com_google_googletest/googletest/src/gtest.cc:2687:5 (libexternal_Scom_Ugoogle_Ugoogletest_Slibgtest.so+0xd4901)
  | [2024-08-27T20:52:49Z]     #9 testing::TestInfo::Run() /proc/self/cwd/external/com_google_googletest/googletest/src/gtest.cc:2836:11 (libexternal_Scom_Ugoogle_Ugoogletest_Slibgtest.so+0xd63c8)
  | [2024-08-27T20:52:49Z]     #10 testing::TestSuite::Run() /proc/self/cwd/external/com_google_googletest/googletest/src/gtest.cc:3015:30 (libexternal_Scom_Ugoogle_Ugoogletest_Slibgtest.so+0xd79e4)
  | [2024-08-27T20:52:49Z]     #11 testing::internal::UnitTestImpl::RunAllTests() /proc/self/cwd/external/com_google_googletest/googletest/src/gtest.cc:5920:44 (libexternal_Scom_Ugoogle_Ugoogletest_Slibgtest.so+0xecc44)
  | [2024-08-27T20:52:49Z]     #12 bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /proc/self/cwd/external/com_google_googletest/googletest/src/gtest.cc:2612:10 (libexternal_Scom_Ugoogle_Ugoogletest_Slibgtest.so+0xec05f)
  | [2024-08-27T20:52:49Z]     #13 bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /proc/self/cwd/external/com_google_googletest/googletest/src/gtest.cc:2648:14 (libexternal_Scom_Ugoogle_Ugoogletest_Slibgtest.so+0xec05f)
  | [2024-08-27T20:52:49Z]     #14 testing::UnitTest::Run() /proc/self/cwd/external/com_google_googletest/googletest/src/gtest.cc:5484:10 (libexternal_Scom_Ugoogle_Ugoogletest_Slibgtest.so+0xebe4c)
  | [2024-08-27T20:52:49Z]     #15 RUN_ALL_TESTS() /proc/self/cwd/external/com_google_googletest/googletest/include/gtest/gtest.h:2317:73 (gcs_client_reconnection_test+0x1113a3)
  | [2024-08-27T20:52:49Z]     #16 main /proc/self/cwd/src/ray/gcs/gcs_client/test/gcs_client_reconnection_test.cc:369:10 (gcs_client_reconnection_test+0x1113a3)
  | [2024-08-27T20:52:49Z]
  | [2024-08-27T20:52:49Z] SUMMARY: ThreadSanitizer: data race /proc/self/cwd/bazel-out/k8-opt/bin/_virtual_includes/grpc_common_lib/ray/rpc/grpc_client.h:157:26 in void ray::rpc::GrpcClient<ray::rpc::InternalKVGcsService>::CallMethod<ray::rpc::InternalKVKeysRequest, ray::rpc::InternalKVKeysReply>(std::__1::unique_ptr<grpc::ClientAsyncResponseReader<ray::rpc::InternalKVKeysReply>, std::__1::default_delete<grpc::ClientAsyncResponseReader<ray::rpc::InternalKVKeysReply> > > (ray::rpc::InternalKVGcsService::Stub::*)(grpc::ClientContext*, ray::rpc::InternalKVKeysRequest const&, grpc::CompletionQueue*), ray::rpc::InternalKVKeysRequest const&, std::__1::function<void (ray::Status const&, ray::rpc::InternalKVKeysReply&&)> const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, long)
  | [2024-08-27T20:52:49Z] ==================

jjyao avatar Aug 27 '24 22:08 jjyao

I treat it as code cleanup instead of performance improvement

jjyao avatar Aug 28 '24 14:08 jjyao

yeah that makse sense. just want to understand what it exactly means by saving memory.

rkooo567 avatar Aug 28 '24 18:08 rkooo567

@wuxibin89 any updates on this?

jjyao avatar Oct 29 '24 06:10 jjyao

This pull request has been automatically marked as stale because it has not had any activity for 14 days. It will be closed in another 14 days if no further activity occurs. Thank you for your contributions.

You can always ask for help on our discussion forum or Ray's public slack channel.

If you'd like to keep this open, just leave any comment, and the stale label will be removed.

github-actions[bot] avatar May 31 '25 00:05 github-actions[bot]

This pull request has been automatically closed because there has been no more activity in the 14 days since being marked stale.

Please feel free to reopen or open a new pull request if you'd still like this to be addressed.

Again, you can always ask for help on our discussion forum or Ray's public slack channel.

Thanks again for your contribution!

github-actions[bot] avatar Jun 14 '25 00:06 github-actions[bot]