datastax
CASSJAVA-97: Let users inject an ID for each request and write to the custom payload
I did integration testing with C* OSS 5.0.2. @lukasz-antoniak helped me add a LoggingQueryHandler and set it as the cassandra.custom_query_handler_class.
I developed a client app using this Java driver with the following config
datastax-java-driver.advanced = {
distributed-tracing.id-generator.class = W3CContextDistributedTraceIdGenerator
distributed-tracing.custom-payload-with-key = "traceparent"
}
Running this client app, I got
17:03:20.860 [s0-io-5] TRACE InFlightHandler - [s0|id: 0xeefaab65, L:/127.0.0.2:52389 - R:/127.0.0.2:9042] Writing 00-d51ed2012c1f31b4434f409e50294da9-25da248d0a23981c-00 on stream id 0
17:03:20.863 [s0-io-5] TRACE CqlRequestHandler$NodeResponseCallback - [00-d51ed2012c1f31b4434f409e50294da9-25da248d0a23981c-00] Request sent on [id: 0xeefaab65, L:/127.0.0.2:52389 - R:/127.0.0.2:9042]
17:03:20.864 [s0-io-5] TRACE CqlRequestHandler$NodeResponseCallback - [00-d51ed2012c1f31b4434f409e50294da9-25da248d0a23981c-00] Speculative execution policy returned -1, no next execution
17:03:20.877 [s0-io-5] DEBUG InFlightHandler - [s0|id: 0xeefaab65, L:/127.0.0.2:52389 - R:/127.0.0.2:9042] Got last response on in-flight stream id 0, completing and releasing
17:03:20.877 [s0-io-5] TRACE InFlightHandler - [s0|id: 0xeefaab65, L:/127.0.0.2:52389 - R:/127.0.0.2:9042] Releasing stream id 0
17:03:20.877 [s0-io-5] TRACE CqlRequestHandler$NodeResponseCallback - [00-d51ed2012c1f31b4434f409e50294da9-25da248d0a23981c-00] Got result, completing
And the debug.log at server side got
DEBUG [Native-Transport-Requests-1] 2025-04-15 17:03:20,870 LoggingQueryHandler.java:44 - Processing CQL statement SelectStatement[aggregationSpecFactory=
,bindVariables=[],isReversed=false,limit= ,orderingComparator= ,parameters=org.apache.cassandra.cql3.statements.SelectStatement$Parameters@5d46ec82,perPartitionLimit= ,restrictions=StatementRestrictions[clusteringColumnsRestrictions=ClusteringColumnRestrictions[allowFiltering=false,comparator=comparator(),restrictions=RestrictionSet[hasAnn=false,hasContains=false,hasIn=false,hasMultiColumnRestrictions=false,hasOnlyEqualityRestrictions=true,hasSlice=false,restrictions={}]],filterRestrictions=IndexRestrictions[customExpressions=[],regularRestrictions=[]],hasRegularColumnsRestrictions=false,isKeyRange=true,nonPrimaryKeyRestrictions=RestrictionSet[hasAnn=false,hasContains=false,hasIn=false,hasMultiColumnRestrictions=false,hasOnlyEqualityRestrictions=true,hasSlice=false,restrictions={}],notNullColumns=[],partitionKeyRestrictions=PartitionKeySingleRestrictionSet[comparator=comparator(org.apache.cassandra.db.marshal.UTF8Type),restrictions=RestrictionSet[hasAnn=false,hasContains=false,hasIn=false,hasMultiColumnRestrictions=false,hasOnlyEqualityRestrictions=true,hasSlice=false,restrictions={}]],table=system.local,type=SELECT,usesSecondaryIndexing=false],selection=SimpleSelection{columns=[key, bootstrapped, broadcast_address, broadcast_port, cluster_name, cql_version, data_center, gossip_generation, host_id, listen_address, listen_port, native_protocol_version, partitioner, rack, release_version, rpc_address, rpc_port, schema_version, tokens, truncated_at], columnMapping={ Columns:[key, bootstrapped, broadcast_address, broadcast_port, cluster_name, cql_version, data_center, gossip_generation, host_id, listen_address, listen_port, native_protocol_version, partitioner, rack, release_version, rpc_address, rpc_port, schema_version, tokens, truncated_at], Mappings:{rack:[rack], cql_version:[cql_version], listen_address:[listen_address], release_version:[release_version], data_center:[data_center], broadcast_port:[broadcast_port], broadcast_address:[broadcast_address], partitioner:[partitioner], host_id:[host_id], gossip_generation:[gossip_generation], listen_port:[listen_port], rpc_address:[rpc_address], schema_version:[schema_version], rpc_port:[rpc_port], truncated_at:[truncated_at], cluster_name:[cluster_name], native_protocol_version:[native_protocol_version], tokens:[tokens], key:[key], bootstrapped:[bootstrapped]} }, metadata=[key(system, local), org.apache.cassandra.db.marshal.UTF8Type][bootstrapped(system, local), org.apache.cassandra.db.marshal.UTF8Type][broadcast_address(system, local), org.apache.cassandra.db.marshal.InetAddressType][broadcast_port(system, local), org.apache.cassandra.db.marshal.Int32Type][cluster_name(system, local), org.apache.cassandra.db.marshal.UTF8Type][cql_version(system, local), org.apache.cassandra.db.marshal.UTF8Type][data_center(system, local), org.apache.cassandra.db.marshal.UTF8Type][gossip_generation(system, local), org.apache.cassandra.db.marshal.Int32Type][host_id(system, local), org.apache.cassandra.db.marshal.UUIDType][listen_address(system, local), org.apache.cassandra.db.marshal.InetAddressType][listen_port(system, local), org.apache.cassandra.db.marshal.Int32Type][native_protocol_version(system, local), org.apache.cassandra.db.marshal.UTF8Type][partitioner(system, local), org.apache.cassandra.db.marshal.UTF8Type][rack(system, local), org.apache.cassandra.db.marshal.UTF8Type][release_version(system, local), org.apache.cassandra.db.marshal.UTF8Type][rpc_address(system, local), org.apache.cassandra.db.marshal.InetAddressType][rpc_port(system, local), org.apache.cassandra.db.marshal.Int32Type][schema_version(system, local), org.apache.cassandra.db.marshal.UUIDType][tokens(system, local), org.apache.cassandra.db.marshal.SetType(org.apache.cassandra.db.marshal.UTF8Type)][truncated_at(system, local), org.apache.cassandra.db.marshal.MapType(org.apache.cassandra.db.marshal.UUIDType,org.apache.cassandra.db.marshal.BytesType)]},table=system.local] with custom payload {traceparent=30302d64353165643230313263316633316234343334663430396535303239346461392d323564613234386430613233393831632d3030}
The value 30302d64353165643230313263316633316234343334663430396535303239346461392d323564613234386430613233393831632d3030 is the hex of the id 00-d51ed2012c1f31b4434f409e50294da9-25da248d0a23981c-00. This shows the capability of tracing a request across client and server.
In order to make this more concrete I've created a PR which clearly states my chief concerns for the current impl (as of this coment) and proposes a concrete implementation which I believe addresses them. This PR is made against a local version of the "request-traceability" branch from @SiyaoIsHiding 's original PR... it seemed like the easiest way to represent the proposed changes.
I also include logging output from a small sample app using this code.... hopefully it clearly demonstrates the goal of these changes.
After discussing the PR containing my proposed changes (referenced above) with @SiyaoIsHiding there seemed to be broad agreement on the structural changes (essentially decoupling the request ID from the session name and treating it as a replacement for the CqlRequestHandler hash code only) but some disagreement on method parameters and a few naming questions. In order to avoid fragmenting the discussion I've merged those changes into this PR (see this commit). We'll continue the discussion on these remaining points here.
I'll be closing the alternate PR with a similar note shortly.
Okay gang, I believe that all outstanding issues on this PR have now been addressed. I'm going to leave this open until sometime around end of day today so that everybody has an opportunity to take one final pass and make sure you can live with what's here... but in the end I want to get this merged so that we can start the process of getting 4.19.1 out the door.
Last call for any final comments on this one. If I don't hear any objections from anybody this is going in sometime this evening U.S. time.
cc/ @SiyaoIsHiding @lukasz-antoniak @aratno
Jenkins run as of https://github.com/apache/cassandra-java-driver/pull/2037/commits/b72d2a183cb0d337baa5d35d293a92239ebafcef looks good. We are (finally) set to call this good and get it merged!
HUGE thanks to @SiyaoIsHiding , @lukasz-antoniak, @aratno and @joao-r-reis for all the help (and persistence!) on this one!