authorjapps
[KAFKA]Cannot inject ZeroCode variables into Validators
Description:
This is my usecase:
Details were already provided initially. Copy pasting it again
- Consuming from Kafka multiple messages which are from different partitions.
- Need to validate the ID field of each message consumed. This ID is Random.Number generated in a previous test (REST POST).
- Trying to access this ID deterministically using validators and it is not working. Below is the validator being used. While consuming we are trying to validate the ID of the message consumed.
{
"name": "consume_from_test-topic",
"url": "kafka-topic:test-topic",
"operation": "consume",
"request": {
"consumerLocalConfigs": {
"recordType": "JSON",
"commitSync": false,
"showRecordsConsumed": true,
"maxNoOfRetryPollsOrTimeouts": 2
}
},
"validators": [
{
"field": "$..journals[?(@.id==${$.send_valid_unique_identifiers.response.body.rawJournals.[0].id})]",
"value": "${$.send_valid_unique_identifiers.response.body.rawJournals.[0].id}"
}
]
}
This is my actual request and actual response
Paste a sample or example here:
Complete logs already attached, as previously requested.
This is what I want to assert or validate
Paste a sample or example here:
Need to validate the value of ID field of each message consumed. This ID is Random.UUID Number generated in a previous test (REST POST)
Further details:
@authorjapps Scenario:
- Consuming from Kafka multiple messages which are from different partitions.
- Need to validate the ID field of each message consumed. This ID is Random.Number generated in a previous test (REST POST).
- Trying to access this ID deterministically using validators and it is not working. Below is the validator being used. While consuming we are trying to validate the ID of the message consumed.
{
"name": "consume_from_test-topic",
"url": "kafka-topic:test-topic",
"operation": "consume",
"request": {
"consumerLocalConfigs": {
"recordType": "JSON",
"commitSync": false,
"showRecordsConsumed": true,
"maxNoOfRetryPollsOrTimeouts": 2
}
},
"validators": [
{
"field": "$..journals[?(@.id==${$.send_valid_unique_identifiers.response.body.rawJournals.[0].id})]",
"value": "${$.send_valid_unique_identifiers.response.body.rawJournals.[0].id}"
}
]
}
Parsing the jsonpath with variables is possible, but this assertion is not working as you would expect.
The field will give a result as '[{"id":"0e42651b-9e4d-4d82-ad77-7c115c947dde"}]' while the value will become '0e42651b-9e4d-4d82-ad77-7c115c947dde'. 😉
Hi @RDBreed - thanks for the response. I am not clear about your answer tho. So is the way I am asserting is wrong? If yes, then what is the right way? Please advise.
... Trying to access this ID deterministically using validators and it is not working.
"validators": [ { "field": "$..journals[?(@.id==${$.send_valid_unique_identifiers.response.body.rawJournals.[0].id})]", ... }
@kt09 , the "field" should be a key(i.e. a field name), not the value. Try this!
or simply use the matchers instead of validators.
"validators": [
{
"field": "$..journals[?(@.fromAccount.accountId='123456')].id",
"value": "${$.send_valid_unique_identifiers.response.body.rawJournals.[0].id}"
}
]
AssertionErrors:
Assertion jsonpath '$' with actual value '["<generated UUID>"]' did not match the expected value '${$.send_valid_unique_identifiers.response.body.rawJournals.[0].id}'
@authorjapps - changed the field to look for id when accountId = 123456 The response from the consumer for this accountId is not deterministic. Sometimes this accountId comes first, sometimes second. Additionally, as you can see from the log snippet above, value is not being able to resolve. And that is the issue. Am I doing something wrong here? If yes, what is the correct way to get to the value of id (which is auto generated using RANDOM.UUID in previous test) from the request or response, of a previous test case with accountID = 123456. Please advise.
Matchers - documentation for this is not clear with not enough examples and context. HelloWorld github also does not have enough examples. I feel that since the documentation has moved to the new website, lot of context is missing and is not clear. Please provide complete context or documentation for Matchers
@kt09 , Issue is not having details what problem you want to solve. Hence no documentation will be helpful in this case. Provide an usecase and make the problem statements clear for the collaborators to answer it.
Can you capture the following into the description?
- What is your usecase?
- What is your actual request and actual response?
- What you want to assert or validate?
Only then we will be able to analyse it further!
Read this please before raising an issue: https://github.com/authorjapps/zerocode/wiki/Guidelines-for-raising-issues
@authorjapps - thanks for the response. All the above points were already addressed. I have updated it again. Given the detailed description and the log file and the follow-up update, is there anything not clear still?
@authorjapps - breaking down the test steps even more!
Test Case steps:
- Test case 1 - Send a POST request which posts 2 records.
- Microservice picks these 2 records.
- Microservice publishes these 2 records on a Kafka topic. Record format is Json.
- Let us assume that the ID of these two records published in Step 3 are ID 1 and ID 2.
- Test case 2 - Consume from this topic and validate that ID 1 and ID 2 fields of the 2 records published in Step 3 are present on the topic.
Issue is on Step 5, where I cannot validate the value of ID field (1 and 2 in the steps above) using the method I have described. Hope this further break down is clear!
@authorjapps - breaking down the test steps even more!
Test Case steps:
- Test case 1 - Send a POST request which posts 2 records.
- Microservice picks these 2 records.
- Microservice publishes these 2 records on a Kafka topic. Record format is Json.
- Let us assume that the ID of these two records published in Step 3 are ID 1 and ID 2.
- Test case 2 - Consume from this topic and validate that ID 1 and ID 2 fields of the 2 records published in Step 3 are present on the topic.
Issue is on Step 5, where I cannot validate the value of ID field (1 and 2 in the steps above) using the method I have described. Hope this further break down is clear!
@kt09 Can you please paste the specific snippet from the log? The attached log file has too many other unrelated details which might be make it difficult for the collaborators to drill down to your specific problem. This looks more like a usage problem than an issue. You can discuss this in Slack too where you may find assistance from wider audience
@officiallysameer - thanks for the response. Please see below as requested. Also, since it has been tagged at "not-an-issue", I am hoping for a quick turnaround on this. Please keep me posted.
INITIAL LOG SNIPPET
Record Key - null , Record value - {"journals":[{"id":"0e42651b-9e4d-4d82-ad77-7c115c947dde", //some additional attributes}, Record partition - 0, Record offset - 10972, Headers - RecordHeaders(headers = [RecordHeader(key = Value.Type, value = [74, 111, 117, 114, 110, 97, 108, 115])], isReadOnly = false)
2020-10-06 09:14:56,360 [main] INFO org.jsmart.zerocode.core.kafka.helper.KafkaConsumerHelper -
Record Key - null , Record value - {"journals":[{"id":"59b61479-8690-4bee-9334-7937b9ac3a0f",//some additioan attributes}, Record partition - 39, Record offset - 10969, Headers - RecordHeaders(headers = [RecordHeader(key = Value.Type, value = [74, 111, 117, 114, 110, 97, 108, 115])], isReadOnly = false)
2020-10-06 09:14:56,371 [main] INFO kafka.PORTKafkaClient - received response from kafka={
"records" : [ {
"key" : null,
"jsonKey" : null,
"value" : {
"journals" : [ {
"id" : "0e42651b-9e4d-4d82-ad77-7c115c947dde",
//some additional attributes
} ],
"systemId" : "testId",
"userId" : "testUser",
"receivedTime" : "2020-10-06T13:14:54.783754"
},
"headers" : {
"Value.Type" : "Journals"
}
}, {
"key" : null,
"jsonKey" : null,
"value" : {
"journals" : [ {
"id" : "59b61479-8690-4bee-9334-7937b9ac3a0f",
//some additional attributes
} ],
"systemId" : "testId",
"userId" : "testUser",
"receivedTime" : "2020-10-06T13:14:54.783754"
},
"headers" : {
"Value.Type" : "Journals"
}
} ],
"size" : 2
}
2020-10-06 09:14:56,372 [main] INFO org.jsmart.zerocode.core.engine.validators.ZeroCodeValidatorImpl - Comparing results via flat validators
com.jayway.jsonpath.InvalidPathException: Expected character: )
2020-10-06 09:14:56,391 [main] ERROR org.jsmart.zerocode.core.runner.ZeroCodeMultiStepsScenarioRunnerImpl - ###Exception while executing a step in the zerocode dsl.
at com.jayway.jsonpath.internal.CharacterIndex.readSignificantChar(CharacterIndex.java:218)
at com.jayway.jsonpath.internal.filter.FilterCompiler.readLogicalANDOperand(FilterCompiler.java:185)
at com.jayway.jsonpath.internal.filter.FilterCompiler.readLogicalAND(FilterCompiler.java:151)
at com.jayway.jsonpath.internal.filter.FilterCompiler.readLogicalOR(FilterCompiler.java:131)
at com.jayway.jsonpath.internal.filter.FilterCompiler.compile(FilterCompiler.java:77)
........
2020-10-06 09:14:56,392 [main] ERROR org.jsmart.zerocode.core.runner.StepNotificationHandler - Exception occurred while executing Scenario:[journals with duplicate identifiers should fail], --> Step:[consume_from_journals.enriched], Details: java.lang.RuntimeException: ZeroCode Step execution failed. Details:com.jayway.jsonpath.InvalidPathException: Expected character: )
2020-10-06 09:14:56,395 [main] WARN org.jsmart.zerocode.core.runner.ZeroCodeMultiStepsScenarioRunnerImpl -
==========================================
LOG SNIPPET AFTER MAKING CHANGE ON FIELD VALIDATION:
"validators": [
{
"field": "$..journals[?(@.fromAccount.accountId='123456')].id",
"value": "${$.send_valid_unique_identifiers.response.body.rawJournals.[0].id}"
}
]
AssertionErrors:
Assertion jsonpath '$' with actual value '["<generated UUID>"]' did not match the expected value '${$.send_valid_unique_identifiers.response.body.rawJournals.[0].id}'
Already did understand what is needed:
- he has a step in which he posts some data with a list to a backend rest server
- this backend application will put this received datalist as separate records on the kafka topic
- next step of the test will get the data records from the kafka topic
The assertion to check here, that the data that was put on the backend rest server, is the same as received from the kafka topic.
That is why it was tried to match the outcome of the kafka topic with the request to the backend rest server and because order may be different for the records, the jsonpath matcher in the "field" was used to get the correct record ($..journals[?(@.id==${$.send_valid_unique_identifiers.response.body.rawJournals.[0].id})] which means get the record with an id that matches with the original request list the first one's id) and then perform a (second imo unnecessary because if the jsonpath matching above is not getting a result, it doesn't exist and if is matches, then we know the outcome too) check with the value (${$.send_valid_unique_identifiers.response.body.rawJournals.[0].id}. which means the value of the original request list the first one's id)..
Right?
@RDBreed - thanks for the response. The assertion is to check on the value of the ID field in the consumed JSON record. This ID is generated by RANDOM.UUID in a previous test case. I am able to get the desired data from kafka, but cannot validate/assert/match on the value of the ID field. Basically, we post 2 ID values and we want to validate/assert/match that those 2 ID values are present on the kafka topic.
Sorry for my lack of knowledge/understanding here, but this is taking way more explaining than I anticipated. It seems like my use case cannot is not supported out-of-the box. If that is the case, please feel free to close this. I will take care of it. Thanks!
hi @kt09 ; @authorjapps ;
I reopen this thread because I actually have the same limitations for my scenario.
I have a use case where
- i produce a kafka message into a first topic with a generated
correlation_idin headers - my app process the message and produce a response in another topic with the same
correlation_id
Here is my sample scenario:
{
"scenarioName": "Issue scenario",
"steps": [
{
"name": "produce_step",
"url": "kafka-topic:${SYSTEM.ENV:FIRST-TOPIC}",
"operation": "produce",
"request": {
"recordType": "JSON",
"records": [
{
"key": null,
"headers": {
"CORRELATION_ID": "${RANDOM.UUID}"
},
"value": {
"test" : "test"
}
}
]
},
"assertions": {
"status": "Ok",
"recordMetadata": {
"topicPartition": {
"topic": "${SYSTEM.ENV:FIRST-TOPIC}"
}
}
}
},
{
"name": "consume the response ",
"url": "kafka-topic:${SYSTEM.ENV:SECOND-TOPIC}",
"operation": "consume",
"request": {
"consumerLocalConfigs": {
"recordType": "JSON"
}
},
"validators": [
{
"field": "$.records[?(@.headers.CORRELATION_ID == '${$.produce_step.request.records[0].headers.CORRELATION_ID}')]",
"value": [
{
"value": {
"errors.SIZE": "$EQ.0",
"details['2'].total": "$GT.0",
"details['2_STORE'].total": "$GT.0"
}
}
]
}
]
}
]
}
Actually, the json path expression "field": "$.records[?(@.headers.CORRELATION_ID == '${$.produce_step.request.records[0].headers.CORRELATION_ID}')]", is not transformed by zerocode during the runtime like this is do with the assertions.
Furthermore, using the Zerocode expressions in this query not working too (like "field": "$.records[?(@.headers.CORRELATION_ID == 'RANDOM.UUID')]",)
I investigate directly in the zerocode core codebase and this is actually not supported for the validator way:
- The assertions are transformed on-the-fly during the
ZeroCodeMultiStepsScenarioRunnerImpl#executeRetry
// ---------------------------------
// Handle assertion section -START
// ---------------------------------
String resolvedAssertionJson = zeroCodeAssertionsProcessor.resolveStringJson(
thisStep.getAssertions().toString(),
scenarioExecutionState.getResolvedScenarioState()
);
// -----------------
// logging assertion
// -----------------
List<FieldAssertionMatcher> failureResults = compareStepResults(thisStep, executionResult, resolvedAssertionJson);
- but not the step validators (
ZeroCodeMultiStepsScenarioRunnerImpl#compareStepResults)
List<FieldAssertionMatcher> failureResults = new ArrayList<>();
// --------------------
// Validators (pyrest)
// --------------------
if (ofNullable(thisStep.getValidators()).orElse(null) != null) {
failureResults = validator.validateFlat(thisStep, actualResult);
}
- The
ZeroCodeValidatorImpl#validateFlatdirectly get the validator field and assert on it
@Override
public List<FieldAssertionMatcher> validateFlat(Step thisStep, String actualResult) {
LOGGER.info("Comparing results via flat validators");
List<FieldAssertionMatcher> failureResults = new ArrayList<>();
List<Validator> validators = thisStep.getValidators();
for (Validator validator : validators) {
String josnPath = validator.getField();
JsonNode expectedValue = validator.getValue();
Object actualValue = JsonPath.read(actualResult, josnPath);
List<JsonAsserter> asserters = zeroCodeAssertionsProcessor.createJsonAsserters(expectedValue.toString());
failureResults.addAll(zeroCodeAssertionsProcessor.assertAllAndReturnFailed(asserters, actualValue.toString()));
}
return failureResults;
}
So there is no chance that it will work!
Now, if we use the same zeroCodeAssertionsProcessor.resolveStringJson method on validators before invoking the asserts ; it works like a charm.
I create a DRAFT PR (https://github.com/authorjapps/zerocode/pull/538) to illustrate the change in the codebase ; i can finalize this PR with the related tests in order to release this fix when you can :)
I have a more basic question: is zerocode framework still being maintained/developed and supported? I noticed that the website is not there anymore
Sent from my iPhone
On Oct 5, 2022, at 9:31 AM, Ludovic Dussart @.***> wrote:
hi @kt09 ; @authorjapps ;
I reopen this thread because I actually have the same limitations for my scenario.
I have a use case where
i produce a kafka message into a first topic with a generated correlation_id in headers my app process the message and produce a response in another topic with the same correlation_id Here is my sample scenario:
{ "scenarioName": "Issue scenario", "steps": [ { "name": "produce_step", "url": "kafka-topic:${SYSTEM.ENV:FIRST-TOPIC}", "operation": "produce", "request": { "recordType": "JSON", "records": [ { "key": null, "headers": { "CORRELATION_ID": "${RANDOM.UUID}" }, "value": { "test" : "test" } } ] }, "assertions": { "status": "Ok", "recordMetadata": { "topicPartition": { "topic": "${SYSTEM.ENV:FIRST-TOPIC}" } } } }, { "name": "consume the response ", "url": "kafka-topic:${SYSTEM.ENV:SECOND-TOPIC}", "operation": "consume", "request": { "consumerLocalConfigs": { "recordType": "JSON" } }, "validators": [ { "field": @.***_ID == '${$.produce_step.request.records[0].headers.CORRELATION_ID}')]", "value": [ { "value": { "errors.SIZE": "$EQ.0", "details['2'].total": "$GT.0", "details['2_STORE'].total": "$GT.0" } } ] } ] } ] }
Actually, the json path expression "field": @._ID == '${$.produce_step.request.records[0].headers.CORRELATION_ID}')]", is not transformed by zerocode during the runtime like this is do with the assertions. Furthermore, using the Zerocode expressions in this query not working too (like "field": @._ID == 'RANDOM.UUID')]",)
I investigate directly in the zerocode core codebase and this is actually not supported for the validator way:
The assertions are transformed on-the-fly during the ZeroCodeMultiStepsScenarioRunnerImpl#executeRetry // --------------------------------- // Handle assertion section -START // --------------------------------- String resolvedAssertionJson = zeroCodeAssertionsProcessor.resolveStringJson( thisStep.getAssertions().toString(), scenarioExecutionState.getResolvedScenarioState() );
// ----------------- // logging assertion // ----------------- List<FieldAssertionMatcher> failureResults = compareStepResults(thisStep, executionResult, resolvedAssertionJson);but not the step validators (ZeroCodeMultiStepsScenarioRunnerImpl#compareStepResults) List<FieldAssertionMatcher> failureResults = new ArrayList<>();
// -------------------- // Validators (pyrest) // -------------------- if (ofNullable(thisStep.getValidators()).orElse(null) != null) { failureResults = validator.validateFlat(thisStep, actualResult); }The ZeroCodeValidatorImpl#validateFlat directly get the validator field and assert on it @Override public List<FieldAssertionMatcher> validateFlat(Step thisStep, String actualResult) { LOGGER.info("Comparing results via flat validators");
List<FieldAssertionMatcher> failureResults = new ArrayList<>(); List<Validator> validators = thisStep.getValidators(); for (Validator validator : validators) { String josnPath = validator.getField(); JsonNode expectedValue = validator.getValue(); Object actualValue = JsonPath.read(actualResult, josnPath); List<JsonAsserter> asserters = zeroCodeAssertionsProcessor.createJsonAsserters(expectedValue.toString()); failureResults.addAll(zeroCodeAssertionsProcessor.assertAllAndReturnFailed(asserters, actualValue.toString())); } return failureResults; }So there is no chance that it will work!
Now, if we use the same zeroCodeAssertionsProcessor.resolveStringJson method on validators before invoking the asserts ; it works like a charm.
I create a DRAFT PR (#538) to illustrate the change in the codebase ; i can finalize this PR with the related tests in order to release this fix when you can :)
— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.
Thank you, great job. Will review and merge soon.
Final bit and help needed when you get chance: Would you mind adding a basic helloword-type example for this please into the Docs?
Doc link:
This will be very useful for most users.