aws
DynamoDbTable.updateItem clears data from table, then only sets non-null field values
Describe the bug
When executing DynamoDbTable.updateItem for a few fields, the execution clear the existing data and only sets new field values from input object.
My understanding, for an existing record, when a Java object populated with primary key and fields to be changed, this operation should affect only the fields to be changed for the primary key record.
Curious to know, how is this updateItem supposed to work?
Expected Behavior
- The
updateItemshould only change the fields which are present in the input - This begs a question what should happen to the default values which are assigned as
null. There is a field level annotation for ignoring null (I tried, it didn't work for me either). Also, need a class level annotation for ignoring all null fields.
Current Behavior
Overwriting existing contents of a record for updateItem operation.
I am using spring-cloud-aws-starter-dynamodb using DynamoDbTemplate AWS Spring DynamoDb, however, it fails when using below code snippet
/*
Existing record:
Car(vin=1234567890, color=white, year=2020, model=Ford Mustang)
*/
final Car car = Car
.builder()
.vin("1234567890")
.color("black") // existing record has white color
.build();
Car updatedCar = dynamoDbEnhancedClient.table("Car", TableSchema.fromBean(Car.class))
.updateItem(car);
assertEquals("black", updatedCar.getColor()); // PASS
assertEquals("2020", updatedCar.getYear()); // FAIL
assertEquals("Ford Mustang", updatedCar.getModel()); // FAIL
Reproduction Steps
- Create a
Cartable with fields vin (primary key), year, model, color - Create Java class named as Car
- Add class level annotation
@DynamoDbBean - Add
@DynamoDbPartitionKeyannotation to a getter method of vin - Add a new record in Car table as mentioned in Current Behavior
- Create an instance of
dynamoDbEnhancedClient - Execute code snippet as mentioned in Current Behavior
Possible Solution
- updateItem should only affect the values in the input
- This begs a question what should happen to the default values which are assigned as
null. There is a field level annotation for ignoring null (it didn't work for me either). Also, need a class level annotation for ignoring all null fields.
Additional Information/Context
- Springboot 3.1.5
- Java 21
- AWS SDK 2.20.153
- spring-cloud-aws-starter-dynamodb 3.0.2
AWS Java SDK version used
2.20.153
JDK version used
21
Operating System and version
Windows 11 Pro x64
Hi @ipsi-apant apologies for the long silence.
Does it work if you set @DynamoDbUpdateBehavior to WRITE_IF_NOT_EXISTS?
https://docs.aws.amazon.com/sdk-for-java/latest/developer-guide/ddb-en-client-adv-features-upd-behavior.html
Hi @debora-ito , thank you for reviewing and your time.
I will give it a try for this option. This is a field level annotation, would like to learn more on, what are the challenges for not having a class level annotation (for all fields)?
I am wondering, for the above example, if this annotation added to the color property, would it still be able to update to 'black' color ?
My understanding from this java doc, if a new value for color is provided, it will be a 'a new record is being written` scenario, right ?
Write the new value if there is no existing value in the persisted record or a new record is being written, otherwise leave the existing value.
Hi @debora-ito, this is not working as expected.
AWS SDK version 2.24.1
It fails during update car operation. This line car.getModel().equals("Ford Mustang"). As per stacktrace it has null.
During debug I have captured screenshots after inserting a record and after update the record by vin number. Attached for review here.
It did update color, however removed model and year.
Also tried using @DynamoDbIgnoreNulls. The result is same.
https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/enhanced/dynamodb/mapper/annotations/DynamoDbIgnoreNulls.html
Car (bean)
@Data
@NoArgsConstructor
@AllArgsConstructor
@Builder
@DynamoDbBean
@JsonIgnoreProperties(ignoreUnknown = true)
@JsonInclude(JsonInclude.Include.NON_NULL)
public class Car {
private String vin;
private String model;
private String color;
private String year;
@DynamoDbPartitionKey
public String getVin() {
return vin;
}
@DynamoDbUpdateBehavior(UpdateBehavior.WRITE_IF_NOT_EXISTS)
public String getModel() {
return model;
}
@DynamoDbUpdateBehavior(UpdateBehavior.WRITE_IF_NOT_EXISTS)
public String getColor() {
return color;
}
@DynamoDbUpdateBehavior(UpdateBehavior.WRITE_IF_NOT_EXISTS)
public String getYear() {
return year;
}
}
Unit test (We are using Springboot Webflux)
@Test
void testCarWithoutUpdateBehaviour() {
// create new
Mono<Car> saved = save(VIN_10, "Ford Mustang", "White", "2020");
StepVerifier.create(saved)
.expectNextMatches(car -> car.getVin()
.equals(VIN_10)
&& car.getModel()
.equals("Ford Mustang")
&& car.getColor()
.equals("White")
&& car.getYear()
.equals("2020")
)
.verifyComplete();
// get car
Mono<Car> gotCar = get(VIN_10);
StepVerifier.create(gotCar)
.expectNextMatches(car -> car.getVin()
.equals(VIN_10)
&& car.getModel()
.equals("Ford Mustang")
&& car.getColor()
.equals("White")
&& car.getYear()
.equals("2020")
)
.verifyComplete();
// update car
Mono<Car> updatedCar = update(VIN_10, "Black");
StepVerifier.create(updatedCar)
.expectNextMatches(car -> car.getVin()
.equals(VIN_10)
&& car.getModel()
.equals("Ford Mustang")
&& car.getColor()
.equals("Black")
&& car.getYear()
.equals("2020")
)
.verifyComplete();
}
private Mono<Car> get(String vin) {
return carService.get(vin);
}
private Mono<Car> save(String vin, String model, String color, String year) {
return carService.save(Car.builder()
.vin(vin)
.model(model)
.color(color)
.year(year)
.build());
}
private Mono<Car> update(String vin, String color) {
return carService.update(Car.builder()
.vin(vin)
.color(color)
.build());
}
Stacktrace
2024-03-25T10:59:59.058+11:00 INFO 55272 --- [token-manager] [ Test worker] CarService : Saving car: Car(vin=VIN10, model=Ford Mustang, color=White, year=2020)
2024-03-25T10:59:59.072+11:00 DEBUG 55272 --- [token-manager] [ Test worker] AnnotationBasedDynamoDbTableNameResolver : Class: Car, DynamoDb table name: Car
2024-03-25T10:59:59.189+11:00 INFO 55272 --- [token-manager] [ Test worker] CarService : Getting car with vin: VIN10
2024-03-25T10:59:59.190+11:00 DEBUG 55272 --- [token-manager] [ Test worker] AnnotationBasedDynamoDbTableNameResolver : Class: Car, DynamoDb table name: Car
2024-03-25T11:12:32.642+11:00 INFO 55272 --- [token-manager] [ Test worker] CarService : Updating car: Car(vin=VIN10, model=null, color=Black, year=null)
2024-03-25T11:12:34.074+11:00 DEBUG 55272 --- [token-manager] [ Test worker] AnnotationBasedDynamoDbTableNameResolver : Class: Car, DynamoDb table name: Car
java.lang.AssertionError: expectation failed (failed running expectation on signal [onNext(Car(vin=VIN10, model=null, color=Black, year=null))] with [java.lang.NullPointerException]:
Cannot invoke "String.equals(Object)" because the return value of "Car.getModel()" is null)
at reactor.test.MessageFormatter.assertionError(MessageFormatter.java:115)
at reactor.test.MessageFormatter.failPrefix(MessageFormatter.java:104)
at reactor.test.MessageFormatter.fail(MessageFormatter.java:73)
at reactor.test.MessageFormatter.failOptional(MessageFormatter.java:88)
at reactor.test.DefaultStepVerifierBuilder$DefaultVerifySubscriber.onExpectation(DefaultStepVerifierBuilder.java:1511)
at reactor.test.DefaultStepVerifierBuilder$DefaultVerifySubscriber.onNext(DefaultStepVerifierBuilder.java:1146)
at reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onNext(FluxOnAssembly.java:539)
at reactor.core.publisher.MonoPeekTerminal$MonoTerminalPeekSubscriber.onNext(MonoPeekTerminal.java:180)
at reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onNext(FluxOnAssembly.java:539)
at reactor.core.publisher.MonoCallable$MonoCallableSubscription.request(MonoCallable.java:156)
at reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.request(FluxOnAssembly.java:649)
at reactor.core.publisher.MonoPeekTerminal$MonoTerminalPeekSubscriber.request(MonoPeekTerminal.java:139)
at reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.request(FluxOnAssembly.java:649)
at reactor.test.DefaultStepVerifierBuilder$DefaultVerifySubscriber.onSubscribe(DefaultStepVerifierBuilder.java:1161)
at reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onSubscribe(FluxOnAssembly.java:633)
at reactor.core.publisher.MonoPeekTerminal$MonoTerminalPeekSubscriber.onSubscribe(MonoPeekTerminal.java:152)
at reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onSubscribe(FluxOnAssembly.java:633)
at reactor.core.publisher.MonoCallable.subscribe(MonoCallable.java:48)
at reactor.core.publisher.Mono.subscribe(Mono.java:4512)
at reactor.test.DefaultStepVerifierBuilder$DefaultStepVerifier.toVerifierAndSubscribe(DefaultStepVerifierBuilder.java:891)
at reactor.test.DefaultStepVerifierBuilder$DefaultStepVerifier.verify(DefaultStepVerifierBuilder.java:831)
at reactor.test.DefaultStepVerifierBuilder$DefaultStepVerifier.verify(DefaultStepVerifierBuilder.java:823)
at reactor.test.DefaultStepVerifierBuilder.verifyComplete(DefaultStepVerifierBuilder.java:690)
at CarTests.testCarWithoutUpdateBehaviour(CarTests.java:79)
at java.base/java.lang.reflect.Method.invoke(Method.java:580)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)
Suppressed: java.lang.NullPointerException: Cannot invoke "String.equals(Object)" because the return value of "Car.getModel()" is null
at CarTests.lambda$testCarWithoutUpdateBehaviour$3(CarTests.java:73)
at reactor.test.DefaultStepVerifierBuilder.lambda$expectNextMatches$11(DefaultStepVerifierBuilder.java:558)
at reactor.test.DefaultStepVerifierBuilder$SignalEvent.test(DefaultStepVerifierBuilder.java:2289)
at reactor.test.DefaultStepVerifierBuilder$DefaultVerifySubscriber.onSignal(DefaultStepVerifierBuilder.java:1529)
at reactor.test.DefaultStepVerifierBuilder$DefaultVerifySubscriber.onExpectation(DefaultStepVerifierBuilder.java:1477)
... 22 more