palantir
Unexpected null body when deserializing empty optional serialized by conjure-undertow
What happened?
com.palantir.logsafe.exceptions.SafeNullPointerException: Unexpected null body is thrown when calling a conjure-undertow server endpoint via conjure-retrofit.
Repro steps:
- conjure an endpoint with return type any
- have the conjure-undertow server return
Optional.empty() - use the conjure-retrofit client to call the endpoint
What did you want to happen?
The conjure-retrofit client should understand the conjure-undertow server when an Optional.empty() is serialized and deserialized.
This endpoint serves both old and new summaries. The old summaries could return raw types like optional but the new summaries return a conjure-defined struct or union type that wraps the primitive types.
Changing the endpoint to return optional<any> is not desirable because our new summaries will never return Optional.empty() and would require an API break.
an empty optional is surely within the domain of any. It seems reasonably that it might not serialize as a 204 and instead as null or whatever else - as long as, knowing to expect an optional<X> in this particular case, the client can deserialize that object as an optional.
FYI we had to go back to conjure-jersey over this as it proved to be a bigger break than we thought
Can I double check what the signature of your retrofit client was? I remember some tricky inconsistencies in null coercion a while ago.
Was it a Call<Object>?
I think the issue was conjure-undertow serializing json null for returns: any when the returned Object is an empty optional. The spec isn't really clear about what we expect here. optional<any> would allow this to work properly without a real wire break, but it would change bindings that may be used by existing consumers, blocking library upgrades.
Honestly the whole thing feels super ambiguous because you can happily define Foo as an alias of optional<T>, so then in a nice typed world the client could always know that 204 should map to Foo.of(Optional.empty()) but when the client doesn't have a useful signature (i.e. any), it's impossible to differentiate between the server returning Foo.of(Optional.empty()), Bar.of(Optional.empty()) and plain Optional.empty().
Isn't it always up to the code calling a conjure client to know what the exact type of an any endpoint is (presumably based on the query they sent) and deserialise properly? If I get an Object foo = endpoint.callAnyMethod() on the other side, and I know it should be optional, I think it's fine if foo is null instead of Optional.empty(); the same way that if callAnyMethod returns a conjure blob, I don't think foo would be of that type, and would be a Map<String, Object> instead?