Tarmil
How to upgrade from 0.17.4 for single case unions and skippable fields?
Hello, we were using the 0.17.4 version of the library and would like to upgrade it to the latest version of this library. However, we have a lot of serialized documents stored that can't be deserialized using the same options that we were using in 0.17.4.
Are we missing some configuration to deserialize as we did in 0.17.4? Or is this an instance of a bug?
Here's an example input that we cannot deserialize:
type SingleCaseUnion = SingleCaseUnion of string
type ToDeserialize =
{
SingleCaseUnion : SingleCaseUnion
SkippableMember: int option
}
let exampleBody = """
{
"singleCaseUnion": {
"Case": "SingleCaseUnion",
"Fields": [
"some value"
]
}
}"""
Here are the options that we used back then:
module Serializer =
let options =
let opt = JsonSerializerOptions(JsonSerializerDefaults.Web)
let jsonUnionEncodings = JsonUnionEncoding.UnwrapFieldlessTags ||| JsonUnionEncoding.UnwrapOption
opt.Converters.Add(JsonFSharpConverter(jsonUnionEncodings, unionTagCaseInsensitive = true))
opt
let deserialize<'a> (body : string) =
JsonSerializer.Deserialize<'a> (body, options)
I see that SkippableOptionsField was added and is now required for the skippable member. If I'm not mistaken, We can use UnionAdjacentTag to deserialize the single case union.
I also see that we have to switch to the Fluent JsonFSharpOptions to be able to use SkippableOptionsField. So I tried with the following configuration:
module Serializer =
let options =
let opt =
JsonFSharpOptions()
.WithUnwrapOption()
.WithUnionUnwrapFieldlessTags()
.WithUnionTagCaseInsensitive()
.WithAllowNullFields()
.WithSkippableOptionFields()
.WithUnionAdjacentTag()
.ToJsonSerializerOptions()
opt.Converters.Add(JsonFSharpConverter())
opt
let deserialize<'a> (body : string) =
JsonSerializer.Deserialize<'a> (body, options)
However, we get this error:
System.Text.Json.JsonException: Missing field for record type Playground.Types+ToDeserialize: SingleCaseUnion
at [email protected](String x)
at System.Text.Json.Serialization.JsonRecordConverter`1.ReadRestOfObject(Utf8JsonReader& reader, Boolean skipFirstRead)
at System.Text.Json.Serialization.JsonRecordConverter`1.Read(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options)
at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value, Boolean& isPopulatedValue)
at System.Text.Json.Serialization.JsonCollectionConverter`2.OnTryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, TCollection& value)
at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value, Boolean& isPopulatedValue)
at System.Text.Json.Serialization.JsonConverter`1.ReadCore(Utf8JsonReader& reader, JsonSerializerOptions options, ReadStack& state)
at System.Text.Json.Serialization.Metadata.JsonTypeInfo`1.DeserializeAsObject(Utf8JsonReader& reader, ReadStack& state)
at System.Text.Json.JsonSerializer.ReadAsObject(Utf8JsonReader& reader, JsonTypeInfo jsonTypeInfo)
at System.Text.Json.Serialization.JsonListConverter`1.Read(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options)
at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value, Boolean& isPopulatedValue)
at System.Text.Json.Serialization.JsonConverter`1.ReadCore(Utf8JsonReader& reader, JsonSerializerOptions options, ReadStack& state)
at System.Text.Json.JsonSerializer.ReadFromSpan[TValue](ReadOnlySpan`1 utf8Json, JsonTypeInfo`1 jsonTypeInfo, Nullable`1 actualByteCount)
at System.Text.Json.JsonSerializer.ReadFromSpan[TValue](ReadOnlySpan`1 json, JsonTypeInfo`1 jsonTypeInfo)
at System.Text.Json.JsonSerializer.Deserialize[TValue](String json, JsonSerializerOptions options)
at Playground.Serializer.deserialize[a](String body) in C:\Users\AlexGagné\RiderProjects\Playground\Playground\Deserializer.fs:line 36
at Playground.Program.main(String[] _arg1) in C:\Users\AlexGagné\RiderProjects\Playground\Playground\Program.fs:line 72
As if the AdjacentTag was being ignored? Or am I misunderstanding something?
I think it's failing to parse the record, rather than the union. The difference is the JsonSerializerDefaults.Web you used in the old version, which includes case-insensitive property names. You need it in the new version too. It's only usable with the constructor for JsonSerializerOptions, so instead of ToJsonSerializerOptions, you can use AddToJsonSerializerOptions:
let options =
let opt = JsonSerializerOptions(JsonSerializerDefaults.Web)
JsonFSharpOptions()
(* all your options here... *)
.AddToJsonSerializerOptions(opt)
opt
You also don't need the extra opt.Converters.Add(JsonFSharpConverter()).
Hello, thanks for your quick response. I think you are correct that the web defaults were missing. However, I still can't seem to deserialize my example.
type SingleCaseUnion = SingleCaseUnion of string
type ToDeserialize =
{
SingleCaseUnion : SingleCaseUnion
SkippableMember: int option
}
let exampleBody = """{"singleCaseUnion":{"Case":"SingleCaseUnion","Fields":["some value"]}}"""
let options =
let opt = JsonSerializerOptions(JsonSerializerDefaults.Web)
JsonFSharpOptions()
.WithUnwrapOption()
.WithUnionUnwrapFieldlessTags()
.WithUnionTagCaseInsensitive()
.WithAllowNullFields()
.WithSkippableOptionFields(SkippableOptionFields.Always, true)
.WithUnionAdjacentTag()
.AddToJsonSerializerOptions(opt)
opt
let deserialize<'a> (body : string) =
JsonSerializer.Deserialize<'a> (body, options)
I then get this exception:
System.Text.Json.JsonException: The JSON value could not be converted to System.String. Path: $ | LineNumber: 0 | BytePositionInLine: 1.
---> System.InvalidOperationException: Cannot get the value of a token type 'StartObject' as a string.
at System.Text.Json.ThrowHelper.ThrowInvalidOperationException_ExpectedString(JsonTokenType tokenType)
at System.Text.Json.Utf8JsonReader.GetString()
at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value, Boolean& isPopulatedValue)
at System.Text.Json.Serialization.JsonConverter`1.ReadCore(Utf8JsonReader& reader, JsonSerializerOptions options, ReadStack& state)
--- End of inner exception stack trace ---
at System.Text.Json.ThrowHelper.ReThrowWithPath(ReadStack& state, Utf8JsonReader& reader, Exception ex)
at System.Text.Json.Serialization.JsonConverter`1.ReadCore(Utf8JsonReader& reader, JsonSerializerOptions options, ReadStack& state)
at System.Text.Json.JsonSerializer.Read[TValue](Utf8JsonReader& reader, JsonTypeInfo`1 jsonTypeInfo)
at System.Text.Json.Serialization.JsonUnwrappedUnionConverter`2.Read(Utf8JsonReader& reader, Type _typeToConvert, JsonSerializerOptions options)
at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value, Boolean& isPopulatedValue)
at System.Text.Json.Serialization.JsonConverter`1.ReadCore(Utf8JsonReader& reader, JsonSerializerOptions options, ReadStack& state)
at System.Text.Json.Serialization.Metadata.JsonTypeInfo`1.DeserializeAsObject(Utf8JsonReader& reader, ReadStack& state)
at System.Text.Json.JsonSerializer.ReadAsObject(Utf8JsonReader& reader, JsonTypeInfo jsonTypeInfo)
at System.Text.Json.Serialization.Helpers.FieldHelper.Deserialize(Utf8JsonReader& reader)
at System.Text.Json.Serialization.JsonRecordConverter`1.ReadRestOfObject(Utf8JsonReader& reader, Boolean skipFirstRead)
at System.Text.Json.Serialization.JsonRecordConverter`1.Read(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options)
at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value, Boolean& isPopulatedValue)
at System.Text.Json.Serialization.JsonConverter`1.ReadCore(Utf8JsonReader& reader, JsonSerializerOptions options, ReadStack& state)
at System.Text.Json.JsonSerializer.ReadFromSpan[TValue](ReadOnlySpan`1 utf8Json, JsonTypeInfo`1 jsonTypeInfo, Nullable`1 actualByteCount)
at System.Text.Json.JsonSerializer.ReadFromSpan[TValue](ReadOnlySpan`1 json, JsonTypeInfo`1 jsonTypeInfo)
at System.Text.Json.JsonSerializer.Deserialize[TValue](String json, JsonSerializerOptions options)
at Playground.Serializer.deserialize[a](String body) in C:\Users\AlexGagné\RiderProjects\Playground\Playground\Deserializer.fs:line 35
at Playground.Program.main(String[] _arg1) in C:\Users\AlexGagné\RiderProjects\Playground\Playground\Program.fs:line 64
Hello @Tarmil
Could you please confirm if this is a bug from the library or if I'm doing something wrong again?
I've tried different combination of options, removing the skippable member, trying different options, but couldn't manage to deserialize the body. I've also tried renaming all of my properties to UpperCamelCase and removing the Web defaults to see if it made a difference, but I got the exact same error (The The JSON value could not be converted to System.String error).
I cannot find a combination that deserializes my example body properly.
Serializing using my options, I see this being serialized:
{"singleCaseUnion":"some value"}
I now see that the Adjacent Tag says that if there are no fields, then the fields property is excluded. I'm guessing this might be a breaking change that was done? If that is the case, is there a plan to re-enable this behaviour in the future? Or would we have to migrate all of our serialized data to follow this new rule?
Hello @Tarmil
Could you please confirm if excluding the "fields" property when there are no fields is an intended breaking change (I couldn't find it in the release notes)? Or maybe including the property if there were no fields was a bug that was fixed? In either case, do you think there should be an option to re-enable it?
I wouldn't mind implementing this myself and contribute to the library, I'd just like a confirmation from you that it would be a welcome change.
Hello @Tarmil Have you had time to take a look at my request? I'm willing to implement it, just wanted to make sure it's actually something that you would agree with.
Hi, sorry for the delay. What you're seeing here is not the result of UnwrapFieldlessTags, since this union does have a field, of type string. Rather, it's the result of UnwrapSingleCaseUnions, which is enabled by default. You can disable it with:
.WithUnionUnwrapSingleCaseUnions(false)
The idea behind this option is that when a union has a single case with a single field, it usually is a simple typed wrapper that people want to treat as its "underlying" type, ie the type of the field.