`to_h` ignores the default value of enum

[ganmacs]

What version of protobuf and what language are you using?

Version: commit b8d356701 Language: Ruby

What operating system (Linux, Windows, ...) and version?

macOS Sonoma 14.6.1

What runtime / compiler are you using (e.g., python version or gcc version)

$ ruby -v                                                                                                                                         
ruby 3.1.2p20 (2022-04-12 revision 4491bb740a) [arm64-darwin21]
$ bundler -v                 
Bundler version 2.4.22

What did you do?

I added some test cases to basic.rb. and run rake test command.

https://github.com/ganmacs/protobuf/commit/4facc4872bb24400f0b8fabecc8d7bf0dcc70e4a

m = MapMessage.new(
  :map_string_int32 => {"a" => 1, "b" => 2},
  :map_string_msg => {"a" => TestMessage2.new(:foo => 1),
                      "b" => TestMessage2.new(:foo => 2)},
  :map_string_enum => {"b" => :Default}
)
expected_result = {
  :map_string_int32 => {"a" => 1, "b" => 2},
  :map_string_msg => {"a" => {:foo => 1}, "b" => {:foo => 2}},
  :map_string_enum => {"b" => :Default}
}
assert_equal expected_result, m.to_h # pass
m = Enumer.new(
  :optional_enum => :Default,
  :repeated_enum => [:A, :C]
)
expected_result = {
  :optional_enum => :Default,
  :repeated_enum => [:A, :C]
}
assert_equal expected_result,  m.to_h # error

What did you expect to see

Enumer.new(:optional_enum => :Default, :repeated_enum => [:A, :C]).to_h returns { :optional_enum => :Default, :repeated_enum => [:A, :C] }

What did you see instead?

Enumer.new(:optional_enum => :Default, :repeated_enum => [:A, :C]).to_h returns { :repeated_enum => [:A, :C] }

$ rake test

...

Failure: test_to_h(BasicTest::MessageContainerTest)
/Users/ganmacs/src/github.com/protocolbuffers/protobuf/ruby/tests/basic.rb:522:in `test_to_h'
     519:         :optional_enum => :Default,
     520:         :repeated_enum => [:A, :C]
     521:       }
  => 522:       assert_equal expected_result,  m.to_h # error
     523:     end
     524:
     525:
<{:optional_enum=>:Default, :repeated_enum=>[:A, :C]}> expected but was
<{:repeated_enum=>[:A, :C]}>

diff:
? {:optional_enum=>:Default, :repeated_enum=>[:A, :C]}

Anything else we should know about your project / environment

The protobuf definitions are

• TestEnum https://github.com/protocolbuffers/protobuf/blob/b8d356701a788951d92a1b4f04881fb539ca62d7/ruby/tests/basic_test.proto#L103-L109
• MapMessage https://github.com/protocolbuffers/protobuf/blob/b8d356701a788951d92a1b4f04881fb539ca62d7/ruby/tests/basic_test.proto#L138-L142
• Enumer https://github.com/protocolbuffers/protobuf/blob/b8d356701a788951d92a1b4f04881fb539ca62d7/ruby/tests/basic_test.proto#L234-L242

[github-actions[bot]]

We triage inactive PRs and issues in order to make it easier to find active work. If this issue should remain active or becomes active again, please add a comment.

This issue is labeled inactive because the last activity was over 90 days ago. This issue will be closed and archived after 14 additional days without activity.

[ganmacs]

Hi @JasonLunn. Any updates?

[JasonLunn]

@ganmacs - could you turn your test into a PR that I can run on our CI tests? That would help expedite an investigation.

[amano-takehiro]

Hi, I'm not directly involved in this issue, but just wanted to note:

This might be related to #6167, which changed the behavior of to_h to omit unset fields, including enum fields with default values.

In this case, I believe optional_enum in Enumer has a default value of :Default, so it may have been omitted from the hash output for that reason.

Apologies if this is off-base — I'm still learning, but thought this might help!

[github-actions[bot]]

We triage inactive PRs and issues in order to make it easier to find active work. If this issue should remain active or becomes active again, please add a comment.

This issue is labeled inactive because the last activity was over 90 days ago. This issue will be closed and archived after 14 additional days without activity.

[shouichi]

This issue prevents us from upgrading the ruby binding (v3 to v4). Is this behavioral change intentional?

[github-actions[bot]]

We triage inactive PRs and issues in order to make it easier to find active work. If this issue should remain active or becomes active again, please add a comment.

This issue is labeled inactive because the last activity was over 90 days ago. This issue will be closed and archived after 14 additional days without activity.

[shouichi]

Please don't close this issue.