`Any::PackFrom` broken in protobuf 33.0 (regression)

[juergbi]

What version of protobuf and what language are you using? Version:33.0 Language: C++

What operating system (Linux, Windows, ...) and version? Linux 6.17

What runtime / compiler are you using (e.g., python version or gcc version) GCC 15.2

What did you do? Test code to reproduce the issue:

#include <iostream>

#include <google/protobuf/any.pb.h>
#include <google/protobuf/wrappers.pb.h>

int main(int argc, char **argv)
{
    google::protobuf::StringValue stringValue1;
    stringValue1.set_value("hello, world");

    google::protobuf::Any any1;
    any1.PackFrom(stringValue1);

    // The following statement works around the issue:
    // any1.mutable_value();

    const auto anySerialized = any1.SerializeAsString();

    google::protobuf::Any any2;
    if (!any2.ParseFromString(anySerialized)) {
        return 2;
    }

    google::protobuf::StringValue stringValue2;
    any2.UnpackTo(&stringValue2);

    std::cout << "Unpacked string: " << stringValue2.value() << std::endl;

    return stringValue2.value() == "hello, world" ? 0 : 1;
}

Compile e.g. with g++ -o anytest $(pkg-config --cflags --libs protobuf) anytest.cc

What did you expect to see Unpacked string: hello, world and exit code 0.

What did you see instead? Unpacked string: and exit code 1.

This is a regression in 33.0, which was introduced in https://github.com/protocolbuffers/protobuf/pull/22956 in combination with the auto-generated commit be95d7788c7305bf2e25eae2f3069929e31f4636. @ClaytonKnittel, please take a look.

As I understand it, the issue is that Any::PackFrom uses _internal_mutable_value(), which used to call SetHasBit() but no longer does with the change mentioned above. This results in serialization skipping the Any.value field even though it is set.

This breaks https://gitlab.com/BuildGrid/buildbox/buildbox/.

[ClaytonKnittel]

Ahh this is interesting, this case was not exercised in our tests internally because we represent Any messages differently. I'll send out a fix soon.

[Chekov2k]

Just to say I am seeing the same issue, where a Protobuf Message stored in an Any field cannot be unpacked properly anymore. For me SerializeAsString results in an empty payload, whereas DebugString returns the correct content

[ClaytonKnittel]

This should be fixed now, let me know if you run into any more issues!

[juergbi]

Thanks for the quick fix! It seems src/google/protobuf/any.pb.h hasn't been regenerated yet, though. Will this happen automatically or do you have to trigger this internally? I assume this will also be cherry-picked to the 33.x branch to get this fix into 33.1.

[ClaytonKnittel]

They are periodically refreshed by a bot, this commit from a few hours ago grabbed these changes.

[raulcd]

Hi! thanks for the quick fix! Do you know if this is going to be cherry-picked to the 33.x branch? This is causing several failures for us on apache/arrow and would love to understand what is the plan for this to be released so we can plan accordingly. Thanks!

[ClaytonKnittel]

@raulcd I'll hopefully have an update on this tomorrow! This will be fixed in 33.1, I'll update this thread when we decide when that will be released.

[ClaytonKnittel]

Version 33.1 will be released tomorrow! That will include the fix to this issue.

[Chekov2k]

Awesome, thank you :)