Protocol Buffers Serialization in Lucee 7

edit Introduced: 7.0

Protocol Buffers Serialization in Lucee 7

Lucee 7's Maven integration makes it easy to work with Protocol Buffers data using Google's official Java library, without needing an extension.

This is an alternative to Adobe ColdFusion's serializeProtoBuf() and deserializeProtoBuf() functions.

Maven Dependency

First, find the latest version at mvnrepository.com. Currently version 4.36.2 is available. You need protobuf-java and protobuf-java-util (for JSON conversion):

implementation("com.google.protobuf:protobuf-java:4.36.2")
implementation("com.google.protobuf:protobuf-java-util:4.36.2")

Simple Example

Protobuf messages are normally defined in a .proto file and compiled with protoc. To avoid needing protoc, this example defines the message type in code, the equivalent of:

syntax = "proto3";
message Customer {
    string firstName = 1;
    string lastName = 2;
    int32 age = 3;
}

The inline component builds the message descriptor, then converts between structs and protobuf bytes using DynamicMessage and JsonFormat:

protobuf = new component javasettings='{"maven":["com.google.protobuf:protobuf-java:4.36.2","com.google.protobuf:protobuf-java-util:4.36.2"]}' {
    import com.google.protobuf.DescriptorProtos$FileDescriptorProto;
    import com.google.protobuf.DescriptorProtos$DescriptorProto;
    import com.google.protobuf.DescriptorProtos$FieldDescriptorProto;
    import com.google.protobuf.DescriptorProtos$FieldDescriptorProto$Type;
    import com.google.protobuf.Descriptors$FileDescriptor;
    import com.google.protobuf.DynamicMessage;
    import com.google.protobuf.util.JsonFormat;
    // fields is an ordered struct of field name -> protobuf type, e.g. "TYPE_STRING"
    function defineMessage(required string name, required struct fields) {
        var message = DescriptorProtos$DescriptorProto::newBuilder().setName(arguments.name);
        var number = 1;
        for (var fieldName in arguments.fields) {
            message.addField(
                DescriptorProtos$FieldDescriptorProto::newBuilder()
                    .setName(fieldName)
                    .setNumber(number++)
                    .setType(DescriptorProtos$FieldDescriptorProto$Type::valueOf(arguments.fields[fieldName]))
                    .build()
            );
        }
        var file = DescriptorProtos$FileDescriptorProto::newBuilder()
            .setName(arguments.name & ".proto")
            .setSyntax("proto3")
            .addMessageType(message.build())
            .build();
        return Descriptors$FileDescriptor::buildFrom(file, []).findMessageTypeByName(arguments.name);
    }
    function serialize(required struct data, required descriptor) {
        var builder = DynamicMessage::newBuilder(arguments.descriptor);
        JsonFormat::parser().merge(serializeJSON(arguments.data), builder);
        return builder.build().toByteArray();
    }
    function deserialize(required binary bytes, required descriptor) {
        var message = DynamicMessage::parseFrom(arguments.descriptor, arguments.bytes);
        return deserializeJSON(JsonFormat::printer().print(message));
    }
};
// Usage
customerType = protobuf.defineMessage("Customer", [
    "firstName": "TYPE_STRING",
    "lastName": "TYPE_STRING",
    "age": "TYPE_INT32"
]);
bytes = protobuf.serialize({"firstName": "John", "lastName": "Doe", "age": 30}, customerType);
customer = protobuf.deserialize(bytes, customerType);
dump(customer); // {firstName: "John", lastName: "Doe", age: 30}

Note that nested Java classes such as DescriptorProtos.FileDescriptorProto are imported with a $, e.g. import com.google.protobuf.DescriptorProtos$FileDescriptorProto;, and are then referenced by that name, e.g. DescriptorProtos$FileDescriptorProto::newBuilder().

As with any proto3 message, fields with default values (empty strings, 0, false) are not written, so they are missing from the deserialized struct. Also note that the protobuf JSON format represents 64-bit integers (TYPE_INT64) as strings.

Using protoc Generated Classes

If you already have Java classes generated by protoc for your .proto files, package them in a jar and load that jar (for example via javasettings loadPaths) together with protobuf-java. You can then use the generated classes directly, e.g. Customer::parseFrom(bytes) and Customer::newBuilder(), instead of building the descriptor in code.

That's all you need to start working with Protocol Buffers in Lucee 7! Maven handles downloading the library and all dependencies automatically.

See also