StackOverFlowError at toConnectSchema

[TobiasKrok]

Hi!

I'm getting this error when trying to start my Kafka connector

      [2021-06-11 14:29:10,173] ERROR Failed to start task mqtt-source-0 (org.apache.kafka.connect.runtime.Worker:560)
      java.lang.StackOverflowError
              at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
              at java.base/java.lang.reflect.Method.invoke(Method.java:566)
              at com.google.protobuf.GeneratedMessageV3.invokeOrDie(GeneratedMessageV3.java:1856)
              at com.google.protobuf.GeneratedMessageV3.access$1100(GeneratedMessageV3.java:79)
              at com.google.protobuf.GeneratedMessageV3$FieldAccessorTable$SingularFieldAccessor$ReflectionInvoker.has(GeneratedMessageV3.java:2221)
              at com.google.protobuf.GeneratedMessageV3$FieldAccessorTable$SingularFieldAccessor.has(GeneratedMessageV3.java:2329)
              at com.google.protobuf.GeneratedMessageV3.hasField(GeneratedMessageV3.java:268)
              at com.google.protobuf.GeneratedMessageV3$ExtendableMessage.hasField(GeneratedMessageV3.java:1258)
              at com.google.protobuf.GeneratedMessageV3.getAllFieldsMutable(GeneratedMessageV3.java:191)
              at com.google.protobuf.GeneratedMessageV3.access$800(GeneratedMessageV3.java:79)
              at com.google.protobuf.GeneratedMessageV3$ExtendableMessage.getAllFields(GeneratedMessageV3.java:1239)
              at com.blueapron.connect.protobuf.ProtobufData.getConnectFieldName(ProtobufData.java:78)
              at com.blueapron.connect.protobuf.ProtobufData.toConnectSchema(ProtobufData.java:216)
              at com.blueapron.connect.protobuf.ProtobufData.toConnectSchema(ProtobufData.java:217)
              at com.blueapron.connect.protobuf.ProtobufData.toConnectSchema(ProtobufData.java:217)
              at com.blueapron.connect.protobuf.ProtobufData.toConnectSchema(ProtobufData.java:217)
              at com.blueapron.connect.protobuf.ProtobufData.toConnectSchema(ProtobufData.java:217)
              at com.blueapron.connect.protobuf.ProtobufData.toConnectSchema(ProtobufData.java:217)
              at com.blueapron.connect.protobuf.ProtobufData.toConnectSchema(ProtobufData.java:217)
              at com.blueapron.connect.protobuf.ProtobufData.toConnectSchema(ProtobufData.java:217)
              at com.blueapron.connect.protobuf.ProtobufData.toConnectSchema(ProtobufData.java:217)
              at com.blueapron.connect.protobuf.ProtobufData.toConnectSchema(ProtobufData.java:217)
              at com.blueapron.connect.protobuf.ProtobufData.toConnectSchema(ProtobufData.java:217)
              at com.blueapron.connect.protobuf.ProtobufData.toConnectSchema(ProtobufData.java:217)
              at com.blueapron.connect.protobuf.ProtobufData.toConnectSchema(ProtobufData.java:217)
              at com.blueapron.connect.protobuf.ProtobufData.toConnectSchema(ProtobufData.java:217)
              at com.blueapron.connect.protobuf.ProtobufData.toConnectSchema(ProtobufData.java:217)
              at com.blueapron.connect.protobuf.ProtobufData.toConnectSchema(ProtobufData.java:217)
              at com.blueapron.connect.protobuf.ProtobufData.toConnectSchema(ProtobufData.java:217)
              at com.blueapron.connect.protobuf.ProtobufData.toConnectSchema(ProtobufData.java:217)
              at com.blueapron.connect.protobuf.ProtobufData.toConnectSchema(ProtobufData.java:217)
              at com.blueapron.connect.protobuf.ProtobufData.toConnectSchema(ProtobufData.java:217)
              at com.blueapron.connect.protobuf.ProtobufData.toConnectSchema(ProtobufData.java:217)
              at com.blueapron.connect.protobuf.ProtobufData.toConnectSchema(ProtobufData.java:217)
              at com.blueapron.connect.protobuf.ProtobufData.toConnectSchema(ProtobufData.java:217)
              at com.blueapron.connect.protobuf.ProtobufData.toConnectSchema(ProtobufData.java:217)
              at com.blueapron.connect.protobuf.ProtobufData.toConnectSchema(ProtobufData.java:217)
              at com.blueapron.connect.protobuf.ProtobufData.toConnectSchema(ProtobufData.java:217)
              at com.blueapron.connect.protobuf.ProtobufData.toConnectSchema(ProtobufData.java:217)
              at com.blueapron.connect.protobuf.ProtobufData.toConnectSchema(ProtobufData.java:217)
              at com.blueapron.connect.protobuf.ProtobufData.toConnectSchema(ProtobufData.java:217)
              at com.blueapron.connect.protobuf.ProtobufData.toConnectSchema(ProtobufData.java:217)
              at com.blueapron.connect.protobuf.ProtobufData.toConnectSchema(ProtobufData.java:217)
              at com.blueapron.connect.protobuf.ProtobufData.toConnectSchema(ProtobufData.java:217)
              at com.blueapron.connect.protobuf.ProtobufData.toConnectSchema(ProtobufData.java:217)
              at com.blueapron.connect.protobuf.ProtobufData.toConnectSchema(ProtobufData.java:217)
              at com.blueapron.connect.protobuf.ProtobufData.toConnectSchema(ProtobufData.java:217)

My Proto file looks like this:

syntax = "proto3";

import "google/protobuf/any.proto";

package org.eclipse.tahu.protobuf;

option java_package         = "org.eclipse.tahu.protobuf";
option java_outer_classname = "SparkplugBProto";

enum DataType {
    // Indexes of Data Types

    // Unknown placeholder for future expansion.
    Unknown         = 0;

    // Basic Types
    Int8            = 1;
    Int16           = 2;
    Int32           = 3;
    Int64           = 4;
    UInt8           = 5;
    UInt16          = 6;
    UInt32          = 7;
    UInt64          = 8;
    Float           = 9;
    Double          = 10;
    Boolean         = 11;
    String          = 12;
    DateTime        = 13;
    Text            = 14;

    // Additional Metric Types
    UUID            = 15;
    DataSet         = 16;
    Bytes           = 17;
    File            = 18;
    Template        = 19;

    // Additional PropertyValue Types
    PropertySet     = 20;
    PropertySetList = 21;

    // Array Types
    Int8Array = 22;
    Int16Array = 23;
    Int32Array = 24;
    Int64Array = 25;
    UInt8Array = 26;
    UInt16Array = 27;
    UInt32Array = 28;
    UInt64Array = 29;
    FloatArray = 30;
    DoubleArray = 31;
    BooleanArray = 32;
    StringArray = 33;
    DateTimeArray = 34;
}

message Payload {

    message Template {

        message Parameter {
            string name        = 1;
            uint32 type        = 2;

            oneof value {
                uint32 int_value        = 3;
                uint64 long_value       = 4;
                float  float_value      = 5;
                double double_value     = 6;
                bool   boolean_value    = 7;
                string string_value     = 8;
                ParameterValueExtension extension_value = 9;
            }

            message ParameterValueExtension {
                repeated google.protobuf.Any extensions = 1;
            }
        }

        string version                       = 1;          // The version of the Template to prevent mismatches
        repeated Metric metrics              = 2;          // Each metric is the name of the metric and the datatype of the member but does not contain a value
        repeated Parameter parameters        = 3;
        string template_ref                  = 4;          // Reference to a template if this is extending a Template or an instance - must exist if an instance
        bool is_definition                   = 5;
        repeated google.protobuf.Any details = 6;
    }

    message DataSet {

        message DataSetValue {

            oneof value {
                uint32 int_value                        = 1;
                uint64 long_value                       = 2;
                float  float_value                      = 3;
                double double_value                     = 4;
                bool   boolean_value                    = 5;
                string string_value                     = 6;
                DataSetValueExtension extension_value   = 7;
            }

            message DataSetValueExtension {
                repeated google.protobuf.Any details = 1;
            }
        }

        message Row {
            repeated DataSetValue elements  = 1;
            repeated google.protobuf.Any details = 2;
        }

        uint64 num_of_columns                = 1;
        repeated string   columns            = 2;
        repeated uint32   types              = 3;
        repeated Row      rows               = 4;
        repeated google.protobuf.Any details = 5;
    }

    message PropertyValue {

        uint32     type                   = 1;
        bool       is_null                = 2;

        oneof value {
            uint32          int_value              = 3;
            uint64          long_value             = 4;
            float           float_value            = 5;
            double          double_value           = 6;
            bool            boolean_value          = 7;
            string          string_value           = 8;
            PropertySet     propertyset_value      = 9;
            PropertySetList propertysets_value     = 10;      // List of Property Values
            PropertyValueExtension extension_value = 11;
        }

        message PropertyValueExtension {
            repeated google.protobuf.Any details = 1;
        }
    }

    message PropertySet {
        repeated string        keys     = 1;         // Names of the properties
        repeated PropertyValue values   = 2;
        repeated google.protobuf.Any details = 3;
    }

    message PropertySetList {
        repeated PropertySet propertyset = 1;
        repeated google.protobuf.Any details = 2;
    }

    message MetaData {
        // Bytes specific metadata
        bool   is_multi_part   = 1;

        // General metadata
        string content_type    = 2;        // Content/Media type
        uint64 size            = 3;        // File size, String size, Multi-part size, etc
        uint64 seq             = 4;        // Sequence number for multi-part messages

        // File metadata
        string file_name       = 5;        // File name
        string file_type       = 6;        // File type (i.e. xml, json, txt, cpp, etc)
        string md5             = 7;        // md5 of data

        // Catchalls and future expansion
        string description     = 8;        // Could be anything such as json or xml of custom properties
        repeated google.protobuf.Any details = 9;
    }

    message Metric {

        string   name          = 1;        // Metric name - should only be included on birth
        uint64   alias         = 2;        // Metric alias - tied to name on birth and included in all later DATA messages
        uint64   timestamp     = 3;        // Timestamp associated with data acquisition time
        uint32   datatype      = 4;        // DataType of the metric/tag value
        bool     is_historical = 5;        // If this is historical data and should not update real time tag
        bool     is_transient  = 6;        // Tells consuming clients such as MQTT Engine to not store this as a tag
        bool     is_null       = 7;        // If this is null - explicitly say so rather than using -1, false, etc for some datatypes.
        MetaData metadata      = 8;        // Metadata for the payload
        PropertySet properties = 9;

        oneof value {
            uint32   int_value                      = 10;
            uint64   long_value                     = 11;
            float    float_value                    = 12;
            double   double_value                   = 13;
            bool     boolean_value                  = 14;
            string   string_value                   = 15;
            bytes    bytes_value                    = 16;       // Bytes, File
            DataSet  dataset_value                  = 17;
            Template template_value                 = 18;
            MetricValueExtension extension_value    = 19;
        }

        message MetricValueExtension {
            repeated google.protobuf.Any details = 1;
        }
    }

    uint64   timestamp      = 1;        // Timestamp at message sending time
    repeated Metric metrics = 2;        // Repeated forever - no limit in Google Protobufs
    uint64   seq            = 3;        // Sequence number
    string   uuid           = 4;        // UUID to track message type in terms of schema definitions
    bytes    body           = 5;        // To optionally bypass the whole definition above
    repeated google.protobuf.Any details = 6;
}

And my Kafka connector config looks like this:

key.converter=com.blueapron.connect.protobuf.ProtobufConverter
key.converter.schemas.enable=false
value.converter=com.blueapron.connect.protobuf.ProtobufConverter
value.converter.protoClassName=org.eclipse.tahu.protobuf.SparkplugBProto$Payload

Does anyone have an idea what could cause this? Is the Proto file too complex for this converter? I've tried experimenting with different versions of the google protobuf java library as well. Also different protoc versions just to check.

Thank you in advance!

[iqbal-apex]

Facing the same issue. Were you able to solve this?

[itaykat]

Same here... any updates?

[TobiasKrok]

@itaykat No, sorry. I ended up using Avro instead.