Data FAQ

How do I validate a record with default values filled in?

Foo foo = new Foo();
ValidationResult result = ValidateDataAgainstSchema.validate(foo.data(), foo.schema(), new ValidationOptions(RequiredMode.FIXUP_ABSENT_WITH_DEFAULT));
assert(result.isValid());

This will fail if the underlying data is read-only and default values cannot be set for absent fields.

This will also work for partially filled in records. It will only add default values to fields that are absent.

How do I convert a Pegasus data schema to an Avro data schema

Requires data-avro module, or data-avro-*.jar.

At runtime, use

Foo foo = new Foo();
DataSchema pegasusSchema = foo.schema();
org.apache.avro.Schema avroSchema = SchemaTranslator.dataToAvroSchema(pegasusSchema);
String avsoSchemaInJson = avroSchema.toString();

From command line, to create text files with Avro schema from pdsc files (version 0.17.1 or higher)

# syntax:
# java [-Dgenerator.resolver.path=<path>] com.linkedin.data.avro.AvroSchemaGenerator <targetDir> [<pdscFile|schemaFullName>]...
java -Dgenerator.resolver.path=src/main/pegasus com.linkedin.data.avro.generator.AvroSchemaGenerator ../build/main/codegen/avro src/main/pegasus/com/linkedin/foo/*.pdsc
# or
java -Dgenerator.resolver.path=src/main/pegasus com.linkedin.data.avro.generator.AvroSchemaGenerator ../build/main/codegen/avro com.linkedin.foo.Foo

Classpath must be setup to include data-avro.jar and its dependencies.