Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Complex Properties Overhaul #121

Closed
wants to merge 12 commits into from
59 changes: 56 additions & 3 deletions 3.0/vector_tile.proto
Original file line number Diff line number Diff line change
Expand Up @@ -31,11 +31,11 @@ message Tile {
message Feature {
optional uint64 id = 1 [ default = 0 ];

// Tags of this feature are encoded as repeated pairs of
// Attributes of this feature are encoded as repeated pairs of
// integers.
// A detailed description of tags is located in sections
// A detailed description of attributes is located in sections
// 4.2 and 4.4 of the specification
repeated uint32 tags = 2 [ packed = true ];
repeated uint32 attributes = 2 [ packed = true ];

// The type of geometry stored in this feature.
optional GeomType type = 3 [ default = UNKNOWN ];
Expand All @@ -44,6 +44,57 @@ message Tile {
// A detailed description on geometry encoding is located in
// section 4.3 of the specification.
repeated uint32 geometry = 4 [ packed = true ];

// Additional attributes (or all the attributes) of this feature may be
// encoded as repeated pairs of 64-bit integers, to take
// advantage of inline encoding of small values,
// improved compression from use of repeated values,
// and support for list and map values.
//
// This message may only be used if the layer version is >= 3.
//
// The inline_attributes field is much like the attributes field in that it is a pair of
// integers that reference key and value pairs. However, the value reference
// is a "complex_value" that combines a type and an index.
//
// The "key_index" is much like the key index in the use for attributes, but instead
// of pointing into layer.keys, it points into layer.attribute_pool.keys.
//
// An complex value has two parts: the lowest 4 bits are the type bits,
// and the remaining bits are the parameter bits. What is stored in the parameter
// bits is dependent on the contents of the type bits. For example for inline values,
// the parameter field is not an index but simply a value. For other types it is
// an index position into a value storage of the layer.
//
// uint64_t type = complex_value & 0x0F; // Bottom 4 bits
// uint64_t parameter = complex_value >> 4;
//
// Type | Id | Parameter
// ---------------------------------
// string | 0 | index into layer.attribute_pool.string_values
// float | 1 | index into layer.attribute_pool.float_values
// double | 2 | index into layer.attribute_pool.double_values
// int | 3 | index into layer.attribute_pool.signed_integer_values
// uint | 4 | index into layer.attribute_pool.unsigned_integer_values
// inline uint | 5 | value of unsigned integer (values between 0 to 2^60-1)
// inline sint | 6 | value of zigzag-encoded integer (values between -2^59 to 2^59-1)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We probably should change this to be 2^56 for uint and 2^55 for signed due to the way varints are encoded.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we need to differentiate between what encodings are possible and what encodings are recommended. The spec may well say that this or that encoding is recommended becaus it is usually better, but still require readers to understand a different encoding.

// bool/null | 7 | value of 0 = false, 1 = true, 2 = null
// list | 8 | value is the number of sub-attributes to follow:
// | | each item in the list is a complex value
// map | 9 | value is the number of sub-attributes to follow:
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Question on intent of wording here, is the number of sub attributes to follow based on number of key value pairs or the number of keys and values.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I meant it to be the number of pairs, not the total number of words to follow. Thanks. I'll reword.

// | | each item is an index into layer.attribute_pool.keys
// | | followed by a complex_value for the value
repeated uint64 inline_attributes = 5 [ packed = true ];
}

message AttributePool {
repeated string keys = 1;

repeated string string_values = 2;
repeated float float_values = 3 [ packed = true ];
repeated double double_values = 4 [ packed = true ];
repeated sfixed64 signed_integer_values = 5 [ packed = true ];
repeated fixed64 unsigned_integer_values = 6 [ packed = true ];
}

// Layers are described in section 4.1 of the specification
Expand All @@ -69,6 +120,8 @@ message Tile {
// See https://github.com/mapbox/vector-tile-spec/issues/47
optional uint32 extent = 5 [ default = 4096 ];

optional AttributePool attribute_pool = 7;

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggest these should get "logical" names like signed_integer_values or so instead of ones based on the encoding sfixed....

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also fine with me.

extensions 16 to max;
}

Expand Down