Skip to content

Commit

Permalink
Accept any Tables.jl-compatible source to write_parquet
Browse files Browse the repository at this point in the history
The Tables.jl interface is designed so consumers (i.e. `write_parquet`) should never call `Tables.rowaccess` or `Tables.columnaccess`; they just call whatever orientation they need the data in (in this case, `Tables.columns`), and Tables.jl will take care of the rest. Column-oriented sources will be "automatic", while row-oriented sources will have columns "built", so that either way, `write_parquet` will get columns to write out.
  • Loading branch information
quinnj authored Jul 2, 2020
1 parent e0618d4 commit ce3eb58
Showing 1 changed file with 2 additions and 3 deletions.
5 changes: 2 additions & 3 deletions src/writer.jl
Original file line number Diff line number Diff line change
Expand Up @@ -460,9 +460,8 @@ tbl - A Tables.jl columnaccessible table e.g. a DataFrame
compression_code - Default "SNAPPY". The compression codec. The supported
values are "UNCOMPRESSED", "SNAPPY", "ZSTD", "GZIP"
"""
function write_parquet(path, tbl; compression_codec = "SNAPPY")
# tbl needs to be iterable by column as parquet is a columnar format
@assert Tables.columnaccess(tbl)
function write_parquet(path, x; compression_codec = "SNAPPY")
tbl = Tables.columns(x)

# check that all types are supported
sch = Tables.schema(tbl)
Expand Down

0 comments on commit ce3eb58

Please sign in to comment.