Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Occasional error when writing TileDB array #27

Open
DPeterK opened this issue Mar 10, 2020 · 0 comments
Open

Occasional error when writing TileDB array #27

DPeterK opened this issue Mar 10, 2020 · 0 comments

Comments

@DPeterK
Copy link
Contributor

DPeterK commented Mar 10, 2020

There seems to be an occasionally-occurring bug, which only occurs when you create, and re-create an instance of a Writer class in the same session. Here's the IPython traceback:

---------------------------------------------------------------------------
TileDBError                               Traceback (most recent call last)
<ipython-input-11-b6147c7d2b8d> in <module>
      4 
      5 writer = MultiAttrTDBWriter(ncdm, base_fp, array_name=tdb_name, unlimited_dims=append_dim)
----> 6 writer.create_domains()

/home/jovyan/tiledb/tiledb_netcdf/nctotdb/writers.py in create_domains(self, data_array_name)
    513             # Get data vars in this domain and create and populate a multi-attr array.
    514             self.create_multiattr_array(domain_var_names, domain_coord_names,
--> 515                                         group_dirname, data_array_name)
    516             self.populate_multiattr_array(data_array_name, domain_var_names, group_dirname)
    517 

/home/jovyan/tiledb/tiledb_netcdf/nctotdb/writers.py in create_multiattr_array(self, domain_var_names, domain_dims, group_dirname, data_array_name)
    482         array_filename = os.path.join(group_dirname, data_array_name)
    483         # Create an empty array.
--> 484         schema = tiledb.ArraySchema(domain=tdb_domain, sparse=False, attrs=attrs)
    485         tiledb.Array.create(array_filename, schema)
    486 

tiledb/libtiledb.pyx in tiledb.libtiledb.ArraySchema.__init__()

tiledb/libtiledb.pyx in tiledb.libtiledb._raise_ctx_err()

tiledb/libtiledb.pyx in tiledb.libtiledb._raise_tiledb_error()

TileDBError: [TileDB::ArraySchema] Error: Array schema check failed; Attributes and dimensions must have unique names

This might be evidence of implicit ordering in the input NetCDF contents that is being inappropriately relied upon in this library: on the first load the implicit order is used to set either the data model or something in the Writer instance spinup, on the second load (in the same session), the first load is still active in the session but the implicit order of NetCDF contents is different and leads to this odd error. Confirming this is the case will require more exploration, however.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant