Releases: ibis-project/ibis
Releases · ibis-project/ibis
3.0.1
3.0.0
3.0.0 (2022-04-25)
⚠ BREAKING CHANGES
- ir: The following are breaking changes due to simplifying expression internals
ibis.expr.datatypes.DataType.scalar_type
andDataType.column_type
factory
methods have been removed,DataType.scalar
andDataType.column
class
fields can be used to directly construct a corresponding expression instance
(though prefer to useoperation.to_expr()
)ibis.expr.types.ValueExpr._name
andValueExpr._dtype`` fields are not accassible anymore. While these were not supposed to used directly now
ValueExpr.has_name(),
ValueExpr.get_name()and
ValueExpr.type()` methods
are the only way to retrieve the expression's name and datatype.ibis.expr.operations.Node.output_type
is a property now not a method,
decorate those methods with@property
ibis.expr.operations.ValueOp
subclasses must defineoutput_shape
and
output_dtype
properties from now on (note the datatype abbreviationdtype
in the property name)ibis.expr.rules.cast()
,scalar_like()
andarray_like()
rules have been
removed
- api: Replace
t["a"].distinct()
witht[["a"]].distinct()
. - deps: The sqlalchemy lower bound is now 1.4
- ir: Schema.names and Schema.types attributes now have tuple type rather than list
- expr: Columns that were added or used in an aggregation or
mutation would be alphabetically sorted in compiled SQL outputs. This
was a vestige from when Python dicts didn't preserve insertion order.
Now columns will appear in the order in which they were passed to
aggregate
ormutate
- api:
dt.float
is nowdt.float64
; usedt.float32
for the previous behavior. - ir: Relation-based
execute_node
dispatch rules must now accept tuples of expressions. - ir: removed ibis.expr.lineage.{roots,find_nodes} functions
- config: Use
ibis.options.graphviz_repr = True
to enable - hdfs: Use
fsspec
instead of HDFS from ibis - udf: Vectorized UDF coercion functions are no longer a public API.
- The minimum supported Python version is now Python 3.8
- config:
register_option
is no longer supported, please submit option requests upstream - backends: Read tables with pandas.read_hdf and use the pandas backend
- The CSV backend is removed. Use Datafusion for CSV execution.
- backends: Use the datafusion backend to read parquet files
Expr() -> Expr.pipe()
- coercion functions previously in expr/schema.py are now in udf/vectorized.py
- api:
materialize
is removed. Joins with overlapping columns now have suffixes. - kudu: use impala instead: https://kudu.apache.org/docs/kudu_impala_integration.html
- Any code that was relying implicitly on string-y
behavior from UUID datatypes will need to add an explicit cast first.
Features
- add repr_html for expressions to print as tables in ipython (cd6fa4e)
- add duckdb backend (667f2d5)
- allow construction of decimal literals (3d9e865)
- api: add
ibis.asc
expression (efe177e), closes #1454 - api: add has_operation API to the backend (4fab014)
- api: implement type for SortExpr (ab19bd6)
- clickhouse: implement string concat for clickhouse (1767205)
- clickhouse: implement StrRight operation (67749a0)
- clickhouse: implement table union (e0008d7)
- clickhouse: implement trim, pad and string predicates (a5b7293)
- datafusion: implement Count operation (4797a86)
- datatypes: unbounded decimal type (f7e6f65)
- date: add ibis.date(y,m,d) functionality (26892b6), closes #386
- duckdb/postgres/mysql/pyspark: implement
.sql
on tables for mixing sql and expressions (00e8087) - duckdb: add functionality needed to pass integer to interval test (e2119e8)
- duckdb: implement _get_schema_using_query (93cd730)
- duckdb: implement now() function (6924f50)
- duckdb: implement regexp replace and extract (18d16a7)
- implement
force
argument in sqlalchemy backend base class (9df7f1b) - implement coalesce for the pyspark backend (8183efe)
- implement semi/anti join for the pandas backend (cb36fc5)
- implement semi/anti join for the pyspark backend (3e1ba9c)
- implement the remaining clickhouse joins (b3aa1f0)
- ir: rewrite and speed up expression repr (45ce9b2)
- mysql: implement _get_schema_from_query (456cd44)
- mysql: move string join impl up to alchemy for mysql (77a8eb9)
- postgres: implement _get_schema_using_query (f2459eb)
- pyspark: implement Distinct for pyspark (4306ad9)
- pyspark: implement log base b for pyspark (527af3c)
- pyspark: implement percent_rank and enable testing (c051617)
- repr: add interval info to interval repr (df26231)
- sqlalchemy: implement ilike (43996c0)
- sqlite: implement date_truncate (3ce4f2a)
- sqlite: implement ISO week of year (714ff7b)
- sqlite: implement string join and concat (6f5f353)
- support of arrays and tuples for clickhouse (db512a8)
- ver: dynamic version identifiers (408f862)
Bug Fixes
- added wheel to pyproject toml for venv users (b0b8e5c)
- allow major version changes in CalVer dependencies (9c3fbe5)
- annotable: allow optional arguments at any position (778995f), closes #3730
- api: add ibis.map and .struct (327b342), closes #3118
- api: map string multiplication with integer to repeat method (b205922)
- api: thread suffixes parameter to individual join methods (31a9aff)
- change TimestampType to Timestamp (e0750be)
- clickhouse: disconnect from clickhouse when computing version (11cbf08)
- clickhouse: use a context manager for execution ([a471225](https://github.com/ibis-project/ibi...
2.1.1
2.1.0
2.1.0 (2022-01-12)
Bug Fixes
- consider all packages' entry points (b495cf6)
- datatypes: infer bytes literal as binary #2915 (#3124) (887efbd)
- deps: bump minimum dask version to 2021.10.0 (e6b5c09)
- deps: constrain numpy to ensure wheels are used on windows (70c308b)
- deps: update dependency clickhouse-driver to ^0.1 || ^0.2.0 (#3061) (a839d54)
- deps: update dependency geoalchemy2 to >=0.6,<0.11 (4cede9d)
- deps: update dependency pyarrow to v6 (#3092) (61e52b5)
- don't force backends to override do_connect until 3.0.0 (4b46973)
- execute materialized joins in the pandas and dask backends (#3086) (9ed937a)
- literal: allow creating ibis literal with uuid (#3131) (b0f4f44)
- restore the ability to have more than two option levels (#3151) (fb4a944)
- sqlalchemy: fix correlated subquery compilation (43b9010)
- sqlite: defer db connection until needed (#3127) (5467afa), closes #64
Features
- allow column_of to take a column expression (dbc34bb)
- ci: More readable workflow job titles (#3111) (d8fd7d9)
- datafusion: initial implementation for Arrow Datafusion backend (3a67840), closes #2627
- datafusion: initial implementation for Arrow Datafusion backend (75876d9), closes #2627
- make dayofweek impls conform to pandas semantics (#3161) (9297828)
Reverts
- "ci: install gdal for fiona" (8503361)