TFX 1.3.1

Major Features and Improvements

• N/A

For Pipeline Authors

• N/A

For Component Authors

• N/A

Deprecations

• N/A

Bug Fixes and Other Changes

• Fixed Vertex Pusher by passing enable_vertex flag for deploying model.

Documentation Updates

• N/A

TFX 1.3.0

Major Features and Improvements

• TFX CLI now supports runtime parameter on Kubeflow, Vertex, and Airflow.
  Use it with '--runtime_parameter=<parameter_name>=<parameter_value>' flag.
  In the case of multiple runtime parameters, format is as follows:
  '--runtime_parameter=<parameter_name>=<parameter_value> --runtime_parameter
  =<parameter_name>=<parameter_value>'
• Added Manual node in the experimental orchestrator.
• Placeholders support index access and JSON serialization for list type execution properties.
• Added ImportSchemaGen which is a dedicated component to import a
  pre-defined schema file. ImportSchemaGen will replace Importer with
  simpler syntax and less constraints. You have to pass the file path to the
  schema file instead of the parent directory unlike Importer.

Breaking Changes

For Pipeline Authors

• N/A

For Component Authors

• N/A

Deprecations

• The import name of KerasTuner has been changed from kerastuner
  to keras_tuner. The import name of kerastuner is still supported.
  A warning will occur when import from kerastuner, but does not affect
  the usage.

Bug Fixes and Other Changes

• The default job name for Google Cloud AI Training jobs was changed from
  'tfx_YYYYmmddHHMMSS' to 'tfx_YYYYmmddHHMMSS_xxxxxxxx', where 'xxxxxxxx' is
  a random 8 digit hexadecimal string.
• Fix component to raise error if its input required channel (specified from
  ComponentSpec) has no artifacts in it.
• Fixed an issue where ClientOptions with regional endpoint was
  incorrectly left out in Vertex AI pusher.
• CLI now hides passed flags from user python files in "--pipeline-path". This
  will prevent errors when user python file tries reading and parsing flags.
• Fixed missing type information marker file 'py.typed'.
• Depends on apache-beam[gcp]>=2.32,<3.
• Depends on google-cloud-bigquery>=1.28.0,<3.
• Depends on google-cloud-aiplatform>=0.5.0,<2.
• Depends on jinja2>=2.7.3,<4, i.e. now supports Jinja 3.x.
• Depends on keras-tuner>=1.0.4,<2.
• Depends on kfp>=1.6.1,!=1.7.2,<1.8.2 in [kfp] extra.
• Depends on kfp-pipeline-spec>=>=0.1.10,<0.2.
• Depends on ml-metadata>=1.3.0,<1.4.0.
• Depends on struct2tensor>=0.34.0,<0.35.0.
• Depends on tensorflow>=1.15.2,!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.*,!=2.4.*,!=2.5.*,<3.
• Depends on tensorflow-data-validation>=1.3.0,<1.4.0.
• Depends on tensorflow-model-analysis>=0.34.1,<0.35.0.
• Depends on tensorflow-serving-api>=1.15,!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.*,!=2.4.*,!=2.5.*,<3.
• Depends on tensorflow-transform>=1.3.0,<1.4.0.
• Depends on tfx-bsl>=1.3.0,<1.4.0.

Documentation Updates

• N/A

TFX 1.3.0-rc0

Major Features and Improvements

• TFX CLI now supports runtime parameter on Kubeflow, Vertex, and Airflow.
  Use it with '--runtime_parameter=<parameter_name>=<parameter_value>' flag.
  In the case of multiple runtime parameters, format is as follows:
  '--runtime_parameter=<parameter_name>=<parameter_value> --runtime_parameter
  =<parameter_name>=<parameter_value>'
• Added Manual node in the experimental orchestrator.
• Placeholders support index access and JSON serialization for list type execution properties.
• Added ImportSchemaGen which is a dedicated component to import a
  pre-defined schema file. ImportSchemaGen will replace Importer with
  simpler syntax and less constraints. You have to pass the file path to the
  schema file instead of the parent directory unlike Importer.

Breaking Changes

For Pipeline Authors

• N/A

For Component Authors

• N/A

Deprecations

• The import name of KerasTuner has been changed from kerastuner
  to keras_tuner. The import name of kerastuner is still supported.
  A warning will occur when import from kerastuner, but does not affect
  the usage.

Bug Fixes and Other Changes

• The default job name for Google Cloud AI Training jobs was changed from
  'tfx_YYYYmmddHHMMSS' to 'tfx_YYYYmmddHHMMSS_xxxxxxxx', where 'xxxxxxxx' is
  a random 8 digit hexadecimal string.
• Fix component to raise error if its input required channel (specified from
  ComponentSpec) has no artifacts in it.
• Fixed an issue where ClientOptions with regional endpoint was
  incorrectly left out in Vertex AI pusher.
• CLI now hides passed flags from user python files in "--pipeline-path". This
  will prevent errors when user python file tries reading and parsing flags.
• Fixed missing type information marker file 'py.typed'.
• Depends on apache-beam[gcp]>=2.32,<3.
• Depends on google-cloud-bigquery>=1.28.0,<3.
• Depends on google-cloud-aiplatform>=0.5.0,<=1.4.3.
• Depends on jinja2>=2.7.3,<4, i.e. now supports Jinja 3.x.
• Depends on keras-tuner>=1.0.4,<2.
• Depends on kfp>=1.6.1,!=1.7.2,<1.8.2 in [kfp] extra.
• Depends on kfp-pipeline-spec>=>=0.1.10,<0.2.
• Depends on ml-metadata>=1.3.0,<1.4.0.
• Depends on struct2tensor>=0.34.0,<0.35.0.
• Depends on tensorflow>=1.15.2,!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.*,!=2.4.*,!=2.5.*,<3.
• Depends on tensorflow-data-validation>=1.3.0,<1.4.0.
• Depends on tensorflow-model-analysis>=0.34.1,<0.35.0.
• Depends on tensorflow-serving-api>=1.15,!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.*,!=2.4.*,!=2.5.*,<3.
• Depends on tensorflow-transform>=1.3.0,<1.4.0.
• Depends on tfx-bsl>=1.3.0,<1.4.0.

Documentation Updates

• N/A

TFX 1.2.0

Major Features and Improvements

• Added RuntimeParam support for Trainer's custom_config.
• TFX Trainer and Pusher now support Vertex, which can be enabled with
  ENABLE_VERTEX_KEY key in custom_config.

Breaking Changes

For Pipeline Authors

• N/A

For Component Authors

• N/A

Deprecations

• N/A

Bug Fixes and Other Changes

• Fixed the issue that kfp_pod_name is not generated as an execution property
  for Kubeflow Pipelines.
• Fixed issue when InputValuePlaceholder is used as component parameter in
  container based component.
• Depends on kubernetes>=10.0.1,<13
• CsvToExample now supports multi-line strings.
• tfx.benchmarks package was removed from the Python TFX wheel. This package
  is used only for benchmarking and not useful for end users.
• Fixed the issue for fairness_indicator_thresholds support of Evaluator.
• Depends on apache-beam[gcp]>=2.31,<3.
• Depends on kfp-pipeline-spec>=0.1.8,<0.2.
• Depends on ml-metadata>=1.2.0,<1.3.0.
• Depends on struct2tensor>=0.33.0,<0.34.0.
• Depends on tensorflow>=1.15.2,!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.*,!=2.4.*,<2.6.
• Depends on tensorflow-data-validation>=1.2.0,<1.3.0.
• Depends on tensorflow-model-analysis>=0.33.0,<0.34.0.
• Depends on tensorflow-transform>=1.2.0,<1.3.0.
• Depends on tfx-bsl>=1.2.0,<1.3.0.

Documentation Updates

• N/A

TFX 1.2.0-rc0

Major Features and Improvements

• Added RuntimeParam support for Trainer's custom_config.
• TFX Trainer and Pusher now support Vertex, which can be enabled with
  ENABLE_VERTEX_KEY key in custom_config.

Breaking Changes

For Pipeline Authors

• N/A

For Component Authors

• N/A

Deprecations

• N/A

Bug Fixes and Other Changes

• Fixed issue when InputValuePlaceholder is used as component parameter in
  container based component.
• Depends on kubernetes>=10.0.1,<13
• CsvToExample now supports multi-line strings.
• tfx.benchmarks package was removed from the Python TFX wheel. This package
  is used only for benchmarking and not useful for end users.
• Fixed the issue for fairness_indicator_thresholds support of Evaluator.
• Depends on apache-beam[gcp]>=2.31,<3.
• Depends on kfp-pipeline-spec>=0.1.8,<0.2.
• Depends on ml-metadata>=1.2.0,<1.3.0.
• Depends on struct2tensor>=0.33.0,<0.34.0.
• Depends on tensorflow-data-validation>=1.2.0,<1.3.0.
• Depends on tensorflow-model-analysis>=0.33.0,<0.34.0.
• Depends on tensorflow-transform>=1.2.0,<1.3.0.
• Depends on tfx-bsl>=1.2.0,<1.3.0.

Documentation Updates

• N/A

TFX 1.0.0

Major Features and Improvements

• Added tfx.v1 Public APIs, please refer to
  API doc for details.
• Transform component now computes pre-transform and post-transform statistics
  and stores them in new, indvidual outputs ('pre_transform_schema',
  'pre_transform_stats', 'post_transform_schema', 'post_transform_stats',
  'post_transform_anomalies'). This can be disabled by setting
  disable_statistics=True in the Transform component.
• BERT cola and mrpc examples now demonstrate how to calculate statistics for
  NLP features.
• TFX CLI now supports
  Vertex Pipelines.
  use it with --engine=vertex flag.
• Telemetry: Only first-party tfx component's executor telemetry will be
  collected. All other executors will be recorded as third_party_executor.
  For labels longer than 63, keep first 63 characters (instead of last 63
  characters before).
• Supports text type (use proto json string format) RuntimeParam for protos.
• Combined/moved taxi's runtime_parameter, kubeflow_local and kubleflow_gcp
  example pipelines into one penguin_pipeline_kubeflow example
• Transform component now supports passing stats_options_updater_fn directly
  as well as through the module file.
• Placeholders support accessing artifact property and custom property.
• Removed the extra node information in IR for KubeflowDagRunner, to reduce
  size of generated IR.

Breaking Changes

• Removed unneccessary default values for required component input Channels.
• The _PropertyDictWrapper internal wrapper for component.inputs and
  component.outputs was removed: component.inputs and component.outputs
  are now unwrapped dictionaries, and the attribute accessor syntax (e.g.
  components.outputs.output_name) is no longer supported. Please use the
  dictionary indexing syntax (e.g. components.outputs['output_name'])
  instead.

For Pipeline Authors

• N/A

For Component Authors

• Apache Beam support is migrated from TFX Base Components and Executors to
  dedicated Beam Components and Executors. BaseExecutor will no longer embed
  beam_pipeline_args. Custom executors for Beam powered components should
  now extend BaseBeamExecutor instead of BaseExecutor.

Deprecations

• Deprecated nested RuntimeParam for Proto, Please use text type (proto json
  string) RuntimeParam instead of Proto dict with nested RuntimeParam in it.

Bug Fixes and Other Changes

• Forces keyword arguments for AirflowComponent to make it compatible with
  Apache Airflow 2.1.0 and later.
• Fixed issue where passing analyzer_cache to tfx.components.Transform
  before there are any Transform cache artifacts published would fail.
• Included type information according to PEP-561. However, protobuf generated
  files don't have type information, and you might need to ignore errors from
  them. For example, if you are using mypy, see
  the related doc.
• Removed six dependency.
• Depends on apache-beam[gcp]>=2.29,<3.
• Depends on google-cloud-bigquery>=1.28.0,<2.21
• Depends on ml-metadata>=1.0.0,<1.1.0.
• Depends on protobuf>=3.13,<4.
• Depends on struct2tensor>=0.31.0,<0.32.0.
• Depends on tensorflow>=1.15.2,!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.*,!=2.4.*,<3.
• Depends on tensorflow-data-validation>=1.0.0,<1.1.0.
• Depends on tensorflow-hub>=0.9.0,<0.13.
• Depends on tensorflowjs>=3.6.0,<4.
• Depends on tensorflow-model-analysis>=0.31.0,<0.32.0.
• Depends on tensorflow-serving-api>=1.15,!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.*,!=2.4.*,<3.
• Depends on tensorflow-transform>=1.0.0,<1.1.0.
• Depends on tfx-bsl>=1.0.0,<1.1.0.

Documentation Updates

• Update the Guide of TFX to adopt 1.0 API.
• TFT and TFDV component documentation now describes how to
  configure pre-transform and post-transform statistics, which can be used for
  validating text features.

TFX 1.0.0-rc2

Major Features and Improvements

• Added tfx.v1 Public APIs, please refer to
  API doc for details.
• Transform component now computes pre-transform and post-transform statistics
  and stores them in new, indvidual outputs ('pre_transform_schema',
  'pre_transform_stats', 'post_transform_schema', 'post_transform_stats',
  'post_transform_anomalies'). This can be disabled by setting
  disable_statistics=True in the Transform component.
• BERT cola and mrpc examples now demonstrate how to calculate statistics for
  NLP features.
• TFX CLI now supports
  Vertex Pipelines.
  use it with --engine=vertex flag.
• Telemetry: Only first-party tfx component's executor telemetry will be
  collected. All other executors will be recorded as third_party_executor.
  For labels longer than 63, keep first 63 characters (instead of last 63
  characters before).
• Supports text type (use proto json string format) RuntimeParam for protos.
• Combined/moved taxi's runtime_parameter, kubeflow_local and kubleflow_gcp
  example pipelines into one penguin_pipeline_kubeflow example
• Transform component now supports passing stats_options_updater_fn directly
  as well as through the module file.
• Placeholders support accessing artifact property and custom property.
• Removed the extra node information in IR for KubeflowDagRunner, to reduce
  size of generated IR.

Breaking Changes

• Removed unneccessary default values for required component input Channels.
• The _PropertyDictWrapper internal wrapper for component.inputs and
  component.outputs was removed: component.inputs and component.outputs
  are now unwrapped dictionaries, and the attribute accessor syntax (e.g.
  components.outputs.output_name) is no longer supported. Please use the
  dictionary indexing syntax (e.g. components.outputs['output_name'])
  instead.

For Pipeline Authors

• N/A

For Component Authors

• Apache Beam support is migrated from TFX Base Components and Executors to
  dedicated Beam Components and Executors. BaseExecutor will no longer embed
  beam_pipeline_args. Custom executors for Beam powered components should
  now extend BaseBeamExecutor instead of BaseExecutor.

Deprecations

• Deprecated nested RuntimeParam for Proto, Please use text type (proto json
  string) RuntimeParam instead of Proto dict with nested RuntimeParam in it.

Bug Fixes and Other Changes

• Forces keyword arguments for AirflowComponent to make it compatible with
  Apache Airflow 2.1.0 and later.
• Fixed issue where passing analyzer_cache to tfx.components.Transform
  before there are any Transform cache artifacts published would fail.
• Included type information according to PEP-561. However, protobuf generated
  files don't have type information, and you might need to ignore errors from
  them. For example, if you are using mypy, see
  the related doc.
• Removed six dependency.
• Depends on apache-beam[gcp]>=2.29,<3.
• Depends on google-cloud-bigquery>=1.28.0,<2.21
• Depends on ml-metadata>=1.0.0,<1.1.0.
• Depends on protobuf>=3.13,<4.
• Depends on struct2tensor>=0.31.0,<0.32.0.
• Depends on tensorflow>=1.15.2,!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.*,!=2.4.*,<3.
• Depends on tensorflow-data-validation>=1.0.0,<1.1.0.
• Depends on tensorflow-hub>=0.9.0,<0.13.
• Depends on tensorflowjs>=3.6.0,<4.
• Depends on tensorflow-model-analysis>=0.31.0,<0.32.0.
• Depends on tensorflow-serving-api>=1.15,!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.*,!=2.4.*,<3.
• Depends on tensorflow-transform>=1.0.0,<1.1.0.
• Depends on tfx-bsl>=1.0.0,<1.1.0.

Documentation Updates

• Update the Guide of TFX to adopt 1.0 API.
• TFT and TFDV component documentation now describes how to
  configure pre-transform and post-transform statistics, which can be used for
  validating text features.

TFX 0.30.2

Version 0.30.2

Major Features and Improvements

• N/A

Breaking Changes

For Pipeline Authors

• N/A

For Component Authors

• N/A

Deprecations

• N/A

Bug Fixes and Other Changes

• Update resolver query in TFX -> KFP IR compiler with vertex placeholder syntax

Documentation Updates

• N/A

TFX 0.30.1

Major Features and Improvements

• TFX CLI now supports
  Vertex Pipelines.
  use it with --engine=vertex flag.

Breaking Changes

For Pipeline Authors

• N/A

For Component Authors

• N/A

Deprecations

• N/A

Bug Fixes and Other Changes

• Fix resolver artifact filter in TFX -> KFP IR compiler with OP filter syntax.
• Forces keyword arguments for AirflowComponent to make it compatible with
  Apache Airflow 2.1.0 and later.

Documentation Updates

• N/A

TFX 1.0.0-rc1

Major Features and Improvements

• Added tfx.v1 Public APIs, please refer to
  API doc for details.
• Transform component now computes pre-transform and post-transform statistics
  and stores them in new, indvidual outputs ('pre_transform_schema',
  'pre_transform_stats', 'post_transform_schema', 'post_transform_stats',
  'post_transform_anomalies'). This can be disabled by setting
  disable_statistics=True in the Transform component.
• BERT cola and mrpc examples now demonstrate how to calculate statistics for
  NLP features.

Breaking Changes

For Pipeline Authors

• N/A

For Component Authors

• Apache Beam support is migrated from TFX Base Components and Executors to
  dedicated Beam Components and Executors. BaseExecutor will no longer embed
  beam_pipeline_args. Custom executors for Beam powered components should
  now extend BaseBeamExecutor instead of BaseExecutor.

Deprecations

• N/A

Bug Fixes and Other Changes

• TFX CLI now supports
  Vertex Pipelines.
  use it with --engine=vertex flag.
• Forces keyword arguments for AirflowComponent to make it compatible with
  Apache Airflow 2.1.0 and later.
• Removed six dependency.
• Depends on apache-beam[gcp]>=2.29,<3.
• Depends on ml-metadata>=1.0.0,<1.1.0.
• Depends on struct2tensor>=0.31.0,<0.32.0.
• Depends on tensorflow>=1.15.2,!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.*,!=2.4.*,<3.
• Depends on tensorflow-data-validation>=1.0.0,<1.1.0.
• Depends on tensorflowjs>=3.6.0,<4.
• Depends on tensorflow-model-analysis>=0.31.0,<0.32.0.
• Depends on tensorflow-serving-api>=1.15,!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.*,!=2.4.*,<3.
• Depends on tensorflow-transform>=1.0.0,<1.1.0.
• Depends on tfx-bsl>=1.0.0,<1.1.0.

Documentation Updates

• Update the Guide of TFX to adopt 1.0 API.
• TFT and TFDV component documentation now describes how to
  configure pre-transform and post-transform statistics, which can be used for
  validating text features.