Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update all non-major dependencies #10

Merged
merged 1 commit into from
Oct 18, 2024
Merged

Conversation

renovate[bot]
Copy link
Contributor

@renovate renovate bot commented Aug 19, 2024

This PR contains the following updates:

Package Change Age Adoption Passing Confidence Type Update
github.com/apache/beam/sdks/v2 v2.58.0 -> v2.60.0 age adoption passing confidence require minor
go (source) 1.22.6 -> 1.23.2 age adoption passing confidence toolchain minor

Release Notes

apache/beam (github.com/apache/beam/sdks/v2)

v2.60.0: Beam 2.60.0 release

We are happy to present the new 2.60.0 release of Beam.
This release includes both improvements and new functionality.

For more information on changes in 2.60.0, check out the detailed release notes.

Highlights

  • Added support for using vLLM in the RunInference transform (Python) (#​32528)
  • [Managed Iceberg] Added support for streaming writes (#​32451)
  • [Managed Iceberg] Added auto-sharding for streaming writes (#​32612)
  • [Managed Iceberg] Added support for writing to dynamic destinations (#​32565)

New Features / Improvements

  • Dataflow worker can install packages from Google Artifact Registry Python repositories (Python) (#​32123).
  • Added support for Zstd codec in SerializableAvroCodecFactory (Java) (#​32349)
  • Added support for using vLLM in the RunInference transform (Python) (#​32528)
  • Prism release binaries and container bootloaders are now being built with the latest Go 1.23 patch. (#​32575)
  • Prism
    • Prism now supports Bundle Finalization. (#​32425)
  • Significantly improved performance of Kafka IO reads that enable commitOffsetsInFinalize by removing the data reshuffle from SDF implementation. (#​31682).
  • Added support for dynamic writing in MqttIO (Java) (#​19376)
  • Optimized Spark Runner parDo transform evaluator (Java) (#​32537)
  • [Managed Iceberg] More efficient manifest file writes/commits (#​32666)

Breaking Changes

  • In Python, assert_that now throws if it is not in a pipeline context instead of silently succeeding (#​30771)
  • In Python and YAML, ReadFromJson now override the dtype from None to
    an explicit False. Most notably, string values like "123" are preserved
    as strings rather than silently coerced (and possibly truncated) to numeric
    values. To retain the old behavior, pass dtype=True (or any other value
    accepted by pandas.read_json).
  • Users of KafkaIO Read transform that enable commitOffsetsInFinalize might encounter pipeline graph compatibility issues when updating the pipeline. To mitigate, set the updateCompatibilityVersion option to the SDK version used for the original pipeline, example --updateCompatabilityVersion=2.58.1

Deprecations

  • Python 3.8 is reaching EOL and support is being removed in Beam 2.61.0. The 2.60.0 release will warn users
    when running on 3.8. (#​31192)

Bugfixes

  • (Java) Fixed custom delimiter issues in TextIO (#​32249, #​32251).
  • (Java, Python, Go) Fixed PeriodicSequence backlog bytes reporting, which was preventing Dataflow Runner autoscaling from functioning properly (#​32506).
  • (Java) Fix improper decoding of rows with schemas containing nullable fields when encoded with a schema with equal encoding positions but modified field order. (#​32388).

Known Issues

N/A

For the most up to date list of known issues, see https://github.com/apache/beam/blob/master/CHANGES.md

List of Contributors

According to git shortlog, the following people contributed to the 2.60.0 release. Thank you to all contributors!

Ahmed Abualsaud, Aiden Grossman, Arun Pandian, Bartosz Zablocki, Chamikara Jayalath, Claire McGinty, DKPHUONG, Damon Douglass, Danny McCormick, Dip Patel, Ferran Fernández Garrido, Hai Joey Tran, Hyeonho Kim, Igor Bernstein, Israel Herraiz, Jack McCluskey, Jaehyeon Kim, Jeff Kinard, Jeffrey Kinard, Joey Tran, Kenneth Knowles, Kirill Berezin, Michel Davit, Minbo Bae, Naireen Hussain, Niel Markwick, Nito Buendia, Reeba Qureshi, Reuven Lax, Robert Bradshaw, Robert Burke, Rohit Sinha, Ryan Fu, Sam Whittle, Shunping Huang, Svetak Sundhar, Udaya Chathuranga, Vitaly Terentyev, Vlado Djerek, Yi Hu, Claude van der Merwe, XQ Hu, Martin Trieu, Valentyn Tymofieiev, twosom

v2.59.0: Beam 2.59.0 release

We are happy to present the new 2.59.0 release of Beam.
This release includes both improvements and new functionality.
See the download page for this release.

For more information on changes in 2.59.0, check out the detailed release notes.

Highlights

  • Added support for setting a configureable timeout when loading a model and performing inference in the RunInference transform using with_exception_handling (#​32137)
  • Initial experimental support for using Prism with the Java and Python SDKs
    • Prism is presently targeting local testing usage, or other small scale execution.
    • For Java, use 'PrismRunner', or 'TestPrismRunner' as an argument to the --runner flag.
    • For Python, use 'PrismRunner' as an argument to the --runner flag.
    • Go already uses Prism as the default local runner.

I/Os

  • Improvements to the performance of BigqueryIO when using withPropagateSuccessfulStorageApiWrites(true) method (Java) (#​31840).
  • [Managed Iceberg] Added support for writing to partitioned tables (#​32102)
  • Update ClickHouseIO to use the latest version of the ClickHouse JDBC driver (#​32228).
  • Add ClickHouseIO dedicated User-Agent (#​32252).

New Features / Improvements

  • BigQuery endpoint can be overridden via PipelineOptions, this enables BigQuery emulators (Java) (#​28149).
  • Go SDK Minimum Go Version updated to 1.21 (#​32092).
  • [BigQueryIO] Added support for withFormatRecordOnFailureFunction() for STORAGE_WRITE_API and STORAGE_API_AT_LEAST_ONCE methods (Java) (#​31354).
  • Updated Go protobuf package to new version (Go) (#​21515).
  • Added support for setting a configureable timeout when loading a model and performing inference in the RunInference transform using with_exception_handling (#​32137)
  • Adds OrderedListState support for Java SDK via FnApi.
  • Initial support for using Prism from the Python and Java SDKs.

Bugfixes

  • Fixed incorrect service account impersonation flow for Python pipelines using BigQuery IOs (#​32030).
  • Auto-disable broken and meaningless upload_graph feature when using Dataflow Runner V2 (#​32159).
  • (Python) Upgraded google-cloud-storage to version 2.18.2 to fix a data corruption issue (#​32135).
  • (Go) Fix corruption on State API writes. (#​32245).

Known Issues

  • Prism is under active development and does not yet support all pipelines. See #​29650 for progress.
    • In the 2.59.0 release, Prism passes most runner validations tests with the exceptions of pipelines using the following features:
      OrderedListState, OnWindowExpiry (eg. GroupIntoBatches), CustomWindows, MergingWindowFns, Trigger and WindowingStrategy associated features, Bundle Finalization, Looping Timers, and some Coder related issues such as with Python combiner packing, and Java Schema transforms, and heterogenous flatten coders. Processing Time timers do not yet have real time support.
    • If your pipeline is having difficulty with the Python or Java direct runners, but runs well on Prism, please let us know.

For the most up to date list of known issues, see https://github.com/apache/beam/blob/master/CHANGES.md

List of Contributors

According to git shortlog, the following people contributed to the 2.59.0 release. Thank you to all contributors!

Ahmed Abualsaud,Ahmet Altay,Andrew Crites,atask-g,Axel Magnuson,Ayush Pandey,Bartosz Zablocki,Chamikara Jayalath,cutiepie-10,Damon,Danny McCormick,dependabot[bot],Eddie Phillips,Francis O'Hara,Hyeonho Kim,Israel Herraiz,Jack McCluskey,Jaehyeon Kim,Jan Lukavský,Jeff Kinard,Jeffrey Kinard,jonathan-lemos,jrmccluskey,Kirill Berezin,Kiruphasankaran Nataraj,lahariguduru,liferoad,lostluck,Maciej Szwaja,Manit Gupta,Mark Zitnik,martin trieu,Naireen Hussain,Prerit Chandok,Radosław Stankiewicz,Rebecca Szper,Robert Bradshaw,Robert Burke,ron-gal,Sam Whittle,Sergei Lilichenko,Shunping Huang,Svetak Sundhar,Thiago Nunes,Timothy Itodo,tvalentyn,twosom,Vatsal,Vitaly Terentyev,Vlado Djerek,Yifan Ye,Yi Hu

v2.58.1: Beam 2.58.1 release

We are happy to present the new 2.58.1 release of Beam.
This release includes both improvements and new functionality.
See the download page for this release.

New Features / Improvements

  • Fixed issue where KafkaIO Records read with ReadFromKafkaViaSDF are redistributed and may contain duplicates regardless of the configuration. This affects Java pipelines with Dataflow v2 runner and xlang pipelines reading from Kafka, (#​32196)

Known Issues

  • Large Dataflow graphs using runner v2, or pipelines explicitly enabling the upload_graph experiment, will fail at construction time (#​32159).
  • Python pipelines that run with 2.53.0-2.58.0 SDKs and read data from GCS might be affected by a data corruption issue (#​32169). The issue will be fixed in 2.59.0 (#​32135). To work around this, update the google-cloud-storage package to version 2.18.2 or newer.

For the most up to date list of known issues, see https://github.com/apache/beam/blob/master/CHANGES.md

List of Contributors

According to git shortlog, the following people contributed to the 2.58.1 release. Thank you to all contributors!

Danny McCormick

Sam Whittle

golang/go (go)

v1.23.2

v1.23.1

v1.23.0

v1.22.8

v1.22.7


Configuration

📅 Schedule: Branch creation - "before 8am on Monday" (UTC), Automerge - At any time (no schedule defined).

🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.

Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

👻 Immortal: This PR will be recreated if closed unmerged. Get config help if that's undesired.


  • If you want to rebase/retry this PR, check this box

This PR was generated by Mend Renovate. View the repository job log.

@renovate renovate bot requested a review from rjfonseca as a code owner August 19, 2024 00:59
@renovate renovate bot force-pushed the renovate/all-minor-patch branch 2 times, most recently from abca7d0 to 829c584 Compare September 11, 2024 21:39
Copy link
Contributor Author

renovate bot commented Sep 11, 2024

ℹ Artifact update notice

File name: go.mod

In order to perform the update(s) described in the table above, Renovate ran the go get command, which resulted in the following additional change(s):

  • 5 additional dependencies were updated

Details:

Package Change
golang.org/x/net v0.27.0 -> v0.29.0
golang.org/x/sys v0.22.0 -> v0.25.0
golang.org/x/text v0.16.0 -> v0.18.0
google.golang.org/genproto/googleapis/rpc v0.0.0-20240624140628-dc46fd24d27d -> v0.0.0-20240903143218-8af14fe29dc1
google.golang.org/grpc v1.64.1 -> v1.67.0

@renovate renovate bot force-pushed the renovate/all-minor-patch branch from 829c584 to b2aa7fc Compare October 1, 2024 21:21
@renovate renovate bot force-pushed the renovate/all-minor-patch branch from b2aa7fc to efbcfb7 Compare October 18, 2024 02:15
Copy link

@rjfonseca rjfonseca merged commit 0b8403f into main Oct 18, 2024
1 of 4 checks passed
@rjfonseca rjfonseca deleted the renovate/all-minor-patch branch October 18, 2024 22:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant