Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update schema_migration_tools.md #3093

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

d-ganchar
Copy link

Summary

Python tools, link to Thedus

python tools, link to thedus
@CLAassistant
Copy link

CLAassistant commented Jan 15, 2025

CLA assistant check
All committers have signed the CLA.

Copy link
Member

@SpencerTorres SpencerTorres left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for making a migration tool for ClickHouse!

I'm not sure the official policy for adding new tools to this list, but I would like to see some more documentation on how to use this project. It looks like the first commit/release was only a couple days ago. For the sake of security and QA, maybe this should wait a bit until the project matures.

I see you've also contributed to Houseplant. What is the difference between these two tools? I see Houseplant is also new (about a month old), but seems to have some decent documentation available as well as some downloads/stars showing people using/contributing to it.

Could you provide some more info about your project? Thanks!

@d-ganchar
Copy link
Author

d-ganchar commented Jan 16, 2025

Thanks for making a migration tool for ClickHouse!

I'm not sure the official policy for adding new tools to this list, but I would like to see some more documentation on how to use this project. It looks like the first commit/release was only a couple days ago. For the sake of security and QA, maybe this should wait a bit until the project matures.

I see you've also contributed to Houseplant. What is the difference between these two tools? I see Houseplant is also new (about a month old), but seems to have some decent documentation available as well as some downloads/stars showing people using/contributing to it.

Could you provide some more info about your project? Thanks!

Hey!

Thank you for your response.
The reasons why I created another package:

  • Houseplant may have stars and downloads, but if you follow the documentation, it doesn’t work ( 33, 34, 35 )
  • Houseplant uses mocks for interface tests. Thedus runs tests on the built and installed package, ensuring that the interface works fine.
  • Houseplant uses the YAML format for migrations. This is not always a good solution for writing SQL. Thedus uses Python for executing migrations, which provides more flexibility and capabilities. See demo files.
  • Houseplant relies on ReplicatedReplacingMergeTree, FINAL, and OPTIMIZE TABLE to store state. Thedus has no state - it keeps a log of executed migrations and runs.
  • Thedus allows you to see if a migration was skipped due to environment settings. With Houseplant, you will see that the migration was executed, but in reality, no changes were made to the database.

To be honest, I wouldn’t add this to the documentation without cli tests verifying logs and database state(using multiple Py versions)
I think this could be really useful for someone. Moreover, I'll use Thedus in production myself when finish with requirements, cluster design and setup.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants