Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Improvement] Support removing user-specified repartition before writing when using zorder #6560

Open
3 of 4 tasks
huangxiaopingRD opened this issue Jul 24, 2024 · 0 comments · May be fixed by #6561
Open
3 of 4 tasks

Comments

@huangxiaopingRD
Copy link
Contributor

Code of Conduct

Search before asking

  • I have searched in the issues and found no similar issues.

What would you like to be improved?

In our company, users will use the repartition hint to merge small files, and the number of partitions in the repetition may not be reasonable. This will result in the zorder compression rate not being high enough, or because the number of repartition partitions is too small, resulting in a long sort time overhead. So we hope to have a configuration to remove the user-specified repartition.

How should we improve?

Add a resolve rule to remove user-specified repartition before writing when using zorder

Are you willing to submit PR?

  • Yes. I would be willing to submit a PR with guidance from the Kyuubi community to improve.
  • No. I cannot submit a PR at this time.
huangxiaopingRD added a commit to huangxiaopingRD/kyuubi that referenced this issue Jul 24, 2024
huangxiaopingRD added a commit to huangxiaopingRD/kyuubi that referenced this issue Jul 24, 2024
huangxiaopingRD added a commit to huangxiaopingRD/kyuubi that referenced this issue Jul 24, 2024
huangxiaopingRD added a commit to huangxiaopingRD/kyuubi that referenced this issue Jul 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
1 participant