Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

*: only enable new string serdes format in exchange when MppVersion >= MppVersionV3 #9759

Merged
merged 48 commits into from
Jan 10, 2025

Conversation

JinheLin
Copy link
Contributor

@JinheLin JinheLin commented Jan 2, 2025

What problem does this PR solve?

Issue Number: close #9673

Problem Summary:

  • Before this PR, during the rolling upgrade process, data of exchange string type may be incompatible.

What is changed and how it works?

  • tidb will send an MppVersion field to tiflash.

  • When rolling upgrade, tiflash will be upgraded before tidb.

  • During the process of upgrading tiflash, tidb has not been upgraded yet, so all mpp versions received are MppVersionsV2, tiflash still using the old format to exchange data.

  • When upgrading tidb, the new tidb uses MppVersionsV3 to send requests, and tiflash starts using the new format to exchange data (at this point, all tiflash has been upgraded, so there will be no compatibility issues).

  • PR of tidb is copr: add MppVersionV3 tidb#58652.

    • Merge this PR before merging the PR of tidb.
1. Passing mpp_version to CHBlockChunkCodec and CHBlockChunkCodecV1 for encoding: if mpp_version <= MppVersion2, use the legacy format of string.

2. When decoding, CHBlockChunkCodec and CHBlockChunkCodecV1 respect to the type name from encoder.

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
    • In a cluster with two tiflash servers, upgrade one tiflash, and executing queries which exchanging string data.
  • No code

Side effects

  • Performance regression: Consumes more CPU
  • Performance regression: Consumes more Memory
  • Breaking backward compatibility

Documentation

  • Affects user behaviors
  • Contains syntax changes
  • Contains variable changes
  • Contains experimental features
  • Changes MySQL compatibility

Release note

None

@ti-chi-bot ti-chi-bot bot added release-note-none Denotes a PR that doesn't merit a release note. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Jan 2, 2025
@JinheLin
Copy link
Contributor Author

JinheLin commented Jan 2, 2025

PR of tidb is pingcap/tidb#58652.

Merge this PR before merging the PR of tidb.

@JinheLin JinheLin mentioned this pull request Jan 2, 2025
13 tasks
@JinheLin JinheLin changed the title *: only enable new string serdes format when MppVersion >= MppVersionV3 WIP: *: only enable new string serdes format when MppVersion >= MppVersionV3 Jan 2, 2025
@ti-chi-bot ti-chi-bot bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jan 2, 2025
@ti-chi-bot ti-chi-bot bot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Jan 3, 2025
@JaySon-Huang
Copy link
Contributor

@windtalker PTAL

@ti-chi-bot ti-chi-bot bot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Jan 8, 2025
@ti-chi-bot ti-chi-bot bot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Jan 8, 2025
@JinheLin
Copy link
Contributor Author

JinheLin commented Jan 9, 2025

/retest

@JinheLin JinheLin changed the title *: only enable new string serdes format when MppVersion >= MppVersionV3 *: only enable new string serdes format in exchange when MppVersion >= MppVersionV3 Jan 9, 2025
dbms/src/Flash/Coprocessor/CHBlockChunkCodecV1.h Outdated Show resolved Hide resolved
case MppVersion::MppVersionV3:
return MPPDataPacketVersion::MPPDataPacketV2;
default:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why make MPPDataPacketV1 as the default value? If there is MppVersionV4, will it use MPPDataPacketV1?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I used to follow the code of

auto data_codec_version = mpp_version == MppVersionV0 ? MPPDataPacketV0 : MPPDataPacketV1;

Its default value is MPPDataPacketV1.

But I have just updated the code to make MPPDataPacketV2 as the default value now.

Copy link
Contributor

@windtalker windtalker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

Copy link
Contributor

ti-chi-bot bot commented Jan 10, 2025

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: JaySon-Huang, windtalker

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:
  • OWNERS [JaySon-Huang,windtalker]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ti-chi-bot ti-chi-bot bot added lgtm and removed needs-1-more-lgtm Indicates a PR needs 1 more LGTM. labels Jan 10, 2025
Copy link
Contributor

ti-chi-bot bot commented Jan 10, 2025

[LGTM Timeline notifier]

Timeline:

  • 2025-01-07 09:23:00.248966612 +0000 UTC m=+259123.537798316: ☑️ agreed by JaySon-Huang.
  • 2025-01-10 07:52:24.334846886 +0000 UTC m=+512887.623678587: ☑️ agreed by windtalker.

@ti-chi-bot ti-chi-bot bot merged commit b4e8068 into pingcap:master Jan 10, 2025
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved lgtm release-note-none Denotes a PR that doesn't merit a release note. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Enhance deserialization performance of short string.
3 participants