tests: add `integration2` for tools integration tests #56486

bb7133 · 2024-10-08T21:46:03Z

What problem does this PR solve?

Issue Number: ref #56556

Problem Summary:

Please check #56556.

What changed and how does it work?

This is the first try of #56556. Rather than updating tests/integrationtest, currently I added integrationtest2 so that the existing test scripts/test cases could be stable enough.

In the future, I will try to merge integrationtest2 into integrationtest when this is stable and mature enough.

Check List

Tests

Unit test
Integration test
Manual test (add detailed scripts or steps below)
No need to test
- I checked and no code files have been changed.

Side effects

NA

Documentation

NA

Release note

Please refer to Release Notes Language Style Guide to write a quality release note.

None

codecov · 2024-10-08T22:09:02Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 56.5504%. Comparing base (df821b9) to head (e6c8220).
Report is 40 commits behind head on master.

Additional details and impacted files

@@                Coverage Diff                @@
##             master     #56486         +/-   ##
=================================================
- Coverage   73.2357%   56.5504%   -16.6853%     
=================================================
  Files          1626       1752        +126     
  Lines        448692     636412     +187720     
=================================================
+ Hits         328603     359894      +31291     
- Misses        99977     252351     +152374     
- Partials      20112      24167       +4055

Flag	Coverage Δ
integration	`36.7960% <ø> (?)`
unit	`72.4625% <ø> (+0.0224%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components	Coverage Δ
dumpling	`52.9478% <ø> (ø)`
parser	`∅ <ø> (∅)`
br	`51.0975% <ø> (+5.6261%)`	⬆️

bb7133 · 2024-10-09T16:55:40Z

/retest

bb7133 · 2024-10-09T17:34:09Z

/retest

Defined2014 · 2024-10-11T02:48:07Z

tests/integrationtest2/r/br_integration.result

+br_integration	t1	_tidb_rowid	101	_TIDB_ROWID
+br_integration	t1	_tidb_rowid	1	AUTO_INCREMENT
+Destination	Size	BackupTS	Queue Time	Execution Time
+/tmp/t1_550838d3-0672-45d9-8e57-894755fe0478	3695	453145470351704099	2024-10-11 10:13:06	2024-10-11 10:13:06


Should not print this msg, because it has some unstable result

I used 'enableResultLog' to control it, now it's gone. Thanks!

pingcap/mysql-tester#133

Defined2014 · 2024-10-11T02:48:43Z

tests/integrationtest2/r/br_integration.result

+Destination	Size	BackupTS	Queue Time	Execution Time
+/tmp/t1_550838d3-0672-45d9-8e57-894755fe0478	3695	453145470351704099	2024-10-11 10:13:06	2024-10-11 10:13:06
+Destination	Size	BackupTS	Cluster TS	Queue Time	Execution Time
+/tmp/t1_550838d3-0672-45d9-8e57-894755fe0478	3695	453145470351704099	453145471282315315	2024-10-11 10:13:10	2024-10-11 10:13:10


Defined2014 · 2024-10-11T02:51:04Z

tests/integrationtest2/t/dumpling_import_integration.test

+INSERT INTO t2(a, b) VALUES (1, 1), (2, 2), (3, 3);
+SELECT (c & 31) as inc FROM t2 ORDER BY inc;
+
+--dump_and_import t2 AS tt2


Split it to two commands dump and import is better.

I want it to provide an easiest way to performance 'backup' and 'restore' without any detail(for example, the syntax of BACKUP/RESTORE, the arguments of BR binary, local path of backup data...).

For integration tests with more details, many other ways like our internal testing framework can be used.

But still, maybe we can provide two new commands for dump and import.

Defined2014 · 2024-10-11T02:51:32Z

tests/integrationtest2/t/br_integration.test

+INSERT INTO t1 (b) VALUES (1), (2), (3);
+SHOW TABLE t1 NEXT_ROW_ID;
+
+--backup_and_restore t1 AS tt1


Split it to two commands backup and restore is better.

Ditto as dump and import.

Defined2014 · 2024-10-11T02:56:03Z

tests/integrationtest2/download_integration_test_binaries.sh

@@ -0,0 +1,91 @@
+#! /usr/bin/env bash


This file seems a little bit unreasonable here.

If we want test it on local env, maybe tiup is a better choose. If we want test it on ci env, maybe add it to ci repo is better.

It is a copy of https://github.com/pingcap/tidb/blob/master/br/tests/download_integration_test_binaries.sh

This is used mainly by CI, but I want it available for local env, too.

Also, this script guarantees we always use latest builds for all components, which is hard to archive for TiUP nightly.

But this shell always downloads centos7 image which can't work under MacOS env.

Using tiup means that users can manually set up the test environment through tiup.

I'll refine it later to make it work in MacOS.

And better add third_bin path into .gitignore file

ti-chi-bot · 2024-10-11T06:59:29Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: Defined2014, hawkingrei

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [Defined2014,hawkingrei]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

ti-chi-bot · 2024-10-11T06:59:32Z

[LGTM Timeline notifier]

Timeline:

2024-10-11 04:51:47.205026524 +0000 UTC m=+1194462.625239536: ☑️ agreed by hawkingrei.
2024-10-11 06:59:31.690104219 +0000 UTC m=+1202127.110317228: ☑️ agreed by Defined2014.

bb7133 · 2024-10-11T16:44:48Z

/retest

bb7133 · 2024-10-11T17:30:49Z

/retest

ti-chi-bot bot added do-not-merge/needs-linked-issue do-not-merge/needs-tests-checked release-note-none Denotes a PR that doesn't merit a release note. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Oct 8, 2024

tests/integrationtest: update script

b334ea5

bb7133 force-pushed the bb7133/it branch from 8a0a0ec to b334ea5 Compare October 9, 2024 03:14

bb7133 changed the title ~~(draft) tests/integrationtest: update script~~ (WIP) tests/integrationtest: update script Oct 9, 2024

ti-chi-bot bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Oct 9, 2024

ti-chi-bot bot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Oct 10, 2024

bb7133 force-pushed the bb7133/it branch from cd65f2e to fc2abe4 Compare October 11, 2024 02:42

bb7133 changed the title ~~(WIP) tests/integrationtest: update script~~ (WIP) tests: add integration2 for tools integration tests Oct 11, 2024

bb7133 changed the title ~~(WIP) tests: add integration2 for tools integration tests~~ tests: add integration2 for tools integration tests Oct 11, 2024

ti-chi-bot bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Oct 11, 2024

Defined2014 reviewed Oct 11, 2024

View reviewed changes

ti-chi-bot bot removed do-not-merge/needs-linked-issue do-not-merge/needs-tests-checked labels Oct 11, 2024

hawkingrei approved these changes Oct 11, 2024

View reviewed changes

ti-chi-bot bot added approved needs-1-more-lgtm Indicates a PR needs 1 more LGTM. labels Oct 11, 2024

introduce integrationtest2

3e834e3

bb7133 force-pushed the bb7133/it branch from fc2abe4 to 3e834e3 Compare October 11, 2024 05:56

Defined2014 approved these changes Oct 11, 2024

View reviewed changes

ti-chi-bot bot added lgtm and removed needs-1-more-lgtm Indicates a PR needs 1 more LGTM. labels Oct 11, 2024

add license header

e6c8220

ti-chi-bot bot merged commit 2dc1b98 into pingcap:master Oct 11, 2024
23 checks passed

bb7133 mentioned this pull request Oct 24, 2024

tests/integrationtest2: fix bugs in the tests #56815

Merged

5 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

tests: add `integration2` for tools integration tests #56486

tests: add `integration2` for tools integration tests #56486

bb7133 commented Oct 8, 2024 •

edited

Loading

codecov bot commented Oct 8, 2024 •

edited

Loading

bb7133 commented Oct 9, 2024

bb7133 commented Oct 9, 2024

Defined2014 Oct 11, 2024

bb7133 Oct 11, 2024

Defined2014 Oct 11, 2024

Defined2014 Oct 11, 2024

bb7133 Oct 11, 2024

Defined2014 Oct 11, 2024

bb7133 Oct 11, 2024

Defined2014 Oct 11, 2024

bb7133 Oct 11, 2024

Defined2014 Oct 11, 2024

bb7133 Oct 11, 2024

Defined2014 Oct 11, 2024

ti-chi-bot bot commented Oct 11, 2024

ti-chi-bot bot commented Oct 11, 2024

bb7133 commented Oct 11, 2024

bb7133 commented Oct 11, 2024

tests: add integration2 for tools integration tests #56486

tests: add integration2 for tools integration tests #56486

Conversation

bb7133 commented Oct 8, 2024 • edited Loading

What problem does this PR solve?

What changed and how does it work?

Check List

Release note

codecov bot commented Oct 8, 2024 • edited Loading

Codecov Report

bb7133 commented Oct 9, 2024

bb7133 commented Oct 9, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ti-chi-bot bot commented Oct 11, 2024

ti-chi-bot bot commented Oct 11, 2024

[LGTM Timeline notifier]

bb7133 commented Oct 11, 2024

bb7133 commented Oct 11, 2024

tests: add `integration2` for tools integration tests #56486

tests: add `integration2` for tools integration tests #56486

bb7133 commented Oct 8, 2024 •

edited

Loading

codecov bot commented Oct 8, 2024 •

edited

Loading