diff --git a/data/content/test_splitting_env.schema.yaml b/data/content/test_splitting_env.schema.yaml new file mode 100644 index 0000000000..1b94a0606f --- /dev/null +++ b/data/content/test_splitting_env.schema.yaml @@ -0,0 +1,53 @@ +type: object +properties: + predefined: + type: array + items: + type: object + properties: + name: + type: string + desc: + type: array + items: + type: string + required: + - name + - desc + mandatory: + type: array + items: + type: object + properties: + name: + type: string + desc: + type: array + items: + type: string + required: + - name + - desc + optional: + type: array + items: + type: object + properties: + name: + type: string + desc: + type: array + items: + type: string + default: + type: + - string + - integer + - boolean + required: + - name + - desc +required: + - predefined + - mandatory + - optional diff --git a/data/content/test_splitting_env.yaml b/data/content/test_splitting_env.yaml new file mode 100644 index 0000000000..48cd7ed914 --- /dev/null +++ b/data/content/test_splitting_env.yaml @@ -0,0 +1,57 @@ +predefined: + - name: BUILDKITE_BUILD_ID + desc: + - The UUID of the pipeline build. Test Splitter uses this UUID along with `BUILDKITE_STEP_ID` to uniquely identify the test plan. + - name: BUILDKITE_JOB_ID + desc: + - The UUID of the job in the pipeline's build. + - name: BUILDKITE_ORGANIZATION_SLUG + desc: + - The slug of your Buildkite organization. + - name: BUILDKITE_PARALLEL_JOB + desc: + - The index number of a parallel job created from a parallel build step. + - Ensure you configure `parallelism` in your pipeline definition. Learn more about parallel build steps in [Concurrency and parallelism](https://buildkite.com/docs/pipelines/controlling-concurrency#concurrency-and-parallelism). + - name: BUILDKITE_PARALLEL_JOB_COUNT + desc: + - The total number of parallel jobs created from a parallel build step. + - Ensure you configure `parallelism` in your pipeline definition. Learn more about parallel build steps in [Concurrency and parallelism](https://buildkite.com/docs/pipelines/controlling-concurrency#concurrency-and-parallelism). + - name: BUILDKITE_STEP_ID + desc: + - The UUID of the step group in the pipeline build. Test Splitter uses this UUID along with `BUILDKITE_BUILD_ID` to uniquely identify the test plan. + +mandatory: + - name: BUILDKITE_SPLITTER_API_ACCESS_TOKEN + desc: + - Buildkite API access token with `read_suites`, `read_test_plan`, and `write_test_plan` scopes. You can create an [API access token](https://buildkite.com/user/api-access-tokens) from **Personal Settings** > **API Access Tokens** in the Buildkite interface. + - name: BUILDKITE_SPLITTER_SUITE_SLUG + desc: + - The slug of your Buildkite Test Analytics test suite. You can find the suite slug in the url for your test suite. + - "For example, the slug for the url: `https://buildkite.com/organizations/my-organization/analytics/suites/my-suite` is `my-suite`." + +optional: + - name: BUILDKITE_SPLITTER_DEBUG_ENABLED + default: false + desc: + - A flag to enable more verbose logging. + - name: BUILDKITE_SPLITTER_RETRY_COUNT + default: 0 + desc: + - The number of retries permitted. Test Splitter runs the test command defined in `BUILDKITE_SPLITTER_TEST_CMD`, and retries only the failing tests for a maximum of `BUILDKITE_SPLITTER_RETRY_COUNT` times. For RSpec, the Test Splitter runs `BUILDKITE_SPLITTER_TEST_CMD` with `--only-failures` as the retry command. + - name: BUILDKITE_SPLITTER_SPLIT_BY_EXAMPLE + default: false + desc: + - A flag to enable split by example. When this option is `true`, the Test Splitter will split the execution of slow test files over multiple partitions. + - name: BUILDKITE_SPLITTER_TEST_CMD + default: bundle exec rspec {{testExamples}} + desc: + - The test command to run your tests. The Test Splitter will replace and populate the `{{testExamples}}` placeholder with the test plan. + - name: BUILDKITE_SPLITTER_TEST_FILE_EXCLUDE_PATTERN + desc: + - The glob pattern to exclude certain test files or directories. The exclusion will be applied after discovering the test files using a pattern configured with `BUILDKITE_SPLITTER_TEST_FILE_PATTERN`. + - _This option accepts the pattern syntax supported by the [zzglob](https://github.com/DrJosh9000/zzglob?tab=readme-ov-file#pattern-syntax) library._ + - name: BUILDKITE_SPLITTER_TEST_FILE_PATTERN + default: spec/**/*_spec.rb + desc: + - The glob pattern to discover test files. You can exclude certain test files or directories from the discovered test files using a pattern that can be configured with `BUILDKITE_SPLITTER_TEST_FILE_EXCLUDE_PATTERN`. + - _This option accepts the pattern syntax supported by the [zzglob](https://github.com/DrJosh9000/zzglob?tab=readme-ov-file#pattern-syntax) library._ diff --git a/pages/test_analytics/test_splitting.md b/pages/test_analytics/test_splitting.md index 82afe827aa..da78ad0ac3 100644 --- a/pages/test_analytics/test_splitting.md +++ b/pages/test_analytics/test_splitting.md @@ -24,14 +24,22 @@ The Test Splitter tool uses a number of [predefined](#predefined-environment-var By default, the following predefined environment variables are available to your testing environment and do not need any further configuration. If, however, you use Docker or some other type of containerization tool to run your tests, and you wish to use these predefined environment variables in these tests, you may need to expose these environment variables to your containers. -| Environment Variable | Description| -| -------------------- | ----------- | -| `BUILDKITE_BUILD_ID` | The UUID of the pipeline build. Test Splitter uses this UUID along with `BUILDKITE_STEP_ID` to uniquely identify the test plan. | -| `BUILDKITE_JOB_ID` | The UUID of the job in the pipeline's build. | -| `BUILDKITE_ORGANIZATION_SLUG` | The slug of your Buildkite organization. | -| `BUILDKITE_PARALLEL_JOB` | The index number of a parallel job created from a parallel build step.
Ensure you configure `parallelism` in your pipeline definition. Learn more about parallel build steps in [Concurrency and parallelism](https://buildkite.com/docs/pipelines/controlling-concurrency#concurrency-and-parallelism). | -| `BUILDKITE_PARALLEL_JOB_COUNT` | The total number of parallel jobs created from a parallel build step.
Ensure you configure `parallelism` in your pipeline definition. Learn more about parallel build steps in [Concurrency and parallelism](https://buildkite.com/docs/pipelines/controlling-concurrency#concurrency-and-parallelism). | -| `BUILDKITE_STEP_ID` | The UUID of the step group in the pipeline build. Test Splitter uses this UUID along with `BUILDKITE_BUILD_ID` to uniquely identify the test plan. + + + <% TEST_SPLITTING_ENV['predefined'].each do |var| %> + + + + + <% end %> + +
+ <%= var['name'] %> # + + <% var['desc'].each do |d| %> + <%= render_markdown(text: d) %> + <% end %> +
@@ -39,10 +47,22 @@ By default, the following predefined environment variables are available to your The following mandatory environment variables must be set. -| Environment Variable | Description | -| -------------------- | ----------- | -| `BUILDKITE_SPLITTER_API_ACCESS_TOKEN ` | Buildkite API access token with `read_suites`, `read_test_plan`, and `write_test_plan` scopes. You can create an [API access token](https://buildkite.com/user/api-access-tokens) from **Personal Settings** > **API Access Tokens** in the Buildkite interface. | -| `BUILDKITE_SPLITTER_SUITE_SLUG` | The slug of your Buildkite Test Analytics test suite. You can find the suite slug in the url for your test suite. For example, the slug for the url: `https://buildkite.com/organizations/my-organization/analytics/suites/my-suite` is `my-suite` | + + + <% TEST_SPLITTING_ENV['mandatory'].each do |var| %> + + + + + <% end %> + +
+ <%= var['name'] %> # + + <% var['desc'].each do |d| %> + <%= render_markdown(text: d) %> + <% end %> +
@@ -50,14 +70,26 @@ The following mandatory environment variables must be set. The following optional environment variables can also be used to configure the Test Splitter's behavior. -| Environment Variable | Default Value | Description | -| ---- | ---- | ----------- | -| `BUILDKITE_SPLITTER_DEBUG_ENABLED` | `false` | A flag to enable more verbose logging. | -| `BUILDKITE_SPLITTER_RETRY_COUNT` | `0` | The number of retries permitted. Test Splitter runs the test command defined in `BUILDKITE_SPLITTER_TEST_CMD`, and retries only the failing tests for a maximum of `BUILDKITE_SPLITTER_RETRY_COUNT` times. For RSpec, the Test Splitter runs `BUILDKITE_SPLITTER_TEST_CMD` with `--only-failures` as the retry command. | -| `BUILDKITE_SPLITTER_SPLIT_BY_EXAMPLE` | `false` | A flag to enable split by example. When this option is `true`, the Test Splitter will split the execution of slow test files over multiple partitions. | -| `BUILDKITE_SPLITTER_TEST_CMD` | `bundle exec rspec {{testExamples}}` | The test command to run your tests. The Test Splitter will replace and populate the `{{testExamples}}` placeholder with the test plan. | -| `BUILDKITE_SPLITTER_TEST_FILE_EXCLUDE_PATTERN` | - | The glob pattern to exclude certain test files or directories. The exclusion will be applied after discovering the test files using a pattern configured with `BUILDKITE_SPLITTER_TEST_FILE_PATTERN`.
_This option accepts the pattern syntax supported by the [zzglob](https://github.com/DrJosh9000/zzglob?tab=readme-ov-file#pattern-syntax) library._ | -| `BUILDKITE_SPLITTER_TEST_FILE_PATTERN` | `spec/**/*_spec.rb` | The glob pattern to discover test files. You can exclude certain test files or directories from the discovered test files using a pattern that can be configured with `BUILDKITE_SPLITTER_TEST_FILE_EXCLUDE_PATTERN`.
_This option accepts the pattern syntax supported by the [zzglob](https://github.com/DrJosh9000/zzglob?tab=readme-ov-file#pattern-syntax) library._ | + + + <% TEST_SPLITTING_ENV['optional'].each do |var| %> + + + + + <% end %> + +
+ <%= var['name'] %> # +

+ Default: + <%= var['default'] %> +

+ 		+ <% var['desc'].each do |d| %> + <%= render_markdown(text: d) %> + <% end %> +
### Update the pipeline step