0.8.1

QIT CLI 0.8.1 Release Notes

What’s New

Improved Orchestration in Custom Tests

• Shared Setup/Teardown in Bootstrap Mode
  Additional plugins designated as bootstrap now properly execute their shared setup and teardown steps even if their test suites are skipped.

• Corrected Path in dependencies.json
  Fixed a typo referencing qit/dependencies.json. The correct file path is now bootstrap/dependencies.json, ensuring dependencies load as intended.

• Automatic WooCommerce Dependency Handling
  QIT now ensures that if WooCommerce is a required dependency for your plugin (as determined by WooCommerce.com product metadata - soon to be updated to use your plugin’s "Requires Plugins" header), it is automatically included.

• Action Overrides for Additional Plugins
  CLI overrides specified at runtime (e.g., -p plugin:action or --plugin plugin:action) now reliably take precedence over qit.yml settings.

• Enhanced Parameter Handling (--tunnel, --dependencies)
  QIT now lists all possible values for these parameters, making it easier to discover valid options.

Documentation Overhaul

Complete Documentation Rewrite

The QIT documentation at qit.woo.com has been entirely rewritten from scratch. The new documentation provides clearer explanations, step-by-step guides, and updated references to reflect recent orchestration improvements and configuration refinements.

Support and Feedback

If you have any questions, encounter issues, or have suggestions, please visit our GitHub Issues Page. Your feedback helps us continually improve QIT.

What's Changed

• Fix Custom Tests Orchestration: Improve Plugin Actions, Dependencies, and Configuration Handling by @Luc45 in #227

Full Changelog: 0.8.0...0.8.1

0.8.0

QIT CLI 0.8.0 Release Notes

What’s New

Enhanced Orchestration in Custom End-to-End (E2E) Tests

• Optimized Database Lifecycle Management
  Improved the database (DB) lifecycle handling during Custom E2E tests, particularly for compatibility testing involving multiple plugins.

• Streamlined Lifecycle Steps
  The updated lifecycle now follows these steps:

  1. Shared Setup Execution
     Execute setups that are common to all plugins.
  2. Database Export
     Export the current state of the DB.
  3. Per-Plugin Operations
     For each plugin, perform the following:
     • DB Import
       Import the exported DB state.
     • Plugin-Specific Setup
       Execute setups unique to the plugin.
     • Run Tests
       Conduct the plugin-specific tests.
     • Plugin-Specific Teardown
       Execute teardowns unique to the plugin.
  4. Shared Teardown Execution
     Execute teardowns that are common to all plugins.

• Flexible State Management
  This update allows you to:

  • Share State Across Plugins: Define shared states, such as establishing connections to external services or removing onboarding wizards during the shared setup phase.
  • Isolate Plugin-Specific Configurations: Customize setups unique to each plugin, like installing and activating a custom theme required by a specific plugin.

Documentation

For a detailed understanding of the new lifecycle management, please refer to our Understanding the Lifecycle documentation.

Support and Feedback

If you have any questions, encounter issues, or have suggestions, please visit our GitHub Issues Page. Your feedback is invaluable and helps us continuously enhance the QIT CLI tool.

What's Changed

• 9.4.0 snapshot changes by @Luc45 in #218
• Fix "docker compose not found" by @Luc45 in #219
• Snapshots for Syncing 9.5.0-beta.1 by @Luc45 in #223
• 9.5.0-beta.2 by @Luc45 in #224
• Improve test orchestration for more powerful Compatibility Tests between plugins by @Luc45 in #222

Full Changelog: 0.7.1...0.8.0

0.7.1

QIT CLI 0.7.1 Release Notes

What's New

Environment variables in Custom Tests

• Adds --env and --env_file to run:e2e, run:activation and env:up commands.
• This can be used to pass secrets or configuration that you don't want to hardcode in your tests.

Documentation

Check the documentation for Environment Variables.

For any queries, issues, or suggestions, feel free to visit our Github page at https://github.com/woocommerce/qit-cli/issues. We appreciate your feedback as it helps us to continuously improve our tool.

What's Changed

• Custom Tests: Allow to pass env vars by @Luc45 in #217

Full Changelog: 0.7.0...0.7.1

0.7.0

QIT CLI 0.7.0 Release Notes

We are excited to announce the release of QIT CLI version 0.7.0! This update introduces tunneling support, along with UX/developer experience improvements.

What's New

Tunneling Support

• Easy Configuration: Use our built-in tunneling methods for easy tunneling.
• Improved Testing: Tunneling support enables you to test your extensions with a live URL.
• Multiple Tunneling Methods: Choose the tunneling method that works best for you.

Documentation

You can find the documentation for Tunneling here.

For any queries, issues, or suggestions, feel free to visit our Github page at https://github.com/woocommerce/qit-cli/issues. We appreciate your feedback as it helps us to continuously improve our tool.

What's Changed

• Disable Mac test, add Validation workflow by @Luc45 in #207
• Sync 9.3.2 e2e by @zhongruige in #208
• 9.4.0-beta.2 snapshots by @Luc45 in #211
• Fix self-tests for the Custom Tests by @Luc45 in #212
• Allow to use WooCommerce as the SUT for Activation Test by @Luc45 in #213
• Docker pull only once a day by @Luc45 in #215
• Prioritize extension slugs over local directories and improve Docker volume handling by @Luc45 in #214
• Overwrite scaffolded tests with safety checks by @Luc45 in #216
• Add Tunnel support to enable sites to have a live URL and HTTPs by @Luc45 in #209

Full Changelog: 0.6.0...0.7.0

0.6.0

QIT CLI 0.6.0 Release Notes

We are excited to announce the release of QIT CLI version 0.6.0! This update introduces the Activation Test 2.0, our first smart generic test, along with several enhancements designed to improve your testing workflow.

This is the first test type built on top of our Custom Tests platform!

What's New

Activation Test 2.0

• Improved Coverage: Now visits pages added by your extensions, catching errors that previous tests might have missed.
• Smart Generic Tests: Automatically detects and tests essential functionalities without custom scripts.
• Fast Execution: Completes tests in 50 seconds to 3 minutes, depending on your setup.
• Local Testing: Run tests directly on your machine or CI environment for immediate feedback.
• Error Detection: Captures PHP and JavaScript notices, warnings, and errors, including those on custom plugin pages, as well as Database errors.
• Configurable Environment:
  • PHP Versions: Test with PHP 7.2 up to 8.4 RC 3.
  • WooCommerce & WordPress Versions: Choose any version, including development builds.
• Plugin Dependencies: Activate additional plugins by their slugs to simulate real-world scenarios.
• Notifications: Use --notify to receive immediate alerts when issues are detected.
• Centralized Results: All test results are collected in QIT for easy access and analysis.
• Built on Custom Tests: Leverages existing frameworks for efficiency and easier maintenance.

Backwards Compatibility

• Seamless Integration: Designed to work with existing CI workflows without disruptions.
• Minimal Adjustments: No significant changes required for extensions currently using QIT.

Documentation

You can find the documentation for the Activation Test type here.

For any queries, issues, or suggestions, feel free to visit our Github page at https://github.com/woocommerce/qit-cli/issues. We appreciate your feedback as it helps us to continuously improve our tool.

What's Changed

• Fix --source option by @Luc45 in #196
• Sync 9.2.2 by @zhongruige in #197
• Add self-tests for the Validation test by @Luc45 in #198
• Activation as a custom test by @Luc45 in #200
• 9.3.0-beta.1 sync snapshots by @Luc45 in #201
• Activation/custom test improvements by @Luc45 in #202
• Update snapshots by @Luc45 in #204
• Fix old doc link by @zhongruige in #205
• Update snapshots for validation tests by @Luc45 in #206
• Activation refactor backwards compatibility by @Luc45 in #203
• Support qm logs by @MrJnrman in #195

Full Changelog: 0.5.4...0.6.0

0.5.4

QIT CLI 0.5.4 Release Notes

• Adds support for a QIT_RESULTS_DIR env var which specifies where to store the e2e results when running run:e2e command.

For any queries, issues, or suggestions, feel free to visit our Github page at https://github.com/woocommerce/qit-cli/issues. We appreciate your feedback as it helps us to continuously improve our tool.

What's Changed

• Update snapshots for Security Tests by @Luc45 in #179
• 9.0.0 sync by @Luc45 in #181
• Snapshot updates for WC 9.0.1 by @zhongruige in #182
• Sync e2e snapshots for WC 9.1.0 and 9.0.2 tests by @zhongruige in #183
• Sync WC 9.1.0 and 9.1.1 by @zhongruige in #184
• Wait 10 seconds between each self-test test by @Luc45 in #185
• Configurable poll intervall to prevent 429 by @Luc45 in #186
• Temporary test environment tweaks by @Luc45 in #190
• Sync 9.2.0-beta.1 by @Luc45 in #191
• Fix typo in docker compose version by @Luc45 in #192
• Set Result Dir using ENV vat by @MrJnrman in #194

Full Changelog: 0.5.3...0.5.4

0.5.3

QIT CLI 0.5.3 Release Notes

• This is a follow-up to the 0.5.2 release. Fixes an issue with using "--woo" parameter that would cause a Custom E2E test files to not be found.

For any queries, issues, or suggestions, feel free to visit our Github page at https://github.com/woocommerce/qit-cli/issues. We appreciate your feedback as it helps us to continuously improve our tool.

What's Changed

• Use Input instead of argv, activate Woo instead of bootstrapping when using "--woo" parameter by @Luc45 in #178

Full Changelog: 0.5.2...0.5.3

0.5.2

QIT CLI 0.5.2 Release Notes

• Fix an error that can happen when using both "--woo" and "--plugin=woocommerce" at the same time when running a Custom E2E Test.
• Many fixes and improvements

For any queries, issues, or suggestions, feel free to visit our Github page at https://github.com/woocommerce/qit-cli/issues. We appreciate your feedback as it helps us to continuously improve our tool.

What's Changed

• 9.0.0-RC1 snapshots with skips by @Luc45 in #172
• Custom tests snapshots by @Luc45 in #173
• Increase mass test timeout by @Luc45 in #175
• Sync Woo E2E 8.9.2 snapshots by @Luc45 in #174
• Update snapshots by @Luc45 in #176
• Fix issue with "--plugins=woocommerce" and "--woo" parameter by @Luc45 in #177

Full Changelog: 0.5.1...0.5.2

0.5.1

QIT CLI 0.5.1 Release Notes

• Register a test run in QIT for local custom test runs, with an embedded Allure Report with support for screenshots and videos of failures in E2E. You can share a link to that test report with others.
• Many fixes and improvements

For any queries, issues, or suggestions, feel free to visit our Github page at https://github.com/woocommerce/qit-cli/issues. We appreciate your feedback as it helps us to continuously improve our tool.

What's Changed

• Add woo:validate-zip <path> command by @rcstr in #163
• Upload test artifacts to debug self-tests flakiness by @Luc45 in #167
• Sync WC 8.9.0 by @zhongruige in #168
• 9.0.0-beta.2 sync by @Luc45 in #171
• Send custom tests results to Manager by @Luc45 in #170

New Contributors

• @rcstr made their first contribution in #163

Full Changelog: 0.5.0...0.5.1

0.5.0

QIT CLI 0.5.0 Release Notes

• New Feature: Local Test Environments: Added the ability to create Local Test Environments with a single command. These environments are disposable and fully customizable, making automated testing much simpler. (Read More: Test Environment Documentation)
• New Feature: Custom Tests: Added support for Custom E2E tests, allowing developers to write, share, and run personalized automated tests easily. (Read more: Custom Tests Documentation)
• Many fixes and improvements

For any queries, issues, or suggestions, feel free to visit our Github page at https://github.com/woocommerce/qit-cli/issues. We appreciate your feedback as it helps us to continuously improve our tool.

What's Changed

• Use Composer CA Bundle instead of custom implementation by @Luc45 in #139
• When doing a retry, throw some randomness into the mix by @Luc45 in #140
• Hide deprecated options (except on CI) by @Luc45 in #141
• Xdebug on QIT tests by @Luc45 in #142
• PHPCS Develop by @Luc45 in #145
• Add self-tests for WordPress.DB in Security Tests by @Luc45 in #146
• Custom E2E Tests 1 - Renaming Environment to Backend by @Luc45 in #133
• Custom E2E Tests 2 - Local Test Environment by @Luc45 in #130
• Local test env tweaks by @Luc45 in #147
• Update malware tests by @MrJnrman in #150
• Update snapshots for 8.7.0 sync by @Luc45 in #149
• Add empty extension list exception by @zhongruige in #152
• 8.8.0-beta.1 sync by @Luc45 in #155
• Added nonJsonOutput property by @zhongruige in #153
• Port Fast PHPCS PR to CLI by @Luc45 in #156
• Update WC 8.8 rc test snapshots by @zhongruige in #158
• Readme suggestions by @Luc45 in #159
• Sync 8.8.2 by @Luc45 in #161
• Rename woo.com to woocommerce.com by @zhongruige in #157
• Sync 8.9.0 RC by @zhongruige in #164
• Custom Tests by @Luc45 in #148
• Add more badges to the readme by @Luc45 in #165
• Tweaks to release process by @Luc45 in #166

Full Changelog: 0.4.2...0.5.0