Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ZWE Integration Test Framework #3844

Open
wants to merge 24 commits into
base: feature/v3/jcl
Choose a base branch
from

Conversation

MarkAckert
Copy link
Member

@MarkAckert MarkAckert commented May 31, 2024

Address #3922

This PR creates a new ZWE Integration Test Framework which focuses on delivering clear, easy-to-write tests for ZWE commands and uses Jest snapshots to capture and test command output. Higher fidelity tests could be written later with more intelligent analysis of zwe command output. More will be added to this description and the test README soon.

  • Batteries Included - setup, teardown, smart runner, cfg-yaml autocomplete, test logs, clean snapshots
  • zwe built from local repo, dependencies downloaded from manifest.json.template
  • Limited System Requirements (test user account must be able to run zwe commands)
  • Jest Framework
  • Configuration options to help speed up dev test creation and iteration
  • CI Build and Nightly Build
  • SHORT vs LONG test designation, as defined by requiring new runs of init generate vs sharing a single run. SHORT tests complete on average in ~100s per command group, of which up to ~50s is the single shared init generate command.

Limitations:

  • Time: Each test which requires a modification to the zowe.yaml takes roughly 30s to run, bulk of which is backend job execution.
    • Mitigate via SHORT vs LONG designation and careful test design.
  • Some ZWE commands cause the shell to hang. Still investigating.

To be completed before merge:

  • A stable set of tests with committed snapshots. These are still changing frequently.
  • Investigations into USS / Shell hangs.
  • Verification of portability - run this on both internal systems and Marist environments.
  • Github Action
  • Code review feedback

@Martin-Zeithaml
Copy link
Contributor

Recently the behavior of zwe init command was changed, basically useConfigmgr does not matter.
There was a decision made not to rewrite zwe init certificate due to complexity of the code, in other words keep it in shell.
But this would be a huge limitation to have zowe.yaml as one uss file only and without ECMAS2020 scripting.
There is an internal command zwe internal config output to prepare merged config for zwe init certificate.

It does not matter now, if config is zowe.yaml or FILE(zowe.yaml):PARMLIB(TEST(ZWEYAML)), there is no message ZWEL0316E anymore.

This test (and for other commands) is kind of obsolete.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants