Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable warnings as errors for weekly run in test suite #6217

Merged
merged 102 commits into from
Nov 11, 2024

Conversation

mlxd
Copy link
Member

@mlxd mlxd commented Sep 4, 2024

Before submitting

Please complete the following checklist when submitting a PR:

  • All new features must include a unit test.
    If you've fixed a bug or added code that should be tested, add a test to the
    test directory!

  • All new functions and code must be clearly commented and documented.
    If you do make documentation changes, make sure that the docs build and
    render correctly by running make docs.

  • Ensure that the test suite passes, by running make test.

  • Add a new entry to the doc/releases/changelog-dev.md file, summarizing the
    change, and including a link back to the PR.

  • The PennyLane source code conforms to
    PEP8 standards.
    We check all of our code against Pylint.
    To lint modified files, simply pip install pylint, and then
    run pylint pennylane/path/to/file.py.

When all the above are checked, delete everything above the dashed
line and fill in the pull request template.


Context: This PR enables a weekly run of the test suite with Python-level warnings promoted to errors (exceptions). This will allow the test suite to regularly report when hitting warnings through general execution.

Description of the Change: Add a new workflow to run the test suite with Python warnings as errors on a weekly basis,

Benefits: Enable us to catch warnings when and where they occur during testing.

Possible Drawbacks: Additional overhead of running the CI at the intended occurrence.

Related GitHub Issues:

@mlxd mlxd added the do not merge ⚠️ Do not merge the pull request until this label is removed label Sep 5, 2024
pennylane/devices/tests/pytest.ini Outdated Show resolved Hide resolved
tests/pytest.ini Outdated Show resolved Hide resolved
@mlxd mlxd marked this pull request as ready for review September 12, 2024 19:59
@mlxd mlxd changed the title Enable warnings as errors globally in test suite Enable warnings as errors for weekly run in test suite Sep 12, 2024
@mlxd mlxd requested review from mudit2812 and rashidnhm September 12, 2024 20:14
@mlxd mlxd removed the do not merge ⚠️ Do not merge the pull request until this label is removed label Sep 12, 2024
Copy link
Contributor

@mudit2812 mudit2812 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. Left a bunch of suggestions. I would also suggest adding a pull_request trigger to the workflow once it's reasonably complete so you can test it out in action before the PR gets merged.

tests/pytest.ini Outdated Show resolved Hide resolved
.github/workflows/package_warnings_as_errors.yml Outdated Show resolved Hide resolved
.github/workflows/package_warnings_as_errors.yml Outdated Show resolved Hide resolved
.github/workflows/package_warnings_as_errors.yml Outdated Show resolved Hide resolved
.github/workflows/unit-test.yml Outdated Show resolved Hide resolved
.github/workflows/interface-unit-tests.yml Show resolved Hide resolved
Copy link

codecov bot commented Sep 12, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 99.29%. Comparing base (a6dabca) to head (ba46b7d).
Report is 1 commits behind head on master.

Additional details and impacted files
@@           Coverage Diff           @@
##           master    #6217   +/-   ##
=======================================
  Coverage   99.29%   99.29%           
=======================================
  Files         457      457           
  Lines       43413    43413           
=======================================
  Hits        43109    43109           
  Misses        304      304           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@mlxd mlxd requested a review from rashidnhm October 25, 2024 19:07
Copy link
Contributor

@rashidnhm rashidnhm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

According to GitHub docs fail-fast is a boolean requiring true/false!

Scratch that, this holds true for the current logic in this PR

.github/workflows/interface-unit-tests.yml Outdated Show resolved Hide resolved
.github/workflows/interface-unit-tests.yml Outdated Show resolved Hide resolved
@mlxd mlxd requested a review from rashidnhm October 30, 2024 14:02
@mlxd mlxd requested a review from rashidnhm November 8, 2024 15:01
Copy link
Contributor

@rashidnhm rashidnhm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for all the work on this!

@mlxd mlxd enabled auto-merge (squash) November 11, 2024 19:32
@mlxd mlxd added the urgent Mark a pull request as high priority label Nov 11, 2024
@mlxd
Copy link
Member Author

mlxd commented Nov 11, 2024

Just a note, the accepted suggestions have broken large parts of the action. I intend to roll back to a previous working version, albeit with a more verbose content.

@mlxd mlxd merged commit 12fc37d into master Nov 11, 2024
46 checks passed
@mlxd mlxd deleted the tests/warning_as_error branch November 11, 2024 21:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
urgent Mark a pull request as high priority
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants