Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ci-group arg for OSD integ tests #567

Merged
merged 3 commits into from
Nov 25, 2024

Conversation

rishabh6788
Copy link
Collaborator

Description

Add ci-group arg for OSD integ tests.

Issues Resolved

List any issues this PR will resolve, e.g. Closes [...].

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@rishabh6788 rishabh6788 force-pushed the main branch 5 times, most recently from da8d27a to f186299 Compare November 15, 2024 02:27
@rishabh6788 rishabh6788 marked this pull request as ready for review November 15, 2024 02:28
Copy link

codecov bot commented Nov 15, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 84.79%. Comparing base (e6ec7d1) to head (89ef2bd).
Report is 1 commits behind head on main.

Additional details and impacted files
@@             Coverage Diff              @@
##               main     #567      +/-   ##
============================================
+ Coverage     84.75%   84.79%   +0.04%     
  Complexity      114      114              
============================================
  Files           117      119       +2     
  Lines           682      684       +2     
  Branches         75       75              
============================================
+ Hits            578      580       +2     
  Misses           30       30              
  Partials         74       74              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@@ -0,0 +1,54 @@
/*
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not understanding why we need to add this library separately? What is the difference?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see. How about renaming it to runIntegTestScriptForOSD or runOSDintegTestScript?
Also do you recollect why we have that if/else block for windows? Sorry it being long since this was written.
cc: @peterzhuamazon

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

vars/processAndRunOsdIntegTest.groovy Outdated Show resolved Hide resolved
vars/runIntegTestScript.groovy Show resolved Hide resolved
steps {
script {
processAndRunOsdIntegTest(
localComponent: 'OpenSearch',
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should fail right? Since the lib as the name suggests only tests OSD?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is just a dummy component name to check component without any ci-group.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe pass dashboard component. Just a nit since we would never call this library for OS test manifest. Should we add a check in the lib making sure that test manifest belongs to OSD dist and not OS?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

sh("rm -rf ${WORKSPACE}/artifacts")
}
else {
echo "Not on Windows, unstash repository+artifacts"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be

Suggested change
echo "Not on Windows, unstash repository+artifacts"
echo "Not on Windows, skipping unstashing repository+artifacts"

I dont see any action being taken in else statement. Just the echo.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@peterzhuamazon Any idea?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If it is not a blocker we can revisit this later.

@gaiksaya
Copy link
Member

Can you also please add library definitions to these 2 libraries. Something similar to https://github.com/opensearch-project/opensearch-build-libraries/blob/main/vars/publishDistributionBuildResults.groovy#L10-L19
Looks like we missed a lot of them for old ones. Might be a good idea to use this opportunity to fix these 2 for now.

@rishabh6788 rishabh6788 force-pushed the main branch 2 times, most recently from 3e2eb6a to a2ea550 Compare November 20, 2024 21:28
Comment on lines 56 to 58
buildManifest: "${BUILD_MANIFEST}",
testManifest: "${TEST_MANIFEST}",
localPath: "${WORKSPACE}/${distribution}",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From where are we getting this values from? Are those args as well?
Same goes for ARTIFACT_BUCKET_NAME, artifactPathOpenSearch and artifactPath?
If yes can you add the same to lib description above?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All being set as env variable in the jenkins job, and since the script is being executed in the same environment it has access to all the env variables.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This sounds risky if someone needs to use the lib directly. It wont know what to specify. Those args are Parameters in the workflow thats why we can use it as ENV variable. I believe it we need to add as args and add checks.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So runIntegTestScriptForOSD will have more args than usual one which is fine as it is reducing the complexity in main workflow.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

gaiksaya
gaiksaya previously approved these changes Nov 20, 2024
Copy link
Member

@gaiksaya gaiksaya left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Thanks for getting this in.
Make sure to bump this to 8.x as well and create 8.x based on main.
Thanks!

@gaiksaya gaiksaya dismissed their stale review November 20, 2024 23:30

Adding more comments Sorry!

@gaiksaya gaiksaya merged commit 655ac86 into opensearch-project:main Nov 25, 2024
8 of 9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: ✅ Done
Development

Successfully merging this pull request may close these issues.

2 participants