Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Storage plugin for Dell EMC PowerFlex/ScaleIO (formerly VxFlexOS) #4304

Merged
merged 12 commits into from
Feb 24, 2021
Merged

Storage plugin for Dell EMC PowerFlex/ScaleIO (formerly VxFlexOS) #4304

merged 12 commits into from
Feb 24, 2021

Conversation

sureshanaparti
Copy link
Contributor

@sureshanaparti sureshanaparti commented Sep 3, 2020 •
edited
Loading

Description

Storage plugin for Dell EMC PowerFlex/ScaleIO (formerly VxFlexOS)

  • Added support for PowerFlex/ScaleIO (v3.5 onwards) storage pool as a primary storage in CloudStack (for KVM hypervisor) and enabled VM/Volume operations on that pool (using pool tag).

Please find more details in the FS here:
https://cwiki.apache.org/confluence/x/cDl4CQ

Documentation PR: apache/cloudstack-documentation#169

Primate UI: apache/cloudstack-primate#863

Types of changes

  • Breaking change (fix or feature that would cause existing functionality to change)
  • New feature (non-breaking change which adds functionality)
  • Bug fix (non-breaking change which fixes an issue)
  • Enhancement (improves an existing feature and functionality)
  • Cleanup (Code refactoring and cleanup, that may add test cases)

Screenshots (if appropriate):

How Has This Been Tested?

Manual tests:

  • Deploy VM, with qcow2/raw images (with data disk)
  • Stop / Start / Restart / Reinstall / Destroy VM(s)
  • Attach / Detach / Resize / Destroy disk(s)
  • Create/Revert/Delete volume snapshots (on primary storage)
  • Create / Delete VM Snapshots (for Running & Stopped VMs)
  • Revert VM Snapshots (Stopped VMs)
  • Create template from Volume / Snapshot in ScaleIO pool and Deploy VM using that template
  • Migrate VM (with PowerFlex volumes) within/across Clusters
  • Migrate Volume (Volume Tree or V-Tree, along with snapshots) to other PowerFlex pool on the same PowerFlex/ScaleIO cluster
  • Migrate Volume (with no snapshots) to other PowerFlex pool on different PowerFlex/ScaleIO cluster

Marvin tests:
=== TestName: test_01_create_vm_with_volume | Status : SUCCESS ===

=== TestName: test_02_attach_new_volume_to_stopped_vm | Status : SUCCESS ===

=== TestName: test_03_attach_detach_attach_volume_to_vm | Status : SUCCESS ===

=== TestName: test_04_detach_vol_stopped_vm_start | Status : SUCCESS ===

=== TestName: test_05_attach_volume_to_stopped_vm | Status : SUCCESS ===

=== TestName: test_06_attached_volume_reboot_vm | Status : SUCCESS ===

=== TestName: test_07_detach_volume_reboot_vm | Status : SUCCESS ===

=== TestName: test_08_delete_volume_was_attached | Status : SUCCESS ===

=== TestName: test_09_migrate_volume_to_same_instance_pool | Status : SUCCESS ===

=== TestName: test_10_migrate_volume_to_distinct_instance_pool | Status : SUCCESS ===

@rohityadavcloud rohityadavcloud added this to the 4.16.0.0 milestone Sep 9, 2020
@rohityadavcloud
Copy link
Member

@blueorangutan package

@blueorangutan
Copy link

@rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

@blueorangutan
Copy link

Packaging result: ✖centos7 ✖centos8 ✖debian. JID-1929

@sureshanaparti
Copy link
Contributor Author

@blueorangutan package

@blueorangutan
Copy link

@sureshanaparti a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

@blueorangutan
Copy link

Packaging result: ✔centos7 ✔centos8 ✔debian. JID-1930

@sureshanaparti
Copy link
Contributor Author

@blueorangutan package

@blueorangutan
Copy link

@sureshanaparti a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

@blueorangutan
Copy link

Packaging result: ✔centos7 ✔centos8 ✔debian. JID-2190

@sureshanaparti
Copy link
Contributor Author

@blueorangutan test

@blueorangutan
Copy link

@sureshanaparti a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

@blueorangutan
Copy link

Trillian test result (tid-2976)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 39062 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4304-t2976-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_kubernetes_clusters.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_redundant.py
Intermittent failure detected: /marvin/tests/smoke/test_hostha_kvm.py
Smoke tests completed. 84 look OK, 1 have error(s)
Only failed tests results shown below:

Test Result Time (s) Test File
test_hostha_kvm_host_fencing Error 174.43 test_hostha_kvm.py

@sureshanaparti
Copy link
Contributor Author

@blueorangutan package

@blueorangutan
Copy link

@sureshanaparti a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

@blueorangutan
Copy link

Packaging result: ✔centos7 ✔centos8 ✔debian. JID-2211

@sureshanaparti
Copy link
Contributor Author

@blueorangutan package

@blueorangutan
Copy link

@sureshanaparti a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

@blueorangutan
Copy link

Packaging result: ✔centos7 ✔centos8 ✔debian. JID-2308

@sureshanaparti
Copy link
Contributor Author

@blueorangutan test

@blueorangutan
Copy link

@sureshanaparti a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

@blueorangutan
Copy link

Trillian test result (tid-3105)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 31860 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4304-t3105-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_privategw_acl.py
Smoke tests completed. 86 look OK, 0 have error(s)
Only failed tests results shown below:

Test Result Time (s) Test File

@sureshanaparti
Copy link
Contributor Author

@blueorangutan package

@blueorangutan
Copy link

@sureshanaparti a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

@sureshanaparti
Updated PowerFlex/ScaleIO storage plugin version to 4.16.0.0
2de8944
@sureshanaparti
UI Changes to support storage plugin for PowerFlex/ScaleIO storage pool.
3a7c3db 
- PowerFlex pool URL generated from the UI inputs(Gateway, Username, Password, Storage Pool) when adding "PowerFlex" Primary Storage
- Updated protocol to "custom" for PowerFlex provider
- Allow VM Snapshot for stopped VM on KVM hypervisor and PowerFlex/ScaleIO storage pool

and Minor improvements in PowerFlex/ScaleIO storage plugin code
@sureshanaparti
Added support for PowerFlex/ScaleIO volume migration across different…
71252ea 
… PowerFlex storage instances.

- findStoragePoolsForMigration API returns PowerFlex pool(s) of different instance as suitable pool(s), for volume(s) on PowerFlex storage pool.
- Volume(s) with snapshots are not allowed to migrate to different PowerFlex instance.
- Volume(s) of running VM are not allowed to migrate to other PowerFlex storage pools.
- Volume migration from PowerFlex pool to Non-PowerFlex pool, and vice versa are not supported.
@sureshanaparti
Fixed change service offering smoke tests in test_service_offerings.p…
602a708 
…y, test_vm_snapshots.py
@sureshanaparti
Added the PowerFlex/ScaleIO volume/snapshot name to the paths of resp…
9e92647 
…ective CloudStack resources (Templates, Volumes, Snapshots and VM Snapshots)
@sureshanaparti
Added new response parameter “supportsStorageSnapshot” (true/false) t…
1c16a1f 
…o volume response, and Updated UI to hide the async backup option while taking snapshot for volume(s) with storage snapshot support.
@sureshanaparti
Fix to remove the duplicate zone wide pools listed while finding stor…
ea6f7f1 
…age pools for migration
@sureshanaparti
Updated PowerFlex/ScaleIO volume migration checks and rollback migrat…
dc91003 
…ion on failure
@sureshanaparti
Fixed the PowerFlex/ScaleIO volume name inconsistency issue in the vo…
010dfee 
…lume path after migration, due to rename failure
@sureshanaparti
Copy link
Contributor Author

@blueorangutan package

@blueorangutan
Copy link

@sureshanaparti a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

@blueorangutan
Copy link

Packaging result: ✖centos7 ✔centos8 ✔debian. JID-2742

@rohityadavcloud
Copy link
Member

@blueorangutan package

@blueorangutan
Copy link

@rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

@blueorangutan
Copy link

Packaging result: ✔centos7 ✔centos8 ✔debian. JID-2743

@rohityadavcloud
Copy link
Member

@blueorangutan test matrix

@blueorangutan
Copy link

@rhtyd a Trillian-Jenkins matrix job (centos7 mgmt + xs71, centos7 mgmt + vmware65, centos7 mgmt + kvmcentos7) has been kicked to run smoke tests

@blueorangutan
Copy link

Trillian test result (tid-3577)
Environment: xenserver-71 (x2), Advanced Networking with Mgmt server 7
Total time taken: 32865 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4304-t3577-xenserver-71.zip
Intermittent failure detected: /marvin/tests/smoke/test_scale_vm.py
Smoke tests completed. 85 look OK, 1 have error(s)
Only failed tests results shown below:

Test Result Time (s) Test File
test_01_scale_vm Failure 10.43 test_scale_vm.py

@blueorangutan
Copy link

Trillian test result (tid-3578)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 34138 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4304-t3578-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_kubernetes_clusters.py
Intermittent failure detected: /marvin/tests/smoke/test_kubernetes_supported_versions.py
Smoke tests completed. 86 look OK, 0 have error(s)
Only failed tests results shown below:

Test Result Time (s) Test File

@blueorangutan
Copy link

Trillian test result (tid-3579)
Environment: vmware-65u2 (x2), Advanced Networking with Mgmt server 7
Total time taken: 41150 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4304-t3579-vmware-65u2.zip
Intermittent failure detected: /marvin/tests/smoke/test_human_readable_logs.py
Intermittent failure detected: /marvin/tests/smoke/test_kubernetes_clusters.py
Intermittent failure detected: /marvin/tests/smoke/test_volumes.py
Smoke tests completed. 84 look OK, 2 have error(s)
Only failed tests results shown below:

Test Result Time (s) Test File
test_02_enableHumanReadableLogs Error 0.29 test_human_readable_logs.py
test_03_deploy_and_upgrade_kubernetes_cluster Failure 780.42 test_kubernetes_clusters.py

@rohityadavcloud rohityadavcloud marked this pull request as ready for review February 19, 2021 08:53
@rohityadavcloud
Copy link
Member

Tests LGTM, waiting for manual confirmation from @borisstoyanov

borisstoyanov
borisstoyanov approved these changes Feb 23, 2021
View reviewed changes
Copy link
Contributor

@borisstoyanov borisstoyanov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, manually tested

@rohityadavcloud rohityadavcloud merged commit eba186a into apache:master Feb 24, 2021
@rohityadavcloud rohityadavcloud mentioned this pull request May 21, 2021
Merged
12 tasks
@weizhouapache weizhouapache mentioned this pull request Mar 22, 2022
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@slavkap slavkap slavkap left review comments

@borisstoyanov borisstoyanov borisstoyanov approved these changes

@shwstppr shwstppr shwstppr approved these changes

@rohityadavcloud rohityadavcloud rohityadavcloud approved these changes

Assignees

@sureshanaparti sureshanaparti

Labels
Severity:Major type:new-feature
Projects
None yet
Milestone
4.16.0.0
Development

Successfully merging this pull request may close these issues.
9 participants
@sureshanaparti @rohityadavcloud @blueorangutan @borisstoyanov @shwstppr @slavkap @DaanHoogland @nvazquez @ACSGitBot