Skip to content

Latest commit

 

History

History
922 lines (776 loc) · 25.4 KB

README.rst

File metadata and controls

922 lines (776 loc) · 25.4 KB
Raw

Usage

Cinder provides an infrastructure for managing volumes in OpenStack. Originally, this project was the Nova component called nova-volume and starting from the Folsom OpenStack release it has become an independent project.

This file provides the sample configurations for different use cases:

  • Pillar sample of a basic Cinder configuration:

    The pillar structure defines cinder-api and cinder-scheduler inside the controller role and cinder-volume inside the to volume role.

    cinder:
  controller:
    enabled: true
    version: juno
    cinder_uid: 304
    cinder_gid: 304
    nas_secure_file_permissions: false
    nas_secure_file_operations: false
    cinder_internal_tenant_user_id: f46924c112a14c80ab0a24a613d95eef
    cinder_internal_tenant_project_id: b7455b8974bb4064ad247c8f375eae6c
    default_volume_type: 7k2SaS
    enable_force_upload: true
    availability_zone_fallback: True
    database:
      engine: mysql
      host: 127.0.0.1
      port: 3306
      name: cinder
      user: cinder
      password: pwd
    identity:
      engine: keystone
      host: 127.0.0.1
      port: 35357
      tenant: service
      user: cinder
      password: pwd
    message_queue:
      engine: rabbitmq
      host: 127.0.0.1
      port: 5672
      user: openstack
      password: pwd
      virtual_host: '/openstack'
    backend:
      7k2_SAS:
        engine: storwize
        type_name: slow-disks
        host: 192.168.0.1
        port: 22
        user: username
        password: pass
        connection: FC/iSCSI
        multihost: true
        multipath: true
        pool: SAS7K2
    audit:
      enabled: false
    osapi_max_limit: 500
    barbican:
      enabled: true

cinder:
  volume:
    enabled: true
    version: juno
    cinder_uid: 304
    cinder_gid: 304
    nas_secure_file_permissions: false
    nas_secure_file_operations: false
    cinder_internal_tenant_user_id: f46924c112a14c80ab0a24a613d95eef
    cinder_internal_tenant_project_id: b7455b8974bb4064ad247c8f375eae6c
    default_volume_type: 7k2SaS
    enable_force_upload: true
    my_ip: 192.168.0.254
    database:
      engine: mysql
      host: 127.0.0.1
      port: 3306
      name: cinder
      user: cinder
      password: pwd
    identity:
      engine: keystone
      host: 127.0.0.1
      port: 35357
      tenant: service
      user: cinder
      password: pwd
    message_queue:
      engine: rabbitmq
      host: 127.0.0.1
      port: 5672
      user: openstack
      password: pwd
      virtual_host: '/openstack'
    backend:
      7k2_SAS:
        engine: storwize
        type_name: 7k2 SAS disk
        host: 192.168.0.1
        port: 22
        user: username
        password: pass
        connection: FC/iSCSI
        multihost: true
        multipath: true
        pool: SAS7K2
    audit:
      enabled: false
    barbican:
      enabled: true

Volume vmware related options:

cinder:
  volume:
    backend:
      vmware:
        engine: vmware
        host_username: vmware
        host_password: vmware
        cluster_names: vmware_cluster01,vmware_cluster02

  • The CORS parameters enablement:

    cinder:
  controller:
    cors:
      allowed_origin: https:localhost.local,http:localhost.local
      expose_headers: X-Auth-Token,X-Openstack-Request-Id,X-Subject-Token
      allow_methods: GET,PUT,POST,DELETE,PATCH
      allow_headers: X-Auth-Token,X-Openstack-Request-Id,X-Subject-Token
      allow_credentials: True
      max_age: 86400

  • The client-side RabbitMQ HA setup for the controller:

    cinder:
  controller:
    ....
    message_queue:
      engine: rabbitmq
      members:
        - host: 10.0.16.1
        - host: 10.0.16.2
        - host: 10.0.16.3
      user: openstack
      password: pwd
      virtual_host: '/openstack'
    ....

  • The client-side RabbitMQ HA setup for the volume component

    cinder:
  volume:
    ....
    message_queue:
      engine: rabbitmq
      members:
        - host: 10.0.16.1
        - host: 10.0.16.2
        - host: 10.0.16.3
      user: openstack
      password: pwd
      virtual_host: '/openstack'
    ....

  • Configuring TLS communications.

    Note

    By default, system-wide installed CA certs are used. Therefore, the cacert_file and cacert parameters are optional.

    • RabbitMQ TLS:

      cinder:
  controller, volume:
     message_queue:
       port: 5671
       ssl:
         enabled: True
         (optional) cacert: cert body if the cacert_file does not exists
         (optional) cacert_file: /etc/openstack/rabbitmq-ca.pem
         (optional) version: TLSv1_2

    • MySQL TLS:

      cinder:
  controller:
     database:
       ssl:
         enabled: True
         (optional) cacert: cert body if the cacert_file does not exists
         (optional) cacert_file: /etc/openstack/mysql-ca.pem

    • Openstack HTTPS API:

      cinder:
 controller, volume:
     identity:
        protocol: https
        (optional) cacert_file: /etc/openstack/proxy.pem
     glance:
        protocol: https
        (optional) cacert_file: /etc/openstack/proxy.pem

  • Cinder setup with zeroing deleted volumes:

    cinder:
  controller:
    enabled: true
    wipe_method: zero
    ...

  • Cinder setup with shreding deleted volumes:

    cinder:
  controller:
    enabled: true
    wipe_method: shred
    ...

  • Configuration of policy.json file:

    cinder:
  controller:
    ....
    policy:
      'volume:delete': 'rule:admin_or_owner'
      # Add key without value to remove line from policy.json
      'volume:extend':

  • Default Cinder backend lvm_type setup:

    cinder:
  volume:
    enabled: true
    backend:
      # Type of LVM volumes to deploy; (default, thin, or auto). Auto defaults to thin if thin is supported.
      lvm_type: auto

  • Default Cinder setup with iSCSI target:

    cinder:
  controller:
    enabled: true
    version: mitaka
    default_volume_type: lvmdriver-1
    database:
      engine: mysql
      host: 127.0.0.1
      port: 3306
      name: cinder
      user: cinder
      password: pwd
    identity:
      engine: keystone
      host: 127.0.0.1
      port: 35357
      tenant: service
      user: cinder
      password: pwd
    message_queue:
      engine: rabbitmq
      host: 127.0.0.1
      port: 5672
      user: openstack
      password: pwd
      virtual_host: '/openstack'
    backend:
      lvmdriver-1:
        engine: lvm
        type_name: lvmdriver-1
        volume_group: cinder-volume

  • Cinder setup for IBM Storwize:

    cinder:
  volume:
    enabled: true
    backend:
      7k2_SAS:
        engine: storwize
        type_name: 7k2 SAS disk
        host: 192.168.0.1
        port: 22
        user: username
        password: pass
        connection: FC/iSCSI
        multihost: true
        multipath: true
        pool: SAS7K2
      10k_SAS:
        engine: storwize
        type_name: 10k SAS disk
        host: 192.168.0.1
        port: 22
        user: username
        password: pass
        connection: FC/iSCSI
        multihost: true
        multipath: true
        pool: SAS10K
      15k_SAS:
        engine: storwize
        type_name: 15k SAS
        host: 192.168.0.1
        port: 22
        user: username
        password: pass
        connection: FC/iSCSI
        multihost: true
        multipath: true
        pool: SAS15K

  • Cinder setup with NFS:

    cinder:
  controller:
    enabled: true
    default_volume_type: nfs-driver
    backend:
      nfs-driver:
        engine: nfs
        type_name: nfs-driver
        volume_group: cinder-volume
        path: /var/lib/cinder/nfs
        devices:
        - 172.16.10.110:/var/nfs/cinder
        options: rw,sync

  • Cinder setup with NetApp:

    cinder:
  controller:
    backend:
      netapp:
        engine: netapp
        type_name: netapp
        user: openstack
        vserver: vm1
        server_hostname: 172.18.2.3
        password: password
        storage_protocol: nfs
        transport_type: https
        lun_space_reservation: enabled
        use_multipath_for_image_xfer: True
        nas_secure_file_operations: false
        nas_secure_file_permissions: false
        devices:
          - 172.18.1.2:/vol_1
          - 172.18.1.2:/vol_2
          - 172.18.1.2:/vol_3
          - 172.18.1.2:/vol_4
linux:
  system:
    package:
      nfs-common:
        version: latest

  • Cinder setup with Hitachi VPS:

    cinder:
  controller:
    enabled: true
    backend:
      hus100_backend:
        type_name: HUS100
        backend: hus100_backend
        engine: hitachi_vsp
        connection: FC

  • Cinder setup with Hitachi VPS with defined ldev range:

    cinder:
  controller:
    enabled: true
    backend:
      hus100_backend:
        type_name: HUS100
        backend: hus100_backend
        engine: hitachi_vsp
        connection: FC
        ldev_range: 0-1000

  • Cinder setup with Ceph:

    cinder:
  controller:
    enabled: true
    backend:
      ceph_backend:
        type_name: standard-iops
        backend: ceph_backend
        backend_host: ceph
        pool: volumes
        engine: ceph
        user: cinder
        secret_uuid: da74ccb7-aa59-1721-a172-0006b1aa4e3e
        client_cinder_key: AQDOavlU6BsSJhAAnpFR906mvdgdfRqLHwu0Uw==
        report_discard_supported: True
        image_volume_cache_enabled: False

  • Cinder setup with HP3par:

    cinder:
  controller:
    enabled: true
    backend:
      hp3par_backend:
        type_name: hp3par
        backend: hp3par_backend
        user: hp3paruser
        password: something
        url: http://10.10.10.10/api/v1
        cpg: OpenStackCPG
        host: 10.10.10.10
        login: hp3paradmin
        sanpassword: something
        debug: True
        snapcpg: OpenStackSNAPCPG

  • Cinder setup with Fujitsu Eternus:

    cinder:
  volume:
    enabled: true
    backend:
      10kThinPro:
        type_name: 10kThinPro
        engine: fujitsu
        pool: 10kThinPro
        host: 192.168.0.1
        port: 5988
        user: username
        password: pass
        connection: FC/iSCSI
        name: 10kThinPro
      10k_SAS:
        type_name: 10k_SAS
        pool: SAS10K
        engine: fujitsu
        host: 192.168.0.1
        port: 5988
        user: username
        password: pass
        connection: FC/iSCSI
        name: 10k_SAS

  • Cinder setup with IBM GPFS filesystem:

    cinder:
  volume:
    enabled: true
    backend:
      GPFS-GOLD:
        type_name: GPFS-GOLD
        engine: gpfs
        mount_point: '/mnt/gpfs-openstack/cinder/gold'
      GPFS-SILVER:
        type_name: GPFS-SILVER
        engine: gpfs
        mount_point: '/mnt/gpfs-openstack/cinder/silver'

  • Cinder setup with HP LeftHand:

    cinder:
  volume:
    enabled: true
    backend:
      HP-LeftHand:
        type_name: normal-storage
        engine: hp_lefthand
        api_url: 'https://10.10.10.10:8081/lhos'
        username: user
        password: password
        clustername: cluster1
        iscsi_chap_enabled: false

  • Extra parameters for HP LeftHand:

    cinder type-key normal-storage set hplh:data_pl=r-10-2 hplh:provisioning=full

  • Cinder setup with Solidfire:

    cinder:
  volume:
    enabled: true
    backend:
      solidfire:
        type_name: normal-storage
        engine: solidfire
        san_ip: 10.10.10.10
        san_login: user
        san_password: password
        clustername: cluster1
        sf_emulate_512: false
        sf_api_port: 14443
        host: ctl01
        #for compatibility with old versions
        sf_account_prefix: PREFIX

  • Cinder setup with Block Device driver:

    cinder:
  volume:
    enabled: true
    backend:
      bdd:
        engine: bdd
        enabled: true
        type_name: bdd
        devices:
          - sdb
          - sdc
          - sdd

  • Enable cinder-backup service for ceph

    cinder:
  controller:
    enabled: true
    version: mitaka
    backup:
      engine: ceph
      ceph_conf: "/etc/ceph/ceph.conf"
      ceph_pool: backup
      ceph_stripe_count: 0
      ceph_stripe_unit: 0
      ceph_user: cinder
      ceph_chunk_size: 134217728
      restore_discard_excess_bytes: false
  volume:
    enabled: true
    version: mitaka
    backup:
      engine: ceph
      ceph_conf: "/etc/ceph/ceph.conf"
      ceph_pool: backup
      ceph_stripe_count: 0
      ceph_stripe_unit: 0
      ceph_user: cinder
      ceph_chunk_size: 134217728
      restore_discard_excess_bytes: false

  • Auditing filter (CADF) enablement:

    cinder:
  controller:
    audit:
      enabled: true
  ....
      filter_factory: 'keystonemiddleware.audit:filter_factory'
      map_file: '/etc/pycadf/cinder_api_audit_map.conf'
  ....
  volume:
    audit:
      enabled: true
  ....
      filter_factory: 'keystonemiddleware.audit:filter_factory'
      map_file: '/etc/pycadf/cinder_api_audit_map.conf'

  • Cinder setup with custom availability zones:

    cinder:
  controller:
    default_availability_zone: my-default-zone
    storage_availability_zone: my-custom-zone-name
cinder:
  volume:
    default_availability_zone: my-default-zone
    storage_availability_zone: my-custom-zone-name

    The default_availability_zone is used when a volume has been created, without specifying a zone in the create request as this zone must exist in your configuration.

    The storage_availability_zone is an actual zone where the node belongs to and must be specified per each node.

  • Cinder setup with custom non-admin volume query filters:

    cinder:
  controller:
    query_volume_filters:
      - name
      - status
      - metadata
      - availability_zone
      - bootable

  • public_endpoint and osapi_volume_base_url:

    • public_endpoint
      Used for configuring versions endpoint
    • osapi_volume_base_URL
      Used to present Cinder URL to users

    These parameters can be useful when running Cinder under load balancer in SSL.

    cinder:
  controller:
    public_endpoint_address: https://${_param:cluster_domain}:8776

  • Client role definition:

    cinder:
  client:
    enabled: true
    identity:
      host: 127.0.0.1
      port: 35357
      project: service
      user: cinder
      password: pwd
      protocol: http
      endpoint_type: internalURL
      region_name: RegionOne
    backend:
      ceph:
        type_name: standard-iops
        engine: ceph
        key:
          conn_speed: fibre-10G

  • Barbican integration enablement:

    cinder:
  controller:
    barbican:
      enabled: true

  • Keystone API version specification (v3 is default):

    cinder:
  controller:
    identity:
      api_version: v2.0

Enhanced logging with logging.conf

By default logging.conf is disabled. You can enable per-binary logging.conf by setting the following parameters:

  • openstack_log_appender
    Set to true to enable log_config_append for all OpenStack services
  • openstack_fluentd_handler_enabled
    Set to true to enable FluentHandler for all Openstack services
  • openstack_ossyslog_handler_enabled
    Set to true to enable OSSysLogHandler for all Openstack services

Only WatchedFileHandler, OSSysLogHandler, and FluentHandler are available.

To configure this functionality with pillar:

cinder:
  controller:
    logging:
      log_appender: true
      log_handlers:
        watchedfile:
          enabled: true
        fluentd:
          enabled: true
        ossyslog:
          enabled: true

  volume:
    logging:
      log_appender: true
      log_handlers:
        watchedfile:
          enabled: true
        fluentd:
          enabled: true
        ossyslog:
          enabled: true

Enable x509 and ssl communication between Cinder and Galera cluster.

By default communication between Cinder and Galera is unsecure.

cinder:
volume:
database:
x509:
enabled: True
controller:
database:
x509:
enabled: True

You able to set custom certificates in pillar:

cinder:
controller:
database:
x509:
cacert: (certificate content) cert: (certificate content) key: (certificate content)
volume:
database:
x509:
cacert: (certificate content) cert: (certificate content) key: (certificate content)
You can read more about it here:
https://docs.openstack.org/security-guide/databases/database-access-control.html

Cinder services on compute node with memcached caching and security strategy:

cinder:
  volume:
    enabled: true
    ...
    cache:
      engine: memcached
      members:
      - host: 127.0.0.1
        port: 11211
      - host: 127.0.0.1
        port: 11211
      security:
        enabled: true
        strategy: ENCRYPT
        secret_key: secret

Cinder services on controller node with memcached caching and security strategy:

cinder:
  controller:
    enabled: true
    ...
    cache:
      engine: memcached
      members:
      - host: 127.0.0.1
        port: 11211
      - host: 127.0.0.1
        port: 11211
      security:
        enabled: true
        strategy: ENCRYPT
        secret_key: secret

Upgrades

Each openstack formula provide set of phases (logical bloks) that will help to build flexible upgrade orchestration logic for particular components. The list of phases and theirs descriptions are listed in table below:

State Description
<app>.upgrade.service_running Ensure that all services for particular application are enabled for autostart and running
<app>.upgrade.service_stopped Ensure that all services for particular application disabled for autostart and dead
<app>.upgrade.pkgs_latest Ensure that packages used by particular application are installed to latest available version. This will not upgrade data plane packages like qemu and openvswitch as usually minimal required version in openstack services is really old. The data plane packages should be upgraded separately by apt-get upgrade or apt-get dist-upgrade Applying this state will not autostart service.
<app>.upgrade.render_config Ensure configuration is rendered actual version.
<app>.upgrade.pre We assume this state is applied on all nodes in the cloud before running upgrade. Only non destructive actions will be applied during this phase. Perform service built in service check like (keystone-manage doctor and nova-status upgrade)
<app>.upgrade.upgrade.pre Mostly applicable for data plane nodes. During this phase resources will be gracefully removed from current node if it is allowed. Services for upgraded application will be set to admin disabled state to make sure node will not participate in resources scheduling. For example on gtw nodes this will set all agents to admin disable state and will move all routers to other agents.
<app>.upgrade.upgrade This state will basically upgrade application on particular target. Stop services, render configuration, install new packages, run offline dbsync (for ctl), start services. Data plane should not be affected, only OpenStack python services.
<app>.upgrade.upgrade.post Add services back to scheduling.
<app>.upgrade.post This phase should be launched only when upgrade of the cloud is completed. Cleanup temporary files, perform other post upgrade tasks.
<app>.upgrade.verify Here we will do basic health checks (API CRUD operations, verify do not have dead network agents/compute services)

Documentation and bugs