Terraform deployment scripts for Beekeeper.
Includes support for deploying Beekeeper on ECS and Kubernetes. Also includes deployment scripts for a Lambda which notifies Slack when Beekeeper's dead letter queue receives a message (this module is not required to run Beekeeper).
A database password is defined in aws-secrets-manager. The terraform module will use that password for the database (see property k8s_db_password_secret ).
| Name | Version |
|---|---|
| archive | n/a |
| aws | n/a |
| kubernetes | n/a |
| random | n/a |
| template | n/a |
| Name | Description | Type | Default | Required |
|---|---|---|---|---|
| allowed_s3_buckets | List of S3 Buckets to which Beekeeper will have read-write access. | list(string) |
[] |
no |
| apiary_metastore_listener_arn | ARN of the Apiary Metastore Listener. | string |
n/a | yes |
| aws_region | AWS region to use for resources. | string |
n/a | yes |
| beekeeper_tags | A map of tags to apply to resources. | map(string) |
n/a | yes |
| path_cleanup_docker_image | Beekeeper Path Cleanup docker image. | string |
"expediagroup/beekeeper-path-cleanup" |
no |
| path_cleanup_docker_image_version | Beekeeper Path Cleanup docker image version. | string |
"latest" |
no |
| path_cleanup_ecs_cpu | The amount of CPU used to allocate for the Beekeeper Path Cleanup ECS task. Valid values: https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task-cpu-memory-error.html |
string |
"2048" |
no |
| path_cleanup_ecs_memory | The amount of memory (in MiB) used to allocate for the Beekeeper Path Cleanup container. Valid values: https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task-cpu-memory-error.html |
string |
"4096" |
no |
| metadata_cleanup_docker_image | Beekeeper Metadata Cleanup docker image. | string |
"expediagroup/beekeeper-metadata-cleanup" |
no |
| metadata_cleanup_docker_image_version | Beekeeper Metadata Cleanup docker image version. | string |
"latest" |
no |
| db_backup_retention | The number of days to retain backups for the RDS Beekeeper DB. | string |
10 |
no |
| db_backup_window | Preferred backup window for the RDS Beekeeper DB in UTC. | string |
"02:00-03:00" |
no |
| db_maintenance_window | Preferred maintenance window for the RDS Beekeeper DB in UTC. | string |
"wed:03:00-wed:04:00" |
no |
| db_password_key | Key to acquire the database password for the strategy specified. | string |
n/a | yes |
| db_username | Username for the master DB user. | string |
"beekeeper" |
no |
| docker_registry_auth_secret_name | Docker Registry authentication SecretManager secret name. | string |
"" |
no |
| path_cleanup_dry_run_enabled | Enable Path Cleanup to perform dry runs of deletions only. | string |
"false" |
no |
| metadata_cleanup_dry_run_enabled | Enable Metadata Cleanup to perform dry runs of deletions only. | string |
"false" |
no |
| graphite_enabled | Enable to produce Graphite metrics - true or false. | string |
"false" |
no |
| graphite_host | Graphite metrics host. | string |
"localhost" |
no |
| graphite_port | Graphite port. | string |
"2003" |
no |
| graphite_prefix | Prefix for Graphite metrics. | string |
"" |
no |
| metastore_uri | URI of the metastore where tables to be cleaned-up are located. Required for Beekeeper Metadata Cleanup. | string |
"" |
yes |
| instance_name | Beekeeper instance name to identify resources in multi-instance deployments. | string |
"" |
no |
| instance_type | Service to run Beekeeper on. Supported services: ecs (default), k8s. Leaving this blank will still deploy auxiliary components (e.g. RDS, SQS etc.). |
string |
"ecs" |
no |
| k8s_app_name | Name to give to all Kubernetes resources that are deployed. | string |
"beekeeper" |
no |
| k8s_path_cleanup_cpu | Total cpu to allot to the Beekeeper Path Cleanup pod. | string |
"500m" |
no |
| k8s_path_cleanup_ingress_host | Ingress host name for Beekeeper Path Cleanup. | string |
"" |
no |
| k8s_path_cleanup_ingress_path | Ingress path regex for Beekeeper Path Cleanup. | string |
"" |
no |
| k8s_path_cleanup_liveness_delay | Liveness delay (in seconds) for the Beekeeper Path Cleanup service. | number |
60 |
no |
| k8s_path_cleanup_memory | Total memory to allot to the Beekeeper Path Cleanup pod. | string |
"2Gi" |
no |
| k8s_path_cleanup_port | Internal port that the Beekeeper Path Cleanup service runs on. | number |
8008 |
no |
| k8s_metadata_cleanup_cpu | Total cpu to allot to the Beekeeper Metadata Cleanup pod. | string |
"500m" |
no |
| k8s_metadata_cleanup_ingress_host | Ingress host name for Beekeeper Metadata Cleanup. | string |
"" |
no |
| k8s_metadata_cleanup_ingress_path | Ingress path regex for Beekeeper Metadata Cleanup. | string |
"" |
no |
| k8s_metadata_cleanup_liveness_delay | Liveness delay (in seconds) for the Beekeeper Metadata Cleanup service. | number |
60 |
no |
| k8s_metadata_cleanup_memory | Total memory to allot to the Beekeeper Metadata Cleanup pod. | string |
"2Gi" |
no |
| k8s_metadata_cleanup_port | Internal port that the Beekeeper Metadata Cleanup service runs on. | number |
9008 |
no |
| k8s_image_pull_policy | Policy for the Kubernetes orchestrator to pull images. | string |
"Always" |
no |
| k8s_ingress_enabled | Boolean flag to determine if we should create an ingress or not. (0 = off, 1 = on). | number |
0 |
no |
| k8s_ingress_tls_hosts | List of hosts for TLS configuration of a Kubernetes ingress. | list(string) |
[] |
no |
| k8s_ingress_tls_secret | Secret name for TLS configuration of a Kubernetes ingress. | string |
"" |
no |
| k8s_kiam_role_arn | KIAM role arn to use for creating a K8S IAM role with the correct assume role permissions. | string |
"" |
no |
| k8s_db_password_secret | Name of the Kubernetes secret that would store the db password for beekeeper. | string |
"beekeeper-db-password" |
no |
| k8s_namespace | Namespace to deploy all Kubernetes resources to. | string |
"beekeeper" |
no |
| k8s_node_affinity | Full node_affinity object as per terraform/Kubernetes docs. | object({}) |
{} |
no |
| k8s_node_selector | Full node_selector object as per terraform/Kubernetes docs. | object({}) |
{} |
no |
| k8s_node_tolerations | Full k8s_node_tolerations object as per terraform/Kubernetes docs. | object({}) |
{} |
no |
| k8s_scheduler_apiary_cpu | Total cpu to allot to the Beekeeper Scheduler Apiary pod. | string |
"500m" |
no |
| k8s_scheduler_apiary_ingress_host | Ingress host name for Beekeeper Scheduler Apiary. | string |
"" |
no |
| k8s_scheduler_apiary_ingress_path | Ingress path regex for Beekeeper Scheduler Apiary. | string |
"" |
no |
| k8s_scheduler_apiary_liveness_delay | Liveness delay (in seconds) for the Beekeeper Scheduling service. | number |
60 |
no |
| k8s_scheduler_apiary_memory | Total memory to allot to the Beekeeper Scheduler Apiary pod. | string |
"2Gi" |
no |
| k8s_scheduler_apiary_port | Internal port that the Beekeeper Scheduler Apiary service runs on. | number |
8080 |
no |
| message_retention_seconds | SQS message retention (s). | string |
"604800" |
no |
| scheduler_apiary_docker_image | Beekeeper Scheduler Apiary image. | string |
"expediagroup/beekeeper-scheduler-apiary" |
no |
| scheduler_apiary_docker_image_version | Beekeeper Scheduler Apiary image version. | string |
"latest" |
no |
| scheduler_apiary_ecs_cpu | The amount of CPU used to allocate for the Beekeeper Scheduler Apiary ECS task. Valid values: https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task-cpu-memory-error.html |
string |
"2048" |
no |
| scheduler_apiary_ecs_memory | The amount of memory (in MiB) used to allocate for the Beekeeper Scheduler Apiary container. Valid values: https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task-cpu-memory-error.html |
string |
"4096" |
no |
| prometheus_enabled | Enable to pull metrics using Prometheus - true or false. | string |
"false" |
no |
| queue_name | Beekeeper SQS Queue name. | string |
"apiary-beekeeper" |
no |
| queue_stale_messages_timeout | Beekeeper SQS Queue Cloudwatch Alert timeout for messages older than this number of seconds. | string |
"1209600" |
no |
| rds_allocated_storage | RDS allocated storage in GBs. | string |
10 |
no |
| rds_engine_version | RDS engine version. | string |
"8.0" |
no |
| rds_instance_class | RDS instance class. | string |
"db.t2.micro" |
no |
| rds_max_allocated_storage | RDS max allocated storage (autoscaling) in GBs. | string |
100 |
no |
| rds_parameter_group_name | RDS parameter group. | string |
"default.mysql8.0" |
no |
| rds_storage_type | RDS storage type. | string |
"gp2" |
no |
| rds_subnets | Subnets in which to provision Beekeeper RDS DB. | list(string) |
n/a | yes |
| receive_wait_time_seconds | SQS receive wait time (s). | string |
"20" |
no |
| scheduler_apiary_delay_ms | Delay between each cleanup job that is scheduled in milliseconds. | string |
"300000" |
no |
| slack_channel | Slack channel to which alerts about messages landing on the dead letter queue should be sent. | string |
"" |
no |
| slack_lambda_enabled | Boolean flag to determine if Beekeeper should create a Slack notifying Lambda for the dead letter queue. (0 = off, 1 = on). | number |
0 |
no |
| slack_webhook_url | Slack URL to which alerts about messages landing on the dead letter queue should be sent. | string |
"" |
no |
| subnets | Subnets in which to install Beekeeper. | list(string) |
n/a | yes |
| vpc_id | VPC in which to install Beekeeper. | string |
n/a | yes |
No output.
This project is available under the Apache 2.0 License.
Copyright 2019 Expedia, Inc.