Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ENG-1271 feat(demo): Add AWS to break glass demo (elevated permissions) #33

Open
wants to merge 7 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 42 additions & 0 deletions break_glass/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
# Break Glass Request and Approval Orchestration

This program orchestrates the request and approval process for break glass scenarios, where a developer needs elevated permissions to access sensitive data or perform critical operations beyond their usual access.

## Benefits

1. **Controlled Access**: Ensures sensitive data and operations are only accessible when necessary and with proper approval.
2. **Audit Trail**: Maintains a clear record of who requested and approved access, providing transparency and accountability.
3. **Automated Workflow**: Streamlines the request and approval process through automation, reducing manual overhead and potential for errors.

## How It Works

1. **Request Initiation**:
- A developer initiates the process using a Slack slash command to request break glass approval.

2. **Information Gathering**:
- AutoKitteh sends a form to the developer, requesting details about the reason for the elevated access.
- The developer fills out the form, providing the necessary information and justification.

3. **Verification**:
- The program integrates with Jira to verify the existence of the ticket and ensure the requester is the ticket's assignee.

4. **Approval Request**:
- AutoKitteh sends a notification to the Site Reliability Engineering (SRE) team with an approve/deny message, including the details of the request.
- The SRE team reviews the request and makes a decision.

5. **Notification**:
- AutoKitteh notifies the developer of the decision via Slack, indicating whether the request was approved or denied.

6. **Permission Management**:
- If approved, permissions are granted to the developer, with a set expiration time.
- The system monitors the permissions and automatically removes them once they expire, ensuring minimal risk.

7. **Integration with Services**:
- The program uses Slack for communication and notifications.
- It also integrates with AWS for permission management, Redis for storing timestamps, and Google Sheets for tracking purposes.

## Usage

To use this program, ensure you have the necessary environment variables set, such as `APPROVAL_CHANNEL`, `SHEET_ID`, `SHEETS_RANGE`, and other connection details for AWS, Google Sheets, Jira, Redis, and Slack.

The workflow begins when a developer uses the Slack slash command to initiate a break glass request. The program then follows the outlined steps to ensure controlled, auditable access to sensitive resources.
27 changes: 26 additions & 1 deletion break_glass/autokitteh.yaml
Original file line number Diff line number Diff line change
@@ -1,13 +1,38 @@
# This YAML file is a declarative manifest that describes the setup
# of the AutoKitteh project "Break Glass".
# Break Glass integrates AWS, Jira, Redis, and Slack to streamline
# emergency access requests.
#
# Before applying this file:
# - Modify the values in the project's "vars" section, if desired
# - Modify the Redis connection string, if needed
#
# After applying this file, initialize this AutoKitteh project's
# AWS, Jira and Slack connections.

version: v1

project:
name: break_glass
vars:
- name: APPROVAL_CHANNEL
value: "sre-team"
value: <slack-channel> # Modify this if needed.
- name: AWS_ADMIN_GROUP
value: break-glass-admin # Modify this if needed.
- name: REQUESTER_EMAIL # can be replaced with a database or spreadsheet lookup
value: <your-email>
- name: PERMISSION_EXPIRY
value: 10 # seconds
connections:
- name: aws_connection
integration: aws
- name: jira_connection
integration: jira
- name: redis_connection
integration: redis
vars:
- name: url
value: redis://localhost:6379/0 # Modify this if needed.
- name: slack_connection
integration: slack
triggers:
Expand Down
80 changes: 67 additions & 13 deletions break_glass/program.py
Original file line number Diff line number Diff line change
Expand Up @@ -23,15 +23,23 @@

import os
from pathlib import Path
import time

import autokitteh
from autokitteh.atlassian import atlassian_jira_client
from autokitteh.aws import boto3_client
from autokitteh.redis import redis_client
from autokitteh.slack import slack_client
from requests.exceptions import HTTPError


APPROVAL_CHANNEL = os.getenv("APPROVAL_CHANNEL")
AWS_ADMIN_GROUP = os.getenv("AWS_ADMIN_GROUP")
PERMISSION_EXPIRY = os.getenv("PERMISSION_EXPIRY")
REQUESTER_EMAIL = os.getenv("REQUESTER_EMAIL")
aws = boto3_client("aws_connection", "iam")
jira = atlassian_jira_client("jira_connection")
redis = redis_client("redis_connection")
slack = slack_client("slack_connection")


Expand All @@ -42,7 +50,6 @@ def on_slack_slash_command(event):
slack.views_open(trigger_id=trigger_id, view=request_modal)


@autokitteh.activity
def on_form_submit(event):
reason, issue_key, base_url, requester_id = parse_event_data(event)

Expand All @@ -62,22 +69,26 @@ def on_form_submit(event):
slack.chat_postMessage(channel=requester_id, text="Request sent for approval.")


@autokitteh.activity
def on_approve_deny(event):
"""Processes the approval/denial of the request and notifies the requester."""
action_id = event.data["actions"][0]["action_id"]
_, requester, issue_key = action_id.split(" ")
approver_id = event.data["user"]["id"]
approver_info = slack.users_info(user=approver_id)

if event.data["actions"][0]["value"] == "Approve":
approver_email = approver_info["user"]["profile"]["email"]
jira.issue_add_comment(issue_key, f"Request approved by: {approver_email}")
message = f"Request approved by: <@{approver_info['user']['name']}>"
slack.chat_postMessage(channel=requester, text=message)
else:
print(f"Requester: {requester}")
if event.data["actions"][0]["value"] != "Approve":
message = f"Request denied by: <@{approver_info["user"]["name"]}>"
slack.chat_postMessage(channel=requester, text=message)
return

approver_email = approver_info["user"]["profile"]["email"]
jira.issue_add_comment(issue_key, f"Request approved by: {approver_email}")
message = f"Request approved by: <@{approver_info['user']['name']}>"
slack.chat_postMessage(channel=requester, text=message)

aws_user = lookup_user_by_email(approver_email)
set_permissions(aws_user)
monitor_and_remove_permissions(aws_user, requester)


def send_approval_request(reason, issue_key, base_url, requester_id):
Expand All @@ -94,6 +105,18 @@ def send_approval_request(reason, issue_key, base_url, requester_id):
slack.chat_postMessage(channel=APPROVAL_CHANNEL, blocks=blocks)


def monitor_and_remove_permissions(aws_user, slack_user):
while True:
timestamp = get_user_timestamp(aws_user)
if not timestamp:
return
if float(timestamp) < time.time():
break
time.sleep(10)
expire_permissions(aws_user)
slack.chat_postMessage(channel=slack_user, text="Your permissions have expired.")


def parse_event_data(event):
form_data = event.data["view"]["state"]["values"]
reason = form_data["block_reason"]["reason"]["value"]
Expand All @@ -103,6 +126,25 @@ def parse_event_data(event):
return reason, issue_key, base_url, requester_id


def validate_requester(issue_key, requester):
issue = jira.issue(issue_key)
assignee = issue.get("fields", {}).get("assignee", {})
if assignee is None:
return False
email = assignee.get("emailAddress", "")
return email == requester


def lookup_user_by_email(email):
# this is meant to be a placeholder for a real lookup function
email_to_user = {
REQUESTER_EMAIL: "break-glass-test-user",
"[email protected]": "random-user",
}
return email_to_user.get(email)


@autokitteh.activity
def check_issue_exists(issue_key):
try:
jira.issue(issue_key)
Expand All @@ -112,7 +154,19 @@ def check_issue_exists(issue_key):
return False


def validate_requester(issue_key, requester):
issue = jira.issue(issue_key)
assignee = issue.get("fields", {}).get("assignee", {}).get("emailAddress", "")
return assignee == requester
@autokitteh.activity
def set_permissions(user_name):
aws.add_user_to_group(GroupName=AWS_ADMIN_GROUP, UserName=user_name)
time_alotted = os.getenv("PERMISSION_EXPIRY")
redis.set(user_name, time.time() + float(time_alotted))


@autokitteh.activity
def expire_permissions(user_name):
aws.remove_user_from_group(GroupName=AWS_ADMIN_GROUP, UserName=user_name)
redis.delete(user_name)


@autokitteh.activity
def get_user_timestamp(user_name):
return redis.get(user_name)