Merge branch 'develop' into arh-wor-1718

README.md

@@ -5,18 +5,23 @@
 The crux of IAM in Sam is a policy. A policy says **who** can **do what** to a **thing**. More technically the who is called a **subject** and can be a user or a group of users, the do what is called an **action** such as read or update, and the thing is called a **resource** such as a workspace or project. Resources have types which specify what actions are available for its resources, roles (which are collections of actions) and which role is the "owner" role. The "owner" role should have the appropriate actions to administer a resource. When a resource is created a policy with the owner role is automatically created and the creator is added.
 
 ## Terms
-* Subject - an authenticated user or group
+* Subject - an authenticated user, group or policy (policies contain a set of subjects and can be treated as a group)
 * Resource - something to which access is controlled
 * Action - may be performed on a resource - meant to be as granular as possible
-* Policy - represents the actions a subject may perform on a resource
+* Policy - represents the actions a set of subjects may perform on a resource
 * Role - a collection of actions - meant to aggregate actions into a more meaningful, higher level concept
 * Group - a group of subjects (this can include groups)
 * Resource type - defines a class of resources. Each resource has a type which defines
   * Available actions
   * Available roles and actions for each role
-  * Of the available roles which is the “owner” role - this is used when creating a resource to give the creator ownership access
+  * Of the available roles which is the “owner” role - this is used make sure resources are not orphaned 
 
-## Requirements
+## Best Practices
+* Use roles to aggregate actions into a more meaningful, higher level concept. Changing roles is a configuration change and affects all resources with that role, easy. Changing policies requires an api call or direct database updates and affects only the resource the policy is attached to, hard in bulk.
+* Define actions to be as granular as possible. This allows for better composability of roles.
+* Check only 1 action per api call. Complex checks involving multiple actions or even roles are code smells indicating poorly modeled access control. How a subject gets an action might be complicated (groups, hierarchy, etc.) but the action itself should be simple. Of course, there are exceptions to this rule, such as apis that deal with more than one resource, but they should be a minority.
+
+## Design
 ### Guiding Principles
 There are no special/super users in this system. All api calls authenticate as subjects with access rights determined by policies in the same way. In other words, this system should use its own policy mechanisms internally for any authorization needs. (Note that this does leave the problem of bootstrapping, i.e. how is the first user created, which can be achieved by scripts outside the system with direct data store level access.)
 This system can be publicly facing. This does not mean that it will be in all cases but it should be designed with this in mind.
@@ -29,7 +34,7 @@ Evaluation is the act of determining what a user may access.
 1. Given a user and resource, list all the actions the user may perform on that resource
 1. Given a user and resource, list all the user’s roles on that resource
 
-Of these 1 and 2 are the most important from a performance standpoint. Expect 1 to be called for almost every api call in a system. Expect 2 to be called from UI list pages where users generally want a snappy response.
+Of these 1 and 2 are the most important from a performance standpoint. Expect 1 to be called for almost every api call in a system. Expect 2 to be called from UI list pages where users generally want a snappy response. 2 and 4 should never be used to make access decisions because role definitions may change, they are for informational purposes only.
 
 ### Resource and Policy Management
 A resource may be part of a hierarchy of resources. A parent may be set on a resource. To do so, users must have the set_parent action on the resource and the add_child action on the would be parent. Ancestor resources in the hierarchy control permissions on all descendants. 
@@ -41,16 +46,14 @@ A policy is specific to a resource and a resource may have multiple policies. Ea
 * A set of descendant permissions - roles and actions applicable to descendant resources
 All of the subjects may perform all of the actions/roles in the policy. A policy may also be marked as public effectively meaning all users are members. Each policy has a name that is unique within a resource. Access to actions through policies is additive (i.e. the actions available to a user on a resource is an accumulation of all policies the user is a member of for that resource).
 
-There must be functions to create, delete and manage policies for resources. There must be access control around deleting resources and managing policies. There must be some built-in actions to do so (delete, read-policies, alter-policies). 
-
-There must be functions to create and delete resources. When a resource is created the caller should be the “owner.” The “owner” role generally will include delete action and actions to control sharing but need not always (e.g. if a resource may never be deleted then an owner would not have delete permissions). The actions that make up the “owner” role are defined by the resource type.
+The “owner” role of a resource generally will include delete action and actions to control sharing but need not always (e.g. if a resource may never be deleted then an owner would not have delete permissions). The actions that make up the “owner” role are defined by the resource type.
 
-Resource types define the set of available actions for all resources of that type. It also defines a set of roles and their associated actions. Roles are useful because it can be cumbersome to deal with granular actions and as a point of extensibility (when new actions are added to resource types, they can be added to roles as well, effectively adding the action to all resources with that role). It is not yet necessary to provide apis to create and maintain resource types, this can be achieved through configuration.
+Resource types define the set of available actions for all resources of that type. It also defines a set of roles and their associated actions. Roles are useful because it can be cumbersome to deal with granular actions and as a point of extensibility (when new actions are added to resource types, they can be added to roles as well, effectively adding the action to all resources with that role). Creating and maintaining resource types is achieved through [configuration](src/main/resources/reference.conf).
 
 ### Public Policies
 There are some cases where it is desirable to grant actions or roles to all authenticated users. For example, granting read-only access to public workspaces. In this case a policy can be created that has the appropriate actions or roles and set to public. Resources with public policies show up when listing resources for a user. For this reason it is not always desirable to allow everyone to make public policies. Again, the example is public workspaces. Public workspaces show up for everyone and should be curated.
 
-To change a policy's public status the caller must be able to share the policy (either via `alter_policies` and `share_policy::{policy_name}` actions) _and_ must have the `set_public` action on the resource `resource_type_admin/{resource type name}`. `resource_type_admin` is an internally created resource type. `{resource type name}` is for the resource containing the policy. Note that every resource type in sam has a resource of the same name of type `resource_type_admin` which is automatically created. When these resources are created they do not have owners, permissions must be granted via direct postgres changes.
+To change a policy's public status the caller must be able to share the policy (either via `alter_policies` and `share_policy::{policy_name}` actions) _and_ must have the `set_public` action on the resource `resource_type_admin/{resource type name}`. `resource_type_admin` is an internally created resource type. `{resource type name}` is for the resource containing the policy. Note that every resource type in sam has a resource of the same name of type `resource_type_admin` which is automatically created. When these resources are created they do not have owners, permissions must be granted via admin api calls.
 
 ### User and Group Management
 User - Create, enable, disable, get status. Disabled users should be rejected from any api calls. Enabling a user should reinstate any prior access.
@@ -123,10 +126,6 @@ class SamClient(samBasePath: String) {
 * Proxy groups - each user with access to google resources should have a google group known as a proxy. The proxy is 1-to-1 with the user and the user is member of the proxy. The proxy group should be used in place of the user in Google IAM policies and Google groups. Users should not be added directly. This allows easy enable and disable functionality by adding/removing users to their proxy groups. It also allows creation of service accounts that can act as the user (see pet service accounts below).
 * Pet service accounts - Google Compute Engine requires a service account to run compute. Service account credentials are the default credentials on any GCE instance. This is the best way at this time to provide credentials to any processes running on a GCE instance. Pet service accounts correspond with 1 and only 1 user, are added to the user’s proxy group and can call system apis as the user. In this way a pet service account can act as the user in all respects that can be controlled by the system (resources outside control of the system need to be manually shared by the user with the proxy group).
 
-#### Proposed model for accessing external google resources
-![Data Access](data_access.png)
-
-Note that Sam does not actually launch workflows create VMs but appears to in this diagram in order to simplify interactions. The key concept is the user of service accounts.
 #### Google integration requires
 * a GSuite domain
 * a project with a service account for the sam application

env/local.env

@@ -5,7 +5,7 @@ export ADMIN_SERVICE_ACCOUNT_4="src/main/resources/rendered/admin-service-accoun
 export ADMIN_SERVICE_ACCOUNT_5="src/main/resources/rendered/admin-service-account-5.json"
 export SERVICE_ACCOUNT_ADMINS="[email protected], [email protected]"
 export AZURE_ENABLED="false"
-export AZURE_SERVICE_CATALOG_APPS_ENABLED="false"
+export AZURE_SERVICE_CATALOG_ENABLED="false"
 export AZURE_MANAGED_APP_WORKLOAD_CLIENT_ID="661e243c-5ef9-4a9c-9be3-b7f5585828b3"
 export EMAIL_DOMAIN="dev.test.firecloud.org"
 export ENVIRONMENT="dev"

env/test.env

@@ -6,7 +6,7 @@ export AZURE_MANAGED_APP_CLIENT_SECRET="foo"
 export AZURE_MANAGED_APP_TENANT_ID="foo"
 export AZURE_MANAGED_APP_WORKLOAD_CLIENT_ID="foo"
 export AZURE_ALLOW_MANAGED_IDENTITY_USER_CREATION="false"
-export AZURE_SERVICE_CATALOG_APPS_ENABLED="false"
+export AZURE_SERVICE_CATALOG_ENABLED="false"
 export EMAIL_DOMAIN="dev.test.firecloud.org"
 export ENVIRONMENT="local"
 export GOOGLE_APPS_DOMAIN="test.firecloud.org"

pact4s/src/test/scala/org/broadinstitute/dsde/workbench/sam/provider/SamProviderSpec.scala

@@ -137,7 +137,7 @@ class SamProviderSpec
       when {
         googleExt.getArbitraryPetServiceAccountToken(any[SamUser], any[Set[String]], any[SamRequestContext])
       } thenReturn {
-        Future.successful("aToken")
+        IO.pure("aToken")
       }
     )
   } yield ()

src/main/resources/org/broadinstitute/dsde/sam/liquibase/changelog.xml

@@ -31,4 +31,6 @@
     <include file="changesets/20240417_action_managed_identities.xml" relativeToChangelogFile="true"/>
     <include file="changesets/20240416_add_sam_rac_tables.xml" relativeToChangelogFile="true"/>
     <include file="changesets/20240701_add_group_version_and_last_synchronized_version.xml" relativeToChangelogFile="true"/>
+    <include file="changesets/20240809_sam_user_favorite_resources_table.xml" relativeToChangelogFile="true"/>
+    <include file="changesets/20240820_descendant_auth_domains.xml" relativeToChangelogFile="true"/>
 </databaseChangeLog>

src/main/resources/org/broadinstitute/dsde/sam/liquibase/changesets/20240809_sam_user_favorite_resources_table.xml

@@ -0,0 +1,20 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<databaseChangeLog logicalFilePath="dummy"
+                   xmlns="http://www.liquibase.org/xml/ns/dbchangelog"
+                   xmlns:ext="http://www.liquibase.org/xml/ns/dbchangelog-ext"
+                   xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+                   xsi:schemaLocation="http://www.liquibase.org/xml/ns/dbchangelog-ext http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-ext.xsd http://www.liquibase.org/xml/ns/dbchangelog http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-3.4.xsd">
+
+  <changeSet logicalFilePath="dummy" author="tlangs" id="add_sam_user_favorite_resources_table">
+    <createTable tableName="SAM_USER_FAVORITE_RESOURCES">
+      <column name="sam_user_id" type="VARCHAR">
+        <constraints primaryKey="true" foreignKeyName="FK_SUFR_USER_ID" referencedTableName="SAM_USER" referencedColumnNames="id" deleteCascade="true"/>
+      </column>
+      <column name="resource_id" type="BIGINT">
+        <constraints nullable="false" primaryKey="true" foreignKeyName="FK_SUFR_RESOURCE" referencedTableName="SAM_RESOURCE" referencedColumnNames="id" deleteCascade="true"/>
+      </column>
+      <column name="created_at" type="timestamptz" defaultValueComputed="now()"/>
+    </createTable>
+
+  </changeSet>
+</databaseChangeLog>

src/main/resources/org/broadinstitute/dsde/sam/liquibase/changesets/20240820_descendant_auth_domains.xml

@@ -0,0 +1,27 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<databaseChangeLog logicalFilePath="dummy"
+                   xmlns="http://www.liquibase.org/xml/ns/dbchangelog"
+                   xmlns:ext="http://www.liquibase.org/xml/ns/dbchangelog-ext"
+                   xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+                   xsi:schemaLocation="http://www.liquibase.org/xml/ns/dbchangelog-ext http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-ext.xsd http://www.liquibase.org/xml/ns/dbchangelog http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-3.4.xsd">
+
+    <changeSet logicalFilePath="dummy" author="dvoet" id="populate_descendant_auth_domains" runInTransaction="true">
+        <sql stripComments="true">
+            with recursive
+                ancestor_resource(resource_id, group_id) as (
+                    select ad.resource_id, ad.group_id
+                    from sam_resource_auth_domain ad
+                union
+                    select childResource.id, ancestorResource.group_id
+                    from SAM_RESOURCE childResource
+                    join ancestor_resource ancestorResource on ancestorResource.resource_id = childResource.resource_parent_id
+                )
+
+            insert into sam_resource_auth_domain(resource_id, group_id)
+            select ar.resource_id, ar.group_id
+            from ancestor_resource ar
+            on conflict do nothing
+        </sql>
+    </changeSet>
+
+</databaseChangeLog>

src/main/resources/reference.conf

@@ -36,6 +36,9 @@ resourceTypes = {
       admin_read_summary_information = {
         description = "view summary information on resources of this resource type"
       }
+      admin_specify_acting_user = {
+        description = "specify a different user that is preforming a given action on the resource"
+      }
     }
 
     ownerRoleName = "owner"
@@ -227,6 +230,24 @@ resourceTypes = {
           google-project = ["pet-creator"]
         }
       }
+      rawls = {
+        roleActions = [
+          # workspace clone and delete
+          "read_job_result"
+          # workspace clone - create WDS
+          "create_controlled_user_private"
+          # workspace clone - create storage container
+          "create_controlled_user_shared"
+          # workspace delete - leo checks for this action when deleting runtimes
+          "delete"
+          # workspace delete - WSM ensures there are no children before deleting
+          "list_children"
+          # workspace clone - create WDS
+          "add_child"
+          # workspace clone - get storage container, get cloud context and spend profile id
+          "read"
+        ]
+      }
     }
     authDomainConstrainable = true
     allowLeaving = true
@@ -282,6 +303,14 @@ resourceTypes = {
       reader = {
         roleActions = ["read"]
       }
+      rawls = {
+        roleActions = [
+          # workspace clone - read source workspace storage containers
+          "read"
+          # wds clone needs to write db backup to target workspace storage container
+          "write"
+        ]
+      }
     }
     reuseIds = false
   }
@@ -391,6 +420,12 @@ resourceTypes = {
       reader = {
         roleActions = ["read"]
       }
+      rawls = {
+        roleActions = [
+          # workspace delete
+          "delete"
+        ]
+      }
     }
     reuseIds = false
   }
@@ -738,7 +773,14 @@ resourceTypes = {
           google-project = ["notebook-user"]
         }
       }
+      rawls = {
+        roleActions = [
+          # billing project delete
+          "delete"
+        ]
+      }
     }
+    allowLeaving = true
     reuseIds = true
   }
   notebook-cluster = {
@@ -1099,28 +1141,48 @@ resourceTypes = {
       "read_spend_report" = {
         description = "read spend report for this spend profile"
       }
+      "read_job_result" = {
+        description = "allows reading the result of a job"
+      }
+      "read_profile" = {
+        description = "read spend profile"
+      }
     }
     ownerRoleName = "owner"
     roles = {
       owner = {
         descendantRoles = {
           landing-zone = ["owner"]
         }
-        roleActions = ["update_billing_account", "update_metadata", "delete", "link", "share_policy::owner", "share_policy::user", "share_policy::pet-creator", "read_policies", "add_child", "list_children", "view_journal", "set_managed_resource_group", "create-pet", "read_spend_report"]
+        roleActions = ["update_billing_account", "update_metadata", "delete", "link", "share_policy::owner", "share_policy::user", "share_policy::pet-creator", "read_policies", "add_child", "list_children", "view_journal", "set_managed_resource_group", "create-pet", "read_spend_report", "read_job_result", "read_profile"]
       }
       user = {
         descendantRoles = {
           landing-zone = ["user"]
         }
-        roleActions = ["link", "share_policy::user", "share_policy::pet-creator", "read_policy::pet-creator", "add_child", "create-pet"]
+        roleActions = ["link", "share_policy::user", "share_policy::pet-creator", "read_policy::pet-creator", "add_child", "create-pet", "read_job_result", "read_profile"]
       }
       admin = {
-        roleActions = ["share_policy::owner", "read_policies", "alter_policies", "delete"]
+        roleActions = ["share_policy::owner", "read_policies", "alter_policies", "delete", "read_profile"]
       }
       pet-creator = {
-        roleActions = ["create-pet", "share_policy::pet-creator", "read_policy::pet-creator"]
+        roleActions = ["create-pet", "share_policy::pet-creator", "read_policy::pet-creator", "read_profile"]
+      }
+      system = {
+        roleActions = ["read_profile"]
+      }
+      rawls = {
+        roleActions = [
+          # landing zone creation, billing project delete
+          "read_job_result"
+          # billing project delete
+          "delete"
+          # leonardo creates a pet even for a shared app
+          "create-pet"
+        ]
       }
     }
+    allowLeaving = true
     reuseIds = true
   }
   study = {
@@ -1421,6 +1483,12 @@ resourceTypes = {
       user = {
         roleActions = ["list_resources"]
       }
+      rawls = {
+        roleActions = [
+          # billing project delete
+          "list_resources"
+        ]
+      }
     }
     reuseIds = ${?LANDINGZONES_REUSE_IDS}
   }