Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support secrets in tool requirements #19084

Draft
wants to merge 59 commits into
base: dev
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
59 commits
Select commit Hold shift + click to select a range
2c3bd87
init add secrets to tools
arash77 Oct 29, 2024
ec62a63
add secret requirement in tools schema
arash77 Oct 30, 2024
08adae4
check the required field with user_preferences_extra
arash77 Oct 30, 2024
ccbc10f
validate secret type and store for tool interface in user preferences
arash77 Oct 30, 2024
45ad57e
Add secrets into tools
arash77 Oct 30, 2024
b3c3d6c
Add tests for secrets in tools
arash77 Oct 30, 2024
6770499
Avoid log vault_key
arash77 Oct 31, 2024
336c5af
fix typo
arash77 Oct 31, 2024
e8006e2
cast app for using vault into StructuredApp
arash77 Oct 31, 2024
1b328b3
Add secrets parameter to parse_requirements_and_containers method in …
arash77 Oct 31, 2024
e7c45d4
add secrets into cwl and yml
arash77 Oct 31, 2024
f706a1a
Fix tool parsing test to get secrets
arash77 Oct 31, 2024
e8a527d
Fix tool tests to include secrets
arash77 Oct 31, 2024
7fb353b
Rename 'secrets' to 'credentials' in tool parsing
arash77 Nov 28, 2024
55616de
Refactor test cases to remove unused TestSecretsInExtraUserPreference…
arash77 Dec 2, 2024
49462ba
updating the credentials to the new format
arash77 Dec 3, 2024
5a9b45a
Refactor credential classes (Variable and Secret)
arash77 Dec 3, 2024
9e7a63e
user credential model
arash77 Dec 6, 2024
7500986
Add API and schema for user credentials management
arash77 Dec 6, 2024
08d4ba9
Remove unused Union import from credentials service
arash77 Dec 6, 2024
c8b623d
Add basic ToolCredentials component and related interfaces for managi…
davelopez Dec 5, 2024
c6bbc86
Refactor ToolCredentials component
davelopez Dec 5, 2024
bdfaf7a
Add badges to indicate optional and required credentials in Credentia…
davelopez Dec 5, 2024
bc23994
Add WIP user credentials store
davelopez Dec 5, 2024
ff915d9
Refactor ToolCredentials component to enhance user messaging
davelopez Dec 5, 2024
c19458d
Initialize credentials in ManageToolCredentials to use a copy
davelopez Dec 5, 2024
43607a5
Refactor credential validation logic
davelopez Dec 5, 2024
6caf92c
update OpenAPI schema for credentials endpoints
arash77 Dec 9, 2024
06ee314
Add update credentials API and payload models
arash77 Dec 9, 2024
4b4963b
Refactor credentials API schema
arash77 Dec 9, 2024
6729612
Enhance ToolEvaluator to read secrets from UserVaultWrapper and query…
arash77 Dec 9, 2024
a69dc91
Add new models for user credentials and tool credentials management
davelopez Dec 9, 2024
1f9704b
fixing model name
arash77 Dec 16, 2024
a684644
updating models for user credentials
arash77 Dec 16, 2024
f79092d
partial update for credentials api to match the new changes
arash77 Dec 16, 2024
469557c
fix linting and schema
arash77 Dec 16, 2024
a09cfc9
update credentials api
arash77 Dec 16, 2024
585f2a8
update credentials models
arash77 Dec 16, 2024
1413957
update schema
arash77 Dec 16, 2024
00cd601
fix linting
arash77 Dec 16, 2024
6af9b68
update
arash77 Dec 18, 2024
7881b37
refactor credentials API and schema
arash77 Dec 18, 2024
274dd95
refactor user credentials model and introduce variable and secret model
arash77 Dec 18, 2024
b8b5387
add user credentials, user credentials group,
arash77 Dec 19, 2024
c9353b4
update openapi schema
arash77 Dec 19, 2024
95eb67d
rename variable and secret tables to credential_variable and credenti…
arash77 Dec 19, 2024
d476ecb
add credentials attribute to MockTool class for enhanced testing
arash77 Dec 19, 2024
beec63a
rename parse_requirements_and_containers to parse_requirements,
arash77 Dec 20, 2024
445ce87
remove vault in evaluation and MinimalToolApp
arash77 Dec 20, 2024
c9195d0
Fix credentials access control in API
davelopez Dec 20, 2024
d350d5c
Refactor user credentials store
davelopez Dec 20, 2024
64d382c
Refactor ToolCredentials and related UI components
davelopez Dec 20, 2024
44da9c3
Update user_id type to support "current" in credentials
arash77 Jan 2, 2025
ba1fdc5
Fix user credentials retrieval to handle None case and ensure proper …
arash77 Jan 2, 2025
2a16949
Add integration tests for user credentials API
arash77 Jan 2, 2025
e346936
Fix API endpoint string formatting in integration tests for user cred…
arash77 Jan 2, 2025
be6eb75
Add ondelete cascade to user credential group foreign keys and update…
arash77 Jan 3, 2025
d6b92ae
Refactor credential deletion logic to use lists instead of sets and a…
arash77 Jan 3, 2025
e5c3a59
Add ondelete cascade to foreign keys in user credentials and credenti…
arash77 Jan 3, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
346 changes: 346 additions & 0 deletions client/src/api/schema/schema.ts

Large diffs are not rendered by default.

53 changes: 52 additions & 1 deletion client/src/api/users.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { GalaxyApi } from "@/api";
import { type components, GalaxyApi } from "@/api";
import { toQuotaUsage } from "@/components/User/DiskUsage/Quota/model";
import { rethrowSimple } from "@/utils/simple-error";

Expand Down Expand Up @@ -35,3 +35,54 @@ export async function fetchCurrentUserQuotaSourceUsage(quotaSourceLabel?: string

return toQuotaUsage(data);
}

export type CreateSourceCredentialsPayload = components["schemas"]["CreateSourceCredentialsPayload"];
export type ServiceCredentialPayload = components["schemas"]["ServiceCredentialPayload"];
export type ServiceGroupPayload = components["schemas"]["ServiceGroupPayload"];
export type UserCredentials = components["schemas"]["UserCredentialsResponse"];

// TODO: Change API to directly return the correct type to avoid this transformation and additional type definitions.
export function transformToSourceCredentials(
toolId: string,
toolCredentialsDefinition: ServiceCredentialsDefinition[]
): SourceCredentialsDefinition {
return {
sourceType: "tool",
sourceId: toolId,
services: new Map(toolCredentialsDefinition.map((service) => [service.reference, service])),
};
}

/**
* Represents the definition of credentials for a particular service.
*/
export interface ServiceCredentialsDefinition {
reference: string;
name: string;
optional: boolean;
multiple: boolean;
label?: string;
description?: string;
variables: ServiceVariableDefinition[];
secrets: ServiceVariableDefinition[];
}

/**
* Represents the definition of credentials for a particular source.
* A source can be a tool, a workflow, etc.Base interface for credentials definitions.
* A source may accept multiple services, each with its own credentials.
*/
export interface SourceCredentialsDefinition {
sourceType: string;
sourceId: string;
services: Map<string, ServiceCredentialsDefinition>;
}

/**
* Base interface for credential details. It is used to define the structure of variables and secrets.
*/
export interface ServiceVariableDefinition {
name: string;
label?: string;
description?: string;
}
7 changes: 7 additions & 0 deletions client/src/components/Tool/ToolCard.vue
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ import { useUserStore } from "@/stores/userStore";
import ToolSelectPreferredObjectStore from "./ToolSelectPreferredObjectStore";
import ToolTargetPreferredObjectStorePopover from "./ToolTargetPreferredObjectStorePopover";

import ToolCredentials from "./ToolCredentials.vue";
import ToolHelpForum from "./ToolHelpForum.vue";
import ToolTutorialRecommendations from "./ToolTutorialRecommendations.vue";
import ToolFavoriteButton from "components/Tool/Buttons/ToolFavoriteButton.vue";
Expand Down Expand Up @@ -174,6 +175,12 @@ const showHelpForum = computed(() => isConfigLoaded.value && config.value.enable
</div>
</div>

<ToolCredentials
v-if="props.options.credentials"
:tool-id="props.id"
:tool-version="props.version"
:tool-credentials-definition="props.options.credentials" />

<div id="tool-card-body">
<FormMessage variant="danger" :message="errorText" :persistent="true" />
<FormMessage :variant="props.messageVariant" :message="props.messageText" />
Expand Down
209 changes: 209 additions & 0 deletions client/src/components/Tool/ToolCredentials.vue
Original file line number Diff line number Diff line change
@@ -0,0 +1,209 @@
<script setup lang="ts">
import { BAlert, BButton, BModal } from "bootstrap-vue";
import { computed, ref } from "vue";

import { isRegisteredUser } from "@/api";
import {
type CreateSourceCredentialsPayload,
type ServiceCredentialsDefinition,
type SourceCredentialsDefinition,
transformToSourceCredentials,
type UserCredentials,
} from "@/api/users";
import { useUserCredentialsStore } from "@/stores/userCredentials";
import { useUserStore } from "@/stores/userStore";

import LoadingSpan from "@/components/LoadingSpan.vue";
import ManageToolCredentials from "@/components/User/Credentials/ManageToolCredentials.vue";

interface Props {
toolId: string;
toolVersion: string;
toolCredentialsDefinition: ServiceCredentialsDefinition[];
}

const props = defineProps<Props>();

const userStore = useUserStore();
const userCredentialsStore = useUserCredentialsStore(
isRegisteredUser(userStore.currentUser) ? userStore.currentUser.id : "anonymous"
);

const isBusy = ref(true);
const busyMessage = ref<string>("");
const userCredentials = ref<UserCredentials[] | undefined>(undefined);

const credentialsDefinition = computed<SourceCredentialsDefinition>(() => {
return transformToSourceCredentials(props.toolId, props.toolCredentialsDefinition);
});

const hasUserProvidedRequiredCredentials = computed<boolean>(() => {
if (!userCredentials.value || userCredentials.value.length === 0) {
return false;
}

return userCredentials.value.every((credentials) => areOptional(credentials) || areSetByUser(credentials));
});

const hasUserProvidedAllCredentials = computed<boolean>(() => {
if (!userCredentials.value || userCredentials.value.length === 0) {
return false;
}
return userCredentials.value.every(areSetByUser);
});

const hasSomeOptionalCredentials = computed<boolean>(() => {
for (const credentials of credentialsDefinition.value.services.values()) {
if (credentials.optional) {
return true;
}
}
return false;
});

const hasSomeRequiredCredentials = computed<boolean>(() => {
for (const credentials of credentialsDefinition.value.services.values()) {
if (!credentials.optional) {
return true;
}
}
return false;
});

const provideCredentialsButtonTitle = computed(() => {
return hasUserProvidedRequiredCredentials.value ? "Manage credentials" : "Provide credentials";
});

const bannerVariant = computed(() => {
if (isBusy.value) {
return "info";
}
return hasUserProvidedRequiredCredentials.value ? "success" : "warning";
});

const showModal = ref(false);

/**
* Check if the user has credentials for the tool.
* @param providedCredentials - The provided credentials to check. If not provided, the function will fetch the
* credentials from the store if they exist.
*/
async function checkUserCredentials(providedCredentials?: UserCredentials[]) {
busyMessage.value = "Checking your credentials...";
isBusy.value = true;
try {
if (userStore.isAnonymous) {
return;
}

if (!providedCredentials) {
providedCredentials =
userCredentialsStore.getAllUserCredentialsForTool(props.toolId) ??
(await userCredentialsStore.fetchAllUserCredentialsForTool(props.toolId));
}

userCredentials.value = providedCredentials;
} catch (error) {
// TODO: Implement error handling.
console.error("Error checking user credentials", error);
} finally {
isBusy.value = false;
}
}

function areSetByUser(credentials: UserCredentials): boolean {
return Object.values(credentials.groups).every((set) => {
return set.variables.every((variable) => variable.value) && set.secrets.every((secret) => secret.already_set);
});
}

function areOptional(credentials: UserCredentials): boolean {
const matchingDefinition = credentialsDefinition.value.services.get(credentials.reference);
if (!matchingDefinition) {
return false;
}
return matchingDefinition.optional;
}

function provideCredentials() {
showModal.value = true;
}

async function onSavedCredentials(providedCredentials: CreateSourceCredentialsPayload) {
showModal.value = false;
busyMessage.value = "Saving your credentials...";
try {
isBusy.value = true;
userCredentials.value = await userCredentialsStore.saveUserCredentialsForTool(providedCredentials);
} catch (error) {
// TODO: Implement error handling.
console.error("Error saving user credentials", error);
} finally {
isBusy.value = false;
}
}

checkUserCredentials();
</script>

<template>
<div>
<BAlert show :variant="bannerVariant" class="tool-credentials-banner">
<LoadingSpan v-if="isBusy" :message="busyMessage" />
<div v-else-if="userStore.isAnonymous">
<span v-if="hasSomeRequiredCredentials">
<strong>
This tool requires credentials to access its services and you need to be logged in to provide
them.
</strong>
</span>
<span v-else>
This tool <strong>can use additional credentials</strong> to access its services
<strong>or you can use it anonymously</strong>.
</span>
<br />
Please <a href="/login/start">log in or register here</a>.
</div>
<div v-else class="d-flex justify-content-between align-items-center">
<div class="credentials-info">
<span v-if="hasUserProvidedRequiredCredentials">
<strong>You have already provided credentials for this tool.</strong> You can update or delete
your credentials, using the <i>{{ provideCredentialsButtonTitle }}</i> button.
<span v-if="hasSomeOptionalCredentials && !hasUserProvidedAllCredentials">
<br />
You can still provide some optional credentials for this tool.
</span>
</span>
<span v-else-if="hasSomeRequiredCredentials">
This tool <strong>requires you to enter credentials</strong> to access its services. Please
provide your credentials before using the tool using the
<i>{{ provideCredentialsButtonTitle }}</i> button.
</span>
<span v-else>
This tool <strong>can use credentials</strong> to access its services. If you don't provide
credentials, you can still use the tool, but you will access its services
<strong>anonymously</strong> and in some cases, with limited functionality.
</span>
</div>

<BButton variant="primary" size="sm" class="provide-credentials-btn" @click="provideCredentials">
{{ provideCredentialsButtonTitle }}
</BButton>
</div>
</BAlert>
<BModal v-model="showModal" title="Manage Tool Credentials" hide-footer>
<ManageToolCredentials
:tool-id="props.toolId"
:tool-version="props.toolVersion"
:tool-credentials-definition="credentialsDefinition"
:user-tool-credentials="userCredentials"
@save-credentials="onSavedCredentials" />
</BModal>
</div>
</template>

<style scoped>
.tool-credentials-banner {
margin-bottom: 1rem;
}
</style>
Loading
Loading