Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft: submit dummy Slurm job #239

Draft
wants to merge 27 commits into
base: develop
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from 25 commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
452ce9e
Testing with Slurm API
hannahle Mar 29, 2022
2d3e499
Changed api endpoint
hannahle Mar 29, 2022
a34c96e
Dummy hello world job
hannahle Mar 29, 2022
a12f872
Added working dir
hannahle Mar 29, 2022
cfd8cab
experimenting with job ID
hannahle Mar 31, 2022
99bb91a
Added tested curl request to get access token
hannahle Apr 14, 2022
f1551a6
added curl request for posting data to omsp protected server endpoint
hannahle Apr 14, 2022
e153ef4
experimenting with post request on backend
hannahle Apr 19, 2022
266465b
mocked up slurm subscription
hannahle Apr 20, 2022
af27d1b
working subscription on backend, front end buggy
hannahle Apr 21, 2022
2b2f561
Draft code
hannahle Apr 26, 2022
eeb6464
working communication between front end backend
hannahle Apr 26, 2022
5f861fc
draft for securing endpoint
hannahle Apr 27, 2022
20ed55f
testing keycloak verify
hannahle Apr 28, 2022
83bf685
working authorization
hannahle May 2, 2022
c77b0e8
Merge branch 'develop' into hannah/slurm-dummy
hannahle May 3, 2022
555d086
Added slurm adapter and merge with variants query result
hannahle May 4, 2022
8a94f00
added todo to mutate results on backend
hannahle May 4, 2022
3026901
cleanup
hannahle May 4, 2022
376cbf3
cleaned up unused packages
hannahle May 4, 2022
fdfb7f6
added timeout link
hannahle May 4, 2022
7f8afd7
added variants query subscription
hannahle May 5, 2022
cbd5069
Updating front-end data based on subscription
hannahle May 8, 2022
4f378fc
clean up logs
hannahle May 8, 2022
38ccbea
Merge branch 'develop' into hannah/slurm-dummy
hannahle May 9, 2022
a9860ba
fixed linting issues
hannahle May 9, 2022
df1a4d3
Make working directory an env variable
jennyziyi-xu Jul 7, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -98,6 +98,18 @@ docker exec -i <keycloak-container-name> bash /usr/scripts/bootstrap-keycloak.sh

The keycloak admin portal can be accessed in the browser by navigating to localhost and the port specified by the `KEYCLOAK_PORT` env var, e.g., `localhost:9821`

To request an access token to Keycloak:
```bash
export TOKEN=$(curl --location --request POST 'http://localhost:9821/auth/realms/ssmp/protocol/openid-connect/token' --header 'Content-Type: application/x-www-form-urlencoded' --data-urlencode 'password=secret' --data-urlencode 'username=ssmp-user' --data-urlencode 'client_id=ssmp-backend' --data-urlencode 'realm=ssmp' --data-urlencode 'grant_type=password' | jq -r '.access_token')
```

To access a protected endpoint on the backend:
```bash
curl -X POST http://localhost:5862/graphql -H "Authorization: Bearer '$TOKEN'" \
--header 'Content-Type: application/json' \
--data-raw <JSON>
```

## Mongo

Annotations can be imported into mongo using the following command. Note that that the headers should not be included in the csv and the order of the fields passed to the `fields` argument should match the order of the fields in the csv.
Expand Down
3 changes: 3 additions & 0 deletions docker-compose.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,9 @@ services:
TEST_NODE_SSMP_TOKEN_ENDPOINT:
TEST_NODE_TOKEN_AUDIENCE:
MONGO_CONNECTION_STRING: mongodb://${MONGO_INITDB_ROOT_USERNAME}:${MONGO_INITDB_ROOT_PASSWORD}@mongo/${MONGO_INITDB_DATABASE}?authSource=admin
SLURM_USER:
SLURM_JWT:
SLURM_ENDPOINT:
mongo:
image: mongo:latest
environment:
Expand Down
16 changes: 14 additions & 2 deletions react/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -4,16 +4,24 @@
"private": true,
"proxy": "http://ssmp-server:3000",
"dependencies": {
"@apollo/client": "^3.5.10",
"@apollo/link-error": "^2.0.0-beta.3",
"@apollo/react-hooks": "^4.0.0",
"@graphql-tools/schema": "^8.3.10",
"@react-keycloak/web": "^3.4.0",
"@types/uuid": "^8.3.1",
"apollo-client": "^2.6.10",
"apollo-link-rest": "^0.8.0-beta.0",
"apollo-link": "^1.2.14",
"apollo-link-rest": "^0.9.0-rc.1",
"apollo-link-timeout": "^4.0.0",
"apollo-server-core": "^3.6.7",
"crypto": "^1.0.1",
"crypto-hash": "^2.0.1",
"framer-motion": "^4.1.17",
"gql-tag": "^1.0.1",
"graphql": "^15.5.1",
"graphql-anywhere": "^4.2.7",
"graphql-ws": "^5.8.1",
"keycloak-js": "^15.0.0",
"react": "^17.0.2",
"react-beautiful-dnd": "^13.1.0",
Expand All @@ -25,7 +33,11 @@
"react-router-dom": "^5.2.0",
"react-table": "^7.7.0",
"styled-components": "5.3.0",
"uuid": "^8.3.2"
"uuid": "^8.3.2",
"ws": "^8.5.0"
},
"resolutions": {
"apollo-link": "^1.2.14"
},
"devDependencies": {
"@storybook/addon-actions": "^6.3.4",
Expand Down
58 changes: 51 additions & 7 deletions react/src/apollo/client.ts
Original file line number Diff line number Diff line change
@@ -1,16 +1,28 @@
import {
ApolloClient,
ApolloLink,
createHttpLink,
from,
InMemoryCache,
NextLink,
Observable,
Operation,
QueryHookOptions,
split,
SubscriptionHookOptions,
useLazyQuery,
useQuery,
useSubscription,
} from '@apollo/client';
import { onError } from '@apollo/client/link/error';
import { ApolloLink, QueryHookOptions, useQuery } from '@apollo/react-hooks';

import { GraphQLWsLink } from '@apollo/client/link/subscriptions';

import { getMainDefinition } from '@apollo/client/utilities';
import { onError } from '@apollo/link-error';
import { RestLink } from 'apollo-link-rest';
import ApolloLinkTimeout from 'apollo-link-timeout';
import { DocumentNode } from 'graphql';
import { createClient } from 'graphql-ws';
import { makeGraphQLError, makeNetworkError, makeNodeError } from '../components';
import { useErrorContext } from '../hooks';
import { VariantQueryResponseError } from '../types';
Expand All @@ -27,12 +39,29 @@ export const buildLink = (token?: string) => {
headers: { accept: 'application/json' },
});

const remoteNodeErrorLink = new ApolloLink((operation, forward) => {
const wsLink = new GraphQLWsLink(
createClient({
url: `ws://localhost:5862/graphql`,
})
);

const splitLink = split(
({ query }) => {
const definition = getMainDefinition(query);
return (
definition.kind === 'OperationDefinition' && definition.operation === 'subscription'
);
},
wsLink,
httpLink
);

const remoteNodeErrorLink = new ApolloLink((operation: Operation, forward: NextLink) => {
return new Observable(observer => {
const dispatcherContext = operation.getContext();
const sub = forward(operation).subscribe({
next: response => {
if (!!response?.data?.getVariants.errors.length) {
if (!!response?.data?.getVariants?.errors.length) {
response?.data?.getVariants.errors.forEach(
(e: VariantQueryResponseError) => {
dispatcherContext.dispatch(makeNodeError(e));
Expand All @@ -51,7 +80,7 @@ export const buildLink = (token?: string) => {

const errorLink = onError(({ graphQLErrors, networkError, operation, response, forward }) => {
const { dispatch } = operation.getContext();
const sources = operation.variables.input.sources;
const sources = operation.variables.input.sources || [];

if (graphQLErrors) {
graphQLErrors.forEach(graphQLError => {
Expand All @@ -73,7 +102,7 @@ export const buildLink = (token?: string) => {
return forward(operation);
});

const authLink = new ApolloLink((operation, forward) => {
const authLink = new ApolloLink((operation: Operation, forward: NextLink) => {
operation.setContext(({ headers = {} }) => ({
headers: {
...headers,
Expand All @@ -84,7 +113,7 @@ export const buildLink = (token?: string) => {
return forward(operation);
});

return from([mygeneRestLink, authLink, errorLink, timeoutLink, remoteNodeErrorLink, httpLink]);
return from([mygeneRestLink, authLink, errorLink, timeoutLink, remoteNodeErrorLink, splitLink]);
};

export const client = new ApolloClient<any>({
Expand All @@ -108,6 +137,21 @@ export const useApolloQuery = <T, V>(query: DocumentNode, options: QueryHookOpti
});
};

export const useApolloSubscription = <T, V>(
subscription: DocumentNode,
options: SubscriptionHookOptions<T, V> = {}
) => {
const { dispatch } = useErrorContext();

return useSubscription<T, V>(subscription, {
client,
shouldResubscribe: true,
fetchPolicy: 'network-only',
context: { dispatch },
...options,
});
};

export const useLazyApolloQuery = <T, V>(
query: DocumentNode,
options: QueryHookOptions<T, V> = {}
Expand Down
9 changes: 8 additions & 1 deletion react/src/apollo/hooks/index.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,11 @@
import useFetchAutocompleteQuery from './useFetchAutocompleteQuery';
import useFetchVariantsQuery from './useFetchVariantsQuery';
import useFetchVariantsSubscription from './useFetchVariantsSubscription';
import useSlurmSubscription from './useSlurmSubscriptionQuery';

export { useFetchAutocompleteQuery, useFetchVariantsQuery };
export {
useFetchAutocompleteQuery,
useFetchVariantsQuery,
useFetchVariantsSubscription,
useSlurmSubscription,
};
97 changes: 97 additions & 0 deletions react/src/apollo/hooks/useFetchVariantsSubscription.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,97 @@
import { gql } from '@apollo/react-hooks';
import { CombinedVariantQueryResponse } from '../../types';
import { useApolloSubscription } from '../client';

export const fetchVariantsSubscription = gql`
subscription GetVariantsSubscription($id: String!) {
getVariantsSubscription(id: $id) {
data {
variant {
alt
callsets {
callsetId
individualId
info {
ad
dp
gq
qual
zygosity
}
}
end
info {
af
aaAlt
aaPos
aaRef
cdna
consequence
geneName
gnomadHet
gnomadHom
transcript
}
ref
referenceName
start
variantId
}
individual {
datasetId
diseases {
ageOfOnset {
age
ageGroup
}
description
diseaseId
diseaseLabel
levelSeverity
outcome
stage
}
ethnicity
geographicOrigin
individualId
info {
diagnosis
candidateGene
solved
classifications
}
phenotypicFeatures {
ageOfOnset {
age
ageGroup
}
dateOfOnset
levelSeverity
onsetType
phenotypeId
phenotypeLabel
}
sex
}
contactInfo
source
}
errors {
error {
id
code
message
}
source
}
}
}
`;

const useFetchVariantsSubscription = () =>
useApolloSubscription<{ getVariantsSubscription: CombinedVariantQueryResponse }, {}>(
fetchVariantsSubscription,
{}
);

export default useFetchVariantsSubscription;
42 changes: 42 additions & 0 deletions react/src/apollo/hooks/useSlurmSubscriptionQuery.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
import { gql } from '@apollo/react-hooks';
import { SlurmVariantResponse } from '../../types';
import { useApolloSubscription } from '../client';

export const fetchSlurmSubscription = gql`
subscription OnSlurmResponse {
slurmResponse {
jobId
variants {
start
end
referenceName
ref
alt
Consequence
oAA
nAA
FeatureID
cDNApos
protPos
nhomalt
an
af
filter
transcript
cdna
amino_acids
}
}
}
`;

const useSlurmSubscription = () =>
useApolloSubscription<
{
jobId: number;
variants: SlurmVariantResponse[];
},
{}
>(fetchSlurmSubscription, {});

export default useSlurmSubscription;
Loading