Architecture

The backend applications architecture supports the data ingestion, and the maintenance of the graphQL API.

Data Ingestion

The data ingestion process involves transforming input data from various sources into a standardized output format. Here’s a breakdown of how this process typically works:

Input Data Sources: Data can originate from various researchers/data generators who share the data with the platform. Each source may have its own format and structure.

Data Ingestion: The first step of this is usually bringing the raw data into an AWS environment that is accessible to our ingestion workflows. Once the data is available, it needs to be transformed into a standardized format that the CryoET application can surface uniformly across its different datasets.

Architecture diagram

Please find the link to the architecture diagram here.

Account Setup

The backend of the cryoET is distributed across three different AWS accounts for different purposes:

1. dev account

• it hosts all the data processing
• it includes the dev and staging environments

2. public account

• it hosts the publicly accessible s3 bucket and the data

3. prod account

• it hosts the production environment for the frontend and the graphql API