FCSLOG

This wiki page contains the following contents:

1. Description
2. Features
3. Architecture Overview
4. Frontend
5. Backend
6. Installation and Use
7. Discontinuation

Description

FCSLOG is the official name of the 2023 SM2 intake's implementation of the web application outlined in the Problem Space wiki page. The CODEBASE, written in Flask API (Python), can be found within the fcslog archived zip file.

NOTE: This FCSLOG web application is INCOMPLETE as neither the frontend or FlaskAPI backend were made. The merge process is done via CLI using the main.py file.

This web application is capable of merging raw .fcs files with their corresponding Index Primer spreadsheets to produce an Flow Cytometry Experiment (excel) spreadsheet. This process aims to emulate the parsing capabilities of FlowJo while introducing additional functionalities to improve data integration and analysis.

The Experiment spreadsheet is used during the Sequencing and demultiplexing steps. See the CEL-Seq2 workflow diagram for more information.

Useful resources related to this codebase:

• 2023 SM2 Presentation
• 2023 SM2 Technical Notes
• 2023 SM2 Logbook

Features

• Outputs a Flow Cytometry Experiment Spreadsheet within a minute
• Automatically records the FCS and Index Primer files used to make the Experiment spreadsheet
• Preview functionality for exporting files

Architecture Overview

Frontend

The frontend utilises HTML and CSS.

User Interface & Wireframes

The frontend was designed via wireframes. The website has a total of 5 pages being the Home, Tutorial, Generator and Contact pages. The following are links to the frontend resources:

• Wireframes
• UI Workflow

Input File Rules

• Requires at least 1 FCS file along with 1 corresponding Index Primer file
• FCS files:
  • Naming Convention: FCS_LC(Plate Number).fcs
  • Example: FCS_LC370.fcs
• Index Primer files:
  • Naming Convention: LC(Plate Number)_Primer.xlsx
  • Example: LC370_Primer.xlsx
• NOTE: The corresponding file of FCS_LC370.fcs is LC370_Primer.xlsx as they share the same plate number
• The merging mechanism relies on string comparison to know which files to pair and merge together

Automatic FCS and Index Primer Organisation

After the merge process, the web application will sort the constituent files into an Index Primer or FCS directory, as seen below:

Backend

The backend uses Flask API (Python), Jupyter for testing, and some additional dependencies that vary according to the merge mechanism design. Additionally the backend architecture is as follows:

Merge Mechanism

The merge mechanism is what is responsible for collating the FCS and Index Primer files together to produce the experiment spreadsheet.

The merge process has 2 steps:

1. Merge by Column: merges the corresponding FCS & Index Primer files together
2. Merge by Row: merges the resultant files from step 1 to produce the Experiment spreadsheet

The merge mechanism had 3 backend designs. Design 3 is being used despite it's lack of documentation.

Design 1

Software Dependencies: standard Python libraries (e.g. os and glob), pandas, FlowCytometryTools, fcsparser, openpyxl, and flowkit

Initially, the approach involved using fcsparser to extract data from raw FCS files for merging into an Excel sheet. However, this method's limitations became apparent when comparing its output with FlowJo's analytical results.

Drawback: This design fell short as FlowJo not only parses but also analyses data, affecting the integrity of information derived from raw FCS files. Consequently, it was determined that input files should be pre-processed through FlowJo and exported as CSV files for merging.

Design 2

Software Dependencies: same as design 1 with the addition of numpy

Adopting a refined approach, users now input Excel (.xlsx) files already analyzed by FlowJo into our generator. This method ensures the merged CSV file retains the analytical accuracy and detail necessary for subsequent research phases.

Differences between FlowJo & fcsparser

• Data Transformations: fluorescent values may be manipulated differently
• Compensation Function: used to correct for spectral overlap between fluorochromes in flow cytometry data; functions might be different between both software
• FlowJo Plugins: plugins such as FlowClean can omit or manipulate data

Design 3

Despite being the CURRENT implementation, the 2023 Semester 2 intake did NOT document this implementation. The implementation is found within main.py.

Installation and Use

Please refer to the README.md file within the fcslog archived zip file.

Discontinuation

This implementation requires the input files to be processed by flow cytometry software such as FlowJo. This goes against the specifications mentioned in the Problem Space wiki page which require a streamlined and user friendly interface. Furthermore, the Sister Repository offers a UI instead of CLI.

For the CURRENT implementation visit the GMM wiki page.