Skip to content

Latest commit

 

History

History
75 lines (48 loc) · 3.71 KB

DEV_ENV_WITHOUT_DOCKER.md

File metadata and controls

75 lines (48 loc) · 3.71 KB
Raw

LOCAL DEVELOPMENT ENVIRONMENT WITHOUT DOCKER

Setting up local python virtual environment for WMG development

Install python packages needed for WMG

  1. Make sure that you have successfully completed the prerequisite installation steps

  2. Create and activate a python virtual environment using your preferred method. Some options are venv and miniconda. Here is a cheatsheet of commands to manage your virtual environment with miniconda.

    NOTE: As of this writing on 06/12/2023, you should create your virtual environment with python 3.10 or greater

    • If you have installed miniconda, here is a list of basic virtual environment management commands:

      $ conda create -n <env-name> python=3.10 # creates virtual environment

$ conda activate <env-name> # activates virtual environment

$ conda deactivate # deactivate virtual environment

    • If you are using venv, here is a list of basic virtual environment management commands:

      $ cd single-cell-data-portal # root of this code repo

$ python3 -m venv venv/ # create virtual environment

$ source venv/bin/activate # activate virtual environment

$ deactivate # deactivate virtual environment

  3. Install pygraphviz - There is currently a problem with installing pygraphviz on Mac OSX. This package is needed for the WMG pipeline. The installation problem and the solution is explained here

    • brew install graphviz
    • pip install --global-option=build_ext --global-option="-I$(brew --prefix graphviz)/include/" --global-option="-L$(brew --prefix graphviz)/lib/" pygraphviz==1.11

  4. Install packages for WMG api - pip install -r python_dependencies/backend/requirements.txt

  5. Install packages for WMG pipeline - pip install -r python_dependencies/wmg/requirements.txt

Run unit tests for WMG

  1. Run unit tests for WMG api - pytest -v tests/unit/backend/wmg
  2. Run unit tests for WMG pipeline - pytest -v tests/unit/wmg_processing

Run functional tests for WMG

Run functional tests for WMG api against the dev environment. NOTE: dev environment is a remote environment. These functional tests run locally against a backend in a remote environment called dev.

  1. AWS_PROFILE=single-cell-dev DEPLOYMENT_STAGE=dev pytest -v tests/functional/backend/wmg/test_wmg_api.py

Set up vips

You may run into issues with finding _libvips if you're running a Jupyter notebook locally that calls pyvips, such as when running CXG conversion locally. The error may look like this:

ModuleNotFoundError                       Traceback (most recent call last)
File ~/miniconda3/envs/py11/lib/python3.11/site-packages/pyvips/__init__.py:19
     18 try:
---> 19     import _libvips
     21     logger.debug('Loaded binary module _libvips')

ModuleNotFoundError: No module named '_libvips'

To resolve this, you'll need to install vips with brew install vips, because this is a dependency that pyvips has. If you're using conda, you'll have to also tell your conda environment where homebrew installed vips. You can do this with:

mkdir -p ~/miniconda3/envs/<CONDA_ENV_NAME>/etc/conda/activate.d
touch ~/miniconda3/envs/<CONDA_ENV_NAME>/etc/conda/activate.d/env_vars.sh
echo 'export DYLD_LIBRARY_PATH=/opt/homebrew/lib:$DYLD_LIBRARY_PATH' >> ~/miniconda3/envs/<CONDA_ENV_NAME>/etc/conda/activate.d/env_vars.sh