This project is part of my master thesis (TODO: add title/link) and enables robustness testing of DNN-based lane detection models (semantic segmentation) of PytorchAutoDrive with attacks taken from the Adversarial Robustness Toolbox.
The robustness testing is performed in an end-to-end setting within the MetaDrive simulator.
Currently supported models: RESA (paper)
Currently supported attacks: modified Robust DPatch (paper)
Conda: https://docs.conda.io/projects/miniconda/en/latest/
Tested on Ubuntu Bullseye (SLURM node) with a Nvidia Tesla T4 (headless rendering).
Setup conda environment with:
srun -N 1 -n 1 --gpus-per-node=1 --ntasks-per-node=1 --pty bash -i
conda create -n ld python=3.9
conda activate ld
conda install cuda-toolkit==11.4 cudatoolkit=11.4 -c pytorch -c nvidia
pip install jsonargparse panda3d metadrive-simulator opencv-python numpy pillow torch ==2.0.1 torchvision adversarial-robustness-toolbox[pytorch_image] timm mmcv tensorboard importmagician
pip install numpy --upgrade
pip install -U openmim
mim install mmcv
conda list -e > requirements.txt
python3 -m metadrive.examples.verify_headless_installation --cuda --camera rgb
On Windows 11:
conda create -n ld python=3.11
conda activate ld
git clone [email protected]:alexander-toch/metadrive.git ../metadrive
cd ../metadrive
pip install -e .
cd ../lane-detection-e2e-attack
python -m pip install --upgrade pywin32
pip3 install jsonargparse torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121
pip install panda3d fcntl metadrive-simulator opencv-python numpy pillow torch torchvision adversarial-robustness-toolbox[pytorch_image] timm mmcv tensorboard importmagician cupy-cuda12x
pip install cuda-python PyOpenGL PyOpenGL_accelerate
Currently, the attack is started via rm -rf camera_observations/*.{jpg,npy} && python metadrive_bridge_selfdrive.py