hawk-eye

Code to find targets in aerial imagery.

---

hawk-eye is an end-to-end pipeline for training models to detect aerial imagery. Specifically, the goal is to produce models that perform well in the AUVSI SUAS competition.

We train our models with a fusion of real and synthetic data and package our inferencing pipeline as a python pip package. The output of this project is utilized by UAV Austin's Orchestra team.

Overview

Contents

• Setup
• Data Generation
• Model Training
• Inference
• Testing
• Bazel
• Style

Setup

This project supports Linux, WSL, and Mac systems. To setup one of these environments, run:

hawk_eye/setup/setup_env.sh

This file also takes an optional argument to a python virtual environment:

hawk_eye/setup/setup_env.sh ~/path_to_venv

Upon sucessful termination of this script, you need to then recieve access to Google Cloud Storage. See your lead about gaining permissions, then run

hawk_eye/setup/install_google_cloud.sh

To ensure you are all setup and ready to code, you can test code by running:

bazel test //...

Data Generation

Before we can ever train a model we need data. In our project, we actually create synthetic data using various python libraries. Inside of hawk_eye/data_generation you'll find the scripts related to data processing and more information.

Model Training

Currently we train both classifiers and detection models. This is in flux as the pace of discovery in machine learning is rapid.

See hawk_eye/train/README.md for more information on how to running training jobs.

Inference

Inference on an image will utilize both the classifier and object detector models. The inference script can be run as follows:

PYTHONPATH=$(pwd) inference/find_targets.py \
    --image_path /path/to/image \
    --visualization_dir /path/to/save

The command above will visualize and save the models' predictions. See inference/find_targets.py for the full list of available arguments.

One can also specify the model timestamps if you have a certain model to test.

Testing

Testing is done with bazel. Please see the docs for more information on writing and running tests.

Bazel

Bazel is an open-sourced Google project used for a variety of build environments. Right now, we mainly use it for python testing, but if PyTorch eventually supports building itself as an external third party project, we might start using the C++ PyTorch API.

For now, if you're interested, here is some documentation.

hawk_eye closely models the repository structure of Google's MediaPipe. Please see this project when you have questions about how to structure certain Bazel files.

Style

All python code will be automatically formatted using Black through pre-commit. flake8 is employed to correct any other style errors. Please familiarize yourself with the Google python style guide.