Add system for automatically generating docs

.github/workflows/build_docs.yml

@@ -0,0 +1,34 @@
+name: build docs
+on: push
+
+jobs:
+  build_docs:
+    runs-on: "ubuntu-20.04"
+    steps:
+      - name: Check out repository code
+        uses: actions/checkout@v3
+        with:
+          fetch-depth: 0
+
+      - name: Build docs
+        run: make docs
+
+      - name: Checkout ooni/docs
+        uses: actions/checkout@v2
+        with:
+          repository: "ooni/docs"
+          ssh-key: ${{ secrets.OONI_DOCS_DEPLOYKEY }}
+          path: "ooni-docs"
+
+      - name: Update version of docs
+        run: cp -R dist/docs/* ooni-docs/src/content/backend/
+
+      - name: Commit changes
+        id: commit
+        run: |
+          cd ooni-devops
+          git config --global user.email "[email protected]"
+          git config --global user.name "OONI Github Actions Bot"
+          git add .
+          git commit -m "auto: update backend docs" || echo "No changes to commit"
+          git push origin

.gitignore

@@ -19,3 +19,4 @@ af/fastpath/var/
 af/fastpath/etc/
 .coverage
 coverage.xml
+/dist/

Makefile

@@ -0,0 +1,7 @@
+docs:
+	./scripts/build_docs.sh
+
+clean:
+	rm -rf dist/
+
+.PHONY: docs clean

README.md

@@ -2,5 +2,16 @@
 
 Welcome to the OONI backend!
 
-For up to date documentation about the backend, see:
-https://docs.ooni.org/backend
+We follow the principles of a building a [12 factor app](https://12factor.net/),
+which can be summarized as:
+
+- Use declarative formats for setup automation, to minimize time and cost for
+  new developers joining the project;
+- Have a clean contract with the underlying operating system, offering maximum
+  portability between execution environments;
+- Are suitable for deployment on modern cloud platforms, obviating the need for
+  servers and systems administration;
+- Minimize divergence between development and production, enabling continuous
+  deployment for maximum agility;
+- And can scale up without significant changes to tooling, architecture, or
+  development practices.

ooniapi/README.md

@@ -22,20 +22,28 @@ There is also a `common` folder for all shared logic common to every component.
 ## List of OONI API Services
 
 ### Tier0
-* `ooniprobe`, (aka `probe_services`), where probes send their measurements and
-  get metadata to run experiments.
-* `oonirun`, CRUD for editing OONI Run links, but also for probes to get the
-  descriptors and run tests (this fact is what makes it tier0)
+
+- `ooniprobe`, (aka `probe_services`), where probes send their measurements and
+  get metadata to run experiments;
+- `oonirun`, CRUD for editing OONI Run links, but also for probes to get the
+  descriptors and run tests (this fact is what makes it tier0);
+- `prioritization`, CRUD for editing the priorities of URLs and populating the
+  appropriate clickhouse tables so they can be used by probe;
+- `fastpath`, responsible for taking measurements blobs sent to `ooniprobe`
+  service and storing them in s3;
 
 ### Tier1
-* `data`, (aka OONI Data Kraken), where Explorer and other clients access the
-  observations data and experiment results.
-* `findings`, (aka `incidents`) backend for findings pages on explorer
-* `measurements`, backend for aggregation and list measurement endpoints (note
-  also probe uses this, so it's maybe on the high end of tier1)
+
+- `data`, (aka OONI Data Kraken), where Explorer and other clients access the
+  observations data and experiment results;
+- `findings`, (aka `incidents`) backend for findings pages on explorer;
+- `measurements`, backend for aggregation and list measurement endpoints (note
+  also probe uses this, so it's maybe on the high end of tier1);
+- ``
 
 ### Tier2
-* `testlists`, backend for test-lists.ooni.org
+
+- `testlists`, for proposing changes to the test-lists and submitting a github PR;
 
 ## Developer setup
 
@@ -44,6 +52,7 @@ hatch](https://hatch.pypa.io/1.9/install/), check the `Makefile` for service
 specific commands.
 
 Generally it should work something like:
+
 ```
 make test
 make run

scripts/build_docs.sh

@@ -0,0 +1,22 @@
+#!/bin/bash
+DOCS_ROOT=dist/docs/
+
+mkdir -p $DOCS_ROOT
+
+cat <<EOF>$DOCS_ROOT/00-index.md
+---
+title: OONI Backend
+description: OONI Backend documentation
+slug: backend
+---
+EOF
+cat README.md >> $DOCS_ROOT/00-index.md
+
+cat <<EOF>$DOCS_ROOT/01-ooniapi.md
+---
+title: OONI API
+description: OONI API documentation
+slug: backend/ooniapi
+---
+EOF
+cat ooniapi/README.md >> $DOCS_ROOT/01-ooniapi.md