GradBench

Benchmark for differentiable programming across languages and domains.

Usage

If you have Git installed, you can clone this repository; for instance, via the GitHub CLI:

gh repo clone gradbench/gradbench
cd gradbench

Then if you have Python and Docker installed, you can run any of our evals against any of our tools via our run.py script:

./run.py --eval './eval.sh hello' --tool './tool.sh pytorch'

This will first automatically download our latest nightly Docker images for the given eval and tool, and then run the eval against the tool while printing the entire communication log to the terminal.

Contributing

See CONTRIBUTING.md.

License

GradBench is licensed under the MIT License.