Skip to content

Commit

Permalink
Add some more polish around the usage of the native images (#283)
Browse files Browse the repository at this point in the history
1. Include sha 256's in the outputs
2. Add a bash script that you can use to get started pre-loaded with the URL's and hashes to download things
3. Update the readme with a quick start flow using this stuff to get going.
  • Loading branch information
ianoc authored Mar 2, 2020
1 parent 327891e commit a53ce88
Show file tree
Hide file tree
Showing 6 changed files with 122 additions and 11 deletions.
24 changes: 15 additions & 9 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -54,36 +54,42 @@ jobs:
- run: jabba install [email protected]=tgz+${{ matrix.graal_url }}
- name: Make native image
run: ./ci_scripts/make_native_artifact.sh ${{ matrix.graal_url }}
- name: Prepare outputs from platform run
run: ./ci_scripts/prepare_output.sh ${{ matrix.artifact }} staging-directory
- uses: actions/upload-artifact@master
with:
name: ${{ matrix.artifact }}
path: bazel-deps
path: staging-directory
make_release:
name: Make release
needs: native-image
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Download linux bazel-deps
uses: actions/download-artifact@v1
with:
name: bazel-deps-linux
path: downloads/bazel-deps-linux
path: downloads
- name: Download macos bazel-deps
uses: actions/download-artifact@v1
with:
name: bazel-deps-macos
path: downloads/bazel-deps-macos
- name: mv binaries linux
run: mv downloads/bazel-deps-linux/bazel-deps bazel-deps-linux
- name: mv binaries macos
run: mv downloads/bazel-deps-macos/bazel-deps bazel-deps-macos
path: downloads
- name: show downloads
run : ls -R downloads
- name: Build update_dependencies.sh
run: ./ci_scripts/make_update_dependencies.sh "v0.1-${{ github.run_number }}"
- uses: "marvinpinto/action-automatic-releases@latest"
with:
repo_token: "${{ secrets.GITHUB_TOKEN }}"
automatic_release_tag: "v0.1-${{ github.run_number }}"
prerelease: false
title: "Auto generated release"
files: |
bazel-deps-macos
bazel-deps-linux
downloads/bazel-deps-macos
downloads/bazel-deps-macos.sha256
downloads/bazel-deps-linux
downloads/bazel-deps-linux.sha256
update_dependencies.sh
id: "automatic_releases"
16 changes: 15 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,23 @@
Generate [bazel](https://bazel.build/) dependencies transitively for maven artifacts, with scala
support.

## Fetching/usage
## Quickstart

This repo can be cloned and built locally, or you can download pre-build binaries for MacOS and Linux in the releases page. Automatic releases are generated for every commit against master.
We also include a bash script in the releases which will let you easily download/run on mac/linux a default configuration for running bazel-deps.

A flow like:
1) Download the bash script paired with the release, it has the expected per platform sha256's embedded in it
2) Place in your repo and `chmod +x update_dependencies.sh`, maybe in a scripts folder if you wish.
3) Copy the `dependencies.yaml` from this repo, or write your own
4) Run the script, it should produce some files in `3rdparty`
5) Add to your workspace:
```python
load("//3rdparty:workspace.bzl", "maven_dependencies")
maven_dependencies()
load("//3rdparty:target_file.bzl", "build_external_workspace")
build_external_workspace(name = "third_party")
```

## Usage

Expand Down
1 change: 0 additions & 1 deletion WORKSPACE
Original file line number Diff line number Diff line change
Expand Up @@ -40,5 +40,4 @@ maven_dependencies()

load("//3rdparty:target_file.bzl", "build_external_workspace")
build_external_workspace(name = "third_party")

bind(name = 'io_bazel_rules_scala/dependency/scalatest/scalatest', actual = '//3rdparty/jvm/org/scalatest')
1 change: 1 addition & 0 deletions ci_scripts/make_native_artifact.sh
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ native-image -H:+ReportUnsupportedElementsAtRuntime \
-H:+ReportExceptionStackTraces \
--no-fallback \
-H:IncludeResources='.*bzl$' \
-cp bazel-deps.jar \
-jar bazel-deps.jar

cd ..
Expand Down
75 changes: 75 additions & 0 deletions ci_scripts/make_update_dependencies.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
set -e


MACOS_SHA=$(cat downloads/bazel-deps-macos.sha256)
LINUX_SHA=$(cat downloads/bazel-deps-linux.sha256)
BAZEL_DEPS_VERSION=$1

cat >update_dependencies.sh <<EOL
#!/bin/bash
echo -ne "\033[0;32m"
echo 'Updating bazel dependencies. This will take about five minutes.'
echo -ne "\033[0m"
set -e
if [ "\$(uname -s)" == "Linux" ]; then
BAZEL_DEPS_URL=https://github.com/${GITHUB_REPOSITORY}/releases/download/${BAZEL_DEPS_VERSION}/bazel-deps-linux
BAZEL_DEPS_SHA256=${LINUX_SHA}
elif [ "\$(uname -s)" == "Darwin" ]; then
BAZEL_DEPS_URL=https://github.com/${GITHUB_REPOSITORY}/releases/download/${BAZEL_DEPS_VERSION}/bazel-deps-macos
BAZEL_DEPS_SHA256=${MACOS_SHA}
else
"Your platform \$(uname -s) is unsupported, sorry"
exit 1
fi
# This is some bash snippet designed to find the location of the script.
# we operate under the presumption this script is checked into the repo being operated on
# so we goto the script location, then use git to find the repo root.
SCRIPT_LOCATION="\$( cd "\$( dirname "\${BASH_SOURCE[0]}" )" && pwd )"
cd \$SCRIPT_LOCATION
REPO_ROOT=\$(git rev-parse --show-toplevel)
BAZEL_DEPS_PATH="\$HOME/.bazel-deps-cache/${BAZEL_DEPS_VERSION}"
if [ ! -f \${BAZEL_DEPS_PATH} ]; then
( # Opens a subshell
set -e
echo "Fetching bazel deps."
curl -L -o /tmp/bazel-deps-bin \$BAZEL_DEPS_URL
GENERATED_SHA_256=\$(shasum -a 256 /tmp/bazel-deps-bin | awk '{print \$1}')
if [ "\$GENERATED_SHA_256" != "\$BAZEL_DEPS_SHA256" ]; then
echo "Sha 256 does not match, expected: \$BAZEL_DEPS_SHA256"
echo "But found \$GENERATED_SHA_256"
echo "You may need to update the sha in this script, or the download was corrupted."
exit 1
fi
chmod +x /tmp/bazel-deps-bin
mv /tmp/bazel-deps-bin \${BAZEL_DEPS_PATH}
)
fi
cd \$REPO_ROOT
set +e
\$BAZEL_DEPS_PATH generate -r \$REPO_ROOT -s 3rdparty/workspace.bzl -d dependencies.yaml --target-file 3rdparty/target_file.bzl --disable-3rdparty-in-repo
RET_CODE=\$?
set -e
if [ \$RET_CODE == 0 ]; then
echo "Success, going to format files"
else
echo "Failure, checking out 3rdparty/jvm"
cd \$REPO_ROOT
git checkout 3rdparty/jvm 3rdparty/workspace.bzl
exit \$RET_CODE
fi
\$BAZEL_DEPS_PATH format-deps -d \$REPO_ROOT/dependencies.yaml -o
EOL

chmod +x update_dependencies.sh
16 changes: 16 additions & 0 deletions ci_scripts/prepare_output.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
set -e


ARTIFACT_NAME=$1
OUTPUT_PATH=$2

BINARY=bazel-deps


GENERATED_SHA_256=$(shasum -a 256 $BINARY | awk '{print $1}')


mkdir $OUTPUT_PATH

mv $BINARY $OUTPUT_PATH/${ARTIFACT_NAME}
echo $GENERATED_SHA_256 > $OUTPUT_PATH/${ARTIFACT_NAME}.sha256

0 comments on commit a53ce88

Please sign in to comment.