Skip to content

Commit

Permalink
Port Buildpack author guide to NodeJS
Browse files Browse the repository at this point in the history 
Simplify the buildpack author guide, using NodeJS instead of Ruby.
The switch to NodeJS makes multi-arch demos more striaghtforward
  • Loading branch information
@AidanDelaney
AidanDelaney committed Oct 16, 2023
1 parent 966f944 commit 6dbd551
Show file tree
Hide file tree
Showing 11 changed files with 326 additions and 403 deletions.
142 changes: 50 additions & 92 deletions content/docs/buildpack-author-guide/create-buildpack/adding-bill-of-materials.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ You can find some of this information using `pack` via its `inspect-image` comma

<!-- test:exec -->
```bash
pack inspect-image test-ruby-app
pack inspect-image test-node-js-app
```
<!--+- "{{execute}}"+-->
You should see the following:
Expand All @@ -30,28 +30,27 @@ Run Images:
Buildpacks:
ID VERSION HOMEPAGE
examples/ruby 0.0.1 -
examples/node-js 0.0.1 -
Processes:
TYPE SHELL COMMAND ARGS WORK DIR
web (default) bash bundle exec ruby app.rb /workspace
worker bash bundle exec ruby worker.rb /workspace
web (default) bash node-js app.js /workspace
```

Apart from the above standard metadata, buildpacks can also populate information about the dependencies they have provided in form of a `Bill-of-Materials`. Let's see how we can use this to populate information about the version of `ruby` that was installed in the output app image.
Apart from the above standard metadata, buildpacks can also populate information about the dependencies they have provided in form of a `Bill-of-Materials`. Let's see how we can use this to populate information about the version of `node-js` that was installed in the output app image.

To add the `ruby` version to the output of `pack download sbom`, we will have to provide a [Software `Bill-of-Materials`](https://en.wikipedia.org/wiki/Software_bill_of_materials) (`SBOM`) containing this information. There are three "standard" ways to report SBOM data. You'll need to choose to use one of [CycloneDX](https://cyclonedx.org/), [SPDX](https://spdx.dev/) or [Syft](https://github.com/anchore/syft) update the `ruby.sbom.<ext>` (where `<ext>` is the extension appropriate for your SBOM standard, one of `cdx.json`, `spdx.json` or `syft.json`) at the end of your `build` script. Discussion of which SBOM format to choose is outside the scope of this tutorial, but we will note that the SBOM format you choose to use is likely to be the output format of any SBOM scanner (eg: [`syft cli`](https://github.com/anchore/syft)) you might choose to use. In this example we will use the CycloneDX json format.
To add the `node-js` version to the output of `pack download sbom`, we will have to provide a [Software `Bill-of-Materials`](https://en.wikipedia.org/wiki/Software_bill_of_materials) (`SBOM`) containing this information. There are three "standard" ways to report SBOM data. You'll need to choose to use one of [CycloneDX](https://cyclonedx.org/), [SPDX](https://spdx.dev/) or [Syft](https://github.com/anchore/syft) update the `node-js.sbom.<ext>` (where `<ext>` is the extension appropriate for your SBOM standard, one of `cdx.json`, `spdx.json` or `syft.json`) at the end of your `build` script. Discussion of which SBOM format to choose is outside the scope of this tutorial, but we will note that the SBOM format you choose to use is likely to be the output format of any SBOM scanner (eg: [`syft cli`](https://github.com/anchore/syft)) you might choose to use. In this example we will use the CycloneDX json format.

First, annotate the `buildpack.toml` to specify that it emits CycloneDX:

<!-- test:file=ruby-buildpack/buildpack.toml -->
<!-- test:file=node-js-buildpack/buildpack.toml -->
```toml
# Buildpack API version
api = "0.8"

# Buildpack ID and metadata
[buildpack]
id = "examples/ruby"
id = "examples/node-js"
version = "0.0.1"
sbom-formats = [ "application/vnd.cyclonedx+json" ]

Expand All @@ -65,177 +64,136 @@ Then, in our buildpack implementation we will generate the necessary SBOM metada
```bash
# ...

# Append a Bill-of-Materials containing metadata about the provided ruby version
cat >> "$layersdir/ruby.sbom.cdx.json" << EOL
# Append a Bill-of-Materials containing metadata about the provided node-js version
cat >> "${layersdir}/node-js.sbom.cdx.json" << EOL
{
"bomFormat": "CycloneDX",
"specVersion": "1.4",
"version": 1,
"components": [
{
"type": "library",
"name": "ruby",
"version": "$ruby_version"
"name": "node-js",
"version": "$node-js_version"
}
]
}
EOL
```

We can also add an SBOM entry for each dependency listed in `Gemfile.lock`. Here we use `jq` to add a new record to the `components` array in `bundler.sbom.cdx.json`:
We can also add an SBOM entry for each dependency listed in `package.json`. Here we use `jq` to add a new record to the `components` array in `bundler.sbom.cdx.json`:

```bash
crubybom="${layersdir}/ruby.sbom.cdx.json"
cat >> ${rubybom} << EOL
cnode-jsbom="${layersdir}/node-js.sbom.cdx.json"
cat >> ${node-jsbom} << EOL
{
"bomFormat": "CycloneDX",
"specVersion": "1.4",
"version": 1,
"components": [
{
"type": "library",
"name": "ruby",
"version": "$ruby_version"
"name": "node-js",
"version": "$node-js_version"
}
]
}
EOL
if [[ -f Gemfile.lock ]] ; then
if [[ -f package.json ]] ; then
for gem in $(gem dep -q | grep ^Gem | sed 's/^Gem //')
do
version=${gem##*-}
name=${gem%-${version}}
DEP=$(jq --arg name "${name}" --arg version "${version}" \
'.components[.components| length] |= . + {"type": "library", "name": $name, "version": $version}' \
"${rubybom}")
echo ${DEP} > "${rubybom}"
"${node-jsbom}")
echo ${DEP} > "${node-jsbom}"
done
fi
```

Your `ruby-buildpack/bin/build`<!--+"{{open}}"+--> script should look like the following:
Your `node-js-buildpack/bin/build`<!--+"{{open}}"+--> script should look like the following:

<!-- test:file=ruby-buildpack/bin/build -->
<!-- test:file=node-js-buildpack/bin/build -->
```bash
#!/usr/bin/env bash
set -eo pipefail

echo "---> Ruby Buildpack"
echo "---> NodeJS Buildpack"

# 1. GET ARGS
layersdir=$1
plan=$3

# 2. CREATE THE LAYER DIRECTORY
rubylayer="$layersdir"/ruby
mkdir -p "$rubylayer"

# 3. DOWNLOAD RUBY
ruby_version=$(cat "$plan" | yj -t | jq -r '.entries[] | select(.name == "ruby") | .metadata.version')
echo "---> Downloading and extracting Ruby $ruby_version"
ruby_url=https://s3-external-1.amazonaws.com/heroku-buildpack-ruby/heroku-22/ruby-$ruby_version.tgz
wget -q -O - "$ruby_url" | tar -xzf - -C "$rubylayer"

# 4. MAKE RUBY AVAILABLE DURING LAUNCH
echo -e '[types]\nlaunch = true' > "$layersdir/ruby.toml"

# 5. MAKE RUBY AVAILABLE TO THIS SCRIPT
export PATH="$rubylayer"/bin:$PATH
export LD_LIBRARY_PATH=${LD_LIBRARY_PATH:+${LD_LIBRARY_PATH}:}"$rubylayer/lib"

# 6. INSTALL GEMS
# Compares previous Gemfile.lock checksum to the current Gemfile.lock
bundlerlayer="$layersdir/bundler"
local_bundler_checksum=$((sha256sum Gemfile.lock || echo 'DOES_NOT_EXIST') | cut -d ' ' -f 1)
remote_bundler_checksum=$(cat "$layersdir/bundler.toml" | yj -t | jq -r .metadata.checksum 2>/dev/null || echo 'DOES_NOT_EXIST')
# Always set the types table so that we re-use the appropriate layers
echo -e '[types]\ncache = true\nlaunch = true' >> "$layersdir/bundler.toml"
if [[ -f Gemfile.lock && $local_bundler_checksum == $remote_bundler_checksum ]] ; then
# Determine if no gem dependencies have changed, so it can reuse existing gems without running bundle install
echo "---> Reusing gems"
bundle config --local path "$bundlerlayer" >/dev/null
bundle config --local bin "$bundlerlayer/bin" >/dev/null
else
# Determine if there has been a gem dependency change and install new gems to the bundler layer; re-using existing and un-changed gems
echo "---> Installing gems"
mkdir -p "$bundlerlayer"
cat >> "$layersdir/bundler.toml" << EOL
[metadata]
checksum = "$local_bundler_checksum"
EOL
bundle config set --local path "$bundlerlayer" && bundle install && bundle binstubs --all --path "$bundlerlayer/bin"
node-js_layer="${layersdir}"/node-js
mkdir -p "${node-js_layer}"

fi
# 3. DOWNLOAD node-js
node-js_version=$(cat "$plan" | yj -t | jq -r '.entries[] | select(.name == "node-js") | .metadata.version')
echo "---> Downloading and extracting NodeJS $node-js_version"
node-js_url=https://nodejs.org/dist/v18.18.1/node-v18.18.1-linux-x64.tar.xz
wget -q -O - "$node-js_url" | tar -xzf - -C "${node-js_layer}"

# 4. MAKE node-js AVAILABLE DURING LAUNCH
echo -e '[types]\nlaunch = true' > "${layersdir}/node-js.toml"

# 5. MAKE node-js AVAILABLE TO THIS SCRIPT
export PATH="${node-js_layer}"/bin:$PATH
export LD_LIBRARY_PATH=${LD_LIBRARY_PATH:+${LD_LIBRARY_PATH}:}"${node-js_layer}/lib"

# 7. SET DEFAULT START COMMAND
cat > "$layersdir/launch.toml" << EOL
# 6. SET DEFAULT START COMMAND
cat > "${layersdir}/launch.toml" << EOL
# our web process
[[processes]]
type = "web"
command = "bundle exec ruby app.rb"
command = "node app.js"
default = true
# our worker process
[[processes]]
type = "worker"
command = "bundle exec ruby worker.rb"
EOL

# ========== ADDED ===========
# 8. ADD A SBOM
rubybom="${layersdir}/ruby.sbom.cdx.json"
cat >> ${rubybom} << EOL
# 7. ADD A SBOM
node-jsbom="${layersdir}/node-js.sbom.cdx.json"
cat >> ${node-jsbom} << EOL
{
"bomFormat": "CycloneDX",
"specVersion": "1.4",
"version": 1,
"components": [
{
"type": "library",
"name": "ruby",
"version": "$ruby_version"
"name": "node-js",
"version": "$node-js_version"
}
]
}
EOL
if [[ -f Gemfile.lock ]] ; then
for gem in $(gem dep -q | grep ^Gem | sed 's/^Gem //')
do
version=${gem##*-}
name=${gem%-${version}}
DEP=$(jq --arg name "${name}" --arg version "${version}" \
'.components[.components| length] |= . + {"type": "library", "name": $name, "version": $version}' \
"${rubybom}")
echo ${DEP} > "${rubybom}"
done
fi
```

Then rebuild your app using the updated buildpack:

<!-- test:exec -->
```bash
pack build test-ruby-app --path ./ruby-sample-app --buildpack ./ruby-buildpack
pack build test-node-js-app --path ./node-js-sample-app --buildpack ./node-js-buildpack
```
<!--+- "{{execute}}"+-->

Viewing your bill-of-materials requires extracting (or `download`ing) the bill-of-materials from your local image. This command can take some time to return.

<!-- test:exec -->
```bash
pack sbom download test-ruby-app
pack sbom download test-node-js-app
```
<!--+- "{{execute}}"+-->

The SBOM information is now downloaded to the local file system:

<!-- test:exec -->
```bash
cat layers/sbom/launch/examples_ruby/ruby/sbom.cdx.json | jq -M
cat layers/sbom/launch/examples_node-js/node-js/sbom.cdx.json | jq -M
```

You should find that the included `ruby` version is `3.1.0` as expected.
You should find that the included `node-js` version is `3.1.0` as expected.

<!-- test:assert=contains;ignore-lines=... -->
```text
Expand All @@ -246,7 +204,7 @@ You should find that the included `ruby` version is `3.1.0` as expected.
"components": [
{
"type": "library",
"name": "ruby",
"name": "node-js",
"version": "3.1.0"
},
...
Expand All @@ -260,7 +218,7 @@ Congratulations! You’ve created your first configurable Cloud Native Buildpack

Now that you've finished your buildpack, how about extending it? Try:

- Caching the downloaded Ruby version
- Caching the downloaded NodeJS version
- [Packaging your buildpack for distribution][package-a-buildpack]

[package-a-buildpack]: /docs/buildpack-author-guide/package-a-buildpack/
Loading

0 comments on commit 6dbd551

Please sign in to comment.