Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CI] Adding CI Workflows: checks, changed-files, builds #6

Merged
merged 92 commits into from
Nov 1, 2024
Merged
Show file tree
Hide file tree
Changes from 75 commits
Commits
Show all changes
92 commits
Select commit Hold shift + click to select a range
b100844
Merge pull request #2 from nv-rliu/branch-24.12-setup-repo
nv-rliu Oct 11, 2024
068106c
Add changes
nv-rliu Oct 11, 2024
d09a2a0
Merge branch 'branch-24.12-pr-workflow' of https://github.com/nv-rliu…
nv-rliu Oct 11, 2024
765d8c4
Rename incorrect file
nv-rliu Oct 14, 2024
73191ce
Set recently_updated to false
nv-rliu Oct 14, 2024
74ed2c0
Merge branch 'main' into branch-24.12-pr-workflow
nv-rliu Oct 14, 2024
4e6719e
Change target branch for shared workflow
nv-rliu Oct 14, 2024
db6abf5
Merge branch 'branch-24.12-pr-workflow' of https://github.com/nv-rliu…
nv-rliu Oct 14, 2024
2217ffb
Add conda-python-build to pr-builder needs
nv-rliu Oct 14, 2024
01ef220
Add changed-files
nv-rliu Oct 14, 2024
1b332f7
Add changed-files
nv-rliu Oct 14, 2024
8b92e34
Add execute permissions
nv-rliu Oct 14, 2024
9d12369
Add dependencies.yaml
nv-rliu Oct 15, 2024
3cef8fc
Result of running rapids-dependency-file-generator
nv-rliu Oct 18, 2024
6edc396
Delete rough draft dependencies file
nv-rliu Oct 18, 2024
08e3b16
Merge pull request #3 from nv-rliu/branch-24.12-dependencies-file
nv-rliu Oct 18, 2024
af71bb4
Style
nv-rliu Oct 18, 2024
edeb5c9
Drop breathe and doxygen
nv-rliu Oct 21, 2024
0096b92
Removed unused dask channels
nv-rliu Oct 21, 2024
81f3947
Remove dependencies > cuda
nv-rliu Oct 21, 2024
3302725
Update dependencies.yaml
nv-rliu Oct 21, 2024
81861a0
Add missing lines for cuda suffixed packages
nv-rliu Oct 21, 2024
31a8a15
Revert "Add missing lines for cuda suffixed packages"
nv-rliu Oct 21, 2024
9a877df
Reapply "Add missing lines for cuda suffixed packages"
nv-rliu Oct 21, 2024
1342c49
Revert "Reapply "Add missing lines for cuda suffixed packages""
nv-rliu Oct 21, 2024
a411ddf
Reapply "Reapply "Add missing lines for cuda suffixed packages""
nv-rliu Oct 21, 2024
b8888c6
Revert "Reapply "Reapply "Add missing lines for cuda suffixed package…
nv-rliu Oct 21, 2024
6cc1ac4
Drop unused channels and fields
nv-rliu Oct 21, 2024
52e634c
Add missing lines for CUDA suffixed packages
nv-rliu Oct 21, 2024
ac3cb69
Add test_notebook
nv-rliu Oct 21, 2024
b71635c
Add depends_on_cudf
nv-rliu Oct 21, 2024
5555c90
Merge branch 'branch-24.12-pr-workflow' into branch-24.12-dependencie…
nv-rliu Oct 21, 2024
7e212ca
Merge pull request #4 from nv-rliu/branch-24.12-dependencies-file
nv-rliu Oct 21, 2024
b5a0f40
Style
nv-rliu Oct 21, 2024
e20ae9d
Override CI env variables
nv-rliu Oct 23, 2024
da593bb
Merge branch 'branch-24.12' of https://github.com/rapidsai/nx-cugraph…
nv-rliu Oct 29, 2024
fcdfbcb
git push origin branch-24.12-pr-workflowMerge branch 'rapidsai-branch…
nv-rliu Oct 29, 2024
8574e5b
Change branch name for shared workflow
nv-rliu Oct 29, 2024
85565b7
Update dependencies.yaml
nv-rliu Oct 30, 2024
e747dfa
Rename `cugraph-nx` to `nx-cugraph` (#3840)
eriknw Sep 6, 2023
1094b5a
Setup Consistent Nightly Versions for Pip and Conda (#3933)
divyegala Nov 1, 2023
4c4cf59
Use rapids-build-backend (#4393)
KyleFromNVIDIA Jul 1, 2024
64ea728
add setuptools to host requirements for conda packages that need it (…
jameslamb Jul 31, 2024
ca5c02e
Fix build strings in nx-cugraph (#4639)
bdice Aug 29, 2024
ac67fa5
Verify if builds work
nv-rliu Oct 30, 2024
8e4a1f4
Style
nv-rliu Oct 30, 2024
dacd48d
Updates to build and test `nx-cugraph` wheel as part of CI and nightl…
rlratzel Sep 27, 2023
0157adb
Add wheel build
nv-rliu Oct 30, 2024
4a4aed4
Merge branch 'branch-24.12' into branch-24.12-pr-workflow
nv-rliu Oct 31, 2024
f4e27fe
Style
nv-rliu Oct 31, 2024
938542e
Switch to new wheel building pipeline (#3731)
vyasr Jul 24, 2023
11e7581
Simplify wheel build scripts and allow alphas of RAPIDS dependencies …
vyasr Aug 31, 2023
1efdd22
Updates to build and test `nx-cugraph` wheel as part of CI and nightl…
rlratzel Sep 27, 2023
1bc932c
Setup Consistent Nightly Versions for Pip and Conda (#3933)
divyegala Nov 1, 2023
0961ea9
Use new `rapids-dask-dependency` metapackage for managing `dask` vers…
galipremsagar Nov 14, 2023
e66ff54
build wheels for `cugraph-dgl` and `cugraph-pyg` (#4075)
tingyu66 Jan 9, 2024
c1e7316
Create `cugraph-equivariant` package (#4036)
tingyu66 Jan 29, 2024
4f3ac7e
Treat nx-cugraph, cugraph-pyg, cugraph-dgl, and cugraph-equivariant C…
bdice Mar 7, 2024
cd6c8ee
Remove hard-coding of RAPIDS version where possible (#4217)
KyleFromNVIDIA Mar 13, 2024
314260d
New WholeGraph Feature Store for PyG (#4432)
alexbarghi-nv May 30, 2024
0b1ac5f
Forward merge branch-24.06 into branch-24.08 (#4489)
nv-rliu Jun 19, 2024
dd77017
Use rapids-build-backend (#4393)
KyleFromNVIDIA Jul 1, 2024
70e3e65
Use CUDA math wheels (#4621)
KyleFromNVIDIA Aug 23, 2024
6944999
reduce pip verbosity in wheel builds (#4651)
jameslamb Sep 27, 2024
77df577
Comment out publish wheel
nv-rliu Oct 31, 2024
fc67a76
update python dir structure
nv-rliu Oct 31, 2024
27c7847
- to _
nv-rliu Oct 31, 2024
31b8026
Rename typing.py to _typing.py
nv-rliu Oct 31, 2024
6708049
Update .github/workflows/pr.yaml
nv-rliu Oct 31, 2024
25a1280
Update .github/workflows/pr.yaml
nv-rliu Oct 31, 2024
1bc6cbe
Update .github/workflows/pr.yaml
nv-rliu Oct 31, 2024
2ba777d
Update .github/workflows/pr.yaml
nv-rliu Oct 31, 2024
e57e1dc
Update VERSION
nv-rliu Oct 31, 2024
39a498f
Fix package-dir
nv-rliu Oct 31, 2024
2ebf095
Merge branch 'branch-24.12-pr-workflow' of https://github.com/nv-rliu…
nv-rliu Oct 31, 2024
2fee575
Update .github/workflows/pr.yaml
nv-rliu Oct 31, 2024
908c186
Update .github/workflows/pr.yaml
nv-rliu Oct 31, 2024
6f0ff9b
Update ci/build_python.sh
nv-rliu Oct 31, 2024
09b7a5b
Update ci/build_wheel.sh
nv-rliu Oct 31, 2024
80c00ee
Update .github/workflows/pr.yaml
nv-rliu Oct 31, 2024
bc07d4e
Remove unnecessary checks
nv-rliu Oct 31, 2024
5cbb4c5
Re-enable recently_updated check
nv-rliu Oct 31, 2024
45e23d9
Update ci/build_python.sh
nv-rliu Oct 31, 2024
6f0acdf
Update .github/workflows/pr.yaml
nv-rliu Oct 31, 2024
5c961bb
Merge branch 'branch-24.12-pr-workflow' of https://github.com/nv-rliu…
nv-rliu Oct 31, 2024
1d7cd5d
Change wheel dest
nv-rliu Oct 31, 2024
5d7bdd8
Update ci/build_python.sh
nv-rliu Nov 1, 2024
4f7c0fc
Update ci/build_python.sh
nv-rliu Nov 1, 2024
393d6bf
Update ci/build_python.sh
nv-rliu Nov 1, 2024
4517347
Update version to minor_version
nv-rliu Nov 1, 2024
5b57f22
Specify
nv-rliu Nov 1, 2024
773dacc
Apply suggestions from code review
jameslamb Nov 1, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 7 additions & 1 deletion .github/ops-bot.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -5,5 +5,11 @@ auto_merger: true
branch_checker: true
label_checker: true
release_drafter: true
recently_updated: false
nv-rliu marked this conversation as resolved.
Show resolved Hide resolved
forward_merger: true

# enables/disables the Recently Updated Check
recently_updated: false

# sets the threshold for how many commits behind the pull request must be to trigger a failure.
# defaults to 5 if not set
#recently_updated_threshold: 5
30 changes: 30 additions & 0 deletions .github/workflows/pr.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -12,14 +12,44 @@ concurrency:
jobs:
pr-builder:
needs:
- changed-files
- checks
- conda-python-build
- wheel-build-nx-cugraph
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/[email protected]
if: always()
with:
needs: ${{ toJSON(needs) }}
changed-files:
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/[email protected]
with:
files_yaml: |
test_notebooks:
nv-rliu marked this conversation as resolved.
Show resolved Hide resolved
- '**'
- '!README.md'
- '!docs/**'
test_python:
- '**'
- '!README.md'
- '!docs/**'
- '!notebooks/**'
checks:
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/[email protected]
with:
enable_check_generated_files: false
conda-python-build:
needs: [checks, changed-files]
nv-rliu marked this conversation as resolved.
Show resolved Hide resolved
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/[email protected]
with:
build_type: pull-request
nv-rliu marked this conversation as resolved.
Show resolved Hide resolved
wheel-build-nx-cugraph:
needs: [checks, changed-files]
nv-rliu marked this conversation as resolved.
Show resolved Hide resolved
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/[email protected]
with:
build_type: pull-request
script: ci/build_wheel_nx-cugraph.sh
nv-rliu marked this conversation as resolved.
Show resolved Hide resolved
1 change: 1 addition & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ repos:
- id: check-ast
- id: check-toml
- id: check-yaml
exclude: '^conda/recipes/.*\.yaml$'
- id: debug-statements
- id: end-of-file-fixer
exclude_types: [svg]
Expand Down
32 changes: 32 additions & 0 deletions ci/build_python.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
#!/bin/bash
# Copyright (c) 2022-2024, NVIDIA CORPORATION.

set -euo pipefail

rapids-configure-conda-channels

source rapids-date-string

rapids-print-env

rapids-generate-version > ./VERSION
export RAPIDS_PACKAGE_VERSION=$(head -1 ./VERSION)

nv-rliu marked this conversation as resolved.
Show resolved Hide resolved
rapids-logger "Begin py build"

# TODO: Remove `--no-test` flags once importing on a CPU
# node works correctly

# NOTE: nothing in nx-cugraph is CUDA-specific, but it is built on each CUDA
# platform to ensure it is included in each set of artifacts, since test
# scripts only install from one set of artifacts based on the CUDA version used
# for the test run.
rapids-conda-retry mambabuild \
nv-rliu marked this conversation as resolved.
Show resolved Hide resolved
--no-test \
--channel "${RAPIDS_CONDA_BLD_OUTPUT_DIR}" \
nv-rliu marked this conversation as resolved.
Show resolved Hide resolved
conda/recipes/nx-cugraph

RAPIDS_CUDA_MAJOR="${RAPIDS_CUDA_VERSION%%.*}"

nv-rliu marked this conversation as resolved.
Show resolved Hide resolved
# TODO: uncomment when we're ready to upload artifacts in CI
# rapids-upload-conda-to-s3 python
nv-rliu marked this conversation as resolved.
Show resolved Hide resolved
52 changes: 52 additions & 0 deletions ci/build_wheel.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
#!/bin/bash
# Copyright (c) 2023-2024, NVIDIA CORPORATION.

set -euo pipefail

package_name=$1
package_dir=$2

source rapids-configure-sccache
nv-rliu marked this conversation as resolved.
Show resolved Hide resolved
source rapids-date-string

RAPIDS_PY_CUDA_SUFFIX="$(rapids-wheel-ctk-name-gen ${RAPIDS_CUDA_VERSION})"

rapids-generate-version > ./VERSION

cd "${package_dir}"

python -m pip wheel \
-w dist \
-v \
--no-deps \
--disable-pip-version-check \
--extra-index-url https://pypi.nvidia.com \
.

# pure-python packages should be marked as pure, and not have auditwheel run on them.
if [[ ${package_name} == "nx-cugraph" ]]; then
# NOTE: commenting out upload-wheels until ready
echo "Package name is nx-cugraph"
# RAPIDS_PY_WHEEL_NAME="${package_name}_${RAPIDS_PY_CUDA_SUFFIX}" RAPIDS_PY_WHEEL_PURE="1" rapids-upload-wheels-to-s3 dist
else
case "${RAPIDS_CUDA_VERSION}" in
12.*)
EXCLUDE_ARGS=(
--exclude "libcublas.so.12"
--exclude "libcublasLt.so.12"
--exclude "libcurand.so.10"
--exclude "libcusolver.so.11"
--exclude "libcusparse.so.12"
--exclude "libnvJitLink.so.12"
)
;;
11.*)
EXCLUDE_ARGS=()
;;
esac

mkdir -p final_dist
python -m auditwheel repair -w final_dist "${EXCLUDE_ARGS[@]}" dist/*
# NOTE: commenting out upload-wheels until ready
# RAPIDS_PY_WHEEL_NAME="${package_name}_${RAPIDS_PY_CUDA_SUFFIX}" rapids-upload-wheels-to-s3 final_dist
fi
nv-rliu marked this conversation as resolved.
Show resolved Hide resolved
7 changes: 7 additions & 0 deletions ci/build_wheel_nx-cugraph.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
#!/bin/bash
# Copyright (c) 2023-2024, NVIDIA CORPORATION.

set -euo pipefail

# pkg-name pkg-dir
./ci/build_wheel.sh nx-cugraph .
7 changes: 7 additions & 0 deletions conda/recipes/nx-cugraph/build.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
#!/usr/bin/env bash

# Copyright (c) 2023-2024, NVIDIA CORPORATION.

# This assumes the script is executed from the root of the repo directory

./build.sh nx-cugraph
43 changes: 43 additions & 0 deletions conda/recipes/nx-cugraph/meta.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
# Copyright (c) 2023-2024, NVIDIA CORPORATION.

{% set version = environ['RAPIDS_PACKAGE_VERSION'].lstrip('v') + environ.get('VERSION_SUFFIX', '') %}
{% set minor_version = version.split('.')[0] + '.' + version.split('.')[1] %}
{% set py_version = environ['CONDA_PY'] %}
{% set date_string = environ['RAPIDS_DATE_STRING'] %}

package:
name: nx-cugraph
version: {{ version }}

source:
path: ../../..

build:
number: {{ GIT_DESCRIBE_NUMBER }}
string: py{{ py_version }}_{{ date_string }}_{{ GIT_DESCRIBE_HASH }}_{{ GIT_DESCRIBE_NUMBER }}

requirements:
host:
- python
- rapids-build-backend>=0.3.1,<0.4.0.dev0
- setuptools>=61.0.0
run:
- pylibcugraph ={{ version }}
nv-rliu marked this conversation as resolved.
Show resolved Hide resolved
- networkx >=3.0
- cupy >=12.0.0
- python

tests:
imports:
- nx_cugraph
commands:
- pip check
requires:
- pip

about:
home: https://rapids.ai/
dev_url: https://github.com/rapidsai/nx-cugraph
license: Apache-2.0
license_file: ../../../LICENSE
summary: cuGraph backend for GPU-accelerated NetworkX
File renamed without changes.
Loading