Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build Python ABI3 wheels instead of a wheel per Python version #1064

Merged
merged 30 commits into from
Apr 11, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
d22296f
Start compiling code with ABI
IvanIsCoding Jan 21, 2024
43908ab
Copy code from Qiskit
IvanIsCoding Jan 21, 2024
593c522
Remove duplicates
IvanIsCoding Jan 21, 2024
5da92e4
Trigger action on fork report
IvanIsCoding Jan 21, 2024
519a6b1
Test action
IvanIsCoding Jan 21, 2024
94fc744
Merge remote-tracking branch 'upstream/main' into stable-abi-work
IvanIsCoding Jan 24, 2024
32e6886
Skip musl in ppc64
IvanIsCoding Jan 24, 2024
46daedf
Merge remote-tracking branch 'upstream/main' into stable-abi-work
IvanIsCoding Mar 21, 2024
2a26db6
Restore abi-py38
IvanIsCoding Mar 21, 2024
27df55e
Version bumps
IvanIsCoding Mar 21, 2024
c5046af
Take into account Mac OS ARM changes
IvanIsCoding Mar 21, 2024
11049ac
Testing build process
IvanIsCoding Mar 21, 2024
fb69db8
Re-triggering workflow
IvanIsCoding Mar 22, 2024
fdf058f
Try again
IvanIsCoding Mar 22, 2024
dfd99de
Fix upload v4 use
IvanIsCoding Mar 23, 2024
bbb9a62
Fix all upload uses
IvanIsCoding Mar 23, 2024
6b9191f
Repair wheels for macos and windows as well
IvanIsCoding Mar 23, 2024
13c28d9
Use fixed version of abi3audit
IvanIsCoding Mar 23, 2024
4b49f6a
Use cibuildwheel 2.17.0
IvanIsCoding Mar 23, 2024
fbad224
Handle delocate not being installed
IvanIsCoding Mar 23, 2024
95a50e2
Handle pipx in macOS arm
IvanIsCoding Mar 23, 2024
c9ed25d
Update wheels.yml
IvanIsCoding Mar 23, 2024
808b4f4
Test download-artifact v4
IvanIsCoding Mar 23, 2024
039ceef
Add release notes
IvanIsCoding Mar 23, 2024
4de6364
Merge branch 'main' into stable-abi-work
IvanIsCoding Mar 26, 2024
bdbe0b4
Merge branch 'main' into stable-abi-work
IvanIsCoding Mar 29, 2024
227d429
Merge branch 'main' into stable-abi-work
IvanIsCoding Apr 1, 2024
684d705
Merge branch 'main' into stable-abi-work
IvanIsCoding Apr 2, 2024
f86f698
Merge remote-tracking branch 'upstream/main' into stable-abi-work
IvanIsCoding Apr 2, 2024
f01a7e2
Merge branch 'main' into stable-abi-work
mergify[bot] Apr 11, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
198 changes: 28 additions & 170 deletions .github/workflows/wheels.yml
Original file line number Diff line number Diff line change
Expand Up @@ -29,14 +29,15 @@ jobs:
- uses: actions/setup-python@v5
name: Install Python
with:
python-version: '3.8'
python-version: '3.10'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree it's good to bump this to 3.10 since 3.8 goes eol in october (which is a good reminder for us to start emitting a deprecation warning on 3.8 in 0.15.0). But I'm wondering if there was something that required us to move to 3.10 here.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My reasoning was that macOS Arm only runs 3.10+ and I wanted to make it consistent

- name: Install deps
run: pip install -U setuptools-rust
- name: Build sdist
run: python setup.py sdist
- uses: actions/upload-artifact@v4
with:
path: ./dist/*
name: sdist-rustworkx
- name: Publish package distributions to PyPI
uses: pypa/gh-action-pypi-publish@release/v1
build_wheels:
Expand All @@ -52,18 +53,18 @@ jobs:
- uses: actions/setup-python@v5
name: Install Python
with:
python-version: '3.8'
python-version: '3.10'
- uses: dtolnay/rust-toolchain@stable
- name: Install cibuildwheel
run: |
python -m pip install cibuildwheel==2.16.5
python -m pip install cibuildwheel==2.17.0
- name: Build wheels
run: |
python -m cibuildwheel --output-dir wheelhouse
- uses: actions/upload-artifact@v4
with:
path: ./wheelhouse/*.whl
name: shared-wheel-builds
name: shared-wheel-builds-${{ matrix.os }}
upload_shared_wheels:
name: Upload shared build wheels
runs-on: ubuntu-latest
Expand All @@ -72,9 +73,10 @@ jobs:
id-token: write
needs: ["build_wheels", "build-win32-wheels"]
steps:
- uses: actions/download-artifact@v3
- uses: actions/download-artifact@v4
with:
name: shared-wheel-builds
pattern: shared-wheel-builds-*
merge-multiple: true
path: deploy
- name: Publish package distributions to PyPI
uses: pypa/gh-action-pypi-publish@release/v1
Expand All @@ -95,61 +97,25 @@ jobs:
- uses: actions/setup-python@v5
name: Install Python
with:
python-version: '3.8'
- uses: dtolnay/rust-toolchain@stable
- name: Set up QEMU
uses: docker/setup-qemu-action@v2
with:
platforms: all
- name: Install cibuildwheel
run: |
python -m pip install cibuildwheel==2.16.5
- name: Build wheels
run: |
python -m cibuildwheel --output-dir wheelhouse
env:
CIBW_ARCHS_LINUX: aarch64
CIBW_SKIP: cp36-* cp37-* cp311-* cp312-* pp* *musl*
- uses: actions/upload-artifact@v4
with:
path: ./wheelhouse/*.whl
- name: Publish package distributions to PyPI
uses: pypa/gh-action-pypi-publish@release/v1
with:
packages-dir: wheelhouse/
build_wheels_aarch64_part_2:
name: Build wheels on ${{ matrix.os }}
runs-on: ${{ matrix.os }}
environment: release
permissions:
id-token: write
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest]
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
name: Install Python
with:
python-version: '3.8'
python-version: '3.10'
- uses: dtolnay/rust-toolchain@stable
- name: Set up QEMU
uses: docker/setup-qemu-action@v2
with:
platforms: all
- name: Install cibuildwheel
run: |
python -m pip install cibuildwheel==2.16.5
python -m pip install cibuildwheel==2.17.0
- name: Build wheels
run: |
python -m cibuildwheel --output-dir wheelhouse
env:
CIBW_ARCHS_LINUX: aarch64
CIBW_SKIP: cp36-* cp37-* cp38-* cp39-* cp310-* pp* *musl*
CIBW_SKIP: cp36-* cp37-* pp* *musl*
- uses: actions/upload-artifact@v4
with:
path: ./wheelhouse/*.whl
name: wheel-builds-aarch64
- name: Publish package distributions to PyPI
uses: pypa/gh-action-pypi-publish@release/v1
with:
Expand All @@ -169,63 +135,26 @@ jobs:
- uses: actions/setup-python@v5
name: Install Python
with:
python-version: '3.8'
- uses: dtolnay/rust-toolchain@stable
- name: Set up QEMU
uses: docker/setup-qemu-action@v2
with:
platforms: all
- name: Install cibuildwheel
run: |
python -m pip install cibuildwheel==2.16.5
- name: Build wheels
run: |
python -m cibuildwheel --output-dir wheelhouse
env:
CIBW_ARCHS_LINUX: aarch64
CIBW_SKIP: cp36-* cp37-* cp311-* cp312-* *many*
CIBW_TEST_SKIP: cp37-* cp38-* cp39-* cp310-* cp311-* cp312-* *many*
- uses: actions/upload-artifact@v4
with:
path: ./wheelhouse/*.whl
- name: Publish package distributions to PyPI
uses: pypa/gh-action-pypi-publish@release/v1
with:
packages-dir: wheelhouse/
build_wheels_musl_aarch64_part_2:
name: Build wheels on ${{ matrix.os }}
runs-on: ${{ matrix.os }}
environment: release
permissions:
id-token: write
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest]
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
name: Install Python
with:
python-version: '3.8'
python-version: '3.10'
- uses: dtolnay/rust-toolchain@stable
- name: Set up QEMU
uses: docker/setup-qemu-action@v2
with:
platforms: all
- name: Install cibuildwheel
run: |
python -m pip install cibuildwheel==2.16.5
python -m pip install cibuildwheel==2.17.0
- name: Build wheels
run: |
python -m cibuildwheel --output-dir wheelhouse
env:
CIBW_ARCHS_LINUX: aarch64
CIBW_SKIP: cp36-* cp37-* cp38-* cp39-* cp310-* *many*
CIBW_SKIP: cp36-* cp37-* *many*
CIBW_TEST_SKIP: cp37-* cp38-* cp39-* cp310-* cp311-* cp312-* *many*
- uses: actions/upload-artifact@v4
with:
path: ./wheelhouse/*.whl
name: wheel-builds-musl-aarch64
- name: Publish package distributions to PyPI
uses: pypa/gh-action-pypi-publish@release/v1
with:
Expand All @@ -245,61 +174,25 @@ jobs:
- uses: actions/setup-python@v5
name: Install Python
with:
python-version: '3.8'
- uses: dtolnay/rust-toolchain@stable
- name: Set up QEMU
uses: docker/setup-qemu-action@v2
with:
platforms: all
- name: Install cibuildwheel
run: |
python -m pip install cibuildwheel==2.16.5
- name: Build wheels
run: |
python -m cibuildwheel --output-dir wheelhouse
env:
CIBW_SKIP: cp36-* cp37-* cp39-* cp310-* cp311-* pp* *win32 *musl*
CIBW_ARCHS_LINUX: ppc64le
- uses: actions/upload-artifact@v4
with:
path: ./wheelhouse/*.whl
- name: Publish package distributions to PyPI
uses: pypa/gh-action-pypi-publish@release/v1
with:
packages-dir: wheelhouse/
build_wheels_ppc64le_part2:
name: Build wheels on ${{ matrix.os }}
runs-on: ${{ matrix.os }}
environment: release
permissions:
id-token: write
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest]
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
name: Install Python
with:
python-version: '3.8'
python-version: '3.10'
- uses: dtolnay/rust-toolchain@stable
- name: Set up QEMU
uses: docker/setup-qemu-action@v2
with:
platforms: all
- name: Install cibuildwheel
run: |
python -m pip install cibuildwheel==2.16.5
python -m pip install cibuildwheel==2.17.0
- name: Build wheels
run: |
python -m cibuildwheel --output-dir wheelhouse
env:
CIBW_SKIP: cp36-* cp37-* cp38-* cp312-* pp* *win32 *musl*
CIBW_SKIP: cp36-* cp37-* pp* *win32 *musl*
CIBW_ARCHS_LINUX: ppc64le
- uses: actions/upload-artifact@v4
with:
path: ./wheelhouse/*.whl
name: wheel-builds-ppc64le
- name: Publish package distributions to PyPI
uses: pypa/gh-action-pypi-publish@release/v1
with:
Expand All @@ -319,61 +212,25 @@ jobs:
- uses: actions/setup-python@v5
name: Install Python
with:
python-version: '3.8'
- uses: dtolnay/rust-toolchain@stable
- name: Set up QEMU
uses: docker/setup-qemu-action@v2
with:
platforms: all
- name: Install cibuildwheel
run: |
python -m pip install cibuildwheel==2.16.5
- name: Build wheels
run: |
python -m cibuildwheel --output-dir wheelhouse
env:
CIBW_SKIP: cp36-* cp37-* cp39-* cp310-* cp311-* pp* *win32 *musl*
CIBW_ARCHS_LINUX: s390x
- uses: actions/upload-artifact@v4
with:
path: ./wheelhouse/*.whl
- name: Publish package distributions to PyPI
uses: pypa/gh-action-pypi-publish@release/v1
with:
packages-dir: wheelhouse/
build_wheels_s390x_part2:
name: Build wheels on ${{ matrix.os }}
runs-on: ${{ matrix.os }}
environment: release
permissions:
id-token: write
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest]
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
name: Install Python
with:
python-version: '3.8'
python-version: '3.10'
- uses: dtolnay/rust-toolchain@stable
- name: Set up QEMU
uses: docker/setup-qemu-action@v2
with:
platforms: all
- name: Install cibuildwheel
run: |
python -m pip install cibuildwheel==2.16.5
python -m pip install cibuildwheel==2.17.0
- name: Build wheels
run: |
python -m cibuildwheel --output-dir wheelhouse
env:
CIBW_SKIP: cp36-* cp37-* cp38-* cp312-* pp* *win32 *musl*
CIBW_SKIP: cp36-* cp37-* pp* *win32 *musl*
CIBW_ARCHS_LINUX: s390x
- uses: actions/upload-artifact@v4
with:
path: ./wheelhouse/*.whl
name: wheel-builds-s390x
- name: Publish package distributions to PyPI
uses: pypa/gh-action-pypi-publish@release/v1
with:
Expand All @@ -387,7 +244,7 @@ jobs:
- uses: actions/setup-python@v5
name: Install Python
with:
python-version: '3.8'
python-version: '3.10'
architecture: 'x86'
- uses: dtolnay/rust-toolchain@stable
with:
Expand All @@ -396,7 +253,7 @@ jobs:
run: rustup default stable-i686-pc-windows-msvc
- name: Install cibuildwheel
run: |
python -m pip install cibuildwheel==2.16.5
python -m pip install cibuildwheel==2.17.0
- name: Build wheels
run: |
python -m cibuildwheel --output-dir wheelhouse
Expand All @@ -405,7 +262,7 @@ jobs:
- uses: actions/upload-artifact@v4
with:
path: ./wheelhouse/*.whl
name: shared-wheel-builds
name: shared-wheel-builds-win32
retworkx-compat-build:
name: Build retworkx
runs-on: ubuntu-latest
Expand All @@ -428,5 +285,6 @@ jobs:
- uses: actions/upload-artifact@v4
with:
path: ./dist/*
name: sdist-retworkx
- name: Publish package distributions to PyPI
uses: pypa/gh-action-pypi-publish@release/v1
2 changes: 1 addition & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ rustworkx-core = { path = "rustworkx-core", version = "=0.15.0" }

[dependencies.pyo3]
version = "0.21.1"
features = ["extension-module", "hashbrown", "num-bigint", "num-complex", "indexmap"]
features = ["abi3-py38", "extension-module", "hashbrown", "num-bigint", "num-complex", "indexmap"]

[dependencies.ndarray]
version = "^0.15.6"
Expand Down
5 changes: 5 additions & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@ test-skip = "cp38-*musllinux* *linux_s390x *ppc64le"
[tool.cibuildwheel.linux]
before-all = "yum install -y wget && {package}/tools/install_rust.sh"
environment = 'PATH="$PATH:$HOME/.cargo/bin" CARGO_NET_GIT_FETCH_WITH_CLI="true"'
repair-wheel-command = "auditwheel repair -w {dest_dir} {wheel} && pipx run abi3audit==0.0.9 --strict --report {wheel}"

[[tool.cibuildwheel.overrides]]
select = "*-musllinux*"
Expand All @@ -46,3 +47,7 @@ before-test = 'python -m pip install numpy --config-settings=setup-args="-Dallow

[tool.cibuildwheel.macos]
environment = "MACOSX_DEPLOYMENT_TARGET=10.12"
repair-wheel-command = "brew install pipx && pipx ensurepath && pipx run --spec delocate==0.11.0 delocate-wheel --require-archs {delocate_archs} -w {dest_dir} -v {wheel} && pipx run abi3audit==0.0.9 --strict --report {wheel}"

[tool.cibuildwheel.windows]
repair-wheel-command = "cp {wheel} {dest_dir}/. && pipx run abi3audit==0.0.9 --strict --report {wheel}"
9 changes: 9 additions & 0 deletions releasenotes/notes/stable-abi-python-4c883bc4edaf8354.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
---
features:
- |
rustworkx wheels are now built against Python's stable
Application Binary Interface (ABI). For rustworkx users, this means
that wheels distributed by us via PyPI will continue to work with
newer versions of Python without having to recompile the code.
This change will also simplify the release process for the developers
and reduce the storage size required to mirror rustworkx wheels.
Loading
Loading