Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add cibuildwheel config #270

Merged
merged 3 commits into from
Nov 21, 2023
Merged

Add cibuildwheel config #270

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
6842576
Move most options to pyproject.toml, fix MANIFEST
maxnoe Nov 21, 2023
e9b34b5
Add cibuildwheel config
maxnoe Nov 17, 2023
31a2692
Update action versions in CI config
maxnoe Nov 21, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions .github/workflows/ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,12 +21,12 @@ jobs:
matrix:
python-version: ["3.9", "3.10", "3.11", "3.12"]
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
with:
fetch-depth: 0

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v3
uses: actions/setup-python@v4
with:
python-version: ${{ matrix.python-version }}

Expand All @@ -44,4 +44,4 @@ jobs:
run: |
pytest --cov=eventio --cov-report=xml

- uses: codecov/codecov-action@v1
- uses: codecov/codecov-action@v3
71 changes: 62 additions & 9 deletions .github/workflows/deploy.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,27 +6,80 @@ on:
- 'v*'

jobs:
deploy:
runs-on: ubuntu-latest
build_wheels:
name: Build wheels on ${{ matrix.os }}
runs-on: ${{ matrix.os }}
strategy:
matrix:
os: [ubuntu-20.04, windows-2019, macOS-11]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does it matter that we build everything on ubuntu as a test for linux instead of alma? It's not super relevant for this PR, I am asking because that question was raised in the simtools context.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The base image for the linux build doesn't matter at all. Since the actual build happens another layer down in the "manylinux" docker images, which are actually based on centos:

https://github.com/pypa/manylinux

Copy link
Member Author

@maxnoe maxnoe Nov 21, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't have a choice here, binary wheels delivered to pypi must follow these standards.

We can then test if the wheels build that way run correctly on our reference system, which they should.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, that answers the question also for simtools then. We can do all of the building in ubuntu, even building the containers, and just make sure that the tests (unit and integration) run well in the alma container.

Copy link
Member Author

@maxnoe maxnoe Nov 21, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

simtools itself is not a compiled module, so the story is different (much simpler, since you upload a platform-independet wheel to pypi).

The question how to compile the needed binary tools by simpipe is different.


steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v4
with:
fetch-depth: 0
fetch-tags: true

- uses: actions/setup-python@v4

- name: Install cibuildwheel
run: python -m pip install cibuildwheel

# make sure we have version info
- run: git fetch --tags
- name: Build wheels
run: python -m cibuildwheel --output-dir dist/
env:
# only build cpython
CIBW_BUILD: "cp*"
# exclude 32 bit and musl architectures to reduce build time
# these shouldn't be needed
CIBW_SKIP: "*-win32 *-manylinux_i686 *musllinux*"

- uses: actions/upload-artifact@v3
with:
path: ./dist/*.whl
name: wheels

build_sdist:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
fetch-tags: true

- name: Set up Python
uses: actions/setup-python@v2
uses: actions/setup-python@v4
with:
python-version: 3.8
python-version: 3.9

- name: Install dependencies
- name: Build sdist
run: |
python --version
pip install -U build
python -m build --sdist

- uses: actions/upload-artifact@v3
with:
path: ./dist/*.tar.gz
name: sdist

pypi_upload:
runs-on: ubuntu-latest
needs:
- build_wheels
- build_sdist
steps:
- name: Download artifacts
uses: actions/download-artifact@v3

- name: preparing upload
run: |
ls -l wheels sdist
mkdir dist
cp wheels/* sdist/* dist/

- name: Publish package
uses: pypa/gh-action-pypi-publish@master
uses: pypa/gh-action-pypi-publish@release/v1

with:
user: __token__
password: ${{ secrets.pypi_password }}
3 changes: 2 additions & 1 deletion MANIFEST.in
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1 +1,2 @@
prune eventio/_dev_version
prune src/eventio/_dev_version
prune tests
63 changes: 63 additions & 0 deletions pyproject.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,66 @@
[build-system]
requires = ["setuptools>=64", "Cython", "numpy>=1.25", 'setuptools_scm[toml]>=8']
build-backend = "setuptools.build_meta"

[project]
name = "eventio"
description = "Python read-only implementation of the EventIO file format"

requires-python = '>=3.9'
dependencies = [
'numpy >= 1.21',
'corsikaio ~= 0.3.3',
'zstandard > 0.11.1', # memory leak in zstandard 0.11.1
'setuptools_scm',
]

authors = [
{name = "Maximilian Linhoff", email = "[email protected]"},
{name = "Dominik Neise", email = "[email protected]"},
{name = "Orel Gueta", email = "[email protected]"},
]
license = {text = "MIT"}
readme = "README.rst"

classifiers = [
"Development Status :: 5 - Production/Stable",
"Intended Audience :: Science/Research",
"License :: OSI Approved :: MIT License",
"Natural Language :: English",
"Operating System :: OS Independent",
"Programming Language :: Python",
"Programming Language :: Python :: 3",
"Programming Language :: Python :: 3.9",
"Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.11",
"Programming Language :: Python :: 3.12",
"Programming Language :: Python :: 3 :: Only",
"Topic :: Scientific/Engineering :: Astronomy",
"Topic :: Scientific/Engineering :: Physics",
]

dynamic = ["version"]

[project.urls]
repository = "https://github.com/cta-observatory/pyeventio"
issues = "https://github.com/cta-observatory/pyeventio/issues"
conda-forge = "https://anaconda.org/conda-forge/eventio"

[project.scripts]
eventio_print_structure = 'eventio.scripts.print_structure:main'
eventio_print_simtel_history = 'eventio.scripts.print_simtel_history:main'
eventio_print_simtel_metaparams = 'eventio.scripts.print_simtel_metaparams:main'
eventio_plot_histograms = 'eventio.scripts.plot_hists:main'
eventio_print_object_information = 'eventio.scripts.print_object_information:main'
eventio_cut_file = 'eventio.scripts.cut_eventio_file:main'

[tool.setuptools.packages.find]
where = ["src"]
exclude = ["eventio._dev_version"]

[tool.setuptools_scm]
version_file = "src/eventio/_version.py"


[tool.pytest]
addopts = "-v --durations=10"
47 changes: 0 additions & 47 deletions setup.cfg
View file
Open in desktop

This file was deleted.

20 changes: 0 additions & 20 deletions setup.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
from setuptools import setup, Extension
import os
from Cython.Build import cythonize
import numpy as np

Expand Down Expand Up @@ -33,24 +32,5 @@
]

setup(
use_scm_version={"write_to": os.path.join("src", "eventio", "_version.py")},
ext_modules=cythonize(extensions),
python_requires='>=3.9',
install_requires=[
'numpy >= 1.21',
'corsikaio ~= 0.3.3',
'zstandard > 0.11.1', # memory leak in zstandard 0.11.1
'setuptools_scm',
],
entry_points={
'console_scripts': [
'eventio_print_structure = eventio.scripts.print_structure:main',
'eventio_print_simtel_history = eventio.scripts.print_simtel_history:main',
'eventio_print_simtel_metaparams = eventio.scripts.print_simtel_metaparams:main',
'eventio_plot_histograms = eventio.scripts.plot_hists:main',
'eventio_print_object_information = eventio.scripts.print_object_information:main',
'eventio_cut_file = eventio.scripts.cut_eventio_file:main',
]
},
tests_require=['pytest>=3.0.0'],
)