Repository navigation
Build and test conda packages and wheels - #3151
jharlow-intel wants to merge 1 commit into
Conversation
73735c3 to
f70dbd8
Compare
6aca46e to
e9f220a
Compare
|
View rendered docs @ https://intelpython.github.io/dpnp/pull/3151/index.html |
|
Array API standard conformance tests for dpnp=0.22.0dev2+7.g283e2552c ran successfully. |
| | --- | --- | --- | | ||
| | pre-commit (`Check`) | `.github/workflows/pre-commit.yml` | yes | | ||
| | conda build and tests, Linux and Windows, py3.13-3.15 and 3.14t (`Test (...)`) | `conda-package.yml` | yes | | ||
| | conda packages and wheels built and tested, Linux and Windows, py3.11-3.15 and 3.14t for conda, py3.10-3.14 and 3.14t for wheels (`Test <Platform> Python <version>`, `Test <Platform> Wheels Python <version>`); the conda matrix is `conda-recipe/conda_build_config.yaml`, the wheel matrix the Python classifiers in `pyproject.toml`; see `ci/README.md` | `intel-ci.yml` | no | |
There was a problem hiding this comment.
The wheels are 3.11-3.14 and 3.14t: pyproject.toml has no 3.10 classifier, as #3173 dropped 3.10. Also, the old row was required on master. Is a required check planned for intel-ci.yml, or should branch protection just drop the old Test (...) checks?
There was a problem hiding this comment.
Fixed the wheel range to 3.11-3.14 and 3.14t. The required Test (...) checks come from conda-package.yml, which this PR removes, so branch protection needs them swapped for intel-ci.yml's jobs before this can merge.
There was a problem hiding this comment.
add1e50 has the right wheel range, thanks. If intel-ci.yml's test jobs replace the required Test (...) checks, this row should say yes in the last column, as the old one did.
b53329b to
c93fd5e
Compare
c93fd5e to
44571ef
Compare
Adds a ci/ directory and .github/workflows/intel-ci.yml, which build the conda packages and the wheels for every declared platform and Python and test each with ci/test.py. A pull request builds the oldest and the newest regular Python and the newest free-threaded one, and every push and tag run gets a concurrency group of its own. conda-recipe/ is now the only recipe, the conda-forge one that was conda-recipe-cf/. The Intel-channel recipe and conda-package.yml are removed, and so are conda-package-cf.yml and its conda-build-test.yml, whose builds intel-ci.yml repeats. The dev uploads to anaconda.org and their cleanup, the Array API conformance job and the pybind11 examples move to intel-ci.yml. The examples and the Array API job wait only for the builds of the newest regular and free-threaded Linux packages, which every event builds, and install dpnp from the built packages only. The benchmarks take PyYAML from conda-forge, as PyPI has no Python 3.15 wheel for it yet. ci/build.py refuses two built packages with the same file name, which would overwrite each other in its flat output directory, and takes METADATA's line ending from its headers. conda-build records the commit each conda package came from in info/about.json, and LDFLAGS links the Linux wheels' libraries with full RELRO. ci/test.py reruns a failed suite, twice for the main tests and three times for the tensor tests, and stops a tensor test stuck for 120 s. GH_CI_DEVICE and ONEAPI_DEVICE_SELECTOR reach the tests. The recipe ignores mkl-dpcpp's run export, and the ci/ scripts carry the BSD-3 header. The conda builds and test environments take dpctl's dev builds from dppy/label/dev after conda-forge, with flexible channel priority, and the wheel tests from its pip index after PyPI: ci/build.py and ci/hooks.py get --channel and --extra-index-url from intel-ci.yml. The examples and the Array API job pin dpnp to the exact build. [tool.gh-ci.pin-compatible] maps each runtime wheel to the compiler or oneMKL package the build installed; the Linux before-all and ci/install_icx.py print those versions and ci/build.py requires >= each, < its next major. The upload takes each wheel's version from its file name and fails when there is nothing to upload. Every job sets conda up with .github/actions/setup-miniconda.
44571ef to
add1e50
Compare
| - Every job sets `timeout-minutes`; for a job that calls a reusable workflow, set it inside the called workflow. | ||
| - `actionlint` runs in pre-commit. | ||
| - Set up conda with the local `.github/actions/setup-miniconda` action, which retries a failed setup once, instead of calling `conda-incubator/setup-miniconda` directly; a job that checks dpnp out into a subdirectory uses it from there (`./source/.github/actions/setup-miniconda`). | ||
| - `workflows/intel-ci.yml` is Intel CI: its conda matrix is `conda-recipe/conda_build_config.yaml`, its wheel matrix the Python classifiers in `pyproject.toml`, and its upload and cleanup jobs run only for pushes to `master` and tags in `IntelPython/dpnp`; see `ci/README.md`. |
There was a problem hiding this comment.
After the rebase, line 16 here and the comment at .github/dependabot.yml:21 (both from #3171) still name conda-recipe/rattler_recipe.yaml and conda-recipe-cf/meta.yaml, which this PR removes. With one recipe left, conda-recipe/meta.yaml is a dev-series pin and [build-system].requires the only release-only one. Could both places say that?
Adds
ci/and.github/workflows/intel-ci.yml, which build the conda packages and wheels for Linux and Windows on every declared Python, and test each in a clean environment.conda-recipe/is now the only recipe: the conda-forge one that wasconda-recipe-cf/. It builds against conda-forge, withdpctl's dev builds fromdppy/label/devafter it. The Intel-channel recipe, withrattler_recipe.yaml,conda-package.yml,conda-package-cf.ymlandconda-build-test.ymlare removed; the Array API conformance and pybind11 examples jobs move tointel-ci.yml.dppy/label/devuploads and their cleanup move intointel-ci.yml, for pushes tomasterand tags only.dpctlwheel on PyPI, and require the oneAPI runtime wheels (dpcpp-cpp-rt,intel-sycl-rt,mkl,onemkl-sycl-*), as the PyPI wheels do, each at least the version the build used ([tool.gh-ci.pin-compatible]). Their tests takedpctl's dev wheels from thedppy/label/devindex. PyPI has nodpctlfor 3.15 yet, so the 3.15 classifier is dropped until it does.ci/test.pyrunsdpnp.tests, thendpnp.tests.tensor, against the installed package, and fails when it finds no SYCL device: the OpenCL CPU device in Actions, or a GPU withGH_CI_DEVICE=gpu.The required checks on
mastercome fromconda-package.yml, so they need replacing with this workflow's, such asTest Linux Python 3.11, before this can merge.Validation: run 43919353056 built and tested the conda packages and the wheels for 3.11, the newest regular Python and 3.14t on Linux and Windows, and ran the examples and the Array API job. Not run yet: a GPU, and the uploads, which only
masterand tags do.