Release plans: 2.22 and 3.0

[henryiii]

Description

I'd like to make a proposal for the versioning strategy and contents of the next few cibuildwheel versions. Here's the suggestion:

The next release (2.22) has the following features:

• feat: adding enable #2048 - But the default set of versions is the same if you don't set enable. We can warn if prerelease or free-threading standalone options are set.
• chore: add warning when running cibuildwheel with python<3.11 #2050
• feat: add manylinux armv7l #2052
• feat: support test-groups #2063

The following release of cibuildwheel I propose we call "3.0", with the following features (not a complete list):

• More enable groups, and removing the prerelease and free-threading standalone options. (chore: drop deprecated options related to CIBW_ENABLE #2095)
• feat: use python3.11+ for cibuildwheel driver #1912
• chore: refactor cibuildwheel.utils #2252
• Add a test-sources configuration option. #2062
• Change the default build backend to build
• Show the pyproject.toml config by default

After this point, we can shift our numbering up one, with major releases for adding/removing Pythons, minor release for features, and patch releases for bug fixes. This would allow the much-requested vX GHA tag to be something we can provide.

Thoughts?

[joerick]

The following release of cibuildwheel I propose we call "3.0", with the following features (not a complete list):

• More enable groups, and removing the prerelease and free-threading standalone options.
• feat: use SPEC 0 schedule for cibuildwheel #1912

This sounds good. It especially works well with the introduction of the enable option - we can make some braver decisions there, such as dropping pypy from defaults, or changing the policy on building non-eol pythons by default if we like.

• New version of Split file utils into dedicated submodule #1873

Ah yes. I'm happy to take a pass at this if you haven't started already?

After this point, we can shift our numbering up one, with major releases for adding/removing Pythons, minor release for features, and patch releases for bug fixes. This would allow the much-requested vX GHA tag to be something we can provide.

Here, I'm less sure. Personally, I see the compatibility part of our versioning system to be related to the package and configuration, not so much the Python versions that we build. That's my logic behind not being too worried about bumping major when dropping an old version. I see the specific versions of wheels produced more of an side-effect of the time that version of cibuildwheel was released, not part of the API contract.

Having said all that, we don't follow Semantic versioning anyway, so maybe the specific definition of compatibility is moot.

The bigger point is - do users need to care? When I see a package I use bump major, I'll assume there might be some work I need to do to update - read the release notes, maybe update the config files as a result. But I don't think that applies for Python addition/removal updates. The Python additions require no work (assuming you've already made the package compatible during the beta phase), and the removals are probably so old as to not be of interest (we're still building 3.6!).

Put another way - should we be requiring a major version bump to move an old version of PyPy into the pypy-eol enable group? I think it's better to reserve the major bump for circumstances where packagers need to pay attention, outside of the usual churn of interpreter versions.

[joerick]

An aside, but since we're talking about a new major version, a couple things we should consider-

• Changing the default build-frontend to build - the only reason we're still on pip is inertia, in my opinion. Of course, the build method of doing sdist->wheel builds will break some users, who might have to switch back to pip, but I think build is a better default these days.
• Adding delvewheel as a default option for repair-wheel-command on Windows. (cc @adang1345)

I think I'd be in favour of both, but curious of opinions.

[henryiii]

the build method of doing sdist->wheel builds

That won't affect us at all, since we build with --wheel, which is a direct build, like pip's. If we made SDists, they'd collide.

Yes, more changes is fine, I was just listing what I could quickly think of.

It might be a good time to finally switch the docs to show the TOML option first, with the envvar as second tab.

As for moving to more SemVer-like versioning, it was mostly to get people to stop complaining about a lack of a vX tag. Not required if we'd rather say with the current versioning scheme.

[joerick]

That won't affect us at all, since we build with --wheel, which is a direct build, like pip's. If we made SDists, they'd collide.

Oh, I didn't know that. That's good. So build's wheel builds are 'in-tree', like pip's then?

It might be a good time to finally switch the docs to show the TOML option first, with the envvar as second tab.

Good call.

As for moving to more SemVer-like versioning, it was mostly to get people to stop complaining about a lack of a vX tag. Not required if we'd rather say with the current versioning scheme.

My opinion on this has shifted lately... I think we should just relent and make a vX tag anyway. I think that while it's probably not a good idea for serious, professional packages, there are plenty of hobbyists who just want wheel builds and don't particularly care about the details. That seems valid to me. The number of side-projects I have that get constant dependabot spam for CVEs I don't care about, the dependabot fatigue is real.

[adang1345]

Regarding adding a delvewheel default repair command, I'm not sure. I have noticed that current cibuildwheel users frequently use the --add-path option to tell delvewheel where to find DLL dependencies during the repair process, and the path used is different for each project. It may be better to require the user to manually set the repair command and just put an example command in the documentation such as delvewheel repair -w {dest_dir} {wheel}.

[henryiii]

So build's wheel builds are 'in-tree', like pip's then?

pipx build is not in-tree, it builds a wheel from an SDist. But pipx build --wheel is, as there's no SDist built.

[mayeut]

I added #2052 to the list for 2.22 per #1912 (comment) comment

[EwoutH]

3.0 might also be an oppertunity to bump the default manylinux version:

• feat: move default manylinux build to manylinux_2_28 #1988

[joerick]

Would it make sense to create a v3 branch we can start merging these new features into?

[henryiii]

Aren't we just about ready to release 2.22? We just have #2048 and #2052 left. Then we can move forward with main being version 3, and make a v2 branch for back ports.

[henryiii]

IMO, 2.22 is ready.

[joerick]

I think it's time to get working on v3.0. Because the utils refactor is gonna be very merge-conflicty, I'll hold off on that until there are fewer PRs waiting to go in. I think a good one to start with is #1912.

[mattip]

Any plans to drop older versions of Python? I think 3.6 can be dropped, maybe even 3.7. That would save some CI processing.

[mattip]

Hmm, maybe I am wrong. The urllib3 download statistics claim around 2% for python3.6 and 17% (!!!) for 3.7.

[mayeut]

Any plans to drop older versions of Python? I think 3.6 can be dropped, maybe even 3.7. That would save some CI processing.

It's planned in manylinux: pypa/manylinux#1671
The planning takes into account CI to some extent as this will allow to add Python 3.14 Beta 1 with almost no impact on CI times (replace 3.6/3.7 with 3.14/3.14t).
cibuildwheel could drop before but probably unlikely.

The download statistics for manylinux wheels (Linux is probably where 3.6/3.7 are most used) shows 0.2% downloads for 3.6 and 4.5% for 3.7 over all supported packages at the moment. 11.5% of packages support Python 3.6 (either directly, as py3 or abi3 wheels) and 25% support Python 3.7.

[henryiii]

Quick thought: maybe we could drop 3.6 and 3.7 in 3.0; that timeframe would likely also allow #1538 to make it in for 3.0 too. We'd need to do a 2.22 patch release before then to get #2268 and the manylinux update out sooner.

[joerick]

I think it would make sense to drop 3.6 and 3.7 in this v3.0 release, in fact, it's a good opportunity to do so.

@henryiii I'm happy to make a v2.x branch and do a point/minor release, I'd probably do a minor release for a new PyPy version. Looking at the recent PRs, things I'd like to backport are:

• feature: add PyPy 3.11 #2268
• feat: Add testing and docs for ubuntu-24.04-arm GHA runner #2135
• chore: use pypa/manylinux for manylinux_2_31_armv7l #2269
• Bump to Pyodide 0.27 #2117
• ...and a dependency update

Is there anything else you'd like to see?

[mayeut]

Would 3.0 be a good time to drop support for manylinux1, manylinux2010, manylinux_2_24 & musllinux_1_1 support ? I don't know why I didn't think of this earlier (maybe related to the late drop of python 3.6/3.7, maybe not). This would probably require to add a warning in 2.x ?

[henryiii]

Someone could still work around it by adding the explicit image name, I assume? We'd only be dropping support for the shortcut setting the pins?

[mayeut]

Someone could still work around it by adding the explicit image name, I assume? We'd only be dropping support for the shortcut setting the pins?

Yes, the images aren't going to be removed (at least for now, there might be some clean-up in which version of each image are available at some point though). That being said, manylinux1 images won't be usable after python 3.9 EOL for the before_all step as we use the oldest non EOL python there.

Extended support for RHEL / OL 6 has ended in 2024, support for musl 1.1 / Alpine stopped on 01 May 2022.

So all in all, they should remain usable but unsupported / untested.

[joerick]

I think dropping support for these images would be a good idea, given the base OSs are unsupported. I can see two scenarios for dropping official support,

1. we could put a deprecation notice in the next cibuildwheel release, and drop a few weeks later in v3.0
2. we could deprecate in v3.0 and throw a warning on each build that these images will stop updating soon and strongly recommend that the user should update

I'm more inclined to (2). (1) has a pretty short deprecation period, for an option that I suspect is still in use here and there¹. (2) might be a bit friendlier to cibuildwheel users, while still giving manylinux the freedom to sunset them without worrying that many cibuildwheel users will be left behind.

It does somewhat depend on what the maintenance implications are for manylinux.

Footnotes

1. do we have download/usage stats for the oci images somewhere? it would be useful to have some data on this... ↩

[mayeut]

There are no maintenance implications for manylinux whatsoever as those images are already EOL. (the clean-up mentioned is unrelated to any decision here & won't affect cibuildwheel).

On cibuildwheel side, it's mostly CI time (less tests) & documentation updates.

There are some download stats on quay.io, roughly, only speaking about x86_64, manylinux2014 is around 10-15k downloads/day, then manylinux_2_28 around 6k/d, musllinux_1_2 4k/d, manylinux_2_24 500/d, musllinux_1_1 500/d, manylinux1 <500/d, manylinux2010 250/d.

For python<3.8, the pip version was important w.r.t the manylinux standard. For >= 3.8, almost all default shipped pip support manylinux2014 (I'm seeing 0.1% for manylinux1 downloads).

Given v3.0 is a clear breaking change, I'd rather see (1) but given the short time frame, I'd understand going with (2).

[henryiii]

Regardless, we can go ahead and add a warning for the next patch release, and then decide if we should make it an error or wait longer.

[joerick]

There are some download stats on quay.io, roughly, only speaking about x86_64, manylinux2014 is around 10-15k downloads/day, then manylinux_2_28 around 6k/d, musllinux_1_2 4k/d, manylinux_2_24 500/d, musllinux_1_1 500/d, manylinux1 <500/d, manylinux2010 250/d.

Those numbers for manylinux2010 and manylinux1 are smaller than I expected. That does alter my intuition a bit. I'd be okay fully dropping support if others are in favour.

[henryiii]

I'm okay to drop in 3.0. It's very easy to fix, and people expect a little churn when updating to a major version, especially if they are using an unmaintained image.

[di]

There are some download stats on quay.io

Would it be helpful to see stats on wheel uploads for Python 3.6/3.7? The public dataset should have enough information to do this, I'd be happy to put it together if you'd like.

[joerick]

Hey @di! I tend to look at mayeut's manylinux-timeline for that kind of statistics. It's only manylinux, but that's not a bad constraint since nearly every packager is gonna release manylinux wheels, and it's good that it only considers platform wheels, since that's the segment that cibuildwheel cares about.