Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Release plans: 2.22 and 3.0 #2047

Open
henryiii opened this issue Oct 19, 2024 · 27 comments
Open

Release plans: 2.22 and 3.0 #2047

henryiii opened this issue Oct 19, 2024 · 27 comments

Comments

@henryiii
Copy link
Contributor

henryiii commented Oct 19, 2024

Description

I'd like to make a proposal for the versioning strategy and contents of the next few cibuildwheel versions. Here's the suggestion:

The next release (2.22) has the following features:

The following release of cibuildwheel I propose we call "3.0", with the following features (not a complete list):

After this point, we can shift our numbering up one, with major releases for adding/removing Pythons, minor release for features, and patch releases for bug fixes. This would allow the much-requested vX GHA tag to be something we can provide.

Thoughts?

@joerick
Copy link
Contributor

joerick commented Oct 27, 2024

The following release of cibuildwheel I propose we call "3.0", with the following features (not a complete list):

This sounds good. It especially works well with the introduction of the enable option - we can make some braver decisions there, such as dropping pypy from defaults, or changing the policy on building non-eol pythons by default if we like.

Ah yes. I'm happy to take a pass at this if you haven't started already?

After this point, we can shift our numbering up one, with major releases for adding/removing Pythons, minor release for features, and patch releases for bug fixes. This would allow the much-requested vX GHA tag to be something we can provide.

Here, I'm less sure. Personally, I see the compatibility part of our versioning system to be related to the package and configuration, not so much the Python versions that we build. That's my logic behind not being too worried about bumping major when dropping an old version. I see the specific versions of wheels produced more of an side-effect of the time that version of cibuildwheel was released, not part of the API contract.

Having said all that, we don't follow Semantic versioning anyway, so maybe the specific definition of compatibility is moot.

The bigger point is - do users need to care? When I see a package I use bump major, I'll assume there might be some work I need to do to update - read the release notes, maybe update the config files as a result. But I don't think that applies for Python addition/removal updates. The Python additions require no work (assuming you've already made the package compatible during the beta phase), and the removals are probably so old as to not be of interest (we're still building 3.6!).

Put another way - should we be requiring a major version bump to move an old version of PyPy into the pypy-eol enable group? I think it's better to reserve the major bump for circumstances where packagers need to pay attention, outside of the usual churn of interpreter versions.

@joerick
Copy link
Contributor

joerick commented Oct 27, 2024

An aside, but since we're talking about a new major version, a couple things we should consider-

  • Changing the default build-frontend to build - the only reason we're still on pip is inertia, in my opinion. Of course, the build method of doing sdist->wheel builds will break some users, who might have to switch back to pip, but I think build is a better default these days.
  • Adding delvewheel as a default option for repair-wheel-command on Windows. (cc @adang1345)

I think I'd be in favour of both, but curious of opinions.

@henryiii
Copy link
Contributor Author

the build method of doing sdist->wheel builds

That won't affect us at all, since we build with --wheel, which is a direct build, like pip's. If we made SDists, they'd collide.

Yes, more changes is fine, I was just listing what I could quickly think of.

It might be a good time to finally switch the docs to show the TOML option first, with the envvar as second tab.

As for moving to more SemVer-like versioning, it was mostly to get people to stop complaining about a lack of a vX tag. Not required if we'd rather say with the current versioning scheme.

@joerick
Copy link
Contributor

joerick commented Oct 28, 2024

That won't affect us at all, since we build with --wheel, which is a direct build, like pip's. If we made SDists, they'd collide.

Oh, I didn't know that. That's good. So build's wheel builds are 'in-tree', like pip's then?

It might be a good time to finally switch the docs to show the TOML option first, with the envvar as second tab.

Good call.

As for moving to more SemVer-like versioning, it was mostly to get people to stop complaining about a lack of a vX tag. Not required if we'd rather say with the current versioning scheme.

My opinion on this has shifted lately... I think we should just relent and make a vX tag anyway. I think that while it's probably not a good idea for serious, professional packages, there are plenty of hobbyists who just want wheel builds and don't particularly care about the details. That seems valid to me. The number of side-projects I have that get constant dependabot spam for CVEs I don't care about, the dependabot fatigue is real.

@adang1345
Copy link

Regarding adding a delvewheel default repair command, I'm not sure. I have noticed that current cibuildwheel users frequently use the --add-path option to tell delvewheel where to find DLL dependencies during the repair process, and the path used is different for each project. It may be better to require the user to manually set the repair command and just put an example command in the documentation such as delvewheel repair -w {dest_dir} {wheel}.

@henryiii henryiii pinned this issue Nov 1, 2024
@henryiii henryiii changed the title Next release versioning proposal Release plans: 2.22 and 3.0 Nov 1, 2024
@henryiii
Copy link
Contributor Author

henryiii commented Nov 1, 2024

So build's wheel builds are 'in-tree', like pip's then?

pipx build is not in-tree, it builds a wheel from an SDist. But pipx build --wheel is, as there's no SDist built.

@mayeut
Copy link
Member

mayeut commented Nov 2, 2024

I added #2052 to the list for 2.22 per #1912 (comment) comment

@EwoutH
Copy link
Contributor

EwoutH commented Nov 4, 2024

3.0 might also be an oppertunity to bump the default manylinux version:

@joerick
Copy link
Contributor

joerick commented Nov 7, 2024

Would it make sense to create a v3 branch we can start merging these new features into?

@henryiii
Copy link
Contributor Author

henryiii commented Nov 7, 2024

Aren't we just about ready to release 2.22? We just have #2048 and #2052 left. Then we can move forward with main being version 3, and make a v2 branch for back ports.

@henryiii
Copy link
Contributor Author

IMO, 2.22 is ready.

@joerick
Copy link
Contributor

joerick commented Jan 4, 2025

I think it's time to get working on v3.0. Because the utils refactor is gonna be very merge-conflicty, I'll hold off on that until there are fewer PRs waiting to go in. I think a good one to start with is #1912.

@mattip
Copy link
Contributor

mattip commented Feb 11, 2025

Any plans to drop older versions of Python? I think 3.6 can be dropped, maybe even 3.7. That would save some CI processing.

@mattip
Copy link
Contributor

mattip commented Feb 11, 2025

Hmm, maybe I am wrong. The urllib3 download statistics claim around 2% for python3.6 and 17% (!!!) for 3.7.

@mayeut
Copy link
Member

mayeut commented Feb 12, 2025

Any plans to drop older versions of Python? I think 3.6 can be dropped, maybe even 3.7. That would save some CI processing.

It's planned in manylinux: pypa/manylinux#1671
The planning takes into account CI to some extent as this will allow to add Python 3.14 Beta 1 with almost no impact on CI times (replace 3.6/3.7 with 3.14/3.14t).
cibuildwheel could drop before but probably unlikely.

The download statistics for manylinux wheels (Linux is probably where 3.6/3.7 are most used) shows 0.2% downloads for 3.6 and 4.5% for 3.7 over all supported packages at the moment. 11.5% of packages support Python 3.6 (either directly, as py3 or abi3 wheels) and 25% support Python 3.7.

@henryiii
Copy link
Contributor Author

henryiii commented Feb 12, 2025

Quick thought: maybe we could drop 3.6 and 3.7 in 3.0; that timeframe would likely also allow #1538 to make it in for 3.0 too. We'd need to do a 2.22 patch release before then to get #2268 and the manylinux update out sooner.

@joerick
Copy link
Contributor

joerick commented Feb 23, 2025

I think it would make sense to drop 3.6 and 3.7 in this v3.0 release, in fact, it's a good opportunity to do so.

@henryiii I'm happy to make a v2.x branch and do a point/minor release, I'd probably do a minor release for a new PyPy version. Looking at the recent PRs, things I'd like to backport are:

Is there anything else you'd like to see?

This was referenced Feb 23, 2025
@mayeut
Copy link
Member

mayeut commented Mar 9, 2025

Would 3.0 be a good time to drop support for manylinux1, manylinux2010, manylinux_2_24 & musllinux_1_1 support ? I don't know why I didn't think of this earlier (maybe related to the late drop of python 3.6/3.7, maybe not). This would probably require to add a warning in 2.x ?

@henryiii
Copy link
Contributor Author

henryiii commented Mar 9, 2025

Someone could still work around it by adding the explicit image name, I assume? We'd only be dropping support for the shortcut setting the pins?

@mayeut
Copy link
Member

mayeut commented Mar 10, 2025

Someone could still work around it by adding the explicit image name, I assume? We'd only be dropping support for the shortcut setting the pins?

Yes, the images aren't going to be removed (at least for now, there might be some clean-up in which version of each image are available at some point though). That being said, manylinux1 images won't be usable after python 3.9 EOL for the before_all step as we use the oldest non EOL python there.

Extended support for RHEL / OL 6 has ended in 2024, support for musl 1.1 / Alpine stopped on 01 May 2022.

So all in all, they should remain usable but unsupported / untested.

@joerick
Copy link
Contributor

joerick commented Mar 10, 2025

I think dropping support for these images would be a good idea, given the base OSs are unsupported. I can see two scenarios for dropping official support,

  1. we could put a deprecation notice in the next cibuildwheel release, and drop a few weeks later in v3.0
  2. we could deprecate in v3.0 and throw a warning on each build that these images will stop updating soon and strongly recommend that the user should update

I'm more inclined to (2). (1) has a pretty short deprecation period, for an option that I suspect is still in use here and there1. (2) might be a bit friendlier to cibuildwheel users, while still giving manylinux the freedom to sunset them without worrying that many cibuildwheel users will be left behind.

It does somewhat depend on what the maintenance implications are for manylinux.

Footnotes

  1. do we have download/usage stats for the oci images somewhere? it would be useful to have some data on this...

@mayeut
Copy link
Member

mayeut commented Mar 10, 2025

There are no maintenance implications for manylinux whatsoever as those images are already EOL. (the clean-up mentioned is unrelated to any decision here & won't affect cibuildwheel).

On cibuildwheel side, it's mostly CI time (less tests) & documentation updates.

There are some download stats on quay.io, roughly, only speaking about x86_64, manylinux2014 is around 10-15k downloads/day, then manylinux_2_28 around 6k/d, musllinux_1_2 4k/d, manylinux_2_24 500/d, musllinux_1_1 500/d, manylinux1 <500/d, manylinux2010 250/d.

For python<3.8, the pip version was important w.r.t the manylinux standard. For >= 3.8, almost all default shipped pip support manylinux2014 (I'm seeing 0.1% for manylinux1 downloads).

Given v3.0 is a clear breaking change, I'd rather see (1) but given the short time frame, I'd understand going with (2).

@henryiii
Copy link
Contributor Author

Regardless, we can go ahead and add a warning for the next patch release, and then decide if we should make it an error or wait longer.

@joerick
Copy link
Contributor

joerick commented Mar 10, 2025

There are some download stats on quay.io, roughly, only speaking about x86_64, manylinux2014 is around 10-15k downloads/day, then manylinux_2_28 around 6k/d, musllinux_1_2 4k/d, manylinux_2_24 500/d, musllinux_1_1 500/d, manylinux1 <500/d, manylinux2010 250/d.

Those numbers for manylinux2010 and manylinux1 are smaller than I expected. That does alter my intuition a bit. I'd be okay fully dropping support if others are in favour.

@henryiii
Copy link
Contributor Author

I'm okay to drop in 3.0. It's very easy to fix, and people expect a little churn when updating to a major version, especially if they are using an unmaintained image.

@di
Copy link
Member

di commented Mar 12, 2025

There are some download stats on quay.io

Would it be helpful to see stats on wheel uploads for Python 3.6/3.7? The public dataset should have enough information to do this, I'd be happy to put it together if you'd like.

@joerick
Copy link
Contributor

joerick commented Mar 12, 2025

Hey @di! I tend to look at mayeut's manylinux-timeline for that kind of statistics. It's only manylinux, but that's not a bad constraint since nearly every packager is gonna release manylinux wheels, and it's good that it only considers platform wheels, since that's the segment that cibuildwheel cares about.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

7 participants