PEP 803 – “abi3t”: Stable ABI for Free-Threaded Builds

[encukou]

Please consider PEP 803 – “abi3t”: Stable ABI for Free-Threaded Builds
https://peps.python.org/pep-0803/

This is where we are for 3.15. The concept is stable and proven feasible; naming & packaging compromises are acceptable.

The C API WG voting/commenting is in progress.
Since this is partly a packaging PEP, I've also asked Paul for pronouncement: https://discuss.python.org/t/104976/72

You should be aware of @zooba's competing PEP 809. (Personally, I think it's not feasible in 3.15 but it can be rewritten as a follow-up proposal.)

---

• The PEP has been discussed in threads listed in its Post-History header
• The PEP was announced on Discuss (link in Post-History)
• The PEP includes all relevant Suggested Sections
  (that list changed since the PEP was first written...)
• The PEP includes endorsements from the projects/groups/people it helps
• The PEP has a CODEOWNERS entry

SIG-specific:

• Typing PEPs: link to the Typing Council's recommendation for the PEP:
• Packaging PEPs: don't file the issue here, ask the delegate (Paul Moore) on Packaging Discourse

[eli-schwartz]

You should be aware of @zooba's competing PEP 809. (Personally, I think it's not feasible in 3.15 but it can be rewritten as a follow-up proposal.)

The competing proposal competes on multiple points. Maybe it's not feasible to design and deploy a rolling ABI window, but that shouldn't really impact whether it's feasible to design and deploy "abi4.so" in the filename. It is essentially appending a string to an existing list (of filename patterns that are looked up in turn).

So PEP 809 is not (just) a "followup proposal". It needs to be considered in the context of PEP 803 today in order to determine whether PEP 803 has flaws that require revision and resubmission to go together with "defer PEP 809 as a followup".

(The lack of extension tags is imo a major flaw and will cause a lot of trouble for downstream redistributors of python software. The fact that PEP 803 brushes this aside by essentially saying that it's "an additional technical change and not really necessary, therefore it is rejected in order to keep the PEP smaller" troubles me.)

[ngoldbaum]

In support of this proposal, Quansight has been working on updating packaging libraries, build tools, and bindings generators to work with an opaque PyObject ABI stable ABI. See the tracking issue for the latest: Quansight-Labs/free-threaded-compatibility#310.

This is all based on the _Py_OPAQUE_PYOBJECT macro available in 3.15 alphas.

We've also been working on a set of end-to-end tests for packaging tools and are experimenting with building real-world projects, see: https://github.com/Quansight-Labs/stable-abi-testing/. I'm still in the process of updating the tests to reflect work that's happened in the last couple of weeks, but that should be done by the time the steering council is reviewing the PEP.

So far, we have patched setuptools, packaging, meson-python, CFFI, Cython, and PyO3. We have not encountered any significant issues, although we have raised the need for a PyCriticalSection stable ABI (see discussion here). Separately, the maintainer of nanobind asked for a way to get instance data besides PyObject_GetTypeData(), since that requires knowledge of CPython static types to work correctly. I don't think these are difficult problems to solve, but it does require some action from the C API workgroup. Having some endorsement from the steering council for PEP 803 would probably help drive those discussions forward.

The lack of extension tags is imo a major flaw and will cause a lot of trouble for downstream redistributors of python software.

@eli-schwartz I'm not totally sure what this refers to. Can you elaborate a bit? Or maybe link to where this came up in the discourse discussion ?

[eli-schwartz]

@eli-schwartz I'm not totally sure what this refers to. Can you elaborate a bit? Or maybe link to where this came up in the discourse discussion ?

https://peps.python.org/pep-0803/#naming-this-abi4

Directly called out in my comment above as:

impact whether it's feasible to design and deploy "abi4.so" in the filename

tl;dr the filename of a C extension module in compiled form provides guidance on what it is compatible with (for non stable-ABI extensions it includes the CPython version number) and can among other things be used to ship multiple versions of an extension module and allow the interpreter to select the version it is compatible with. If your running interpreter isn't compatible with any of them, you get a ModuleNotFoundError, instead of an ImportError "unresolved symbol" or segfault. Debian is an example of a platform where multiple versions of the interpreter all load extensions (including abi3 extensions) from the same site-packages directory. ¹

Basically it's exactly the same general concept as the "4" in "libcurl.so.4", the low-level dynamic loader machinery uses it to try to catch some compatibility issues earlier rather than later.

As for "discourse" discussion I don't really know. That website is IMO pretty unpleasantly designed / difficult to use and I'm not a user of said website. This isn't really the right place to have this discussion, but suffice to say,

my personal stance

is that it is a genuine travesty of FOSS that people feel they need to move to such an ill-designed software stack purely because they feel it's the only modern maintained FOSS forum software of any description that they feel comfortable deploying as community infrastructure rather than because they think it's any good, and as a radical FOSS advocate it makes me sad that web forums is the one place where I sincerely believe the right FOSS decision would rather be to bite the bullet and pay for proprietary nonfree software if the alternative is this. On the other hand, it's not like I have decided to make a career out of developing and offering an alternative option so 🤷 ymmv.

I'm not really sure what would be a good place to discuss this, in part because I'm not sure my random bystander opinion is gonna have any influence or impact on anything. (I guess you could email me if you're truly interested in discussing it with me?) The fact that the Python developer community is also very attached to Discord, despite the numerous obvious problems with FOSS relying on such a service, and the fact that several of those concerns have been vindicated by the new age verification stuff, simply highlights to me that there's apparently a huge gap between how I and the communities I participate in think about social interactions, and how the CPython community thinks about social interactions. And I don't think it's my responsibility to tell a bunch of apparently very happy people that they need to change everything ASAP because I don't feel warmly welcomed in an environment where I'm not a pre-existing regular anyway.

It just means that I'm not really plugged into what people are saying there, and my input doesn't really get solicited, and I try not to lose any sleep over it. ;) And at the current point in time I do not consider myself especially likely to be convinced to change myself either.

---

But my point isn't really about trying to argue the pros and cons of using "abi4.so", merely to point out on procedural grounds that "abi4.so" is a genuine difference between the two PEPs that cannot be treated as "it is advisable to ignore the other PEP for now and implement it as a followup". Changing ABI and keeping the same filename because it is assumed that people only look at wheel tags, is a concrete action that cannot be undone, it's something the competing PEP naturally precludes, and...

... PEP pronouncement should therefore consider both PEPs before pronouncing for either one.

Footnotes

1. which if I recall correctly they did so that it would be possible to change Debian itself to a different CPython version compatible with that Debian release -- usually upgrading, but downgrading is possible in principle -- without reinstalling all abi3 or pure python packages (you still need to reinstall the normal version-specific extension modules of course). ↩

[encukou]

Yeah, by can be rewritten as a follow-up proposal I didn't mean just simple wording changes. Sorry for not being clear there.

My e-mail is in the PEP header; happy to discuss there. You can get new PEPs via RSS. If you can stomach enough Discourse for one-time setup, you can set it up for mailing list mode (which is how I read it, and occasionally reply).

About Debian: AFAIK, the impression from CPython core dev side is that they're not reachable/available, and their patches tend to break things, and it's leading people to recommend against system Python in all distros. If someone from Debian is interested in discussing their on-disk layouts (and the needs that they have, and ways we can make thinks better the Python users they serve), that would be great -- whatever space or platform they'd like to use.
AFAICS, the layout already doesn't handle multiple abi3 wheels, as in what cryptography does. Using .abi3.so for abi3.abi3t wheels doesn't add a new issue -- though it will probably make an existing issue more common.

[dstufft]

FWIW I don't think that what Debian is doing is wrong, and I think CPython's current behavior under free threading is wrong here. Basically I agree with @eli-schwartz I think.

abi3 is intended to provide forwards compatibility, if you build against CPython 3.10 with the Py_Limited API, then you will be ABI compatible with Python 3.11+, including Python 3.15. If free threading breaks that, then it should not be calling what it produces abi3. To do that breaks 15 years of documented behavior of what abi3 even is.

I think that PEP 803 realizes that in some cases, because it doesn't want to redefine what abi3 means in a Python wheel filename, and it instead introduces a abi3t tag there, and it explicitly says that abi3t is an inherently different thing than abi3:

The new ABI (abi3t) will conceptually be treated as separate from the existing Stable ABI (abi3).

I'm not particularly sure why PEP 803 is treating extension filenames as unique, where it's the only place it conflates abi3 and abi3t, and everywhere else it explicitly treats them as distinct ideas.

AFAICT the right solution (as far as I can tell) is:

1. abi3 is not supported under free threading. Full stop.
2. abi3t is distinct from abi3 ¹ and is supported by both free threading and not free threading.
3. Python Free Threading is adjusted to not load .abi3.so modules, because it is not compatible with abi3.
4. Python (free threading or not) is adjusted to also load .abi3t.so modules.

That I think gives you the correct behavior. Free threading will not load .abi3.so (and will not support abi3 wheel tags).

GIL builds will load .abi3.so and .abi3t.so, and will support both abi3 and abi3t tags.

No breaking compatibility, no redefinition of what abi3 means. Everything just works correctly.

The only downside to this is that Python 3.13 and 3.14 have .abi3.so in their importlib.machinery.EXTENSION_SUFFIXES, but I'd argue that was just a bug/overlooked, as none of the PEPs around free threading have made any claims about supporting the stable ABI (and they in fact have explicitly disclaimed support for the stable ABI).

Whether a change to remove .abi3.so gets back ported or not I don't personally care about, but I'm strongly against trying to redefine what abi3 means.

Footnotes

1. I don't care if you call it abi3t or abi4 or abicoolstuff. It's just an opaque string identifier. ↩

[dstufft]

Oh, It's also unclear to me whether PEP 803 is considering abi3t to be stable (in the same way that abi3 is stable) or not? The PEP also explicitly calls out that right now abi3t is a subset of abi3, but doesn't guarantee that. Can abi3 and abi3t diverge? Can I add a function only to one but not the other? Am I limited in which one I'm allowed to add a function to?

[dstufft]

Sorry, not to triple post here, but I realized I forgot to mention: PEP 3147 and PEP 3149 are what introduced the tagged filenames for Python extension modules (as well as .pyc files) and they were explicitly added to support sharing a directory between different versions of Python. After all, if "just don't share directories" is the solution we think is appropriate, then any ABI tagging in the .so files is superfluous, because that should be handled by the directory location, not by the filename.

If the stance of Python is that sharing a directory like this is wrong, then that feels like something that should go through the normal backwards compatibility process at a minimum, if not require a PEP specifically to remove support for that.

At a very bare minimum, PEP 803 should accurately indicate in it's backwards compatibility section that it is breaking the PEP 3149 mechanisms. It sort of mentions this, by saying that abi3t is not backwards compatible with abi3, but by using that wording it makes it sound like (to me anyways) that it just means that you can't "cross the streams" between abi3 and abi3t. However, the reality is that PEP 803 requires "crossing the streams" if you're using PEP 3149 as intended, and it would affect the entire wheel ecosystem as well except that PEP 803 does the right thing with wheel tags (but not filenames).

At the moment, PEP 803 appears to gloss over this breaking change it's introducing.

[eli-schwartz]

About Debian: AFAIK, the impression from CPython core dev side is that they're not reachable/available, and their patches tend to break things, and it's leading people to recommend against system Python in all distros. If someone from Debian is interested in discussing their on-disk layouts (and the needs that they have, and ways we can make thinks better the Python users they serve), that would be great -- whatever space or platform they'd like to use.

I'm surprised to hear this impression -- I'm sure that the current maintainers of Debian's python are actually rather known for reaching out to CPython including, so I was told, having an active Discourse presence. cc @stefanor who I suspect will also be surprised to hear about this impression.

You are also, I suspect, warmly encouraged to proactively reach out via email to Debian Python if you really feel like your collaboration needs are being overlooked and you'd like to get into communication. (Same is true for Gentoo by the way -- feel very free to directly email me @ https://wiki.gentoo.org/wiki/User:Eschwartz or use the project alias at https://wiki.gentoo.org/wiki/Project:Python if you prefer)

AFAICS, the layout already doesn't handle multiple abi3 wheels, as in what cryptography does. Using .abi3.so for abi3.abi3t wheels doesn't add a new issue -- though it will probably make an existing issue more common.

I'm very confused by this statement, since the whole point of abi3 wheels is that you only need one. They are stable. If they weren't stable then the Stable ABI would have no reason to exist and this PEP is definitely unneeded.

Cryptography appears to be doing an entirely ordinary thing, which is 100% their right, and building multiple wheels with different optimization levels, all of which are compatible with python 3.14 running on glibc 2.42. Now, mind you, the entire point of a Stable ABI is often, by many people, said to be because you can just "build once run everywhere" and packages with a compute-heavy build don't have to build 70+ wheels. Cryptography is, nonetheless, building one set of wheels that is abi3 / py38+, and another overlapping set of wheels that is abi3 / py311+, and obviously the former work everywhere the latter work, but the latter are faster because they can use newer stable ABIs.

They are also separately building slow glibc 2.28+ packages and fast glibc 2.34+ packages, which are the same idea, and they are even also building a really slow glibc 2.17+ package.¹ Entirely their choice.

This is an x86-64 matrix of 6 different wheels that are all compatible with glibc 2.34 + python 3.11 (and anything that is newer along either axis). They are probably spending approximately as much time doing this as they'd spend just building non-abi3 wheels once for each CPython release. But less time than they'd spend building 3 different glibc versions for each CPython release. ;)

On Debian, none of them are used because Debian builds from source, and they can simply build for the oldest versions of glibc and python that they support, which is one build that works everywhere and includes dpkg dependency descriptors that require >= whichever glibc and python version it was compiled against -- as is the entire purpose of abi3 (and libc6 for glibc). If someone were to (for mysterious reasons) choose to manually unpack multiple of these six wheels into Debian's dpkg managed directory, the last one installed would fully replace the previous one(s) and whichever one that is would still be compatible with Debian oldstable / Bookworm, which has glibc 2.36 and python 3.11. (Only some of those wheels are compatible with Debian oldoldstable / Bullseye, which has glibc 2.31 and python 3.9.) And it would follow the documented property of continuing to work with all newer versions of python and glibc, e.g. after upgrading Debian to a new version.

The existing issue is extremely real and cryptography has no bearing on the topic.

Honestly I would expect an understanding of the way all this works to be one of the first things a person should dig into if they care about binary compatibility stories. None of this is even Debian specific.

Footnotes

1. The word "slow" is doing a lot of heavy lifting. ↩

[dstufft]

For what it’s worth, while I’ve not always agreed with every decision Debian has made, and there is certainly some old wounds that occasionally crop up— I’ve never found them completely unwilling to work with me on whatever issue is at hand, as long as I was also willing to be reciprocal and care about their concerns too. I don’t think that’s a unique to Debian constraint either, since humans in general tend to be more willing to care about your problems when you care about their problems back :) .

This included working with them to handle the bundled wheel files from ensurepip, a practice that directly flies in the face of everything Debian normally allows… and we were able to work out a solution that gave them some of what they wanted and some of what we (or well, me at least) wanted. I think that’s a pretty reasonable torture test for working with downstream and it went perfectly fine.

[dstufft]

AFAICS, the layout already doesn't handle multiple abi3 wheels, as in what cryptography does. Using .abi3.so for abi3.abi3t wheels doesn't add a new issue -- though it will probably make an existing issue more common.

I've been thinking about this statement since @eli-schwartz called it out and it's been bothering me, and I think I finally figured out why it's bothering me.

It's thinking about abi3, particularly in the filenames, "backwards".

The .abi3.so is not there so you can install multiple independent copies of a particular library. It's there so you can install one copy of a library, and have that work with multiple versions of Python all pointing to the same directory, as long as you ensure that the .abi3.so wheel was compiled with a version of Python at least as old as the oldest you want to point at that shared directory.

Even though a library like cryptography might ship multiple independent wheels, each with their own copy of the .abi3.so file that's an immaterial fact, because you're only expected to install one of them at a time, but it will work with all Pythons >= the version it was compiled against.

The .cpython-315-x86_64-linux-gnu.so extension is used in sort of the opposite way, but for the same goal: I can point multiple versions of Python at the same directory, and they will only attempt to dlopen an extension that has a matching ABI.

The .cpython-315-x86_64-linux-gnu.so extension has more information, because you are expected to provide multiple copies of those extensions, one for each "unstable" ABI for the versions of Python that you're pointing at that shared directory, so they require more information in the filename to allow them to be co-located with each other.

All of these come together to support the goals of PEP 3147/3149:

Point multiple versions of Python at a shared directory, and have it correctly dlopen the file that matches the ABI that it understands.

Python FT does not understand abi3, and thus it should not attempt to dlopen an extension with the suffix .abi3.so. It's semantically equivalent to having Python 3.15 dlopen a .cpython-313-x86_64-linux-gnu.so file, since Python 3.15 also does not understand the Python 3.13 unstable ABI.

[stefanor]

Thanks @dstufft and @eli-schwartz for raising many of the issues I have hear (and @dstufft for bringing it to my attention, I had only been following the competing PEP).

Changing ABI and keeping the same filename because it is assumed that people only look at wheel tags

From Debian's point of view, that's a problem. We are only concerned about the filenames, not wheels. Unlike other Linux distributions, that I know of, we take advantage of PEP3147/PEP3149 to share a single filesystem across multiple Python versions. So, this affects us directly, where it may be more theoretical for other distros.

I'm concerned that this PEP will cause fundamental incompatibilities there that will require us to patch Python to behave in a way that respects our shared dist-packages directory. We have had to do this for abi3 too (python/cpython#122931) I don't like to have to carry patches to Python affecting core behaviour like this. I'd much rather these changes landed upstream.

If someone from Debian is interested in discussing their on-disk layouts (and the needs that they have, and ways we can make thinks better the Python users they serve), that would be great -- whatever space or platform they'd like to use.

I'm always happy to have that conversation, and to bring discussions like this to the Debian Python community. With the caveat that this conversation can't be forcing me to commit Debian to a complete redesign overnight. We don't have the resources to support large-scale change on short notice.

About Debian: AFAIK, the impression from CPython core dev side is that they're not reachable/available, and their patches tend to break things, and it's leading people to recommend against system Python in all distros

My impression is that there are a couple of people in CPython core dev, who push this view hard. I've been trying to break it down, but it's hard to do from the outside. You're obviously free to recommend whatever you think is best, your goals are different from ours. We're building an operating system, where you're typically helping new developers getting started with a programming language. We'd love to be able to provide solutions to all of these needs, of course.

I value a good relationship with our upstreams. And I'd say it's enshrined in Debian's social contract.

---

Footnote:

• Our layout isn't quite as insane as it sounds. In practice, we only support multiple Python versions in our development release (which yes, many people use as a daily driver, and we expect it to work reliably). We ship a single Python release in our stable releases. But we do support PyPy3 (maybe not for much longer, given stagnation) and hopefully soon free-threaded Python too, that all share a tree. Sharing a tree has made migrating to new Python releases significantly easier than it used to be for us. We wouldn't be able to keep up with Python's release cadence without it, or significantly more manpower.

[encukou]

I answered on the packaging thread.

I guess there's is a question larger than this PEP, which the SC could answer (or delegate answering!). Do we support such overlapping installations?

The question is asked on Discourse; also see: PEP-3147 & PEP-3149 mentioned above; but also PEP-739, which assumes -- and is one “UNLESS” away from requiring -- that each stdlib directory corresponds to a single interpreter.

I'll try to make it a non-issue for PEP 803, but it does look like we need some steering here.

Heads up to the SC -- PEP 803 might change. (I have a draft PR with the changes requested here.)
Still, I'd appreciate if you consider the PEP. Features may change in the beta period, and this is one where it'd be helpful to have something solid to base changes on.

---

If someone from Debian is interested in discussing their on-disk layouts (and the needs that they have, and ways we can make thinks better the Python users they serve), that would be great -- whatever space or platform they'd like to use.

Alright! Where? This is not the right place; I guess Discourse isn't a good place either if it excludes @eli-schwartz / Gentoo (and it looks like you missed overlapping installations topic there...)

For context: personally, I'd be on your side; I think “overlapping” installations are great and you should continue chipping away at the rough edges.
But, I think I'm a bit too biased, so when it comes to packaging, I also mostly stay away, and otherwise I try to follow other core devs rather than my own opinions (e.g. I yielded to Victor in #122917). I guess I get your “couple of people in CPython core dev, who push [an antagonistic] view hard”; thanks for calibrating my impression there!

Our layout isn't quite as insane as it sounds. In practice, we only support multiple Python versions in our development release

That makes me think you could be fine with building version-specific extensions for free-threaded Pythons (i.e. ignoring/patching out PEP 803) until you can drop 3.14.

---

You are also, I suspect, warmly encouraged to proactively reach out via email to Debian Python if you really feel like your collaboration needs are being overlooked and you'd like to get into communication. (Same is true for Gentoo by the way -- feel very free to directly email me @ https://wiki.gentoo.org/wiki/User:Eschwartz or use the project alias at https://wiki.gentoo.org/wiki/Project:Python if you prefer)

Thanks! I'll keep it in mind -- though again, it might be a while before the next time I'm involved with a packaging PEP.

[nascheme]

AFAICT the right solution (as far as I can tell) is

1. abi3 is not supported under free threading. Full stop.

2. abi3t is distinct from abi3 and is supported by both free threading and not free threading.

3. Python Free Threading is adjusted to not load .abi3.so modules, because it is not compatible with abi3.

4. Python (free threading or not) is adjusted to also load .abi3t.so modules.

This seems reasonable and quite simple to me. Can someone explain why this should not be the design? Item 1 is obviously true and is not going to change. I think the rest follows from that. We can bikeshed what tag to use rather than abi3t but I think it's about as good as anything else.

[stefanor]

Alright! Where? This is not the right place;

I'll be at PyCon US, this year for the first time in many years. We had a successful distro discussion during the developers summit at PyCon US 21 (online), leading to PEP 668. I think this is something we should try to do more frequently. Sadly I can't make EuroPython this year, which has the 26 developers summit.

But... that's in-person. Discourse absolutely can work. If we want to discuss Debian's layouts specifically, our debian-python mailing list is probably the canonical place to do it. If we were to have a discourse discussion, I'd post on there to ask people to follow & contribute.

I am not currently a Python Core Dev, so I don't have access to your discord, but I think it would be helpful if there was some more interactive space available to raise issues as they come up. Could I suggest a distros room that you don't have to be a core-dev to join? Can we get more people from the distros to become core devs?

(and it looks like you missed overlapping installations topic there...)

FWIW, I was well aware of that, it came out of a conversation I'd had with Filipe on the PyPA discord. My feeling from that discussion was that it was "your side" deciding how you want to treat "us", and my input would not be appreciated on it. It certainly wasn't inviting any input from the distros.