[sysvabi64] Add chapter on Thread Local Storage by smithp35 · Pull Request #311 · ARM-software/abi-aa

smithp35 · 2025-02-20T17:37:00Z

The thread local storage chapter contains:

A description of Thread Local Storage based on addenda32
The key design decisions of AArch64 TLS such as tls variant, tls dialect, TCB size.
The ABI required code sequence for TLSDESC that must be emitted exactly, as GNU ld requires it to be.
Sequences for the different code-models.
Relaxations for GD->IE, GD->LE and IE->LE.
Synchronization requirements for Lazy TLSDESC. With advice not to support it due to overhead of synchronization.

MaskRay · 2025-02-21T06:19:34Z

sysvabi64/sysvabi64.rst

+and ``PT_TLS`` as the program header with type PT_TLS. ``PAD`` must be
+the smallest positive integer that satisfies the following congruence:
+
+``TP + TCB + PAD ≡ PT_TLS.p_vaddr (modulo PT_TLS.p_align)``


TP+TCB+PAD on the left could be confusing, as TCB is placed before TP. Perhaps mention the requirement of TP first (= 0 (modulo p_align)), then describe PAD and this formula.

I'll see if I can word it better. I've found it difficult to try and explain the formula intuitively.

sysvabi64/sysvabi64.rst

MaskRay · 2025-02-21T06:30:37Z

sysvabi64/sysvabi64.rst

+    add  xn, tp, :tprel_hi12:var, lsl #12 // R_AARCH64_TLSLE_ADD_TPREL_HI12 var
+    ldr  xn, [xn, #:tprel_lo12_nc:var]    // R_AARCH64_TLSLE_LDST64_TPREL_LO12_NC var
+
+Static link time TLS Relaxations


Perhaps call this Optimization to be consistent with x86/ppc and "Relocation optimization" (ADRP) and leave the term "relocation relaxation" for RISC-V style section shrinking.

For TLS specifically I'd prefer to keep relaxation as that's what its been referred to in all the previous literature such as Drepper's ELF Handling for Thread Local Storage and the TLSDESC paper too. It should help people searching in the references.

I take the point that it ought to have been called optimization. I'll add a sentence to say that we're using relaxation as a term from the existing literature.

sysvabi64/sysvabi64.rst

smithp35

Thanks very much for the review.

I've updated based on this and some comments I received internally.

sysvabi64/sysvabi64.rst

smithp35 · 2025-02-21T14:58:49Z

sysvabi64/sysvabi64.rst

+    add  xn, tp, :tprel_hi12:var, lsl #12 // R_AARCH64_TLSLE_ADD_TPREL_HI12 var
+    ldr  xn, [xn, #:tprel_lo12_nc:var]    // R_AARCH64_TLSLE_LDST64_TPREL_LO12_NC var
+
+Static link time TLS Relaxations


For TLS specifically I'd prefer to keep relaxation as that's what its been referred to in all the previous literature such as Drepper's ELF Handling for Thread Local Storage and the TLSDESC paper too. It should help people searching in the references.

I take the point that it ought to have been called optimization. I'll add a sentence to say that we're using relaxation as a term from the existing literature.

sysvabi64/sysvabi64.rst

smithp35 · 2025-02-21T15:29:01Z

sysvabi64/sysvabi64.rst

+and ``PT_TLS`` as the program header with type PT_TLS. ``PAD`` must be
+the smallest positive integer that satisfies the following congruence:
+
+``TP + TCB + PAD ≡ PT_TLS.p_vaddr (modulo PT_TLS.p_align)``


I'll see if I can word it better. I've found it difficult to try and explain the formula intuitively.

sysvabi64/sysvabi64.rst

stuij · 2025-06-20T13:45:26Z

sysvabi64/sysvabi64.rst

+
+AArch64 TLS SystemV design choices
+
+* AArch64 uses variant 1 TLS as described in ELFTLS_.


Perhaps mention ELFTLS when doing the TLS introduction as a for more in-depth info resource.

ACK. I'll mention that the introduction in the ABI is only sufficient to describe the terms used like Thread Control Block. A general introduction can be found in ELFTLS_

sysvabi64/sysvabi64.rst

smithp35

Thanks very much for the comments. I'll hopefully have a new patch tomorrow.

sysvabi64/sysvabi64.rst

smithp35 · 2025-06-23T16:07:12Z

sysvabi64/sysvabi64.rst

+
+AArch64 TLS SystemV design choices
+
+* AArch64 uses variant 1 TLS as described in ELFTLS_.


ACK. I'll mention that the introduction in the ABI is only sufficient to describe the terms used like Thread Control Block. A general introduction can be found in ELFTLS_

sysvabi64/sysvabi64.rst

smithp35 · 2025-06-23T17:13:41Z

sysvabi64/sysvabi64.rst

+knows that the TLS variable is defined in the same module as the code
+that is accessing the variable. In this case the offset of the TLS
+variable from the start of the module's TLS block is a static link
+time constant. Instead of dynamically calculating the offset of the


sysvabi64/sysvabi64.rst

smithp35

Thanks for the review comments. I've made the following updates:

Simple NFC text changes.
Better description of deferred TLS and generation count.
Reworded the padding size derivation.
Moved paragraphs around to make the flow a bit easier.

Should be visible as 4 separate commits

sysvabi64/sysvabi64.rst

yury-khrustalev · 2026-03-20T13:03:53Z

sysvabi64/sysvabi64.rst

+*  (Most local) Automatic data (stack variables, instanced once per function
+   activation, per thread).
+
+Rules governing thread local storage on AArch64


This section probably should be named "Scope" and the part about thread_local and __thread should probably elsewhere.

Will have a think to see where best to split out the source parts.

I've decided to remove the source parts as they are out of scope, and everyone knows what they are anyway.

yury-khrustalev · 2026-03-20T13:31:13Z

sysvabi64/sysvabi64.rst

+      return dtv[module_id][offset];
+    }
+
+The calculation in __tls_get_addr is the most general and it can be


nit:

s/__tls_get_addr/``__tls_get_addr``

yury-khrustalev · 2026-03-20T13:32:56Z

sysvabi64/sysvabi64.rst

+thread's DTV is updated, and the TLS for the ``module_id`` is
+allocated if it is not present.
+
+In pseudo code


I think this pseudo code listing is not necessary since we already have the description of the operation above. If both are retained, the code and the description may diverge over time.

OK, I wanted to include the pseudo code in case the description wasn't good enough, but if it is then I can remove it. There are other sources to find pseudo code.

yury-khrustalev · 2026-03-20T13:33:50Z

sysvabi64/sysvabi64.rst

+    }
+
+The calculation in __tls_get_addr is the most general and it can be
+applied to both static and dynamic TLS. There are four defined models


There are four defined models

This probably should start new paragraph and probably a new section

yury-khrustalev · 2026-03-20T13:34:31Z

sysvabi64/sysvabi64.rst

+The calculation in __tls_get_addr is the most general and it can be
+applied to both static and dynamic TLS. There are four defined models
+of accessing TLS that trade off generality for performance. In order
+of descending generality:


descending generality

nit: maybe "descending" is better?

Will have a think. Perhaps In descending order of generality:

yury-khrustalev · 2026-03-20T13:37:47Z

sysvabi64/sysvabi64.rst

+   4. Local Exec, can be used in the executable for TLS variables
+      defined in the executables static TLS block.
+
+SystemV AArch64 TLS addressing


SystemV AArch64 TLS addressing

The title of this section is very similar to the previous one. I think it makes it a bit difficult to navigate this chapter.

Will have a think to see if I can find a better one.

yury-khrustalev · 2026-03-20T13:41:22Z

sysvabi64/sysvabi64.rst

+  only the descriptor dialect as this is the default dialect for GCC
+  and the only dialect supported by clang.
+
+* The thread pointer (TP) is always accessible via the ``TPIDR_EL0``


The thread pointer (TP)

nit:

s/TP/``TP``

as TP is used like inline code below.

yury-khrustalev · 2026-03-20T13:43:39Z

sysvabi64/sysvabi64.rst

+(``PADsize``) between the TCB and the executable's TLS Block. Using
+``TCBsize`` as the size of the TCB (16 bytes), the following expression can be used to calcluate ``PADsize`` from the ``PT_TLS`` program header.
+
+``PADsize = (PT_TLS.p_vaddr - TCBsize) mod PT_TLS.p_align``.


style: add a .. code-block:: instead

Same for other formulae in this section. I think it will make reading easier.

yury-khrustalev · 2026-03-20T13:47:19Z

sysvabi64/sysvabi64.rst

+resolver function.
+
+The static relocations with a prefix of ``R_AARCH64_TLSDESC_``
+targeting TLS symbol ``var``, instruct the static linker to create a


nit: superfluous comma after var?

yury-khrustalev · 2026-03-20T14:01:13Z

sysvabi64/sysvabi64.rst

+
+.. code-block:: asm
+
+    adrp xn, :gottprel: var            // R_AARCH64_TLSIE_ADR_GOTTPREL_PAGE21 var


nit: superfluous space before var

yury-khrustalev

I've added a few minor comments. Otherwise this LGTM and should be approved and merged after the small fixes as per the comments. I think this chapter is very useful and provides good level of detail suitable for this ABI documentation.

smithp35

Thanks for the comments, I'll prepare a new patch, most likely going to be Wednesday.

sysvabi64/sysvabi64.rst

smithp35 · 2026-03-23T22:58:54Z

sysvabi64/sysvabi64.rst

+*  (Most local) Automatic data (stack variables, instanced once per function
+   activation, per thread).
+
+Rules governing thread local storage on AArch64


Will have a think to see where best to split out the source parts.

smithp35 · 2026-03-23T23:02:31Z

sysvabi64/sysvabi64.rst

+thread's DTV is updated, and the TLS for the ``module_id`` is
+allocated if it is not present.
+
+In pseudo code


OK, I wanted to include the pseudo code in case the description wasn't good enough, but if it is then I can remove it. There are other sources to find pseudo code.

smithp35 · 2026-03-23T23:03:06Z

sysvabi64/sysvabi64.rst

+      return dtv[module_id][offset];
+    }
+
+The calculation in __tls_get_addr is the most general and it can be


smithp35 · 2026-03-23T23:05:26Z

sysvabi64/sysvabi64.rst

+   4. Local Exec, can be used in the executable for TLS variables
+      defined in the executables static TLS block.
+
+SystemV AArch64 TLS addressing


Will have a think to see if I can find a better one.

smithp35 · 2026-03-23T23:06:00Z

sysvabi64/sysvabi64.rst

+  only the descriptor dialect as this is the default dialect for GCC
+  and the only dialect supported by clang.
+
+* The thread pointer (TP) is always accessible via the ``TPIDR_EL0``


smithp35 · 2026-03-23T23:06:29Z

sysvabi64/sysvabi64.rst

+(``PADsize``) between the TCB and the executable's TLS Block. Using
+``TCBsize`` as the size of the TCB (16 bytes), the following expression can be used to calcluate ``PADsize`` from the ``PT_TLS`` program header.
+
+``PADsize = (PT_TLS.p_vaddr - TCBsize) mod PT_TLS.p_align``.


smithp35 · 2026-03-23T23:06:55Z

sysvabi64/sysvabi64.rst

+resolver function.
+
+The static relocations with a prefix of ``R_AARCH64_TLSDESC_``
+targeting TLS symbol ``var``, instruct the static linker to create a


smithp35 · 2026-03-23T23:09:11Z

sysvabi64/sysvabi64.rst

+
+.. code-block:: asm
+
+    adrp xn, :gottprel: var            // R_AARCH64_TLSIE_ADR_GOTTPREL_PAGE21 var


The thread local storage chapter contains: * A description of Thread Local Storage based on addenda32 * The key design decisions of AArch64 TLS such as tls variant, tls dialect, TCB size. * The ABI required code sequence for TLSDESC that must be emitted exactly, as GNU ld requires it to be. * Sequences for the different code-models. * Relaxations for GD->IE, GD->LE and IE->LE. * Synchronization requirements for Lazy TLSDESC. With advice not to support it due to overhead of synchronization.

* Edits to split up the bullet points in How to denote TLS in source. * Changed program-own state to process-state as the thread-id may not be stored separately from the programs data. * Removed typically from some of the descriptions as the typically will almost always be the case for a sysvabi platform. * Linked alignment padding to the definition. * Provided a bit more information about generation counters.

* Rearranged formulas and used TCBsize to make it clearer. * Taken out "significant" from a significant number of dynamic linkers. * Give reason for using relaxation rather than optimization. * Clarify that there is no requirement to implement any TLSDESC resolver given in the sysvabi.

Change the input register in add xn, xn, :tprel_hi12:var, lsl ARM-software#12 to the thread pointer tp. We want to calculate the offset from the thread pointer so it needs to be an input of the add.

Document the decision in the GCC mailing list thread TLSDESC clobber ABI stability/futureproofness? https://gcc.gnu.org/legacy-ml/gcc/2018-10/msg00112.html TLSDESC resolver functions assume that any registers added by an extension are caller saved for a TLSDESC call. A brief summary: Dynamic TLS may be lazy allocated upon the first use of a TLSDESC resolver. This may involve calls to heap allocation functions provided by the user, which may use registers from extensions like SVE and SME. As the resolver function can't know what is saved it would have to save all SVE and SME state. This would be way more expensive than a caller save, and an older libc written prior to the introduction of the extension would be unaware of them so the caller has to do the save. * The SVE and SME state is already

Include a pseudo code description of __tls_get_addr with deferred TLS for dynamic modules.

Use integers modulo m to avoid excess use of (modulo m). Explain the congruence symbol. Put expression first so derivation is optional.

are used.

The TLSDESC resolver functions are not ABI so we can move them out of the sysvabi64 document. Providing some examples that can be used by a dynamic linker is still useful so move this to the design documents section. Add a comment about DTV surplus TLS that permits a dynamic loader to dlopen a DSO with initial-exec TLS. There can be a small number of performance critical shared-libraries that use initial exec TLS, but are expected to be opened via dlopen, particularly by scripting languages like python.

* Added `` `` to some variables. * Added some more section headings. * Used code-blocks for formula. * Fixed reference to design document.

smithp35 · 2026-03-25T11:36:20Z

Made changes to address review comments and rebase.

Previous review comments changed name of a section.

smithp35 force-pushed the sysvabitls branch from a261722 to 98ab4d7 Compare February 20, 2025 17:47

MaskRay reviewed Feb 21, 2025

View reviewed changes

smithp35 commented Feb 21, 2025

View reviewed changes

This was referenced Mar 18, 2025

[AArch64] Add an option to get the TLS pointer from software llvm/llvm-project#130932

Open

[rtabi64?] Add method for Software-based TLS Pointer Retrieval #316

Open

smithp35 mentioned this pull request Apr 23, 2025

[lld][ICF] Prevent merging two sections when they point to non-globals llvm/llvm-project#136641

Open

stuij reviewed Jun 23, 2025

View reviewed changes

smithp35 commented Jun 23, 2025

View reviewed changes

smithp35 force-pushed the sysvabitls branch from 1ed7570 to e46f8bf Compare June 27, 2025 12:43

smithp35 commented Jun 27, 2025

View reviewed changes

sysvabi64/sysvabi64.rst Outdated Show resolved Hide resolved

sysvabi64/sysvabi64.rst Outdated Show resolved Hide resolved

smithp35 force-pushed the sysvabitls branch from 2df785e to 8c21467 Compare August 15, 2025 14:06

smithp35 added the sysvabi64 label Feb 10, 2026

smithp35 requested review from yury-khrustalev February 19, 2026 15:31

yury-khrustalev reviewed Mar 20, 2026

View reviewed changes

sysvabi64/sysvabi64.rst Show resolved Hide resolved

yury-khrustalev reviewed Mar 20, 2026

View reviewed changes

smithp35 commented Mar 23, 2026

View reviewed changes

smithp35 added 2 commits March 25, 2026 09:48

smithp35 added 10 commits March 25, 2026 09:48

fix CI due to broken reference

75882be

[sysvabi64] Correct local exec instruction to use tp as input

cec90b5

Change the input register in add xn, xn, :tprel_hi12:var, lsl ARM-software#12 to the thread pointer tp. We want to calculate the offset from the thread pointer so it needs to be an input of the add.

[sysvabi64] Simple fixes from Ties' review [NFC]

cd67e16

Update description of the generation counter and __tls_get_addr

1c4b738

Include a pseudo code description of __tls_get_addr with deferred TLS for dynamic modules.

Reword derivation of TLS padding to make it easier to understand

48e730b

Use integers modulo m to avoid excess use of (modulo m). Explain the congruence symbol. Put expression first so derivation is optional.

Move around some paragraphs to introduce some topics before they

555d8af

are used.

[sysvabi64] Yury's review comments

0834e04

* Added `` `` to some variables. * Added some more section headings. * Used code-blocks for formula. * Fixed reference to design document.

smithp35 force-pushed the sysvabitls branch from 8c21467 to 0834e04 Compare March 25, 2026 11:35

[sysvabi64] Fixup broken link [NFC]

5076042

Previous review comments changed name of a section.


		AArch64 TLS SystemV design choices

		* AArch64 uses variant 1 TLS as described in ELFTLS_.


		.. code-block:: asm

		adrp xn, :gottprel: var // R_AARCH64_TLSIE_ADR_GOTTPREL_PAGE21 var

Conversation

smithp35 commented Feb 20, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

smithp35 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

smithp35 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

smithp35 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!