Add target_toolchain_roots and make sysroot more hermetic#441
Add target_toolchain_roots and make sysroot more hermetic#441AustinSchuh wants to merge 16 commits intobazel-contrib:masterfrom
Conversation
It is useful when cross compiling to have 1 LLVM build used as the compiler, and multiple others used for their libc++/target libraries. This concept forces us to be much more robust about what comes from the target toolchain, and what comes from the compiler. Leave toolchain_roots for the compiler roots, but add a new flag, target_toolchain_roots with the same semantics for target toolchains. This necesitates being much more clear internally about what comes from what. This isn't fully enough to make everything hermetic, clang still has some autodetection logic left over when using sysroots. We want to very explicity specify the resource directory it should use, as well as where the standard C/C++ libraries come from so it finds only what we want it to find. When pulling from a sysroot, we need to know the version of stdc++ to find includes. Use the stdlib variable for that by supporting "stdc++-12" for example, and parsing the number out.
AustinSchuh
force-pushed
the
target_toolchain_roots
branch
from
170ca55 to
4291cb6
Compare
|
bazelbuild/rules_cc#304 needs to land and be available before this can merge. But I would appreciate feedback on this before then as that could take a bit. We used features before for asan/msan/cuda/etc, but I think we can use flags + select instead and avoid needing to change rules_cc again. Feedback welcome on the best way to do all that. |
# Conflicts: # toolchain/cc_toolchain_config.bzl
|
@AustinSchuh Your goals (being very explicit about the compiler resource dir and compiler-rt vs libc++/etc) are very aligned with https://github.com/hermeticbuild/hermetic-llvm. You might want to give that one a shot if you haven't tried it yet (or let us know what's missing for you!) |
|
Thanks @dzbarsky. Your goals sound similar, but I've got numerous yocto sysroots I need to cross compile for, and will need to add CUDA support for those sysroots too. The docs on your repo suggest that is not the direction you want to go. Long long ago (2019 or earlier), we patched the old grailbio toolchains to do all this, and never got around to upstreaming our changes. This is one patch in a series attempting to get us back inline with upstream. |
AustinSchuh
force-pushed
the
target_toolchain_roots
branch
from
8513815 to
cb2e60f
Compare
|
OK, all the tests pass, and there are no merge conflicts with main. I'd love an actual review now. |
3 participants
It is useful when cross compiling to have 1 LLVM build used as the compiler, and multiple others used for their libc++/target libraries.
This concept forces us to be much more robust about what comes from the target toolchain, and what comes from the compiler. Leave toolchain_roots for the compiler roots, but add a new flag, target_toolchain_roots with the same semantics for target toolchains. This necesitates being much more clear internally about what comes from what.
This isn't fully enough to make everything hermetic, clang still has some autodetection logic left over when using sysroots. We want to very explicity specify the resource directory it should use, as well as where the standard C/C++ libraries come from so it finds only what we want it to find.
When pulling from a sysroot, we need to know the version of stdc++ to find includes. Use the stdlib variable for that by supporting "stdc++-12" for example, and parsing the number out.