Arm Community
Arm Community
  • Site
  • User
  • Site
  • Search
  • User
Arm Community blogs
Arm Community blogs
Architectures and Processors blog The LLVM AArch64 Backend
  • Blogs
  • Mentions
  • Sub-Groups
  • Tags
  • Jump...
  • Cancel
More blogs in Arm Community blogs
  • AI blog

  • Announcements

  • Architectures and Processors blog

  • Automotive blog

  • Embedded and Microcontrollers blog

  • Internet of Things (IoT) blog

  • Laptops and Desktops blog

  • Mobile, Graphics, and Gaming blog

  • Operating Systems blog

  • Servers and Cloud Computing blog

  • SoC Design and Simulation blog

  • Tools, Software and IDEs blog

Tell us what you think
Tags
  • OpenCL
  • clang
  • c_language
  • C++
  • compiler
  • LLVM
  • Armv8
  • open_source
  • Linux
Actions
  • RSS
  • More
  • Cancel
Related blog posts
Related forum threads

The LLVM AArch64 Backend

Kristof Beyls
Kristof Beyls
October 3, 2013
8 minute read time.

In this blogpost, Gabor Ballabas and I share why and how we’ve set up a public buildbot for the LLVM AArch64 backend. This buildbot is part of the LLVM community’s continuous integration infrastructure.

LLVM is an open source project providing a set of code generation libraries which are increasingly used to generate code for platforms with ARM® CPUs. These libraries are responsible for the code generation in most OpenCL drivers, Renderscript and the Clang C and C++ compiler.  FreeBSD and Tizen use Clang as the default platform compiler and Debian buildbots regularly build all of Debian with Clang. Clang is included in the Android NDK. The LLVM Linux project targets building the Linux kernel with LLVM.

To ensure first-class software support for the ARMv8 architecture, a well-functioning ARMv8 AArch64 backend for LLVM is a significant and necessary step. As of today, the ARMv8 AArch64 backend can compile regular C and C++ code and support for NEON™ instructions and intrinsics is being added. Furthermore, the addition of support for ARMv8 AArch32 in LLVM is in progress. The combination of the AArch64 and the AArch32 work ensures that LLVM will fully support the ARMv8 architecture. More information about ARM terminology can be found here.

One of our main concerns while implementing support for the ARMv8 architecture in LLVM is to make sure that you can trust LLVM to produce AArch64 object code correctly for C, C++, OpenCL, or any other programming language. One scary aspect of an open source compiler project is that every day, hundreds of patches get committed to the code base; often by developers focussing mostly on other architectures and instruction sets. Therefore, we've put great emphasis on adding high quality regression tests, blocking any changes to the LLVM open source project that would severely regress the AArch64 backend. Our efforts are paying off: the AArch64 backend is already one of the backends in LLVM that are best-covered by regression tests.

In addition to the regression tests, which are used by all LLVM developers to verify they are not introducing regressions when changing the code base, we have also set up a buildbot running on an ARMv8 Foundation Model. This buildbot continuously cross-compiles Clang and LLVM itself using the top-of-trunk sources. This way, every few hours, the buildbot tests whether Clang can still correctly cross-compile itself, targeting an AArch64 Linux platform.

In the paragraphs below, Gabor explains how this buildbot has been set up, including the underlying technologies used. This helps to enable developers to recreate a similar setup to test LLVM; but this can also be used as a guideline to set up similar testing environments for other software projects that you want to port to AArch64.

The main steps in setting up the buildbot consists of (a) setting up an ARMv8 Foundation Model with Linux running on it; (b) setting up a Linaro cross-compilation toolchain targeting AArch64 Linux; (c) cross-compiling Clang; (d) running the regression tests in the foundation model with the cross-compiled Clang and (e) integrating the setup with LLVM's buildbot infrastructure. This is explained in detail below.

(a) Setting up an ARMv8 Foundation Model with Linux

  • The ARMv8 Foundation Model required to run the AArch64 binaries, produced by the LLVM AArch64 backend, can be downloaded free of charge from here (after registration): http://www.arm.com/products/tools/models/fast-models/foundation-model.php
  • The next step is getting a boot image and a rootfs. There are some solutions to choose from:
    1. Linaro provides cross compilation toolchain for AArch64, along with a boot image and rootfs for ARMv8 foundation model, which is accessible via  https://wiki.linaro.org/HowTo/HelloAarch64
    2. Fedora has an AArch64 port at http://fedoraproject.org/wiki/Architectures/ARM/AArch64
    3. Debian has a solution too: https://wiki.debian.org/Arm64Port

In the following steps, we shall use the boot image and rootfs provided by Linaro. The Linaro solution is based on Ubuntu and it comes with the most important packages. It currently lacks some of the necessary packages (for example, Python lacks modules needed for running the LLVM test suite) but this can be worked around as described in section d).

  • Starting the Foundation Model with the Linaro boot image and rootfs:
      Foundation_v8pkg/models/Linux64_GCC-4.1/Foundation_v8 --image path/to/linaro/boot/image --block-device path/to/linaro/rootfs
  • Setting up external communication through network:
    The Foundation Model can use NAT or Ethernet bridge for networking. The NAT is easier to set up but it is much slower than the bridge networking. Configuring bridged networking on Linux is distribution specific. We will outline the necessary steps on Ubuntu, which is a popular Linux distribution:
    1. Install tunctl and brctl with "sudo apt-get install uml-utilities bridge-utils"
    2. These commands can be used to set up the tun/tap device on the Foundation Model:
        tunctl -u <user_id>           # create tap0 for user you have credential in the network.
        brctl addbr br0               # create network bridge
        ifconfig eth0 0.0.0.0 promisc # enable promiscuous mode
        brctl addif br0 eth0          # add eth0 to bridge br0
        dhclient br0                  # using dhcp to get ip address
        brctl addif br0 tap0          # add tap0 to bridge br0
        ifconfig tap0 up              # enable tap0
        chmod 666 /dev/net/tun        # set up tun permission
    3. After these commands, you can start the Foundation Model with the following arguments, and use the tap0 device:
        Foundation_v8pkg/models/Linux64_GCC-4.1/Foundation_v8 --image path/to/linaro/boot/image --block-device path/to/linaro/rootfs --network=bridged --network-bridge=tap0

(b) Setting up a Linaro cross-compiling toolchain

  • You’ll need a Linaro cross-compiling toolchain targeting AArch64 Linux to start with. The instructions to download and set up the Linaro tool-chain can be found here: https://wiki.linaro.org/HowTo/HelloAarch64
  • After the bin directory of the tool-chain is added to your PATH, you can use the various binaries, e.g.: aarch64-linux-gnu-gcc –v
  • If you prefer to use Clang for cross-compiling, the sysroot of the Linaro tool-chain must be passed to the compiler (the same with clang++):
      clang -target aarch64-linux-gnu -gcc-toolchain $TOOLCHAIN_PATH --sysroot=$TOOLCHAIN_PATH/aarch64-linux-gnu/libc
    If the command fails because of missing libs, we need to create two symlinks in the tool-chain's aarch64-linux-gnu/libc/usr/lib directory:
      ln -s ../../../lib/libgcc_s.so
      ln -s ../../../lib/libstdc++.so

(c) Cross-compiling Clang targeting AArch64

  • Get the LLVM source.  You will only need to check out LLVM and Clang. There is a git mirror for the svn repository, and both git and svn should work to check out the source code. To build the clang compiler, make sure clang is in the llvm/tools folder. More information can be found at: http://www.llvm.org/docs/GettingStarted.html#getting-started-quickly-a-summary
    1. LLVM:
        GIT: git clone http://llvm.org/git/llvm.git
        SVN: svn co http://llvm.org/svn/llvm-project/llvm/trunk
    2. Clang:
        GIT: git clone http://llvm.org/git/clang.git
        SVN: svn co http://llvm.org/svn/llvm-project/cfe/trunk clang
  • Configure LLVM from a separate build directory (cross-compiling doesn't work directly inside the LLVM source directory):
      path/to/llvm-src/configure --host=aarch64-linux-gnu
  • Run make.
  • If you want to use Clang instead of gcc to cross-compile LLVM you will need two wrapper scripts (or aliases) since the configure script will look for aarch64-linux-gnu-clang and aarch64-linux-gnu-clang++. Thus you need to invoke clang or clang++ with the arguments described in the Cross-compiling environment for AArch64 target section above.

(d) Running the LLVM test suite on the ARMv8 Foundation Model

  • NFS directory sharing:
    After cross-compiling LLVM we have to run it inside the Foundation Model. NFS is the most convenient solution for this task. The LLVM build system has a fixed directory layout, and we need to provide the same layout when the tests are running.
  • The Python package used in the Linaro rootfs lacks some modules required for running the LLVM test suite. Thus we need a custom compiled Python for running tests:
    1. Get the source code and unpack it:
        wget http://www.python.org/ftp/python/2.7.3/Python-2.7.3.tar.bz2
        tar xvjf Python-2.7.3.tar.bz2
    2. Configure and make a python for the host system:
        ./configure
        make python Parser/pgen
        mv python hostpython
        mv Parser/pgen Parser/hostpgen
        make distclean
    3. Get the patch that makes cross-compiling possible and apply it:
        wget http://randomsplat.com/wp-content/uploads/2012/10/Python-2.7.3-xcompile.patch
        patch -p1 < Python-2.7.3-xcompile.patch
    4. Configure for the cross-target:
      Make sure that the Linaro AArch64 toolchain is in your path!
        CC=aarch64-linux-gnu-gcc \
          CXX=aarch64-linux-gnu-g++ \
          AR=aarch64-linux-gnu-ar \
          RANLIB=aarch64-linux-gnu-ranlib \
          ./configure --host=aarch64-linux-gnu --build=${YOUR_SYSTEMS_TRIPLE_HERE} --prefix=${WHEREVER_YOU_WANT_IT}
      For example, on an x86-64 host machine, you should set YOUR_SYSTEMS_TRIPLE_HERE to x86_64-linux-gnu.
    5. Make:
        make HOSTPYTHON=./hostpython \
          HOSTPGEN=./Parser/hostpgen \
          LDSHARED="aarch64-linux-gnu-gcc -shared" \
          CROSS_COMPILE=aarch64-linux-gnu- \
          CROSS_COMPILE_TARGET=yes \
          HOSTARCH=aarch64-linux-gnu \
          BUILDARCH=${YOUR_SYSTEMS_TRIPLE_HERE}
      NOTE: Ignore the following message that says: "Python build finished, but the necessary bits to build these modules were not found"  and a list of those modules. We don't need them (at least for the LLVM test framework).
    6. Make install:
        make install HOSTPYTHON=./hostpython \
          BLDSHARED="aarch64-linux-gnu-gcc -shared" \
          CROSS_COMPILE=aarch64-linux-gnu- CROSS_COMPILE_TARGET=yes \
          prefix=${WHEREVER_YOU_WANT_IT}
    7. Now we can copy the content of the directory where Python is installed to the Foundation Model and change the PATH to make it the default python.
  • There is a kernel bug that makes a couple of tests fail. The fix is available in the 3.12 kernel, but the latest Linaro AArch64 kernel is only 3.10 at the moment. If you want to run those tests as well your best option is patching the Linaro kernel following this howto: https://wiki.linaro.org/HowTo/BuildArm64Kernel. The patch itself can be downloaded from here: https://git.kernel.org/cgit/linux/kernel/git/cmarinas/linux-aarch64.git/commit/?id=db6f41063cbdb58b14846e600e6bc3f4e4c2e888
  • Running the tests:
    If you NFS-mirrored the LLVM on the Foundation Model - as described above - you can simply use 'make check-lit' in the LLVM build directory to run all tests.

(e) LLVM AArch64 buildbot

  • Link to the buildbot: http://lab.llvm.org:8011/waterfall?show=llvm-aarch64-linux
  • Add a new buildslave to the LLVM builbot system, see: http://llvm.org/docs/HowToAddABuilder.html
  • Compiling the LLVM source code in the Foundation Model is time consuming, so the AArch64 buildslave uses cross-compilation. The buildslave runs on the host machine, and transmit commands to the Foundation Model.

Co-Author:

Gabor_Ballabas.jpgGabor Ballabas, Developer - University of Szeged, is a Developer at the Software Engineering Department in the University of Szeged, Hungary. Currently he is the maintainer of the LLVM Aarch64 buildbot.

Anonymous
  • Alban Rampon
    Alban Rampon over 11 years ago

    Hey Software Development Tools, this article shows an example of utilisation of the ARMv8 Foundation Model.

    George, I saw your Fast Models introduction (Fast Models (快速模型) 简介) article which Google Translate allowed me to somewhat read. If you are also working on ARMv8, this article should interest you!

    • Cancel
    • Up 0 Down
    • Reply
    • More
    • Cancel
Architectures and Processors blog
  • Introducing GICv5: Scalable and secure interrupt management for Arm

    Christoffer Dall
    Christoffer Dall
    Introducing Arm GICv5: a scalable, hypervisor-free interrupt controller for modern multi-core systems with improved virtualization and real-time support.
    • April 28, 2025
  • Getting started with AARCHMRS Features.json using Python

    Joh
    Joh
    A high-level introduction to the Arm Architecture Machine Readable Specification (AARCHMRS) Features.json with some examples to interpret and start to work with the available data using Python.
    • April 8, 2025
  • Advancing server manageability on Arm Neoverse Compute Subsystem (CSS) with OpenBMC

    Samer El-Haj-Mahmoud
    Samer El-Haj-Mahmoud
    Arm and 9elements Cyber Security have brought a prototype of OpenBMC to the Arm Neoverse Compute Subsystem (CSS) to advancing server manageability.
    • January 28, 2025