Compiling Twice as Fast: Native Clang for Windows on Arm

I use hundreds of different tools in my day job at Arm Research, but the three most important are probably the same for any programmer: a good web browser, a really good text editor, and a really good compiler. Following Arm’s work on Electron and Chromium, Microsoft released both the native Edge browser, and recently, the Electron-based Visual Studio Code. However, we were still missing a good compiler... until now.

Many developers for Windows on Arm (WoA) would like to have a high-performance open-source C/C++ compiler option, in addition to MSVC (Microsoft Visual C compiler). A native compiler like Clang gives the highest compilation performance and can offer the best compatibility with existing open-source projects.

To ensure that our build of Clang benefits from the optimizations in LLVM, it should be built using the recommended process for building a release of Clang. We use the pre-built x86 version of Clang as our stage 1 compiler to build our own stage 2 compiler to run natively on Arm.

Why use a native Clang build?

• For WoA devices, using a native Clang instead of an x86 build of Clang, makes compilation twice as fast. On my Surface Pro X, the x86 build of Clang takes 1 hour and 28 minutes to bootstrap Clang, but the native Arm build only took 42 minutes.
• Clang includes a compatibility layer for MSVC. You can use clang-cl.exe as a drop-in replacement for cl.exe (the MSVC compiler executable). You can also get native performance in projects that already use MSVC. In some follow up blog posts, we’ll explain how you can build your existing windows apps for Arm using clang-cl.
• And, of course, you don’t need to cross-compile for Arm, so there’s less complexity in your build flow.

If you would rather not build your own compiler, then that is fine, we have provided a pre-built version of Clang for WoA right here for you. For those that do want to build Clang from scratch, you can find the following instructions.

llvm-package.zip

Enough talk, how do I compile Clang?

For those that want to get Clang compiling now, you can follow the following steps. For those who want the details, you can also read slightly about this process in the section at the end, after these instructions.

There are two steps to this build. The first one is installing all the required tools, and the second is to simply run the script provided here to fetch and build Clang automatically.

Step one (1) Installing prerequisites

The build depends on a few things which you need to install if you do not already have them. Some install settings need changing from their defaults so follow the instructions carefully, so you do not miss anything.

At the time of writing, none of these projects provide WoA binaries yet. So for now you need to download the Windows binaries listed as “Windows 32-bit” or “Windows x86” for each one.

Git

This is the download page: https://git-scm.com/download/win

If you have already got git, you can skip this step, otherwise download and run the 32-bit windows installer. When you run the git installer, at some point you will reach this:

The default option here is fine, but if you prefer a different option you can change it. This should not be a problem for the build. This also means that if you have already got git installed then you do not need to change this setting. Leave all of the other settings on their defaults.

CMake

This is the download page: https://cmake.org/download/#latest

If you do not already have CMake installed then download and run the 32 bit Windows CMake installer. The following image shows the first stage of the CMake Setup Wizard:

During the installation, you will be given the option to put CMake on the system PATH:

The provided build script expects CMake to be on the path. You can choose either of the two options for putting CMake on the path.

If you have already installed CMake previously, but did not put CMake on the PATH, you will need to manually add it to your system PATH. You can check if CMake is on the PATH by opening powershell and typing cmake --version.

If you see a CMake version number printed, then you have CMake on the PATH. You can even see where it is installed by using gcm cmake. If that is the case, then you can skip to the next step: installing Python. See image below.

If CMake is not on the path, you will need to find the install path, which will likely match the screenshot above: C:\Program Files (x86)\CMake\bin\cmake.exe

Once you have the path to CMake, you need to add it to your system PATH. To do this, search for and launch “Advanced system settings” from the start menu, click “Environment Variables…”, select “Path” in the “System variables” pane, click “Edit”, and then click “New”. Copy and paste the path into the new item in the list, but make sure not to include “cmake.exe” at the end of the path. If you have used the default install directory then this should be: C:\Program Files (x86)\CMake\bin\

Python3

This is the download page: https://www.python.org/downloads/windows/

The Python install should be fairly straightforward. If you already have a fairly recent version of Python 3 installed, then you can skip this step. Otherwise download and run the 32 bit Windows installer. See the following image.

Note once again that by default this installer does not add anything to the system PATH. But you can tick the checkbox at the bottom of the installer window to make it add Python to the PATH automatically. If you have already installed Python 3 previously, but did not add it to the PATH, you can follow steps similar to the ones for CMake in the previous step. This makes sure you have Python on your system PATH.

Clang

This is the download page: https://releases.llvm.org/download.html

Since we are bootstrapping Clang using the x86 Windows release, you will also need to install that.

For some users, Windows presents a SmartScreen security warning when downloading the Clang installer, as it is not signed using the standard Microsoft executable signature system. Make sure you download it from the official llvm website. You then confirm that you trust the download and choose the option to run the file anyway on the security warning dialog. The download page includes links to signatures and a PGP key.

Opposed to all the other dependencies we have installed so far is the case of the Clang/LLVM install. In this instance, it is preferable not to add LLVM to the system PATH. If you really want to then you can choose to deviate from the default and add LLVM to the system PATH. However, we are about to compile our own version of Clang, which we want to use as the default for everything we build with Clang on this machine. Therefore, it is easier to avoid ending up using the wrong instance of Clang if it’s not on the PATH.

The build script provided here looks for Clang in the default install directory, so be sure not to change the install directory. It does not attempt to find Clang on the system PATH.

Visual Studio

Now we need to make sure that we have all the headers and libraries needed for compiling for WoA. To do this, we need to use the Visual Studio installer.

We are going to install Visual Studio 2019 Community Edition. If you have already got that installed, then that is great, but you still need to run the installer to make sure you have everything you will need. The Visual Studio installer manages components of Visual Studio for you, and there are a few Arm specific components that we need to make sure we have installed.

If you do not already have it, get the Visual Studio installer from here: https://visualstudio.microsoft.com/downloads/

Make sure you get the 2019 Community Edition.

Once you open the installer, you should see this…

If you already have Visual Studio installed, then you will see something slightly different. You will see that it says you already have Visual Studio installed, and that the installer will allow you to modify the existing installation.

The easiest way to make sure you have all the right components installed is to use the configuration file provided here:

woa.vsconfig

In the Visual Studio installer, you can click on the “More” button in the Visual Studio Community 2019 pane, and then click “Import configuration” and select the downloaded configuration file. See the following.

If you prefer to select the components manually, perhaps because you want to make sure your existing configuration is not broken by importing this one, then you can modify your existing install manually as follows:

Make sure you have “Desktop development with C++” selected in the workloads tab, and then use the “Individual components” tab to select the following Arm64 components for Visual Studio:

C++ Universal Windows Platform support for v142 build tools (ARM64)

MSVC v142 - VS 2019 C++ ARM64 build tools (v14.26)

MSVC v142 - VS 2019 C++ ARM64 Spectre-mitigated libs (v14.26)

C++ ATL for latest v142 build tools (ARM64)

C++ ATL for latest v142 build tools with Spectre Mitigations (ARM64)

Windows 10 SDK (10.0.19041.0)

With the configuration loaded, or the components selected, now press install.

It will take a while, as it fetches and installs all the components you need.

Visual C++ Redistributable

This is the download page: https://support.microsoft.com/en-gb/help/2977003/the-latest-supported-visual-c-downloads

This is the last thing we need to install. Make sure you download and install the Arm64 version of the Visual C++ Redistributable: vc_redist.arm64.exe

Step two (2) Run the build script

Now you should be able to simply run the provided Python script, and it will automatically fetch and build Clang for you.

clang_woa_build.py

Using cmd or powershell, you can run the Python script from the directory where you saved it.

I would recommend doing this using powershell in a modern terminal emulator, such as Windows Terminal. Other terminal emulators for Windows are also available.

To do this, use the following commands, changing the CD path to match where you saved the file:

cd C:\users\YOU\folder_containing_clang_woa_build_dot_py\

python .\clang_woa_build.py

After a while you should see “Build completed successfully”. The script will have created a folder called llvm-project, and another called ninja (unless you already had ninja installed)

You will now find 64-bit Arm versions of the Clang binaries in llvm-project\build\bin. Configure your project to build using these, and you will be building native 64-bit Arm binaries for WoA, using a native compiler. Congratulations.

The details, for those who like details

What is this magical Python script doing?

The Python script starts off by making sure it can find all of the required software. The first time it runs, it will probably fail to find ninja. Sometimes an install of Visual Studio includes a ninja binary, but this might not be compatible with the system. So ninja needs to be compiled from source. If you see an error about your version of Windows not being supported when the script tries to find ninja, this is why.

The script fetches ninja and uses the provided bootstrapping Python script to build ninja. This is fairly simple and does not take too long to do. This is only done if there was not already a working ninja on the device.

The next step is the important one - building Clang. The script fetches Clang, makes sure to check out the release/10.x branch, and proceeds to generate the ninja build files with CMake. If you would like to build a different branch you can specify it with the --branch option. You can also run the Python script with --help to see the details on all the other options to the script.

To make sure that the build system can find all of the required libraries and header files, the script makes use of the vcvars scripts provided with visual studio. Although we are compiling Clang using Clang, we need to use the libraries provided with MSV. Therefore, using this to create the environment for the build system is the easiest way to get a working build environment. This is the reason why we needed to install Visual Studio with all of those extra components, as these components are libraries to support C and C++ development for Arm on Windows.

The script also makes sure to use clang-cl as the compiler, and not plain clang. This is because the environment set by vcvars is designed to work with Microsoft’s cl.exe compiler. Therefore, we use clang-cl, which is the version of Clang that presents an interface compatible with cl.exe.

The script also currently applies a patch to the LLVM repo to fix an issue with a CMake build script that would prevents us from building on WoA. But apart from that, it then just runs a regular build configured for Arm. Once that issue is fixed upstream, this script will still work by simply adding the --no-patches option.

So that is it. Hopefully, we have made this really easy, and hope to see some interesting projects built for WoA.

[CTAToken URL = "https://developer.arm.com/solutions/os/windows-on-arm" target="_blank" text="Learn more about developing for WoA" class ="green"]