CheriBSD Morello FVP Pure-Capability - How to cross-compile from source?

Hi there,

I am trying to cross-compile curl from source to CheriBSD Morello FVP Pure-Capability. I have made a note of all the library and package dependencies listed here: https://packages.ubuntu.com/focal/curl and downloaded the original source files for each using the download links provided.

Normally, when compiling for the host system I'm on, I would just use the ./configure scripts provided with each source download, then run make to compile, and make install to install the software. How would I go about cross-compiling curl for CheriBSD Pure-Capability with the Morello FVP? I heard it is possible to make a new target for cheribuild, but looking at the wiki and the guide there on how to do so, I have no idea where to start or what is required. Or maybe it is still possible to use the ./configure scripts, but cross-compile with them?

Thanks in advance!

Parents Reply Children
  • Thanks so much for this information. Because I will be cross-compiling quite large projects, I think using the generated ./configure script is the best way to go for me. So I've cloned the curl git repo and ran autoconf -fi to generate all the configuration files. Now, let's say I'm wanting to compile for morello purecap, would I run something similar to:

    ./configure CC=aarch64-unknown-freebsd-clang --build=x86_64 --host=aarch64-unknown-freebsd13 --sysroot=/home/ubuntu/cheri/output/rootfs-morello-purecap

    Or have I got the wrong idea here?

    Yeah, that's the right kind of idea.

    CC="$HOME/cheri/output/morello-sdk/bin/clang --target=aarch64-unknown-freebsd13 --sysroot=$HOME/cheri/output/rootfs-morello-purecap" CFLAGS="-march=morello+c64 -mabi=purecap -femulated-tls" LDFLAGS="-fuse-ld=lld" PKG_CONFIG_PATH= PKG_CONFIG_LIBDIR= ../curl/configure --host=aarch64-unknown-freebsd13 --with-openssl --with-ca-path=/etc/ssl/certs

    works for me on my Mac, as a from-scratch set of minimal flags needed for curl. Note that --build is autodetected and --sysroot is an argument to Clang not configure. The PKG_CONFIG_* being set to the empty string are to avoid the configure script's calls to pkg-config finding my native system libraries for various things it searches for.

    (Note also the use of `-femulated-tls`, which is a temporary workaround; until recently the thread-local storage ABI for Morello was entirely undocumented so we couldn't implement the run-time side of it in CheriBSD and instead resorted to emulated TLS. There's a pull request against the spec now to document the current ABI so we could support it in CheriBSD but the current ABI is more experimental than the rest of the toolchain and I raised some concerns with the current approach, so given emulated TLS works we're sticking with that for now until things are more settled. If you don't have `-femulated-tls` you may see the odd undefined symbol reference, in this case `_ThreadRuneLocale`.)

    Just for my own learning going forward, could you walk me through the steps you took to add the target to cheribuild? There are a lot more pieces of software I wish to add in the future, so knowing this process would mean it's less likely for me to have to keep asking questions here on this topic :)

    The short answer is "find an existing target that does what you want and copy the bits you need from it". At this point, most things you could want to do are already done somewhere in cheribuild. In this case, I knew curl was using autotools as its build system so copied the skeleton for an autotools-based project from elsewhere, in this case libffi. We've never really documented cheribuild before because it's been a tool we've developed that users have previously just needed to use as-is, without needing new targets. But the general principles are:

    1. Define a new class BuildMyTarget that inherits from CrossCompileFooProject, for some build system Foo (they're all in pycheribuild/projects/cross/crosscompileproject.py if you want to see what's supported).

    2. Set the repository member to the Git repository

    3. If needed, set the target member to the name of your target (by default, BuildMyTarget_Name gives a target name of mytarget-name, so in fact I didn't need to specify it for curl)

    4. Set supported_architectures to CompilationTargets.ALL_FREEBSD_AND_CHERIBSD_TARGETS + [CompilationTargets.NATIVE] (hopefully self-explanatory) unless there's a good reason why that won't work

    5. If your target depends on other targets (other than the implied default SDK targets, i.e. a compiler and sysroot) then set dependencies to a list of target names *without any architecture suffix*. If you're in one of the rare situations where the list of dependencies changes based on the architecture being compiled for then you can also declare dependencies as a @classmethod that returns the list and can dynamically create the list as needed.

    6. Override the setup method if you need to pass additional environment variables, configure arguments or make arguments; self.configure_arguments is a list you can add to, some build system superclasses give self.add_foo_options methods (e.g. self.add_cmake_options(FOO="bar") will pass -DFOO=bar to cmake), self.configure_environment is a dictionary, self.make_args has a set method (for normal arguments) and an env_vars dictionary field (for environment variables, as you might guess, though slightly weird it's under a thing called make_args). If you just grep the sources for those various things it'll hopefully become clear

    7. If you need to modify any of the configure, build or install steps beyond just arguments and environment variables then configure, compile (not build) and install are methods you can override to do whatever you need (e.g. see curl where configure needs to run autoreconf before running the normal configure method).

    8. If you need to query information (where various directories are, what the target is, compiler flags) for something not automatically handled for you, the base SimpleProject class has a whole load of methods you can call (see pycheribuild/projects/project.py). self.target_info and self._xtarget (prefer the former if it has the information you need) also exist for additional information about the architecture you're compiling for (I always forget the exact difference between them, so just grep for examples of using both to find the information you need).

    We have some ideas for how to make this easier (e.g. potentially adding a special set of `cheribuild build-current-directory-as-autotools-morello-purecap` targets so you can reuse the standard CrossCompileFooProject infrastructure in cheribuild without having to write a target for your project), but none of that exists today unfortunately.

  • Once I have created the myproject.py file in cheribuild/pycheribuild/projects/cross, do I have to run the script with ./myproject.py

    That will not work, and that's not how you use any of the other targets so that's not how you use an additional target you've added.

    or does the file only have to be in that directory so it can be used by cheribuild?

    Any .py file in projects/ and projects/cross/ is automatically picked up.

    Another question, is there a way I can keep the machine online, so that I can close the terminal and not have to wait 10 minutes for ./cheribuild run-fvp-morello-purecap to run when I want to boot up again?

    screen/tmux? There's nothing special about cheribuild in this regard.

  • Another question, is there a way I can keep the machine online, so that I can close the terminal and not have to wait 10 minutes for ./cheribuild run-fvp-morello-purecap to run when I want to boot up again?

    screen/tmux? There's nothing special about cheribuild in this regard.

    Although you may be interested in our Morello QEMU port. cheribuild morello-qemu will build it, and cheribuild run-morello-purecap (no -fvp) will run it using the same disk image. It is still a bit experimental but it has been able to run CheriBSD for a couple of months now. There are still some bugs in more obscure parts of the ISA, but rarely anything that should affect userspace development, and given it is many times faster than the FVP you may find it a much more pleasant environment to work in. We are using it in our day-to-day work rather than Arm's FVP, and it is being used in our CI environment for testing CheriBSD changes.