Has anyone tried using KVM with the Juno board? It doesn't look like the bootloader starts the cores in hypervisor mode, so it's not clear to me whether the current system software will support a hypervisor at the moment, even with the appropriate kernel options selected. Are there any plans to support it?
The Trusted firmware does drop you into UEFI at EL2. So no modification to the boot sequence is required to support virtualization. You may need to do some work to enable KVM, but I believe that much of the basic support is already upstream. (I haven't tried this myself on Juno, but I know various colleagues have successfully booted KVM with an AArch64 kernel).
jwalters wrote:
Has anyone tried using KVM with the Juno board?
[Sudeep] Yes you can run KVM on JUNO atleast on Cortex-A57
It doesn't look like the bootloader starts the cores in hypervisor mode
[Sudeep] Not sure what you mean by this, IIUC UEFI starts Linux in EL2(at-least in the release I am using)
so it's not clear to me whether the current system software will support a hypervisor at the moment,
[Sudeep] Surely it does
even with the appropriate kernel options selected.
[Sudeep] AFAIK, most of the required options are enabled in the upstream kernel(v3.17-rc*), you can pick defconfig from there for KVM
Are there any plans to support it?
[Sudeep] It's already supported.
If you run and get into troubles, post the log messages which will help us to guide you.
You can refer [1] but just use [2] kernel tree to build kvmtool
Regards,
Sudeep
[1] https://github.com/penberg/linux-kvm/blob/master/tools/kvm/README
[2] https://git.kernel.org/cgit/linux/kernel/git/will/kvmtool.git/log/?h=kvmtool/arm
Sorry for the late follow-up. It looks to me like KVM is trying to start up, but that perhaps the A53s aren't supported. I see in dmesg the following:
[ 0.542916] kvm [1]: Error, CPU 0 not supported!
I see from the most recent firmware release that the trusted firmware supports "nominating any CPU as the primary CPU without recompiling the firmware." Would it help to make CPU 4 (in my case, the first A57) the primary CPU? I don't see any documentation for how I would do that. Any suggestions?
Yes A53 is not yet added in the kvmtool. I should have been explicit about this in
my earlier reply. I just mentioned that I have tested on A57, sorry for that.
Instead of taking troubles to make CPU4 boot cpu, how about using taskset to
affine the task to A57. I use it in that way, something like:
$ taskset 0x30 ./lkvm run <arg_list_you_are_using_now>
In my case it appears that KVM isn't initializing at all. I'm missing /dev/kvm (which I assumed was related to the dmesg output I posted earlier). In your experiments, do you have the /dev/kvm device? I haven't tried kvmtool because I assumed that nothing would work without the /dev/kvm device.
It is not initializing because you are probably using an ancient kernel (pre 3.16), which doesn't recognize A53 as a valid CPU for virtualization. Please use a recent enough kernel.
As Sudeep mentioned above, you will also need an updated userspace (or use the taskset workaround).
Thanks,
M.
With some help from sudeep.holla, I also have KVM running on Juno in R&D, and am currently running some experiments with it. Happy to help from a first-timer's perspective if you're stuck.
Hi Aniudipi,
If you don't mind sharing what you had to do to get this running, I'd appreciate it. I had to step away from this for a bit, hence the slowness in my reply, but I just pulled the most recent mainline kernel and wasn't able to get it booting on my Juno board. This is kernel version 3.18-rc3, so it's not from Linaro or ARM. Can you share what kernel version you're using and/or what changes were needed to get things going?
I'm successfully running ARM's 3.15.0-rc8, but as the other posters imply, that's apparently too old to support the A53 cores.
JP
Hi JP,
You can try the Juno support patches posted by Liviu on ALKML[1] on top of v3.18-rc2
[1] https://lkml.org/lkml/2014/10/31/372
I believe that I'm very nearly there, but the KVM tool is segfaulting. As suggested, I'm running 3.18.0-rc2 with the patches. The kernel builds fine and I see that /dev/kvm is exposed. I separately pulled down the kvm-tool and built it for my kernel. I run it as follows:
taskset -c 4 ./lkvm run --kernel Image -d disk_oe64.img -m 512 --console virtio --params "earlyprintk=smh console=hvc0 root=/dev/vda init=/bin/sh"
Unfortunately, I immediately see a kernel oops. The full output is pasted here:
Unable to handle kernel NULL pointer dereference at virtual address 00000000 pg - Pastebin.com
I pulled down the guest kernel binary and file system from one of the Virtual Open Systems guides:
guide to set up a KVM development environment on 64-bit ARMv8 processors
Any thoughts?
Thanks!
I have no idea as what kernel you are running. So my suggestion would
be to run the same host kernel as guest and check first.
It could be issue with your guest kernel.