I am now writing a hyper-visor based on arm-v8 aarch64. and now Linux kernel can boot successfully on this hyper-visor. and I test the code on DS5. But there is a cache issue.
The issue is:
after Linux kernel modify the content of translation table (such as called io_remap to map some device memory), it need to flush the related cache, otherwise when accessing the memory space which mapped by io_remap will cause an EL1 data abort.
I found in the DS5 cache view, when executed in EL1， only the content of translation table are cached, but other EL1 memory are not. so question is:
1: what the EL1 cache state when EL2 is implemented?
2: why it is need do cache flush after change the translation table ? what am I missing ?
the issue seems that CPU think the cache is disabled in EL1, but the translation table content is cached, so when do translation walk, cpu will not fetch content from the cache.
Thanks
更正下：
后来经过我debug发现，由于我HCR_EL2.ID和HCR_EL2.CD都设置成了1，导致stage 2 translation 的cache 被disable 了。虽然stage2 translation 的cache被disable了，但从cache内容来看，整个EL1的页表还是被cache了（就是linux的页表），所以就会出现下面这个状况，导致需要手动刷cache:
Hi Lemin,
I got your point. May I know a little details about armv8 platform, is it a juno board? or other develop board. The mvisor is a type-1 hypervisor?
How many vcpus created for the guest VM? Have you tried to create 1 cpu for guest VM.
Is the guest OS ioremap to normal memory for virtual IO devices?
Could you check the translation table in cache for this issue, is it a translation entry guest OS just created?
Don't care about the English, if you ask in English, more arm expert can give you suggestion :)
Hi Steven
May I know a little details about armv8 platform, is it a juno board? or other develop board
---------- the platform I tested is ARM FVP, and i test it on DS5
The mvisor is a type-1 hypervisor?
------------ Yes it is a type-1 hepervisor
------------- I created 4 vcpus, and do not try 1vcpu for guest VM. But this issue is also exist when only 1 vcpu is online
------------ it is remap to the real HW device. but Whether it is mapping normal memory space or device memory space, all need to flush the cache, since the translation table is cached in the cache
----------- Yes, I have checked the content is the guest OS just created. and only the translation table of the guest VM is cached, all other memory are not if I set HCR_EL2.CD to 1.
So, why the translation table will cached even if I disable the stage 2 translation cache, does it is a normal behavior ?
lemin said:So, why the translation table will cached even if I disable the stage 2 translation cache, does it is a normal behavior ?
HCR_EL2.CD
For the Non-secure EL1&0 translation regime, forces all stage 2 translations for data accesses and translation table walks to Normal memory to be Non-cacheable.
Non-cacheable only for translation table walks to Normal memory.
From KVM implementation for aarch64, HCR_EL2.ID, HCR_EL2.CD set to 0.