Hello Experts!
I am writing a kernel module which makes some low level experiments, and I've noticed that the HW prefetcher is interrupting them. I want to disable it while the experiemnts run. I am running on Pixel 8 (Shiba), with CalyxOS installed with the follwoing uname -a: Linux localhost 5.15.137-android14-11-gbc062a78e195-ab12057991 #1 SMP PREEMPT Mon Jul 8 12:34:46 UTC 2024 aarch64 Toybox.
I read that I can disable it using the PF_DIS bit in the IMP_CPUECTLR_EL1, CPU Extended Control Register. I have never written to this register before. As I understand, in order to modify it I need to write to S3_0_C15_C1_4, which TBH I am not sure what is this name.
Whenever I write into it I get HYP_PANIC. I've checked the pseudo code from arm_cortex_x3_core_trm_101593_0102_07_en (could be also found here: https://developer.arm.com/documentation/101593/0101/AArch64-System-registers/Generic-system-control-register-summary/IMP-CPUECTLR-EL1--CPU-Extended-Control-Register?lang=en):
if PSTATE.EL == EL0 then UNDEFINED; elsif PSTATE.EL == EL1 then if EL2Enabled() && HCR_EL2.TIDCP == '1' then AArch64.SystemAccessTrap(EL2, 0x18); elsif EL2Enabled() && ACTLR_EL2.ECTLREN == '0' then AArch64.SystemAccessTrap(EL2, 0x18); elsif ACTLR_EL3.ECTLREN == '0' then AArch64.SystemAccessTrap(EL3, 0x18); else IMP_CPUECTLR_EL1 = X[t]; elsif PSTATE.EL == EL2 then if EL2Enabled() && ACTLR_EL2.ECTLREN == '0' then AArch64.SystemAccessTrap(EL2, 0x18); elsif ACTLR_EL3.ECTLREN == '0' then AArch64.SystemAccessTrap(EL3, 0x18); else IMP_CPUECTLR_EL1 = X[t]; elsif PSTATE.EL == EL3 then IMP_CPUECTLR_EL1 = X[t];
And as it could be seen above, I think it gets into the seconf 'if' and traps.
As I've read, it is caused because I am running in EL1 (Kernel) and not in EL2 (Hypervisor).
As I've read, it may somehow be related to pKVM hypervisor but unfortunetly I have no clue what does this actualy mean, and how can I fix it.
Logs from the Panics:
RAMDUMP_MSG.txt: reset message: KP: HYP panic: UUID: a76a9e17-965d-a740-a259-36d6491b17a0 last kernel version: 5.15.137-android14-11-gb60f0c2a8346-ab12076200 aosp kernel version: 5.15.137-android14-11-gbc062a78e195-ab12057991 build: google/shiba/shiba:14/AP2A.240905.003/12231197:user/release-keys RST_STAT: 0x80 - SYSTEM_SWRESET_SYSTEM GSA_RESET_STATUS: 0x10 - GSA_INTERMEDIATE_RESET Reboot reason: 0xbaba - Kernel PANIC Reboot mode: 0x0 - Normal Boot
and also:
[ 194.018818][ T3944] kvm [3944]: nVHE hyp BUG at: [<ffffffc009484cb0>] __kvm_nvhe_handle_trap+0xec/0x570! [ 194.018875][ T3944] kvm [3944]: Cannot dump pKVM nVHE stacktrace: !CONFIG_PROTECTED_NVHE_STACKTRACE [ 194.018878][ T3944] kvm [3944]: Hyp Offset: 0xffffff5008000000 [ 194.018884][ T3944] Kernel panic - not syncing: HYP panic: [ 194.018884][ T3944] PS:824003c9 PC:0000007001484cb0 ESR:f2000800 [ 194.018884][ T3944] FAR:515bb77cfbbe6b3d HPFAR:80000000adc0ff10 PAR:0f0000ce88c868b2 [ 194.018884][ T3944] VCPU:0000000000000000 [ 194.018889][ T3944] CPU: 7 PID: 3944 Comm: insmod Tainted: G S W OE 5.15.137-android14-11-gbc062a78e195-ab12057991 #1 [ 194.018895][ T3944] Hardware name: ZUMA SHIBA MP based on ZUMA (DT) [ 194.018899][ T3944] Call trace: [ 194.018902][ T3944] dump_backtrace+0xf8/0x1e8 [ 194.018912][ T3944] dump_stack_lvl+0x74/0xa4 [ 194.018934][ T3944] panic+0x180/0x430 [ 194.018942][ T3944] nvhe_hyp_panic_handler+0x140/0x160 [ 194.018954][ T3944] initialize_executor+0x78/0x148 [revizor_executor] [ 194.018975][ T3944] init_module+0x128/0xf7c [revizor_executor] [ 194.018985][ T3944] do_one_initcall+0xe8/0x338 [ 194.018991][ T3944] do_init_module+0x58/0x774 [ 194.018999][ T3944] load_module+0x1708/0x18fc [ 194.019003][ T3944] __arm64_sys_finit_module+0x178/0x18c [ 194.019008][ T3944] invoke_syscall+0x58/0x138 [ 194.019015][ T3944] el0_svc_common+0xb0/0xe8 [ 194.019019][ T3944] do_el0_svc+0x20/0x7c [ 194.019023][ T3944] el0_svc+0x28/0x9c [ 194.019049][ T3944] el0t_64_sync_handler+0x7c/0xe4 [ 194.019054][ T3944] el0t_64_sync+0x1b4/0x1b8
What can I do to somehow disable the prefetcher? BTW, If this possible to be HW implementation independent it would be even better :)
Is it possible to do it without recompilation?
There is an old question from from 6 years ago ( Disable data prefetching in a Cortex-A53 running Android ) which somehow made this work, but (A) it is very HW specific and (B) Unfortunatly I don't understand how he made it actually work.
Any help would be great!
Thank you all.