We are running a survey to help us improve the experience for all of our members. If you see the survey appear, please take the time to tell us about your experience if you can.
Hello Experts!
I am writing a kernel module which makes some low level experiments, and I've noticed that the HW prefetcher is interrupting them. I want to disable it while the experiemnts run. I am running on Pixel 8 (Shiba), with CalyxOS installed with the follwoing uname -a: Linux localhost 5.15.137-android14-11-gbc062a78e195-ab12057991 #1 SMP PREEMPT Mon Jul 8 12:34:46 UTC 2024 aarch64 Toybox.
I read that I can disable it using the PF_DIS bit in the IMP_CPUECTLR_EL1, CPU Extended Control Register. I have never written to this register before. As I understand, in order to modify it I need to write to S3_0_C15_C1_4, which TBH I am not sure what is this name.
Whenever I write into it I get HYP_PANIC. I've checked the pseudo code from arm_cortex_x3_core_trm_101593_0102_07_en (could be also found here: https://developer.arm.com/documentation/101593/0101/AArch64-System-registers/Generic-system-control-register-summary/IMP-CPUECTLR-EL1--CPU-Extended-Control-Register?lang=en):
if PSTATE.EL == EL0 then UNDEFINED; elsif PSTATE.EL == EL1 then if EL2Enabled() && HCR_EL2.TIDCP == '1' then AArch64.SystemAccessTrap(EL2, 0x18); elsif EL2Enabled() && ACTLR_EL2.ECTLREN == '0' then AArch64.SystemAccessTrap(EL2, 0x18); elsif ACTLR_EL3.ECTLREN == '0' then AArch64.SystemAccessTrap(EL3, 0x18); else IMP_CPUECTLR_EL1 = X[t]; elsif PSTATE.EL == EL2 then if EL2Enabled() && ACTLR_EL2.ECTLREN == '0' then AArch64.SystemAccessTrap(EL2, 0x18); elsif ACTLR_EL3.ECTLREN == '0' then AArch64.SystemAccessTrap(EL3, 0x18); else IMP_CPUECTLR_EL1 = X[t]; elsif PSTATE.EL == EL3 then IMP_CPUECTLR_EL1 = X[t];
And as it could be seen above, I think it gets into the seconf 'if' and traps.
As I've read, it is caused because I am running in EL1 (Kernel) and not in EL2 (Hypervisor).
As I've read, it may somehow be related to pKVM hypervisor but unfortunetly I have no clue what does this actualy mean, and how can I fix it.
Logs from the Panics:
RAMDUMP_MSG.txt: reset message: KP: HYP panic: UUID: a76a9e17-965d-a740-a259-36d6491b17a0 last kernel version: 5.15.137-android14-11-gb60f0c2a8346-ab12076200 aosp kernel version: 5.15.137-android14-11-gbc062a78e195-ab12057991 build: google/shiba/shiba:14/AP2A.240905.003/12231197:user/release-keys RST_STAT: 0x80 - SYSTEM_SWRESET_SYSTEM GSA_RESET_STATUS: 0x10 - GSA_INTERMEDIATE_RESET Reboot reason: 0xbaba - Kernel PANIC Reboot mode: 0x0 - Normal Boot
and also:
[ 194.018818][ T3944] kvm [3944]: nVHE hyp BUG at: [<ffffffc009484cb0>] __kvm_nvhe_handle_trap+0xec/0x570! [ 194.018875][ T3944] kvm [3944]: Cannot dump pKVM nVHE stacktrace: !CONFIG_PROTECTED_NVHE_STACKTRACE [ 194.018878][ T3944] kvm [3944]: Hyp Offset: 0xffffff5008000000 [ 194.018884][ T3944] Kernel panic - not syncing: HYP panic: [ 194.018884][ T3944] PS:824003c9 PC:0000007001484cb0 ESR:f2000800 [ 194.018884][ T3944] FAR:515bb77cfbbe6b3d HPFAR:80000000adc0ff10 PAR:0f0000ce88c868b2 [ 194.018884][ T3944] VCPU:0000000000000000 [ 194.018889][ T3944] CPU: 7 PID: 3944 Comm: insmod Tainted: G S W OE 5.15.137-android14-11-gbc062a78e195-ab12057991 #1 [ 194.018895][ T3944] Hardware name: ZUMA SHIBA MP based on ZUMA (DT) [ 194.018899][ T3944] Call trace: [ 194.018902][ T3944] dump_backtrace+0xf8/0x1e8 [ 194.018912][ T3944] dump_stack_lvl+0x74/0xa4 [ 194.018934][ T3944] panic+0x180/0x430 [ 194.018942][ T3944] nvhe_hyp_panic_handler+0x140/0x160 [ 194.018954][ T3944] initialize_executor+0x78/0x148 [revizor_executor] [ 194.018975][ T3944] init_module+0x128/0xf7c [revizor_executor] [ 194.018985][ T3944] do_one_initcall+0xe8/0x338 [ 194.018991][ T3944] do_init_module+0x58/0x774 [ 194.018999][ T3944] load_module+0x1708/0x18fc [ 194.019003][ T3944] __arm64_sys_finit_module+0x178/0x18c [ 194.019008][ T3944] invoke_syscall+0x58/0x138 [ 194.019015][ T3944] el0_svc_common+0xb0/0xe8 [ 194.019019][ T3944] do_el0_svc+0x20/0x7c [ 194.019023][ T3944] el0_svc+0x28/0x9c [ 194.019049][ T3944] el0t_64_sync_handler+0x7c/0xe4 [ 194.019054][ T3944] el0t_64_sync+0x1b4/0x1b8
What can I do to somehow disable the prefetcher? BTW, If this possible to be HW implementation independent it would be even better :)
Is it possible to do it without recompilation?
There is an old question from from 6 years ago ( Disable data prefetching in a Cortex-A53 running Android ) which somehow made this work, but (A) it is very HW specific and (B) Unfortunatly I don't understand how he made it actually work.
Any help would be great!
Thank you all.