I'm trying to write a driver to DMA from the host memory to the FPGA device connected to the PCI bus. I'm facing a translation fault 0x10 thrown by the SMMU v3. (arm-smmu-v3.4 arm-smmu-v3.4.auto event 0x10 received)
After inspecting the registers and dumping event queue records I found out that it's a stage 1 translation fault.
The register and class details are below:
Event Queue records
Stream ID - 0x00008500
Substream ID - Unknown
PnU - Underprivileged access
S2 - 0 => Stage 1 translation fault
Class = TTD
TTBR0/1_EL1 registers T0SZ - 5'b01000 T1SZ - 5'b0100; TCR_EL1 - 0x34B5503510 => Translation table walks enabled for both stages (EPD0 and EPD1 = 0'b0)
The ARM SMMU v3 documentation says the F_TRANSLATION fault occurs when the address provided to a stage of translation failed the range check defined by TxSZ/SLx, the address was within a disabled TTBx, or a valid translation table descriptor was not found for the address. My DMA input addresses lie in the range of 0xF000000 - 0xFFFFFFF (which is within the range defined by TxSZ). Also, I ran the same application with a Infiniband Mellanox RDMA switch and it worked!! (The DMA address ranges for RDMA were within the range of 0xE000000 - 0xEF00000). This is really puzzling.
How do I check the address lies within a disabled TTBx or a valid translation table descriptor isn't found for the address? Is this a known issue that can be fixed by patches? What am I missing out on? Any help would be great!
Let me know if you need additional information.
Update -
I was able to fix the issue. There was a synchronization lapse between the IOMMU and the DMA mapping. There were no valid descriptors found for the mapped DMA addresses in the SMMU TTB0.
I used dma_alloc_coherent(SZ_2M) to get a buffer region and use IOMMU domain ops to map the IOVA to the SMMU.
int ret = iommu_domain->ops->map(domain, IOVA, size, phys_addr, PROT)
Now the SMMU can fetch the descriptors and translate the IOVA.
For some reason, dma_map_single(..) doesn't work with my current implementation. I have to investigate why a streaming DMA API doesn't work.