Hi,
I am trying to profile Mali GPU counters on Odroid XU3 using Streamline. I have downloaded DS-5 Ultimate Edition trial for this purpose and built and run the gator daemon on the board ( using the steps from https://github.com/ARM-software/gator )
1. I am having some trouble with the GATOR_WITH_MALI_SUPPORT=MALI_MIDGARD flags, make does not seem to recognize this option. Without it I am able to compile and run.
2. I am able to see the Cortex A15/A7 stats but no GPU counters are visible on Streamline. Is this related to the previous step?
3. Does DS-5 Community Edition not support Streamline? I initially used that and it said there was no valid license.
4. Is there a way to read/log the GPU counter data on the board itself without using a host PC app like Streamline? If yes, could you please point me to documentation to enable this?
Thanks,
Manoj
Hi Stephen,
I reverted the kernel and now I can see the Mali counters available from the userspace gator available on Streamline.
When I started a capture however, the target crashed (I had posted another post with dmesg error logs, but I guess I cannot put the log in the thread. I can attach a log if you need it.
I do remember reading something about the Odroid XU3 freezing on using Streamline. I found it on (https://community.arm.com/tools/f/discussions/5787/ds5-streamline-with-odroid-xu3). However this seems to be due to the gator.ko compilation issue. I should not be having this problem since I am using userspace daemon.
[ 324.140990] [c0] .rt_nr_running : 0 [ 324.140994] [c6] CPU PMU: CPU6 reading wrong counter -1 [ 324.141002] [c0] .rt_throttled : 0 [ 324.141006] [c5] CPU PMU: CPU5 reading wrong counter -1 [ 324.141013] [c7] CPU PMU: CPU7 reading wrong counter -1 [ 324.141020] [c5] CPU PMU: CPU5 reading wrong counter -1 [ 324.141026] [c6] CPU PMU: CPU6 reading wrong counter -1 [ 324.141033] [c0] .rt_time : 0.000000 [ 324.141038] [c5] CPU PMU: CPU5 reading wrong counter -1 [ 324.141045] [c7] CPU PMU: CPU7 reading wrong counter -1 [ 324.141051] [c6] CPU PMU: CPU6 reading wrong counter -1 [ 324.141060] [c0] .rt_runtime : 0.000000 [ 324.141065] [c0] [ 324.141065] rt_rq[7]:/ [ 324.141069] [c4] CPU PMU: CPU4 reading wrong counter -1 [ 324.141076] [c7] CPU PMU: CPU7 reading wrong counter -1 [ 324.141082] [c0] .rt_nr_running : 0 [ 324.141088] [c6] CPU PMU: CPU6 reading wrong counter -1 [ 324.141095] [c7] CPU PMU: CPU7 reading wrong counter -1 [ 324.141101] [c0] .rt_throttled : 0 [ 324.141108] [c7] CPU PMU: CPU7 reading wrong counter -1 [ 324.141116] [c0] .rt_time : 0.797500 [ 324.141123] [c0] .rt_runtime : 950.000000 [ 324.141133] [c0] [ 324.141133] runnable tasks: [ 324.141133] task PID tree-key switches prio exec-runtime sum-exec sum-sleep [ 324.141133] ---------------------------------------------------------------------------------------------------------- [ 324.141194] [c0] [ 324.141314] [c6] CPU6: stopping [ 324.141327] [c6] CPU: 6 PID: 304 Comm: systemd-journal Not tainted 3.10.96+ #22 [ 324.141349] [c6] Backtrace: [ 324.141370] [c6] [<c0012434>] (dump_backtrace+0x0/0x114) from [<c0012770>] (show_stack+0x20/0x24) [ 324.141415] [c6] r6:00000006 r5:00000000 r4:c0913384 r3:00000000 [ 324.141430] [c6] [<c0012750>] (show_stack+0x0/0x24) from [<c0617c48>] (dump_stack+0x20/0x28) [ 324.141442] [c6] [<c0617c28>] (dump_stack+0x0/0x28) from [<c0013ef8>] (handle_IPI+0x188/0x1bc) [ 324.141457] [c6] [<c0013d70>] (handle_IPI+0x0/0x1bc) from [<c0008584>] (gic_handle_irq+0x64/0x6c) [ 324.141516] [c6] r8:b6f552ac r7:f0002000 r6:dd2e5fb0 r5:c091f284 r4:f000200c [ 324.141516] r3:00001005 [ 324.141531] [c6] [<c0008520>] (gic_handle_irq+0x0/0x6c) from [<c061e6c0>] (__irq_usr+0x40/0x60) [ 324.141541] [c6] Exception stack(0xdd2e5fb0 to 0xdd2e5ff8) [ 324.141553] [c6] 5fa0: be9eac50 b6e35d15 00000000 b6ee057d [ 324.141567] [c6] 5fc0: b6f75238 17da790d 00000001 b6ed1000 b6f552ac b6f565b4 b6f565b4 be9eacee [ 324.141578] [c6] 5fe0: b6f68f04 be9eac40 b6e35f21 b6ee057c a00f0030 ffffffff [ 324.141621] [c6] r7:b6ed1000 r6:ffffffff r5:a00f0030 r4:b6ee057c [ 324.141639] [c6] Unable to handle kernel NULL pointer dereference at virtual address 00000000 [ 324.141650] [c6] pgd = dd384000 [ 324.141670] [c6] [00000000] *pgd=00000000 [ 324.141687] [c6] Internal error: Oops: 805 [#1] PREEMPT SMP ARM [ 324.141802] [c6] Modules linked in: binfmt_misc cfg80211 rfkill smsc95xx usbnet joydev ads7846 spidev ina231_sensor spi_s3c64xx i2c_s3c2410 fuse [ 324.141816] [c6] CPU: 6 PID: 304 Comm: systemd-journal Not tainted 3.10.96+ #22 [ 324.141827] [c6] task: dde035c0 ti: dd2e4000 task.ti: dd2e4000 [ 324.141839] [c6] PC is at exynos_ss_save_context+0x38/0x224 [ 324.141852] [c6] LR is at exynos_ss_save_context+0x20/0x224 [ 324.141865] [c6] pc : [<c00278a0>] lr : [<c0027888>] psr: 200f0193 [ 324.141865] sp : dd2e5f48 ip : dd2e5f48 fp : dd2e5f64 [ 324.141875] [c6] r10: b6f565b4 r9 : b6f565b4 r8 : dd2e5fb0 [ 324.141886] [c6] r7 : 00000005 r6 : c09116a0 r5 : c091f058 r4 : 600f0193 [ 324.141895] [c6] r3 : 00000000 r2 : 01528000 r1 : 10c5387d r0 : 00000006 [ 324.141906] [c6] Flags: nzCv IRQs off FIQs on Mode SVC_32 ISA ARM Segment user [ 324.141917] [c6] Control: 10c5387d Table: 5d38406a DAC: 00000015 [ 324.141925] [c6] CPU 2 / CLUSTER 0 [ 324.141935] [c6] DFSR: 00000805, ADFSR: 00000000, IFSR: 00000007, AIFSR: 00000000 [ 324.141944] [c6] CPUMERRSR: 00000000_00000000, L2MERRSR: 00000000_00000000 [ 324.141955] [c6] [ 324.141955] PC: 0xc0027820: [ 324.142045] [c6] 7820 c098ad34 c07e1c44 e1a0c00d e92dd818 e24cb004 e92d4000 e8bd4000 e3010c54 [ 324.142132] [c6] 7840 e30c11ec e34c007e e34c1062 e3a04000 eb17b49e e3a034f9 e5834010 ebffdbaa [ 324.142218] [c6] 7860 e1a00004 e89da818 e1a0c00d e92dd878 e24cb004 e92d4000 e8bd4000 e10f4000 [ 324.142302] [c6] 7880 f10c0080 eb095b80 e59f51f0 e59f61f0 e1a03006 e7952100 e7923003 ee111f10 [ 324.142387] [c6] 78a0 e5831000 ee121f10 e5831004 ee121f30 e5831008 ee121f50 e583100c ee131f10 [ 324.142472] [c6] 78c0 e5831010 ee151f10 e5831014 ee161f10 e5831018 ee151f30 e583101c ee161f50 [ 324.142554] [c6] 78e0 e5831020 ee1a1f12 e583102c ee1a1f32 e5831030 ee1d1f10 e5831034 ee1d1f30 [ 324.142638] [c6] 7900 e5831038 ee1d1f50 e583103c ee1d1f70 e5831040 ee1d1f90 e5831044 eb095b5a [ 324.142648] [c6] [ 324.142648] LR: 0xc0027808: [ 324.142734] [c6] 7808 e3a0300a e7c53006 e3a034f9 e5832014 e24bd028 e89daff0 c098ad34 c07e1c44 [ 324.142817] [c6] 7828 e1a0c00d e92dd818 e24cb004 e92d4000 e8bd4000 e3010c54 e30c11ec e34c007e [ 324.142899] [c6] 7848 e34c1062 e3a04000 eb17b49e e3a034f9 e5834010 ebffdbaa e1a00004 e89da818 [ 324.142987] [c6] 7868 e1a0c00d e92dd878 e24cb004 e92d4000 e8bd4000 e10f4000 f10c0080 eb095b80 [ 324.143069] [c6] 7888 e59f51f0 e59f61f0 e1a03006 e7952100 e7923003 ee111f10 e5831000 ee121f10 [ 324.143153] [c6] 78a8 e5831004 ee121f30 e5831008 ee121f50 e583100c ee131f10 e5831010 ee151f10 [ 324.143238] [c6] 78c8 e5831014 ee161f10 e5831018 ee151f30 e583101c ee161f50 e5831020 ee1a1f12 [ 324.143322] [c6] 78e8 e583102c ee1a1f32 e5831030 ee1d1f10 e5831034 ee1d1f30 e5831038 ee1d1f50 [ 324.143332] [c6] [ 324.143332] SP: 0xdd2e5ec8: [ 324.143418] [c6] 5ec8 c002f358 c0266b5b dd2e5ef0 00000008 dd2e5ef0 dd2e5f44 c00278a0 200f0193 [ 324.143502] [c6] 5ee8 ffffffff dd2e5f34 dd2e5f64 dd2e5f00 c061e498 c00083a4 00000006 10c5387d [ 324.143584] [c6] 5f08 01528000 00000000 600f0193 c091f058 c09116a0 00000005 dd2e5fb0 b6f565b4 [ 324.143669] [c6] 5f28 b6f565b4 dd2e5f64 dd2e5f48 dd2e5f48 c0027888 c00278a0 200f0193 ffffffff [ 324.143753] [c6] 5f48 00000000 c0913384 00000000 00000006 dd2e5f8c dd2e5f68 c0013efc c0027874 [ 324.143841] [c6] 5f68 00001005 f000200c c091f284 dd2e5fb0 f0002000 b6f552ac dd2e5fac dd2e5f90 [ 324.143924] [c6] 5f88 c0008584 c0013d7c b6ee057c a00f0030 ffffffff b6ed1000 00000000 dd2e5fb0 [ 324.144010] [c6] 5fa8 c061e6c0 c000852c be9eac50 b6e35d15 00000000 b6ee057d b6f75238 17da790d [ 324.144021] [c6] [ 324.144021] IP: 0xdd2e5ec8: [ 324.144106] [c6] 5ec8 c002f358 c0266b5b dd2e5ef0 00000008 dd2e5ef0 dd2e5f44 c00278a0 200f0193 [ 324.144192] [c6] 5ee8 ffffffff dd2e5f34 dd2e5f64 dd2e5f00 c061e498 c00083a4 00000006 10c5387d [ 324.144275] [c6] 5f08 01528000 00000000 600f0193 c091f058 c09116a0 00000005 dd2e5fb0 b6f565b4 [ 324.144356] [c6] 5f28 b6f565b4 dd2e5f64 dd2e5f48 dd2e5f48 c0027888 c00278a0 200f0193 ffffffff [ 324.144435] [c6] 5f48 00000000 c0913384 00000000 00000006 dd2e5f8c dd2e5f68 c0013efc c0027874 [ 324.144521] [c6] 5f68 00001005 f000200c c091f284 dd2e5fb0 f0002000 b6f552ac dd2e5fac dd2e5f90 [ 324.144601] [c6] 5f88 c0008584 c0013d7c b6ee057c a00f0030 ffffffff b6ed1000 00000000 dd2e5fb0 [ 324.144689] [c6] 5fa8 c061e6c0 c000852c be9eac50 b6e35d15 00000000 b6ee057d b6f75238 17da790d [ 324.144700] [c6] [ 324.144700] FP: 0xdd2e5ee4: [ 324.144779] [c6] 5ee4 200f0193 ffffffff dd2e5f34 dd2e5f64 dd2e5f00 c061e498 c00083a4 00000006 [ 324.144865] [c6] 5f04 10c5387d 01528000 00000000 600f0193 c091f058 c09116a0 00000005 dd2e5fb0 [ 324.144951] [c6] 5f24 b6f565b4 b6f565b4 dd2e5f64 dd2e5f48 dd2e5f48 c0027888 c00278a0 200f0193 [ 324.145039] [c6] 5f44 ffffffff 00000000 c0913384 00000000 00000006 dd2e5f8c dd2e5f68 c0013efc [ 324.145121] [c6] 5f64 c0027874 00001005 f000200c c091f284 dd2e5fb0 f0002000 b6f552ac dd2e5fac [ 324.145202] [c6] 5f84 dd2e5f90 c0008584 c0013d7c b6ee057c a00f0030 ffffffff b6ed1000 00000000 [ 324.145291] [c6] 5fa4 dd2e5fb0 c061e6c0 c000852c be9eac50 b6e35d15 00000000 b6ee057d b6f75238 [ 324.145379] [c6] 5fc4 17da790d 00000001 b6ed1000 b6f552ac b6f565b4 b6f565b4 be9eacee b6f68f04 [ 324.145389] [c6] [ 324.145389] R5: 0xc091efd8: [ 324.145469] [c6] efd8 c091efd8 c091efd8 00000000 00000000 00000000 00000000 00000000 00000000 [ 324.145555] [c6] eff8 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 324.145642] [c6] f018 000000c8 00000008 00000009 00009000 0000000f c1e4b100 c1e4b0c0 c1e4b080 [ 324.145724] [c6] f038 c1e03000 00000000 00000007 0000002c c1e4b000 c1e4b040 00000001 00001000 [ 324.145808] [c6] f058 014f2000 014fb000 01504000 0150d000 01516000 0151f000 01528000 01531000 [ 324.145895] [c6] f078 0004f200 00000001 0005f772 000be9ff 0000e44f 00002000 00000000 00000032 [ 324.145977] [c6] f098 0000fffa 00000001 df019c00 00000001 00000000 c061039c c0131324 c012eeac [ 324.146061] [c6] f0b8 c012c554 00000000 c012fad8 c012c538 c012c440 00000000 00000000 c012aea0 [ 324.146070] [c6] [ 324.146070] R6: 0xc0911620: [ 324.146151] [c6] 1620 00080000 00000001 0001f30a 00000004 00000000 00000000 00000001 000d1ea5 [ 324.146228] [c6] 1640 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 324.146310] [c6] 1660 00000000 887a999d 00000000 00000000 00000000 00000000 00000000 00000000 [ 324.146396] [c6] 1680 0000001c a6211578 00000000 00000000 559e7d8a 32652688 00000000 00000000 [ 324.146483] [c6] 16a0 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 324.146564] [c6] 16c0 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 324.146645] [c6] 16e0 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 324.146729] [c6] 1700 000081a4 00000744 559e7d88 559e7d8a 558348a1 00000000 00010000 00000008 [ 324.146739] [c6] [ 324.146739] R8: 0xdd2e5f30: [ 324.146825] [c6] 5f30 dd2e5f48 dd2e5f48 c0027888 c00278a0 200f0193 ffffffff 00000000 c0913384 [ 324.146909] [c6] 5f50 00000000 00000006 dd2e5f8c dd2e5f68 c0013efc c0027874 00001005 f000200c [ 324.146992] [c6] 5f70 c091f284 dd2e5fb0 f0002000 b6f552ac dd2e5fac dd2e5f90 c0008584 c0013d7c [ 324.147077] [c6] 5f90 b6ee057c a00f0030 ffffffff b6ed1000 00000000 dd2e5fb0 c061e6c0 c000852c [ 324.147160] [c6] 5fb0 be9eac50 b6e35d15 00000000 b6ee057d b6f75238 17da790d 00000001 b6ed1000 [ 324.147247] [c6] 5fd0 b6f552ac b6f565b4 b6f565b4 be9eacee b6f68f04 be9eac40 b6e35f21 b6ee057c [ 324.147335] [c6] 5ff0 a00f0030 ffffffff 00000000 00000000 dd2e6040 dc40fa80 dcbaa0a8 c092f72c [ 324.147423] [c6] 6010 dd2e6010 dd2e6010 00000000 dead4ead ffffffff ffffffff c0d8abc4 00000000 [ 324.147435] [c6] Process systemd-journal (pid: 304, stack limit = 0xdd2e4238) [ 324.147446] [c6] Stack: (0xdd2e5f48 to 0xdd2e6000) [ 324.147456] [c6] 5f40: 00000000 c0913384 00000000 00000006 dd2e5f8c dd2e5f68 [ 324.147468] [c6] 5f60: c0013efc c0027874 00001005 f000200c c091f284 dd2e5fb0 f0002000 b6f552ac [ 324.147478] [c6] 5f80: dd2e5fac dd2e5f90 c0008584 c0013d7c b6ee057c a00f0030 ffffffff b6ed1000 [ 324.147488] [c6] 5fa0: 00000000 dd2e5fb0 c061e6c0 c000852c be9eac50 b6e35d15 00000000 b6ee057d [ 324.147499] [c6] 5fc0: b6f75238 17da790d 00000001 b6ed1000 b6f552ac b6f565b4 b6f565b4 be9eacee [ 324.147511] [c6] 5fe0: b6f68f04 be9eac40 b6e35f21 b6ee057c a00f0030 ffffffff 00000000 00000000 [ 324.147532] [c6] Backtrace: [ 324.147548] [c6] [<c0027868>] (exynos_ss_save_context+0x0/0x224) from [<c0013efc>] (handle_IPI+0x18c/0x1bc) [ 324.147588] [c6] r6:00000006 r5:00000000 r4:c0913384 r3:00000000 [ 324.147599] [c6] [<c0013d70>] (handle_IPI+0x0/0x1bc) from [<c0008584>] (gic_handle_irq+0x64/0x6c) [ 324.147655] [c6] r8:b6f552ac r7:f0002000 r6:dd2e5fb0 r5:c091f284 r4:f000200c [ 324.147655] r3:00001005 [ 324.147668] [c6] [<c0008520>] (gic_handle_irq+0x0/0x6c) from [<c061e6c0>] (__irq_usr+0x40/0x60) [ 324.147680] [c6] Exception stack(0xdd2e5fb0 to 0xdd2e5ff8) [ 324.147692] [c6] 5fa0: be9eac50 b6e35d15 00000000 b6ee057d [ 324.147705] [c6] 5fc0: b6f75238 17da790d 00000001 b6ed1000 b6f552ac b6f565b4 b6f565b4 be9eacee [ 324.147717] [c6] 5fe0: b6f68f04 be9eac40 b6e35f21 b6ee057c a00f0030 ffffffff [ 324.147754] [c6] r7:b6ed1000 r6:ffffffff r5:a00f0030 r4:b6ee057c [ 324.147766] [c6] Code: e1a03006 e7952100 e7923003 ee111f10 (e5831000) [ 324.147789] [c6] ---[ end trace 052283e3befd3ff6 ]--- [ 326.843716] [c0] SMP: failed to stop secondary CPUs [ 335.098918] [c1] CPU PMU: CPU1 reading wrong counter -1 [ 335.104118] [c1] CPU PMU: CPU1 reading wrong counter -1 [ 335.109324] [c1] CPU PMU: CPU1 reading wrong counter -1 [ 335.114711] [c1] BUG: soft lockup - CPU#1 stuck for 27s! [watchdog/1:12] [ 335.121211] [c1] Modules linked in: binfmt_misc cfg80211 rfkill smsc95xx usbnet joydev ads7846 spidev ina231_sensor spi_s3c64xx i2c_s3c2410 fuse [ 335.134141] [c1] [ 335.136056] [c1] CPU: 1 PID: 12 Comm: watchdog/1 Tainted: G D 3.10.96+ #22 [ 335.143772] [c1] task: df2e4b40 ti: df2f4000 task.ti: df2f4000 [ 335.149589] [c1] PC is at _raw_spin_unlock_irq+0x34/0x5c [ 335.154878] [c1] LR is at _raw_spin_unlock_irq+0x30/0x5c [ 335.160170] [c1] pc : [<c061df3c>] lr : [<c061df38>] psr: 60000013 [ 335.160170] sp : df2f5de8 ip : df2f5de8 fp : df2f5dfc [ 335.172057] [c1] r10: de26e8d0 r9 : c091f058 r8 : df2f4000 [ 335.177700] [c1] r7 : 00000000 r6 : ddada040 r5 : c1e0f280 r4 : c1e0f280 [ 335.184639] [c1] r3 : 000012d4 r2 : 00000001 r1 : 000004a8 r0 : c1e0f280 [ 335.191582] [c1] Flags: nZCv IRQs on FIQs on Mode SVC_32 ISA ARM Segment kernel [ 335.199301] [c1] Control: 10c5387d Table: 5c6f806a DAC: 00000015 [ 335.205462] [c1] CPU 1 / CLUSTER 1 [ 335.208847] [c1] DFSR: 00000817, ADFSR: 00000000, IFSR: 00000007, AIFSR: 00000000 [ 335.216309] [c1]