• Locked Locked
  • Replies 14 replies
  • Subscribers 136 subscribers
  • Views 16323 views
  • Users 0 members are here
Options
Related
How was your experience today?
This discussion has been locked.
You can no longer post new replies to this discussion. If you have a question you can start a new discussion

Understanding Mali GPU Hardware Counters

maasa

Hi ,

I have read your blog on Mali GPU Hardware Counters. I have a few questions. 

The Mali Job Manager Cycles:GPU cycles counter gives the total amount of cycles, the GPU was active. If I execute a compute workload (not graphics), I should be able to predict the execution time of the kernel should be from Tripipe cycles counter.  There is always a differnce in value  between the Mali Job Manager Cycles:GPU cycles and Mali Core Cycles:Tripipe cycles. What does this extra cycles signify. I know that the values reported by streamline is average value across all the shader cores but still what does this extra cycles signify?

I also would like to know what exactly does the  Mali Core Cycles:Compute cycles and Mali Compute Threads:Compute cycles awaiting descriptors counters report ??.

This is because I ran a OpenCL benchamark with zero arithmetic instructions but still the values of Mali Core Cycles:Compute cycles and Mali Compute Threads:Compute cycles awaiting descriptors  are not zero while Mali Compute Threads:Compute tasks and Mali Compute Threads:Compute threads started were zero.

Also the tripipe cycles counter value should be equal to the maximum of cycles spent in Arithmetic/LS-pipeline/Texture pipeline but even when there are no texture and Arithmetic instructions, the value of Mali Core Cycles:Tripipe cycles is not the same as Mali Load/Store Pipe:LS instruction issues counter. Why this is happening? If I am executing only memory instructions, Mali Core Cycles:Tripipe cycles should be equal to Mali Load/Store Pipe cycles instead I see that  Mali Core Cycles:Compute cycles , Mali Compute Threads:Compute cycles awaiting descriptors and  Mali Core Cycles:Tripipe cycles have similar values??

 

It would be helpful if you can give some insights to these behaviours?

P.S. I am doing an academic project and i am modeling the performance of opencl kernel on Mali GPUs.

P.P.S.I am not an android developer looking at optimizations

Parents

  • maasa said:
    There is always a differnce in value  between the Mali Job Manager Cycles:GPU cycles and Mali Core Cycles:Tripipe cycles. What does this extra cycles signify.

    The Job Manager is responsible for turning a compute dispatch into smaller pieces of work which can be distributed over the shader cores in the system, so it has to load some control structures from main memory and ensure memory coherency with the CPU's view of the world before starting any work and ensure any results are memory coherent with the CPU's view of the world at the end of the dispatch.

    This Job Manager overhead is normally a small (< 10K cycles) fixed cost on any compute work submissions, and will be amortized and pipelined if you submit a compute queue containing multiple kernels to execute. 

    maasa said:
    I also would like to know what exactly does the  Mali Core Cycles:Compute cycles and Mali Compute Threads:Compute cycles awaiting descriptors counters report ??.

    Compute cycles increments any time that a shader core is processing any part of a compute job (or a non-fragment job for graphics). This includes any cycle where something is either in the fixed function thread setup unit, or in the shader core itself. Descriptors are the control-plane state for the work being submitted - so the other counter shows how long the front-end is waiting to load descriptors. Note, for a pipelined compute submission containing multiple kernels this pipelines, so a cycle spent waiting here doesn't necessarily mean a lost cycle of tripipe processing time in non-trivial applications.

    In general:

    "Job Manager: GPU Active Cycles" > "Shader Core: Compute Active Cycles" > "Shader Core: Tripipe Active Cycles"

    The tripipe is the heart of the system for running code, but there are a few thin layers of overhead which are incurred to determine how to drive it.

    There is some overhead to determine that we don't need to do anything at all (e.g. we don't spend hardware to optimize cases which don't occur in real world scenarios - sending zero sized jobs is an application issue really), which is why you see some Compute Active time even if you don't spawn any threads.

    Also the tripipe cycles counter value should be equal to the maximum of cycles spent in Arithmetic/LS-pipeline/Texture pipeline but even when there are no texture and Arithmetic instructions, the value of Mali Core Cycles:Tripipe cycles is not the same as Mali Load/Store Pipe:LS instruction issues counter

    maasa said:
    the value of Mali Core Cycles:Tripipe cycles is not the same as Mali Load/Store Pipe:LS instruction issues counter. Why this is happening?

    The architectural best case is one cycle per instruction for the load/store pipe, but that assumes cache hits for either reads or writes. Real-world performance will vary depending on how your addresses are spread across memory and how those addresses land temporally. If you are hitting external memory (e.g. L1 and L2 caches both miss) a lot then the performance issues may be beyond Mali - we don't control the external memory system linking the GPU to the DDR memory controller.

    Also note that a lot also depends on how big you workloads are - it sounds like you are running some small test workloads if you are running zero sized kernels. Remember GPUs are massively multithreaded and rely on running hundreds of threads concurrently in each shader core to ensure that the functional units stay busy, and the 1 cycle per pipe throughput assumes full thread occupancy for a reasonably large workload. If you are running very small test loads then it is likely that you are losing a lot of utilization to ramp up and ramp down time, and may simply lack enough threads to keep the hardware busy. Very rough rule of thumb - aim for compute kernels which are at least hundreds of thousands of work items.

    maasa said:
    Mali Core Cycles:Compute cycles , Mali Compute Threads:Compute cycles awaiting descriptors and  Mali Core Cycles:Tripipe cycles have similar values??

    Ideally Compute cycles == Tripipe cycles; this shows that the workload is mostly spending its time running in the shader core (e.g. there is minimal overhead from the fixed function thread setup units). I wouldn't worry too much about the waiting for descriptor counter; because of how it loads pipeline in parallel to the tripipe running it's not really showing you anything particularly useful.

    HTH, 

    Pete

Reply

  • maasa said:
    There is always a differnce in value  between the Mali Job Manager Cycles:GPU cycles and Mali Core Cycles:Tripipe cycles. What does this extra cycles signify.

    The Job Manager is responsible for turning a compute dispatch into smaller pieces of work which can be distributed over the shader cores in the system, so it has to load some control structures from main memory and ensure memory coherency with the CPU's view of the world before starting any work and ensure any results are memory coherent with the CPU's view of the world at the end of the dispatch.

    This Job Manager overhead is normally a small (< 10K cycles) fixed cost on any compute work submissions, and will be amortized and pipelined if you submit a compute queue containing multiple kernels to execute. 

    maasa said:
    I also would like to know what exactly does the  Mali Core Cycles:Compute cycles and Mali Compute Threads:Compute cycles awaiting descriptors counters report ??.

    Compute cycles increments any time that a shader core is processing any part of a compute job (or a non-fragment job for graphics). This includes any cycle where something is either in the fixed function thread setup unit, or in the shader core itself. Descriptors are the control-plane state for the work being submitted - so the other counter shows how long the front-end is waiting to load descriptors. Note, for a pipelined compute submission containing multiple kernels this pipelines, so a cycle spent waiting here doesn't necessarily mean a lost cycle of tripipe processing time in non-trivial applications.

    In general:

    "Job Manager: GPU Active Cycles" > "Shader Core: Compute Active Cycles" > "Shader Core: Tripipe Active Cycles"

    The tripipe is the heart of the system for running code, but there are a few thin layers of overhead which are incurred to determine how to drive it.

    There is some overhead to determine that we don't need to do anything at all (e.g. we don't spend hardware to optimize cases which don't occur in real world scenarios - sending zero sized jobs is an application issue really), which is why you see some Compute Active time even if you don't spawn any threads.

    Also the tripipe cycles counter value should be equal to the maximum of cycles spent in Arithmetic/LS-pipeline/Texture pipeline but even when there are no texture and Arithmetic instructions, the value of Mali Core Cycles:Tripipe cycles is not the same as Mali Load/Store Pipe:LS instruction issues counter

    maasa said:
    the value of Mali Core Cycles:Tripipe cycles is not the same as Mali Load/Store Pipe:LS instruction issues counter. Why this is happening?

    The architectural best case is one cycle per instruction for the load/store pipe, but that assumes cache hits for either reads or writes. Real-world performance will vary depending on how your addresses are spread across memory and how those addresses land temporally. If you are hitting external memory (e.g. L1 and L2 caches both miss) a lot then the performance issues may be beyond Mali - we don't control the external memory system linking the GPU to the DDR memory controller.

    Also note that a lot also depends on how big you workloads are - it sounds like you are running some small test workloads if you are running zero sized kernels. Remember GPUs are massively multithreaded and rely on running hundreds of threads concurrently in each shader core to ensure that the functional units stay busy, and the 1 cycle per pipe throughput assumes full thread occupancy for a reasonably large workload. If you are running very small test loads then it is likely that you are losing a lot of utilization to ramp up and ramp down time, and may simply lack enough threads to keep the hardware busy. Very rough rule of thumb - aim for compute kernels which are at least hundreds of thousands of work items.

    maasa said:
    Mali Core Cycles:Compute cycles , Mali Compute Threads:Compute cycles awaiting descriptors and  Mali Core Cycles:Tripipe cycles have similar values??

    Ideally Compute cycles == Tripipe cycles; this shows that the workload is mostly spending its time running in the shader core (e.g. there is minimal overhead from the fixed function thread setup units). I wouldn't worry too much about the waiting for descriptor counter; because of how it loads pipeline in parallel to the tripipe running it's not really showing you anything particularly useful.

    HTH, 

    Pete

Children