About mali-g76 MP12 GPU and micro-architecture

sooyeon said:

1) Do they context switch between warps to hide memory access latency when the kernel has memory operations ??

Massively multithreaded architectures like GPUs shader cores context switch all the time.

BUT, don't think of this like the heavy-weight context switching like a CPU, that's simply not how this type of hardware works. All the contexts for all threads are live in registers all the time, and the GPU can pick any active warp any clock cycle. 

sooyeon said:

and as I know, they have 8 lanes (8-wide warp) per execution engine. how they can run 768 threads simultaneously?? (with context switching? or  are they more lanes?...)

The first point is don't confuse issue width (new operations issued per cycle) with execution capacity (total number of in-flight operations). Most operations will have a latency of multiple cycles, so you can have have more than 1 warp live in a singe hardware unit at the same time due to pipeline length. Some of the fixed function units (texture unit, for example) have very long pipelines. 

The other point is that the purpose of having this may threads is to hide the latency of memory fetches. A significant portion of the 768 are likely to be idle waiting for data from memory; the others keep the hardware busy.

sooyeon said:

3) If they can run 768 threads simultaneously and the work-group size is only 24, do they run 24 warps(8-wide warps*3 engine) with same work_group id per core?

If your work group size is 24 threads, you'll get 3 warps (24 / 8) per work group, and 32 (768 / 24) different workgroups running in each core at the same time. 

sooyeon said:

if work-group size is 8, remain lanes(24 lanes-8 = 16 lanes) don't work?

If the work-group size is 8, then you get 1 warp per work group, and 96 different workgroups running in each core at the same time. The hardware is three independent 8-wide units, so an 8-wide work group is fine, you'll just issue three different workgroups in parallel.

What you want to avoid is work groups narrower than 8. A 4-wide workgroup would only use half of the 8-wide hardware, so keep workgroups larger than 8 (and on newer hardware, larger than 16).

HTH, 

Pete

sooyeon said:

1) Do they context switch between warps to hide memory access latency when the kernel has memory operations ??

Massively multithreaded architectures like GPUs shader cores context switch all the time.

BUT, don't think of this like the heavy-weight context switching like a CPU, that's simply not how this type of hardware works. All the contexts for all threads are live in registers all the time, and the GPU can pick any active warp any clock cycle. 

sooyeon said:

and as I know, they have 8 lanes (8-wide warp) per execution engine. how they can run 768 threads simultaneously?? (with context switching? or  are they more lanes?...)

The first point is don't confuse issue width (new operations issued per cycle) with execution capacity (total number of in-flight operations). Most operations will have a latency of multiple cycles, so you can have have more than 1 warp live in a singe hardware unit at the same time due to pipeline length. Some of the fixed function units (texture unit, for example) have very long pipelines. 

The other point is that the purpose of having this may threads is to hide the latency of memory fetches. A significant portion of the 768 are likely to be idle waiting for data from memory; the others keep the hardware busy.

sooyeon said:

3) If they can run 768 threads simultaneously and the work-group size is only 24, do they run 24 warps(8-wide warps*3 engine) with same work_group id per core?

If your work group size is 24 threads, you'll get 3 warps (24 / 8) per work group, and 32 (768 / 24) different workgroups running in each core at the same time. 

sooyeon said:

if work-group size is 8, remain lanes(24 lanes-8 = 16 lanes) don't work?

If the work-group size is 8, then you get 1 warp per work group, and 96 different workgroups running in each core at the same time. The hardware is three independent 8-wide units, so an 8-wide work group is fine, you'll just issue three different workgroups in parallel.

What you want to avoid is work groups narrower than 8. A 4-wide workgroup would only use half of the 8-wide hardware, so keep workgroups larger than 8 (and on newer hardware, larger than 16).

HTH, 

Pete

sooyeon said:

1) how many work registers per engine? or per core? (in datasheet, max 64 work register per one thread, and if I use 64 register, mali runs 384 threads, and if I use 32 register, mali runs 768 threads. is it right if I use more than 32 registers, performance is down?)

There are enough registers in the hardware for 32 * 768 to be allocated concurrently. If you use more than 32 then the allocation size for a thread doubles to 64, so we can only allocate half as many threads.

This _might_ impact performance, but it depends on what you are doing and the ratio of the workloads though the different pipes, so it's not a simple "yes / no" answer I'm afraid. In general if you have a high percentage of texturing in a shader you need more threads, because the texture unit is relatively high latency, but for vertex and compute workloads you might not see much slowdown at all with 64 registers. 

sooyeon said:

2) there are 8 lanes (8 threads) per engine. so, 8 * 64register = 512 registers per the engine?

8192 (768 / (3 * 32)) per engine, 24567 in per shader core (768 / 32). 

sooyeon said:

3) what is unfirom register?(it is increased when I used buffer)

A read-only register for storing uniforms and other draw-time constants that the shader might need

sooyeon said:

4) how many uniform register per engine? or per core?

All threads in a draw share the uniform register file. It's a fixed size, so there isn't any step change as you see with work registers. The size is not publicly documented, but in general I'd say it's large enough you don't have to worry. 

sooyeon said:

5) in micro architecture, quad manager exist. in mali-g76 have 8 lanes. quad manager pack the threads in 4 or 8???

In the general case all GPUs fragment shade in blocks of 2x2 pixels, due to the need for derivatives. Hardware will pack out warps as needed to fill the available width. 

HTH,
Pete