Hi,
I've gotten most of my code working with the Chromebook development platform. Simple kernels run fine and show a nice performance speedup. However more complex kernels perform poorly.
As per the suggestions of some members of this forum, I've reorganized one of my OpenCL kernels to avoid using local memory and to use the float4 vectors. The routine I was optimizing was a simple covariance calculation for large data sets. Since this is essentially a modified matrix multiply, I used the provided sgemm.cl as an example. This resulted in a pretty substantial speed up in my code (about a factor of 2-3).
Since I can't guarantee that a client's system has a function OpenCL system, I have a standard CPU version of the same function. Running both of these on the chromebook I get the following performance figures for a data set:
Using local memory (i.e. optimized for nVidia): 24.5 seconds
Using vectors and no local memory (i.e. optimized for Mali): 9.26 seconds
Using CPUs with calls to BLAS (no OpenCL): 6.99 seconds
So here despite optimization effort, it is still quicker to use the CPU to do the processing.
Just checking to see if I did anything wrong, I checked the provided sgemm.cpp file. As a comparison, I ran the BLAS CPU version of sgemm (installed using apt-get) against the provided OpenCL example. For the default matrix size of (2048x2048), I get the following run times:
OpenCL sgemm: 84 seconds
CPU BLAS sgemm: 3 seconds
So the single core CPU version is abut 30 times faster than the OpenCL version!
I've checked an the output is the same for both cases.
I suspect that performance is being choked in the OpenCL code by the sheer number of loads from global memory. On the systems with Local memory, this can be cached so performance doesn't choke. I assume the CPU is caching data fairly well and speeding this up.
If this is indeed the case, performance will be bad on anything other than extremely simple kernels.
Am I making some sort of mistake here? Is there anything I can do to mitigate this? Essentially 90% of the processing time in most of my stuff is a matrix multiply.
Thanks for all of your help.
--Mike
Hi Mike,
Just a further update which you might find useful when writing your own kernels.
The SGEMM sample in the SDK doesn't set the workgroup size when queueing the kernel - it just passes NULL as the workgroup size parameter. We made this configurable and found a sweet spot with a workgroup size of 4 x 16 or 8 x 16. We believe this modifies the memory accesses sufficiently to reduce the cache misses... in our tests the SGEMM sample was then running in around 7.5s. It's certainly worth experimenting with this, particularly for kernels with many load/store operations.
Regards,
Tim
And a further update, we have managed to get the SGEMM kernel example down to ~2.5 seconds runtime by transposing the matrix, removing the 4 scalar loads and replacing them with a vector load, and changing the work-group size to 4x16. This significantly improves the cache problems which were crippling it before. In the future the SDK examples should be updated to include these changes.
So, faster AND lower power than the CPU
Hope this helps,
Chris
Thanks,
This is good news. My workflow is almost all matrix multiplies or matrix multiply-like operations.
I look forward to seeing the new SDK code.
Thanks!
I'm just changing this to a discussion rather than a question as I think there is no one right answer here. Please continue with the discussion if anything more comes up.