This discussion has been locked.
You can no longer post new replies to this discussion. If you have a question you can start a new discussion

This very simple shader fails to link on only some Mali GPUs

Some of our shaders fail to 'link' on some specific devices with Mali GPUs. These shaders have not failed on any non Mali GPU in at least last 13 months. On analyzing our debug logs we found that all of our shader failures in last few months are from Mali GPU devices only. Some of these devices report Android 6.0 as their OS, so they are not running very old software.

Some of the GPUs which fail: Mali-T624, Mali-T628, Mali-T720, Mali-T760 and Mali-T860.

Some of the Android devices which fail: Aqua 4G+, SM-J120A, SM-J200M, SM-T800, CP8298_I00, CRR-L09, E5506, F3216, F670S, P1ma40, SM-G903W, SM-G920P, SM-J120A, SM-J200GU, SM-J200M, SM-J700F, SM-J700M, TAB 2 A8-50F, Z820, SM-A510F

It is interesting that shaders work fine on some devices with these same GPUs and fail to link on other devices with same GPUs, I guess it could be due to different GPU driver versions etc...

 

I have tried the offline compiler using different parameters, but that succeeds as expected. Compiling succeeds on actual devices too, it is the linking which fails.

Here is the simplest of our shaders that fails to link. The shader compiler and linker logs are empty.

Here is the shader source, including the 'additions' done to shader code by cocos2dx game engine, We don't really use those extra uniforms added by cocos as we are not attaching these shaders to a cocos2dx object but its part of the final strings sent to GPU to be compiled and linked.

Vertex Shader:

precision highp float;
 precision highp int;
uniform mat4 CC_PMatrix;
uniform mat4 CC_MVMatrix;
uniform mat4 CC_MVPMatrix;
uniform mat3 CC_NormalMatrix;
uniform vec4 CC_Time;
uniform vec4 CC_SinTime;
uniform vec4 CC_CosTime;
uniform vec4 CC_Random01;
uniform sampler2D CC_Texture0;
uniform sampler2D CC_Texture1;
uniform sampler2D CC_Texture2;
uniform sampler2D CC_Texture3;
//CC INCLUDES END


        attribute vec4 a_position;
        attribute vec2 a_texCoord;
        varying vec2 v_texCoord;
        void main() {
            gl_Position = a_position;
            v_texCoord = a_texCoord;
        }

Fragment Shader:

precision mediump float;
 precision mediump int;
uniform mat4 CC_PMatrix;
uniform mat4 CC_MVMatrix;
uniform mat4 CC_MVPMatrix;
uniform mat3 CC_NormalMatrix;
uniform vec4 CC_Time;
uniform vec4 CC_SinTime;
uniform vec4 CC_CosTime;
uniform vec4 CC_Random01;
uniform sampler2D CC_Texture0;
uniform sampler2D CC_Texture1;
uniform sampler2D CC_Texture2;
uniform sampler2D CC_Texture3;
//CC INCLUDES END


        varying vec2 v_texCoord;
        uniform sampler2D texture0;
        void main() {
            vec4 color = texture2D(texture0, v_texCoord);
            color.rgb *= color.w;
            color.a = 1.0;
            gl_FragColor = color;
        }

 

We have already spent more than a week digging into why this could be, but we don't have a device which would fail. Only debug logs from an app published on play store. We can really use some help here.

Parents
  • Well, using the informations provided in the glLinkProgram manual page, I'll split the linking failure problems in two categories :

    Syntax errors

    • A vertex shader and a fragment shader are not both present in the program object.
    • The main function is missing for the vertex shader or the fragment shader.
    • A varying variable actually used in the fragment shader is not declared in the same way (or is not declared at all) in the vertex shader.
    • A reference to a function or variable name is unresolved.
    • A shared global is declared with two different types or two different initial values.
    • One or more of the attached shader objects has not been successfully compiled (via glCompileShader) or loaded with a pre-compiled shader binary (via glShaderBinary).

    GPU resources related errors

    • The number of active attribute variables supported by the implementation has been exceeded.
    • The storage limit for uniform variables has been exceeded.
    • The number of active uniform variables supported by the implementation has been exceeded.
    • Binding a generic attribute matrix caused some rows of the matrix to fall outside the allowed maximum of GL_MAX_VERTEX_ATTRIBS.
    • Not enough contiguous vertex attribute slots could be found to bind attribute matrices.

    The Syntax related problems should provoke a failure, no matter what.

    The GPU resources related can be figured out by running the OpenGL program on a development phone, or on a computer, and analyse it using an OpenGL tracing program like Mali Graphics Debugger or Apitrace . The former might provide more Mali GPU specific informations about the current GPU resources usage.

    You can then add informations about the current GPU GL_MAX_??? limits in the logs when glLinkProgram fails, using glGet, and compare them with the values of working systems.

    Also, don't forget to use glGetString and query the GL_RENDERER, and use Android methods to get the current Android version used, along with the current ROM name. It might be due to the current Android ROM abusing the GPU for some special desktop effects, or misconfiguring the GPU driver and leaving it with almost no system memory.

Reply
  • Well, using the informations provided in the glLinkProgram manual page, I'll split the linking failure problems in two categories :

    Syntax errors

    • A vertex shader and a fragment shader are not both present in the program object.
    • The main function is missing for the vertex shader or the fragment shader.
    • A varying variable actually used in the fragment shader is not declared in the same way (or is not declared at all) in the vertex shader.
    • A reference to a function or variable name is unresolved.
    • A shared global is declared with two different types or two different initial values.
    • One or more of the attached shader objects has not been successfully compiled (via glCompileShader) or loaded with a pre-compiled shader binary (via glShaderBinary).

    GPU resources related errors

    • The number of active attribute variables supported by the implementation has been exceeded.
    • The storage limit for uniform variables has been exceeded.
    • The number of active uniform variables supported by the implementation has been exceeded.
    • Binding a generic attribute matrix caused some rows of the matrix to fall outside the allowed maximum of GL_MAX_VERTEX_ATTRIBS.
    • Not enough contiguous vertex attribute slots could be found to bind attribute matrices.

    The Syntax related problems should provoke a failure, no matter what.

    The GPU resources related can be figured out by running the OpenGL program on a development phone, or on a computer, and analyse it using an OpenGL tracing program like Mali Graphics Debugger or Apitrace . The former might provide more Mali GPU specific informations about the current GPU resources usage.

    You can then add informations about the current GPU GL_MAX_??? limits in the logs when glLinkProgram fails, using glGet, and compare them with the values of working systems.

    Also, don't forget to use glGetString and query the GL_RENDERER, and use Android methods to get the current Android version used, along with the current ROM name. It might be due to the current Android ROM abusing the GPU for some special desktop effects, or misconfiguring the GPU driver and leaving it with almost no system memory.

Children
  • Can you please review the shader code to see what might be wrong?

    We have been through these basics and can't see anything that is out of place. The shader above uses all resources way below the minimums in standard - 1 varying, 1 uniform, 1 texture read, etc. And it is never failing on any non-Mali device.

    I think that the reason it happens on only some devices with Mali-T720 and not on other devices with same Mali-T720 is that it might be some old driver bug that is fixed in newer driver versions. But I am just guessing.

    It will also really help if someone can try it on one of the above devices.
  • I will try on a Mali device. However, trying it on my PC, using MESA drivers (Gallium / ATI Radeon 7950) yielded the following error :

    Problem was : error: declarations for uniform `CC_PMatrix` have mismatching precision qualifiers

    Removing the duplicated projections uniforms solved the problem. Another way to solve the problem will be to use the same precision qualifiers.
  • Aha, that makes sense.

    Those uniforms are unused so they should have been optimized away. Probably thats why it doesn't effect other GPUs.

    Drivers on these devices with Mali GPUs probably don't optimize them away (or might be doing these precision checks before the optimization step).
  • We will prepare a new build and try to roll it out for beta testing. I will report back my results, but it can take a few days for the build to roll out and reach new users with target devices (all old users would have uninstalled the app due to this bug).

    On a side note, can you please point me to some reference page on how I can setup and run my tests using Mesa drivers? Some way to only use them for compiling/linking shaders like Mali's offline compiler?
  • I've prepared a little example using a corrected versions of your shaders : https://github.com/Miouyouyou/Buggy-Shaders .

    Grab a Linux distribution Live CD (Ubuntu, Arch Linux, Fedora, ...) and :

    • Run it
    • Install git, the "build essentials" (GCC, make, ...) and CMake
    • Install the mesa development libraries
    • Clone the Git repository

    Mesa is the default OpenGL implementation on Linux distributions.

    Then do something like:

    cd /tmp
    mkdir build_dir
    cd build_dir
    cmake /path/to/project/sources -DMYY_X11=ON
    make

    You might be able to do all of that within Vmware or VirtualBox VM. Try to install the "mesa-utils" and "mesa-utils-extra" and run "eglgears" to be sure that the OpenGL ES 3D rendering works fine.

     

    Note that you can also try the Android version by just cloning the repository, go to the apk folder and try to launch ./gradlew installDebug . You'll need CMake though.

     

    I have not tried to install Mesa 3D on Windows, however there are instructions here : http://mesa3d.org/README.WIN32

  • If you can build Mesa on your current system, you might be able to use their standalone "glsl-compiler". More informations on this page : www.mesa3d.org/shading.html