Mali offline compiler doesn't seem to handle #include

Okay so I added the following to the top of our .vert file.  Typically shaderc adds these, so it would be nice if malioc did the same for the Vulkan case.

malioc -c Mali-G76 --vulkan --name main DebugMesh.vert

#version 310 es <- had to add

#extension GL_GOOGLE_include_directive: require <- had to add

#include "Shader.h"

void main()

{

   code

}

But now I just get the following.  Shader.h is in the same folder as the shader I'm compiling.

ERROR: 0:4: X0000: '#include' : Could not process include directive for header name: Shader.h. <- why ?

ERROR: 0ssing entry point: X0000: Each stage requires one entry point. <- I've got --name above, it's just stock GLSL void main()

Hi! 

Yes, it's not very practical to keep complex shaders within a single file. However, normally include directives are resolved either at runtime, before passing the shader source to the compiler, or when using an offline compiler tool to produce a binary, such as SPIR-V.

You can pass multiple files to Mali Offline Compiler at once, but they will be just concatenated in the order you specify:

https://developer.arm.com/documentation/101863/0703/Using-Mali-Offline-Compiler/Compiling-OpenGL-ES-shaders?lang=en

Could you try to replace #include "Shader.h" with the actual contents of the file and see if it helps?

Seems like we can hand off spv files to the compiler, and those already have everything concatenated.  I need to be able to process several hundred shader variants with minimal effort, so modifying the sources isn't possible.   Now I can automate it off the spv files.  I couldn't tell what formats were inputs vs. outputs in the help.

Now at least we can get a little analysis on the shaders.

 --spirv

     Compile a SPIR-V binary module.

Shouldn't the compiler tell me how many varyings a shader uses if that's critical to the 180MB limit on Mali driver?  I don't see anything related to that in the output.  Or is that "Work registers"?

Varying variant

---------------

Work registers: 30

Uniform registers: 102

Stack spilling: false

16-bit arithmetic: 0%

Hi Alec,

Thanks for the feedback. 

Alec said:

ERROR: 0X0000:  SPIR-V require version 310 or higher. <- fixed below

The 0X* errors are directly reported from the Khronos glslangValidator application, which we use for converting GLSL into SPIR-V.

The extension support needed for include directives is part of glslangValidator, which is why we don't report it directly, as it's not a SPIR-V extension that our compiler would see. I'll see what is needed to get this working (suspect it might be looking in the current working directory, not the source directory, so we might need to add a way to specify and forward include paths to the child process). 

We could probably automatically inject the #version specifier if it's missing, and add a warning that we did so. As Pavel mentioned we just concatenate N input files, so you could just create a single utility file containing the version and include preamble you need, and specify this first on the command line.

Alec said:

ERROR: 0ssing entry point: X0000: Each stage requires one entry point <- printing problem

Thanks, looks like we're not hiding some of the original log from the glslangValidator, so we're getting to processes writing out to the console.  Will raise a bug for this one. 

Alec said:

ERROR: 0ssing entry point: X0000: Each stage requires one entry point.  <- how to specify this ?

Hmm, shaders with a default "main" should work. Do the non-spirv samples we ship in the samples/vulkan/ directory work for you? If it's possible can you share the problem shader (drop me an email, as I assume you can't share here)?

Alec said:

Shouldn't the compiler tell me how many varyings a shader uses if that's critical to the 180MB limit on Mali driver?  I don't see anything related to that in the output.

At the moment we don't emit this. Agree it would be useful. Will raise an enhancement with the team.

Cheers, 
Pete

We use shaderc for spriv compilation, and it performs much better than glslangValidator.  It injects the version and extension depending on the compile parameters, so we can't have #version at the top of each file.  Tried to switch our codebase over to glslangValidator since that supported Vulkan version number, and hit numerous problems with that compiler. 

One of the biggest issues was that error formatting isn't clickable coming out of glslangValidator.  So now I maintain a shader project that can generate all shader for all platforms and clickthrough to errors/warnings.

It's fine that I can pass the .spv files.   The downside is we often transpile to Metal, and spirv-opt strips all commentary from the files.  That despite that it has embedded shaders with comments and OpLine directives in the compile source that it could reference to grab the comments.

A bit more investigation here on the Mali Offline Compiler 7.4 (will ship at the end of the month).

Using includes does work for glslang (with the #extension added), but only the current working directory is on the include search path. Any includes relative to the source shader location will only work if the source shader is in the current working directory when the offline compiler is invoked.

HTH,
Pete

We do have all our sources/headers in a single folder.  But for now we're using spv for the analysis since it's already spliced together into a single file.  Thanks for the followup.