How to fill the object shaped holes in your scene

Ordinarily the blog posts I write relate directly to whatever I have been working on recently. This one was inspired, or rather, specifically requested from my friend and colleague in Developer Relations, chrisvarns. He called me over the other day and said “You know this thing people keep doing?”

I replied that yes, I did know, as we'd seen it several times whilst setting demos up at exhibitions.

To which he responded “Can you teach them not to?”

Interlude:

Did you know that male sparrows have markings on their chests to indicate their ranking in the literal pecking order of sparrow society? They look like little downward pointing triangles and they can only be seen with a special camera receptive to ultraviolet light.

Similarly, a standard video camera can be easily confused by shining a bright infrared light at it. Invisible to the human eye, infrared will still activate the CCD sensor array and make the picture come out with nothing but a bright white glare.

These two scenarios are real-life situations which show just how little we really pay attention to things that fall outside of our visual spectrum, until some kind of technology is introduced to make it visible.

Kind of like if you have an application showing graphics perfectly normally on a tablet, and then you plug it into a HDMI port and portions of the image on the TV screen are just black. Sometimes whole objects, sometimes rectangular portions, sometimes curious nonsensical shapes formed by the overlapping of scene elements. Since we first spotted it we’ve also seen the problem arise on certain display driver hardware for the device screen, now that we know what to look for.

Developer Relations is often contacted by developers and hardware manufacturers with images that look like these, thinking that there is something wrong with either the chip or device. In fact it is neither. You may never have encountered this problem - I had been working in graphics for years before I saw it - and it can arise from the strangest of circumstances. If you code a lot of graphical applications I’d recommend you continue reading this blog post, because one day you may just suffer this problem, and the solution is simple – if you know what the cause is. Using a tool like the ARM® Mali™ Graphics Debugger you can take a frame buffer direct from the GPU driver and look at the things going on in the alpha channel using image editing software. This is how we first spotted the problem.

People often think of alpha channels as a useful part of a texture and ignore them at all other times but the actual display frame buffer can have an alpha channel too. Most of the time you never see it because the display driver simply ignores it, but it is still there and not EVERY display driver ignores it.

So if you ever find yourself seeing parts of your image inexplicably blacked out or darkened, you should take a look at the following areas in your code:

1. Blend Modes

When doing alpha blending you probably use the line:

glBlendFunc(GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA);

Which is fair enough, except that it can have an interesting effect on the resultant alpha in the frame buffer. If your source is semi-transparent, say 1.0, 1.0 ,1.0, 0.5 (RGBA) and your destination is RGBA 0.0, 0.0, 0.0, 1.0 the resultant blend will be 0.5, 0.5, 0.5, 0.75.

That 0.75 will be your undoing.

Most of the time, unless you’re reading back from the tile to do clever post processing or including GL_DST_ALPHA in your blend function, you don’t want the alpha channel of the frame buffer to ever be anything but a fully opaque 1.0.

Therefore you’d be better served by the line:

glBlendFuncSeparate(GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA, GL_ZERO, GL_ONE);

Which sets up standard alpha blending on RGB but maintains the destination alpha, which should be cleared to 1.0.

2. Shader Fixes

So when you’re done alpha blending, or if you never did it, chances are your blend mode has been set to:

glBlendFunc(GL_ONE, GL_ZERO);

So whatever you output from your shader is going directly onto the frame buffer. At this point, anything you output to your framebuffer’s alpha channel other than 1.0 will pollute it with unwanted transparency. Some people make the mistake of writing shaders which calculate the RGB values perfectly and then pad it out into an RGBA value thusly:

gl_FragColor = vec4(rgbCol,1.0);

This is totally fine, so long as the alpha at the end is 1.0.

If, however, you’ve accidentally written:

gl_FragColor = vec4(rgbCol,0.0);

On the majority of display drivers will look no different, until you hook your tablet up to a monitor and the HDMI encoder decides to use the alpha channel for its own nefarious purposes (usually just blending alpha to black).

3. The Thermonuclear Option

There comes a time in the development life cycle of every project where the code is, frankly, getting a bit big. This is usually near the end, and may be around the point where you first start testing platform compatibility to discover that, connected to HDMI, some devices black out bits of the scene. If you’ve been paying attention, you will probably know by now that this is an alpha problem, but if your code base is particularly massive you’ll want to check that alpha really is to blame before picking your code apart.

What I’m about to suggest will work but may have a minor performance impact, as you’re essentially fixing the alpha in post processing. The fix is this:

glClearColor(0.0f, 0.0f, 0.0f, 1.0f);

glColorMask(GL_FALSE, GL_FALSE, GL_FALSE, GL_TRUE);

glClear(GL_COLOR_BUFFER_BIT);

Adding these three lines at the end of the frame will leave the RGB of the scene intact but completely replace the alpha channel with full opacity. This will immediately fix all alpha problems.  Obviously you’ll need to turn the mask and clear colours back again before you draw the next frame. If this works but you don’t want that extra draw step (it’s not a massive hit, but it’s not completely free in performance terms), get rid of this step and fix them properly.

Finally, just in case I don't get to post again before it, I look forward to seeing you at GDC for more graphics techniques and tips.