I'm finding that ETC1 textures (.pkm) are taking a rather large amount of data up on my storage device. What options are there for further reducing the file size, such as compression with a third party like zlib, anyone know how to do that? Is there any other techniques to reduce the file size without compromising quality?
Depending on the device you're targeting you may be able to step up from ETC into ASTC, which has more options for better compression and higher visual fidelity.
If you're stuck to ETC for legacy reasons zlib is an option, but you would need to link the library into your package and call it manually when loading textures. ETC is a lossy compression but it decompresses in hardware, whereas zlib is a lossless compression which you would need to spend CPU time decoding when the files are loaded at application start up (or during if textures are loaded on the fly). The way zlib works is by dictionary based redundancy, and ETC is designed for blocks of pixels to be independent so you would get best results if your image contained areas where multiple blocks look pretty much the same.
You haven't said what platform you're working on. If it's android based the assets contained in the APK are compressed anyway, and may be getting unpacked into storage at load time. Depending on how much control you have over your asset pipeline (Assuming you're coding it yourself and not using a 3rd party library or an engine like Unity or Unreal) you could unpack assets one at a time, load them into the GPU, and then delete the unpacked files.
If none of these techniques are right for you, you could also look into your actual usage of textures. For example, do you need them to have such a high resolution? examining the screen space filled by these textures, you may find that they contain more information than is needed. Without seeing the application or the assets I can't really advise any further.
Hi stacysmith apologies for not providing enough information. I'm using a third party sdk called Cocos2D-x that is c++ based and compiles to various different platforms, the renderer it uses is OpenGL. The platforms I'm targeting are Android, iOS and Windows Phone, but when I refer to ETC1 I'm talking about the Android build version specifically. The engine doesn't abstract too much away from the developer and it's perfectly feasible to control the pipeline.
Sorry if this is a silly question but is ASTC the same as ATITC (ATC) format? as described here <supports-gl-texture> | Android DevelopersI chose ETC1 mainly because of it's universal compatibility with devices, I guess that is the same thing as "legacy reasons".
I'm fairly sure the library for decompressing zlib is already included in Cocos2D-x, but I'm not sure how to compress them in the first place, does it have command line tools or do you write a program to do it. My textures are loaded during a loading screen before each level then unloaded at the end of the level.
I didn't know the assets are compressed by the apk, that's really interesting. Does that mean it would be a pointless exercise to compress them with zlib only to be further compressed by the .apk. Can they be compressed twice? Compressed textures, being compressed and then compressed again lol. Sorry I'm not too knowledgable about the whole texture compression.
I'm not sure what you mean by delete the unpacked files, you mean delete the unpacked zlib textures which are stored in the ram after they are loaded to the gpu? I'm only concerned with the size of the .apk right now not the performance.
I will have a few sets of assets that scale according to screen size, (sd, hd, 2xhd) etc.
I have several very large backgrounds with tileable assets that a repeated, packed up in to sprite atlases, we're talking at least 20+ (2048x2048) textures.
Let me know what you think,
It's really good to get an insight from a graphics expert.
ASTC is a new cross platform standard, described here ASTC Texture Compression - OpenGL.org
There are also a couple of blog posts about it from myself:ASTC does it and ASTC Does It - Part II: How To Use It
It's fairly new so older devices might not be compatible, that said the window of what you can reasonably be expected to support is always moving forward.
zlib compression is the same compression used when you create a compressed folder on you PC, also known as zip. Without getting into the details of how compression works, compressing files which are already compressed has a certain degree of diminishing returns, each time you edge closer to entropy (a fancy word for the minimum possible size something can be compressed to without losing data). The compression of APKs isn't the most exhaustive compression available so you may get minor returns although it depends on the files themselves, which will already be nearer to entropy than a raw RGB file due to the texture compression.
During my time on the Demo team we would extract assets from the APK onto the SD storage of the device the first time the application was run, then loaded the decompressed textures into the GPU (the files were still ETC compressed, but they were no longer compressed in the APK). I'm was assuming this was similar to your setup such that the assets were taking up too much space decompressed, and so deleting the SD cache after loading could help. Since the size of the APK is your concern that wouldn't help much.
If you're sure you need all the textures and they're all appropriately sized in their atlases (which I can't really comment on, I can just assume you're being honest with yourself as to how much is needed), the only real course of action is to have the textures stored elsewhere, such as hosted on a webserver contacted by the application on the first run to download them into the SD storage.
-Stacy
It is worth pointing out that this latter method of separating the 'assets' from the apk is a very common thing done with many games that are large in size. Originally because of the limitation of APK size for the Google Play Store, but is now a standard practise amongst all games that require large amounts of assets.
Google even allow you to separate them and store them on their servers, these files are usually referred to as 'obb' files. I believe you get 2 as a maximum, each being no larger than 2GB in size. If your game needs more however, you will need to have your own server infrastructure in place that your game can connect to and download.
A further benefit of having your own server, is to allow you to separate the assets by device and/or performance of the device. For example regarding textures, you can therefore have ETC1 texture pack, as well as an ASTC texture pack, and upon first launching the game, it can detect if ASTC is available, and if so, download that pack, falling back to ETC1 if that isn't the case. This means you don't require everyone to download all of your different compressed textures, when most will not be used for their device, further reducing the overall size of the fully installed game.
For further information on these techniques, please google e.g: APK Expansion Files | Android Developers
I hope you find this useful.
Kind Regards,
Michael McGeagh