Starship was formed to use our extensive experience developing software for games & simulations and apply it to market segments that hadn’t yet been exposed to the transformative power of digital technology.
One of the markets we quickly identified was cooking: people are obsessed with celebrity chefs, cooking shows and recipe books, but they haven’t really taken advantage of the latest software features when transferring across to the app space - most recipe apps are, at best, a glorified PDF, and cooking games are rendered in a cartoon style. We were sure we could do a lot better than that!
Our primary technical worry, though, was the steep “uncanny valley” drop off. Just like when looking at human faces, the brain has evolved to be able to spot fake/bad food a mile off. If we wanted realism, it wouldn’t be computationally cheap. On the plus side, our initial UX experiments immediately found the fun: on tablet devices where we can be tactile and the size format closes matches the pan & plates we wanted to represent.
CyberCook's objective then was to achieve a realistic 3D simulation of how food looks and behaves, all while running on tablet (and mobile) hardware at 30fps.
In general, food has pretty similar requirements to human skin to look realistic, which meant we could use the plentiful skin shading research as a reference. As we found, translucency, subsurface scattering, energy conserving BRDFs, IBL reflections, linear lighting and depth of field are all required to render believable food, while being quite a tall order for the mobile GPUs at the time of development.
A physically based solution would have been the ideal choice, but we couldn't afford it on mobile. we opted instead for a physically inspired solution, carefully testing which features made the most difference to the end results and letting go of the energy conserving requirement outside of the main BRDF.
The base intuition we took from our preliminary research on the task was that Depth of Field and Linear lighting are essential to the perception of skin and organic materials as realistic. The typical gamma falloff is ingrained in our mind, after a couple of decades of 3D games, and it screams "I'm fake".
Starship graphics programmer Claudia Doppioslash (doppioslash) had the tricky job of picking the right techniques that would enable the artists to create the assets they needed:
"Linear lighting is not available for mobile in Unity, so we had to implement it from scratch. While it's a simple matter of making sure all your textures and colours inputs are wrapped in a pow(<colour>, 2.2) and having a full screen effect that sets it back to gamma at the end, it's also fiddly, takes up computing power and it was confusing for the artists. At that time full screen effects were not supported on Unity's scene view, so they had to edit the scene in gamma, while taking care to preview their work in a game camera with the full screen effect on.
Depth of Field, while being an expensive full screen effect we paid for in performance, really helped the perception of the image as having been taken from a real camera belonging to a tv cooking show or a professional food photographer. Our artists researched extensively the look of food photography to apply it to CyberCook."
"The choice of BRDFs was at the heart of achieving realism. We know the Blinn-Phong look all too well and have associated it with video games. The moment you see it your brain is subconsciously reminded that you are looking at a game. It's especially bad for organic matter and it wasn't much good as a simulation of the food being coated in oil, either.
We relegated it to be used for non-organic, non-metallic materials in the kitchen environment. The main specular BRDF used for food, metal, and wood is an energy conserving Cook-Torrance with a GGX distribution. It can give the soft quality necessary to believe that something is organic and also the smooth one necessary for the metal objects, and is, all in all, a very flexible BRDF."
"We also used the anisotropic Ashikhmin-Shirley BRDF for the oil coating specular on the food and for the most important metal objects, such as the pan and the hob. The food oil coating was a ripe ground for experiments, as it's not a problem many games have. Ashikhmin-Shirley is expensive but the results are miles away from the alternatives.
Having different BRDFs made it hard to achieve IBL coherent with the lighting of the scene. We used the technique in the Black Ops 2014 Siggraph presentation [1], but it was meant to be used with Blinn-Phong as a distribution. Nevertheless it worked well enough for our requirements.
Last but not least, we used a number of diffuse BRDFs: Phong, of course, then Oren-Nayar was used for some vegetables which didn't look good enough with Phong. Our implementation of Subsurface Scattering follows Penner's pre-integrated skin shading 2011 Siggraph talk [2].
We were forced by the complexity of how prawns look in real life to implement a very approximated translucency effect inspired by the DICE 2011 GDC talk [3]."
[1] Getting More Physical in Call of Duty: Black Ops II
[2] Eric Penner's Pre-Integrated Skin Shading
[3] Approximating Translucency for a Fast, Cheap and Convincing Subsurface Scattering Look
The food looks so realistic I'm actually hungry for a Chinese sweet and sour curry now!