SpriteKit and Normal Maps

Since iOS 8 SpriteKit now has some rudimentary lighting capabilities which has been a feature that I wanted for quite a while in a 2D engine.  Below you'll find my thoughts and details related to performance and aesthetics.

First, there are many potential pitfalls that may destroy the frame rate of your application on devices that do not have an A7, or later, chip.  The issues I discuss below tend to only crop up on low end devices.  We will look at batching, the fragment shader, and potential optimizations.

Batching, as I have discovered, can be broken by having various sprites at various angles when a light shines upon them.  As far as I can tell, the issue is that the implementation uses a single uniform to describe the angle of rotation when converting normals from view space to world space.

The fragment shader is hefty.  On older devices, multiple lights can take a significant portion of the time used to render a frame.  My intuition tells me that these devices are having trouble with the branching and looping found within the shader.

As an optimization, one option is to have lights affect as few objects as possible through the light mask.  Another option, which is a bit more extreme, would be to have a custom fragment program that is fine tuned for the number of lights present in the scene.  If you need to go further, then you may find yourself outgrowing SpriteKit and using Unity or a custom engine which tessellates around the image to minimize the number of times the fragment shader is run.  And if you go that far, you might want to consider moving to a deferred pipe where lights are much less expensive, even if the frame rate would be passable at best on the low end devices.

If you are using batching (a folder with a .atlas extension), have a separate atlas for the colour and normal maps.  If you don't, I've noticed SpriteKit generates its own rather than use the provided ones.

Second, aesthetics plays a pivotal role in whether this capability should be used.  My comments are divided between the shadows and the lights.

Shadows are not as expensive as I would expect them to be.  Shadows can be enabled through a mask to ensure only the most important objects are affected.  If you have a series of images that share edges, like a series of blocks piled upon one another, then be careful if there is anti-aliasing in the image since the shadow extends the alpha portion of the image which may lead to weird lines that are undesired (a break in what should be a continuous flat shadow).  Also, be sure to test on device as the algorithm seems to be slightly different. (It might be using the bounding box rather than the silhouette)

In terms of looks, I found that having a few lights led to some interesting results with little effort.  Unfortunately, I believe much more could be done.  What if the light is behind a sprite illuminating but its edges?  At the moment it feels like it's slightly above the sprites thus illuminating them directly.  A light is like a 3D object in this 2D world, limiting it to two axis reduces the number of possible effects.

In the end, SpriteKit introduces a very rudimentary concept of lighting.  It is not sophisticated, but sufficient to add a little extra detail to an application.  On the down side, it is too easy to hit the limits of what the implementation can do and be forced to change technology.

Edited on February 27th, 2015:  edited the text for legibility.  It seems to have been written when I was exhausted, hence barely comprehensible.  Certain details I did not have offhand (how shadows behave on device versus on the Mac), and will fill that in later when I get the chance.  Doesn't deviate from the fact that testing on device is always a good idea.

I have written a quick set of thoughts when SpriteKit first came out.