Sunday, January 22, 2012

KF Simple Benchmark v0.5

(Android Market link)

So, I was reading reviews of some Android phones late last night and noticed that a lot of places are still testing OpenGL performance with NeoCore.  It's old enough that it basically runs at 60fps on every device out there, and is actually OpenGL 1.1 based besides.  It seems like it's kind of past its prime.

As we have a lot of OpenGL using wallpapers, I figured I'd take a stab at putting together a simple benchmark program using some of the same materials.  This flew together pretty fast once I started going with it, it's definitely not really finished yet but it works and delivers logical results.  I figured I'd get it out there and see if anybody's interested in using it.

So, you'll see there are three tests it runs, then averages the results.  The logic I went with was to try to stress something different with each test.  Here's what each one does:

The Bamboo test leans very heavily on geometry and vertex shader performance.  It renders 75 models, each of which is about 2000 vertices.  The vertex shader uses sin/cos to bend the stalks based on the vertex's Z height, then calculates a texture coordinate based on a light vector.  They're rendered front to back with depth buffering using an interleaved model, with a very simple pixel shader that does one texture sample.

The Wavescape test leans very heavily on fillrate and texture samples.  The geometry here amounts to maybe a thousand vertices, but the post-processing requires two levels of full-screen render to texture, with the light ray trails doing 12 texture samples to create the smooth blur effect.  Screen resolution vs GPU speed makes a huge difference here.

Finally, the Galactic Core test stresses CPU and the driver's handling of draw calls.  Primarily, it's a particle system consisting of 1024 particles orbiting a center point.  Each particle spins based on its distance from the center, and each particle is its own small model, resulting in a very large number of draw calls and uniform updates.

I had a good time putting this together, and will hopefully have some time to nurture it into a more proper product given some time.  Meanwhile, hopefully some folks will get use out of it.

I tested it on several devices, here are some of the averages:

Galaxy Nexus: 13.59 & 93 frames
Incredible 2: 26.138 & 183 frames
Motorola Xoom: 20.492 & 196 frames
Motorola Droid3: 23.97 & 163 frames

I feel a bit sorry for the Nexus, that screen resolution is more than its GPU can really handle filling.  It does rate the highest of any of these on the Bamboo test though.


  1. Hi Jeremy, forgive the newbie question but it relates to this post.

    I'm having trouble getting a full screen Gaussian blur to work with any sort of performance above 10fps on my Galaxy Nexus. It's a naive 2-pass (render-to-texture) 9 step gaussian over the entire screen.

    Is my problem likely to simply be the low fill rate on the Galaxy Nexus and the 16 dependent texture reads required per pixel or am I misunderstanding the effects of a low fill rate?

    I've been racking my brain trying to find the problem (assuming it was my fault) but would you expect this to be hitting the Galaxy Nexus performance limits?



    1. I'd very much expect that to be hitting the performance limits of the Galaxy Nexus, yes. It's an utterly fill-rate starved device that can barely manage 2 samples per pixel at a high frame-rate in my experience.

      Both it and the current crop of tablets have this problem -- the GPU isn't any faster than a standard phone but the screen has three times as many pixels. For almost all our live wallpapers a device like the Xoom or Galaxy Nexus is unquestionably the slowest supported platform. I'm optimistic that once we start seeing some Tegra3 based tablets they'll finally at least be on par with a mid-range phone like an Incredible2.

      There's not a lot of getting around this unfortunately. It really sucks because customers are constantly told how fast these devices are, and they are quick in most regards, but when it comes to filling the screen they're the bottom of the totem pole.

    2. Oh, also... the reason WaveScape runs well in its wallpaper form is for a few reasons. Firstly, it does fewer samples when doing the blurring for the rays (8, I think). Secondly, the rays are rendered at a quarter or less screen-res, while the benchmark version here is full-res. Thirdly, we artificially limit the framerate on the wallpaper to 20fps and skip updating if we get a request sooner than that, so it gives the home screen more CPU time to itself.

      If you compare the benchmark version of Wavescape with the wallpaper version you can really see the fidelity difference in comparison.

    3. Thanks so much. I thought I was struggling in my first step into GLSL for a while but turns out I wasn't making any huge mistakes other than expecting too much of the Galaxy Nexus.

      I managed to borrow a Samsung Galaxy S2 today, and the same code ran at 60fps (artificially limited).

      I guess I'll be doing some dirty hacks like downsampling :) I'll probably be easily able to reduce the 9 steps without too much visible degradation also.

      Keep the technically oriented posts coming!