Benchmarking with your heart

[Johnny Rook]

All over the Net forums dedicated to GPU cards, GPU benchmarking and related subjects, I've seen lots of users disappointed with their 3DMark, Aquamark -and all other "marks" for that matter- scores. Sometimes they are really in dispair, cursing and damning the day they bought their high-end cards.
I think people overvalues benchmarking and what it stands for.
We like to be in control of everything in our life and our hardware is no exception. But, what we generally forget is that our hardware is controled by software - the drivers. The drivers, like everything man-made is faulty and we reached a point in time that software struggles to keep-up with hardware. It seems that the "driver" is drunk. Therefore, is no surprise that high-end cards apperar not to deliver their full power and in truth, they don't. Not because of the cards itselfs but, because of those pieces of code that are in control.
When we open a benchmark application like 3DMark06, we are passing the control of our rigs to its code but, we don't know how far that code works as expected with cards for which it wasn't designed for (not to mention DirectX, OS, CPU, etc). All in all, benchmarking is very subjective so, why trust in pieces of code? I rather trust my own feelings!
Having said this, and within the highly subjective area of feelings, the new cards can nevertheless, deliver an incredible gaming experience. Forget the bench results! Open a game and let your mind, your eyes, your ears and your heart do the benchmarkings!
I've passed the last week doing just that.
I've "benchmarked" NFS Underground, FarCry 2, CODWAW, Assassin's Creed, GTA IV, FIFA09, Crisys Warhead and GRID.
Feelings out, if we can somehow measure the system performance during a game is by measuring the Frames Per Second (FPS) your system can render. Say the experts that under 30 FPS we get a poor gaming experience (games barely playable or not playable at all); 30-45 the game is playable; 45-60 FPS are equal to a good gaming experience, from 60 to 100 FPS, a very good gaming experience and >100 FPS, well... heaven's the limit or you have a very old game.
I used my system, both GPU and CPU @ factory specs. No OC'ing.
I've pumped-up each game definitions to the maximum my 22" allows (1920x1080) and let the games be in control of AA, V-Sync and Ansiotropic Filtering. As far as ATI CCC 3D options are concerned, I've disabled AAA and enabled Trilple buffering (when V-Sync enable by the game).
Then I used a little application called FRAPS.

The results:
NFS Undercover: 64-73 FPS
FarCry 2: 47-61 FPS
GTA IV: 40-60 FPS
CODWAW: 59-61 FPS
Crisys: 44-57
FIFA09: 60 FPS most of the time
A. Creed: 60 FPS most of the time
GRID: 60-83 FPS

Those were the numbers but, what did my eyes saw, what did my mind thought? Well, my eyes saw wonderfulls, my mind went to other worlds, other times and dimensions. I had a great gaming experience! The games ran smooth, even GTA IV and Crisys were fast @ fourties FPS!

Of course, that are somethings we can control until a certain extent and Vertical Sync (V-Sync) is one of them. So, when I've disabled V-Sync, the numbers raised like a condor in some games!

The results:
NFS Undercover: 83-120 FPS
FarCry 2: 60-67 FPS
GTA IV: 60 FPS most of the time
CODWAW: 60-61 FPS
Crisys: 50-60
FIFA09: 60 FPS most of the time
A. Creed: 80-106 FPS
GRID: 83-173 FPS

The curious is that my eyes barely noticed the differences and apart NFSU, the games remained as fast as they were.

Ok. That's it for now. I hope I gave you guys disappointed with your high-end cards benchmarking scores some reasons to smile and wait for better drivers. For all the others who gave themselves the trouble to read this thread, thank you so much!


Johnny Rook