PDA

View Full Version : Bug in 3DMark 2003, cheating is imminent?



Mr.Tweak
02-15-2003, 08:43 AM
While I was fooling around with the latest and greatest from FutureMark, I stumbled upon a bug which can really open the gates of cheating if it isn't fixed pronto. Here's the steps to reproduce it :

1) Set your 3D card at default clock with your favourite overclocking utility
2) Open 3D Mark 2003
3) Start benchmarking, then press Escape so that benchmarking can't finish.
4) Now open your favourite 3D card overclocker again and overclock it like there's no tomorrow
5) Benchmark again, but this time let the benchmark finish
6) Publish your score to the Online Browser for the world to see

If you look carefully, you will see that the 3D Card clock that is being reported is the one you had when you started 3D Mark, and not the actual overclocked values that you used. So it seems that 3D Mark checks for the clock when it starts and NOT when it benchmarks! As you can easily realize, unless this flaw is patched, there is no way you can tell whether the scores you see are coming from overclocked cards or not. I guess FutureMark should move swiftly now and patch this version ASAP.

http://www.neowin.net/comments.php?id=9379&category=main

Wiggo
02-15-2003, 08:54 AM
It seems that none of the "good" sites around will be usin' this as a benchmark as such as there seems to be quite a few nuisances with it that make it very questionable in it's results. :?:
<center>:cheers:</center>

Mr.Tweak
02-15-2003, 09:05 AM
It seems that none of the "good" sites around will be usin' this as a benchmark as such as there seems to be quite a few nuisances with it that make it very questionable in it's results. :?:
<center>:cheers:</center>

We will likely use it for the sack of thoroughness but won't depend on it totally.

urslow
02-15-2003, 10:06 AM
Wow!!! You guys just saved me $40.00. I was all set to buy it until I read this thread. If 3Dmark03 isn't the benchmark of choice, I wonder what will take it's place?

I'm looking to you sage guys for an answer.

Al :confused:

PersianImmortal
02-15-2003, 10:10 AM
Good find, and one that needs to be patched ASAP.

Looks like Futuremark really dropped the ball with 3DMark2003 IMHO.

You can read about the reasons on all the various hardware sites, but the bottom line is 3DMark2003 should be used along with a whole range of other benchmarks and never in isolation.

Even comparisons between similar machines can be pointless because the CPU test for example actually varies depending on which graphics card you're using - something which makes no sense at all!

It's a great looking demo, and I personally think it has a lot of use as a stress test (shame the looping function needs purchase), and also to benchmark the same PC before and after tweaking, but not for comparative purposes.

Wiggo
02-15-2003, 10:12 AM
Until something relies on DX9 then 3DMark2001SE will have to do till 3Dmark03SE comes along. :D
<center>:cheers:</center>

Andy72
02-15-2003, 06:26 PM
Until something relies on DX9 then 3DMark2001SE will have to do till 3Dmark03SE comes along. :D
<center>:cheers:</center>


Agreed. We are just now starting to get games that really use Dx8.1 to the fullest like Unreal 2.
Also, I wonder which will run Doom3 better, high power 8.1 card or mediacore Dx9.0 card like Radeon 9500.

I will wait the next patch of Dx9.0 cards before I make my purchase.



Andy

rugbydude
02-16-2003, 03:07 AM
Thats not the only bug in the new benchmark for futuremark. I downloaded it hopeing to get at least one benchmark from my direct x 7 card but it crashes when i benchmark. And i don't even get the demo. Its currently a useless app on my desktop:angryfire

Wiggo
02-16-2003, 03:14 AM
Ya not the only one with that prob by all reports. :(

:beer: :beer: :beer:

Onyx
02-17-2003, 05:33 PM
I personally don't see it as a major threat to the integrity of 3DMark03 as a benchmark. I mean, it still lists your vid card - that's unchangeable. All it means is when reading reviews/articles we learn to disregard the speed settings of the vid card, and conservatively assume it's set at its highest stable setting.

Wiggo
02-17-2003, 06:21 PM
I'm certainly not goin' to use it as a benchmark myself but I've added it to my stress testin' software (it's not all together a lost cause). :D
<center>:cheers:</center>