RSS Feed

HCW Tech Blog

For the latest info on computer hardware, tech, news, video games, software tips, and Linux, check out our new improved front page: HCW Tech Blog

By: Bryan Pizzuti [06.06.03]

Discuss this article in the forum!
Registration NOT Required!


Yes, you all may have heard about both NVIDIA being naughty about tweaking their drivers' quality settings with this mark, and Futuremark denouncing the action, but then releasing a joint statement with NVIDIA saying it was OK. If you haven't, here's the latest press release (the joint one):

Following is a joint statement from Futuremark and NVIDIA relating to 3DMark03 and optimizations.

Futuremark Statement

For the first time in 6 months, as a result of Futuremark's White Paper on May 23rd, 2003, Futuremark and NVIDIA have had detailed discussions regarding NVIDIA GPUs and Futuremark's 3DMark03 benchmark.

Futuremark now has a deeper understanding of the situation and NVIDIA's optimization strategy. In the light of this, Futuremark now states that NVIDIA's driver design is an application specific optimization and not a cheat.

The world of 3D Graphics has changed dramatically with the latest generation of highly programmable GPUs. Much like the world of CPUs, each GPU has a different architecture and a unique optimal code path. For example, Futuremark's PCMark2002 has different CPU test compilations for AMD's AthlonXP and Intel's Pentium4 CPUs.

3DMark03 is designed as an un-optimized DirectX test and it provides performance comparisons accordingly. It does not contain manufacturer specific optimized code paths. Because all modifications that change the workload in 3DMark03 are forbidden, we were obliged to update the product to eliminate the effect of optimizations identified in different drivers so that 3DMark03 continued to produce comparable results.

However, recent developments in the graphics industry and game development suggest that a different approach for game performance benchmarking might be needed, where manufacturer-specific code path optimization is directly in the code source. Futuremark will consider whether this approach is needed in its future benchmarks.

NVIDIA Statement

NVIDIA works closely with developers to optimize games for GeForceFX. These optimizations (including shader optimizations) are the result of the co-development process. This is the approach NVIDIA would have preferred also for 3DMark03.

Joint NVIDIA-Futuremark Statement

Both NVIDIA and Futuremark want to define clear rules with the industry about how benchmarks should be developed and how they should be used. We believe that common rules will prevent these types of unfortunate situations moving forward.

I have no clue where this whole deal is going, but I know i'm not sure how much to trust 3DMark03 anymore.  Supposedly the 330 patch eliminates the tweaks that NVIDIA added to their drivers to drop quality levels when 3DMark03 is detected.  ATI has also owned up to optimizations for 3DMark03, however those optimizations didn't change a single pixel on screen. Currently, we're working on finding another way to benchmark DX9 performance.

Right now, I know that Futuremark is allowing optimization for NVIDIA's codepaths, but it doesn't tell me if they're allowing the same for ATI's optimizations as well (ATI has already said they would be removing them from future releases). And what about SiS, aspiring to increase their market share with Xabre and Xabre2?  Or S3, with their upcoming DeltaChrome 8-pipeline GPU? And to what degree?  When I see a joint statement from Futuremark and ATI, I'll know that the test is once again fair between the two heavy hitters.  Preferably, I'd like to see a statement from Futuremark indicating they are optimizing for SiS and S3 codepaths in addition to NVIDIA and ATI (Hey, it can't hurt, even if their GPUs end up sucking, right?  Or is the fact that they seem to suck a result of people optimizing only for NVIDIA and ATI? Hmmm...).  But the last thing I want to do is present you readers with a supposedly level benchmark that is actually tilted to favor one manufacturer. Neither you nor I need that.

Next Page: (2)