Gaming Challenge Rematch: Intel vs. AMD
July 31, 2006
Real Gaming Challenge. In it, we built a couple PCs using processors that typical PC users can afford, and measured their performance in games using a different methodology—real gameplay, instead of canned, recorded demo playbacks. Now, with the availability of new dual-core Athlon 64 CPUs and Core 2 Duo processors from Intel—both with very aggressive pricing—it's time for a rematch.
In standard gaming benchmarks, whether you're testing Doom 3, Half-Life 2, or Far-Cry, most game benchmarks are made from the "Quake Timedemo" mold. They run through a sequence of recorded gameplay or simply move the player along a predetermined path, counting frames and time to give you an average frame rate. Every time you run the benchmark, the exact same thing is displayed on-screen. This elimination of the variables introduced by normal gameplay is a very useful part of performance evaluation. It's especially useful in testing graphics cards, where you want to get CPU dependency and variability out of the way as much as possible and stress just that one component. It's also less time-consuming to perform each test, which means we can perform tests on a greater number of games and give a broader picture of graphics performance.
Unfortunately, testing that way doesn't always paint an accurate picture of what happens when you really play a game. When playing back a standard timedemo-style recorded benchmark, many of the game's systems either don't operate or function in a controlled, pre-determined fashion. AI, physics, and much of the core game logic can entirely "turn off" when back recorded benchmark demos are playing. These are CPU-intensive tasks, and removing them from the picture can be useful in graphics benchmarking, but what if you want to see which CPUs perform best in real-world gaming scenarios?
In this feature, we'll be using a popular program called FRAPS to measure performance during real live gameplay in six different games across multiple genres. We'll look at how the games run faster and slower over time, and get into a bit of a discussion about "how many FPS is enough." The point is to figure out whether Pentium 4 or Athlon 64 processors deliver a better gaming platform, and to this end, we'll compare two modern, affordable CPUs that won't set you back an arm and a leg. Continued...
Digg this story! Post to del.icio.us Post to Slashdot
For the uninitiated, FRAPS is a popular performance-monitoring program that can display a little overlay in the corner of your screen, showing you the current frames-per-second you're running at all times. It can do much more than that, though. It's also a screen capture utility, and can record movies of your gameplay sessions. The feature we're most interested is its ability to record a log file of the frame rate over time.
The way it works is this: FRAPS counts buffer flips. Your monitor displays what is in your video card's "front buffer," while the video card actively draws the next frame to the "back buffer." When the back buffer is full, it flips to become the front buffer. Counting these buffer flips is an accurate way of measuring how many frames per second are displayed. When recording the frame rate over time, you can set FRAPS to record the frame rate each second for a specified period of time.
In our tests, we'll play each game for 5 minutes, so we set FRAPS to record the frame rate for 300 seconds. Each second, FRAPS will record to a log file the number of buffer flips, so we can get a good look at how performance changes as the gameplay gets more intense. In other words, we'll have a frames-per-second measurement for every second of the 5 minutes we play.
The upside to using FRAPS to benchmark in this way is clear: You get real performance data from real gameplay. FRAPS doesn't introduce any overhead, so the benchmark data you see is exactly what happens when you really play the game. Using FRAPS is not without its drawbacks, though. Chief among these is a lack of reproducibility: It's very hard to play a game exactly the same way twice. In fact, the more "interesting CPU stuff" that's going on (lots of physically active objects, dynamic AI, and so on), the more impossible it becomes to get the same thing to happen from one test to the next. We'll try to reduce this effect by running each 5-minute test three times. Continued... To compare a recent yet affordable Intel based system with a similar AMD based system, we built two machines as identically as possible—changing only the CPU and motherboard.
Intel processor test system AMD Processor test system
Processor Core 2 Duo E6600 (2.4GHz) Athlon 64 X2 5000+ (2.6 GHz, Socket AM2)
Motherboard (chipset) ASUS P5B Deluxe (Intel P965 chipset) ASUS M2N32-SLI Deluxe (nForce 590 SLI chipset)
Memory 2 x 1GB Corsiar XMS2 6400 (CAS 4-4-4-12) 2 x 1GB Corsiar XMS2 6400 (CAS 4-4-4-12)
Graphics card XFX GeForce 7900GT XFX GeForce 7900GT
Hard drive Seagate 7200.8 160GB SATA Drive Seagate 7200.8 160GB SATA Drive
Optical drive ATAPI DVD-RW Drive ATAPI DVD-RW Drive
Audio Sound Blaster Audigy 2 Sound Blaster Audigy 2
Operating system Windows XP Professional with SP2 Windows XP Professional with SP2
CPU gaming benchmarks are often performed on top-of-the-line models, sometimes even overclocked. That's always interesting, but at $800 to $1,000 a pop, those CPUs don't necessarily represent what the mass market can afford or chooses to buy. In making an effort to represent "your next computer" to a wide audience, we went with CPUs that cost $300 to $350. The Athlon 64 X2 5000+ now sells in volume for $301, while the E6600 model Core 2 Duo is $316. Each of the motherboards can be found for between $200 and $250 (depending on wireless configuration and a couple of other optional features). The graphics card costs less than $300. In other words, these are systems you could build yourself for around $1,500, far from the $2,500 or more you would have to spend on "extreme" gaming systems. These are systems that are within the reach of the everyman.
We're running our tests using six modern games across multiple genres. In each game, we try to perform the same actions each time we test, but this is nearly impossible. Running each test three times per platform helps give us a better overall sampling of performance. Each game was patched with the latest publicly available update.
We'll be looking at overall performance, but we're not only concerned with average frames per second this time around. We're looking at how consistently each CPU is able to maintain a frame rate that makes for a smooth experience. The minimum frame rate required for a smooth and enjoyable game varies from one genre to the next, and depends on a variety of factors. 45fps is a good low threshold for many games, but a real-time strategy game, for example, requires a lot of interaction with a static interface and can be perfectly enjoyed as long as the frame rate stays above 30fps. Top-down isometric RPGs in the vein of Diablo 2 or the new Titan Quest are the same way. Some competitive online games are a bit more demanding. As we get into the performance analysis of each game, we'll discuss what frame-rate target makes sense for each game. The point to take away is that it's critically important how well each CPU is able to maintain performance above these target frame rates.
With each game, we'll present three graphs. The first will show the results (frame rate over time) for all three benchmark runs on the Athlon 64 X2 system, the second the Pentium 4 system. The third graph takes the average of all three runs for each system and presents them on the same graph, so you can easily compare the overall performance of one system against another. Continued... All games are run at a resolution of 1280x1024, with details turned up high. We wanted to test by running the games the way real gamers do—at a reasonably high resolution with all the eye candy turned on. The vast majority of monitors sold these days are either 17" or 19", and 1280x1024 is almost always the native resolution for these displays. We're using a high-quality, speedy, but affordable graphics card: a GeForce 7900 GT (currently costing less than $300). While we want to play the games the way people expect to be able to—with the graphics options turn up high—we didn't want the graphics card to be the limiting performance factor, so we never enabled anti-aliasing or anisotropic filtering. And at last, here are the six games we tested with:
Half-Life 2: Episode One: Valve's episodic add-on to Half-Life 2 is slightly more demanding of your hardware than the original Half-Life 2. We played the Exit 17 chapter, specifically the section where you have to escort several waves of civilians back and forth across a battle zone. We turned the graphics details up to full, including enabling HDR rendering.
Oblivion: The latest chapter in the Elder Scrolls RPG saga is one of the most demanding games on the market. The AI system, physics, and large outdoor environments push your CPU and memory subsystem, while the lush detailed graphics and HDR rendering push the GPU. We set the graphics to "high," maxed out the detail and distance sliders, and ran through the "Killing Fields" quests to save a farm from waves of goblin invaders. Then we roamed the forest toward some ruins, taking out a few wolves along the way.
Battlefield 2: The incredible success of Digital Illusions' online shooter has secured its place as one of the most-played games on the market, but it's the advanced engine and intense combat we're after. Playing a game online makes it especially hard to repeat the benchmark in a similar way, so we fired up a single-player game on the Gulf of Oman map with 15 computer-controlled bots to play with. This is less stressful, in a way, than those massive 64-player online games, but it does add the stress of running bot AI.
Rise of Legends: Real-time strategy games can actually be some of the most CPU-intensive games on the market, and Microsoft's latest RTS really pushes things to the limit. Our test begins late in a single-player battle, where we have several large towns and a massive army built up. We attack the enemy's base, tilting the camera to show as much action as possible, while popping back to our own cities every now and then to upgrade and defend them.
Titan Quest: This modern RPG takes the Diablo 2 mold and applies it to an ancient Greece setting. The graphics are much improved over Blizzard's classic, though. The full 3D graphics, with swaying grass, real-time physics, and hordes of enemies can stress your system. Our test begins just outside the Spartan War Camp, as we take our character and two summoned wolves across the fields, battling swarms of satyrs and centaurs, ultimately taking out the centaur chieftain.
World of Warcraft: It would be enough that WoW has somewhere between 7 and 8 million subscribers worldwide, but these people generally put some serious hours into the game. This may be the most-played game on the market, and in some places it can stress your system. Crowded areas like major cities and battlegrounds push the CPU hardest. It's even harder to have a repeatable experience in this game than with most, due to the massively multiplayer online nature of it. We ran around Orgrimmar, the crowded Horde city, focusing our attention mostly on the crowded bank and auction house area of the Valley of Strength. WoW has a particular issue with dual-core systems that we'll detail on the following pages. Continued...
In high-action first person shooters, you generally want as high a frame rate as possible. Once you get above 60 frames per second, you're reaching the point of diminishing returns. Let's see how AMD and Intel stack up:
The extremely dynamic nature of this scene made it very hard to get consistent benchmarks from one run to the next. Sure, we could have picked a more linear map with more predictable enemy encounters, but what you really want to test is the difficult scenario—lots of physics, lots of AI. Those are the very things that make the gameplay different each time. Both systems deliver fantastic results. We rarely dipped below 60fps on the Athlon 64 X2 system, and never went below that frame rate on the Core 2 Duo box. Both systems spent a whole lot of time, and we mean a lot, over 100fps. Clearly, this is a case of a game where, even with these affordable systems, gamers can expect to enable anti-aliasing or anisotropic filtering for an even better visual experience.
Looking at the average of our three runs for each system, Intel's CPU comes out way ahead—almost a third faster—on the whole. Both are delivering a completely smooth experience that almost never dips below 60fps, but the Core 2 Duo has more headroom. If Valve were to introduce larger battles or greater use of physics in future episodes, the Core 2 Duo would likely be able to handle the load better. Continued... This is a game that really stresses your graphics card, so we were curious to see whether the CPU made any difference in performance. It's very hard to play the game exactly the same each time, but we did our best run the same path and fight the same battles the same way. Repeated three times and averaged together, we think our method produces a pretty good overall picture of performance in this first-person role-playing game. Because this is a slower-paced RPG and the combat isn't as frenetic as a competitive shooter, we think a frame rate that stays above 30fps represents a pretty smooth play experience.
Clearly, such a graphics-intensive game depends less on CPU speed and platform speed and more on the performance of your graphics card. It's interesting that, with a single sub-$300 graphics card and the graphics settings turned up high (including enabling HDR), we still see some difference in performance based entirely on the CPU platform. Both systems hover between 30 and 50fps, but the Core 2 Duo–based setup dips below that 30fps threshold a little less often, and extends up near 60fps more frequently.
Average out all three runs, and we see the Core 2 Duo system maintaining a higher frame rate through longer stretches of gameplay. It's the low points we should focus on, though—the worst dips in performance are the points that stand out and make a game feel sluggish. Both systems get down near 30fps to 35fps more often than we'd like, though the Core 2 Duo system does so a bit less often. It would be interesting to see how performance differs with a really powerful dual-graphics solution, but that's not the focus of our testing here today. Continued... Battlefield 2 is starting to age a bit, but it's still an extremely popular online game and one of the more demanding shooters on the market. As a competitive online shooter, albeit a more deliberate and strategic one than something like Unreal Tournament, we feel that it's important to maintain a high frame rate at all times. With all the quick snapping the camera around to scout for enemies, you really want to keep the frame rate above 60fps. Yes, it's certainly playable at much lower frame rates, but the enjoyability suffers.
The Athlon 64 X2 looks good, rarely dipping below 60fps and usually hovering closer to 80fps, but the Core 2 Duo puts it to shame. Never, not once, did the Core 2 Duo system drop below 60fps. Through three 5-minute tests, the Intel-based system stayed between 80 and 100fps the vast majority of the time.
Average out those three runs to remove the peaks and troughs and you get a clear picture of how much better that $316 Core 2 Duo chip runs Battlefield 2. It's not just that the average frame rate is almost 20% faster, it's the consistency that impresses us. Not only is it that much faster on average, it's that much faster the whole time, and that's an important distinction. Continued... Frame rate is decidedly less important in most real-time strategy (RTS) games than it is in many other genres. The pace can be frantic, but you're usually frantically clicking on static interface elements and tapping on keyboard shortcuts, rather than moving and rotating the camera constantly. You can run most RTS games at about 30fps before a low frame rate starts to impact your enjoyment. With a lower threshold than all the other games, will both CPUs pass with flying colors?
You can see that it's extremely difficult to maintain consistency from one 5-minute test to the next, even though all tests begin with the same saved game and involve the same groups of units attacking the same base. RTS games are very CPU driven, now with plenty of real-time physics on explosions, particle effects, path-finding algorithms running on dozens of units, and more. Both systems, at times, would dip below 30fps and get a little bit choppy, though the game was still quite playable. It's clear from comparing the two charts that the Athlon 64 X2 ran below 30fps quite a bit more often than the Core 2 Duo.
Averaging out three runs makes it easy to compare the two systems, but it can also remove those high and low spikes, and sometimes knowing that a game has occasional dramatic dips in performance can be useful. The Core 2 Duo wins again, averaging a higher frame rate more often. In this particular game, the graphs above that show all three runs are frankly more important, because you don't want to average out that highly variable gameplay, you want to see how wildly performance can vary. Continued... An overhead isometric RPG in the vein of Diablo 2, Titan Quest combines RPG gameplay with fast-paced clicking action. It falls somewhere between an RPG or RTS and action game in terms of the minimum performance required for a really smooth and enjoyable game. If it gets too slow, the fast-paced clicking on enemies starts to suffer. But since the camera doesn't move around a lot, and there's a fair amount of clicking on a static interface (or pressing keyboard hotkeys), you don't need the kind of 60fps smoothness required of most first person shooters. We figure 45fps is a good cut-off line for minimum desired frame rate, but you would probably have a good time as long as you stayed above 30fps.
The Athlon 64 X2 5000+ based system stays above the 45fps cut-off line almost all the time, and usually runs between 50fps and 70fps. The Core 2 Duo E6600–based rig almost never went below 45fps, and consistently remained in the 60fps to 80fps range, with occasional bursts up near 100fps. Both systems deliver fantastic, silky smooth gameplay.
On the whole, the Core 2 Duo system is about 10% faster. It's worth noting that it gets that higher average frame rate mostly during the really fast moments, where it runs at perhaps 80fps while the Athlon 64 X2 runs at around 70. Both systems spend most of the time at 60fps or better, and neither one has a strong advantage in how often they dip down to that minimum 45fps line. So while the Core 2 Duo has more headroom, you're not really going to "feel" a difference in this title. Continued... Here's a really interesting case. There's a well-known bug in World of Warcraft with dual-core CPUs (check the tech support forum for more information). WoW is multithreaded by nature, with as many as seven or eight threads at once, although the bulk of the CPU work is done in a single "main thread" so the performance benefit of a dual-core system is limited. Well, on dual-core systems, a bug in the WoW client code limits your frame rate to 64fps—a strange number. What's more, many users report graphic stuttering and jerking on dual-core systems. We didn't experience this, but the 64fps limit affects every dual-core user. There are apparently some fixes in the upcoming 1.12 patch, but we tried that patch on the public test realms and noticed absolutely no difference.
What's more, WoW runs so well on modern CPUs that it's hard to find an area that will stress one enough. Places where lots of players gather together seem best suited to this purpose, so we concentrated our play in the most crowded area of the Horde city of Orgrimmar, near the bank and auction house.
As you can see, the bulk of our play time was spent stuck at that 64fps upper limit imposed by WoW's dual-core bug. We could sometimes dip below 50fps on the Athlon 64 X2, but these spikes are relatively infrequent. Even so, you can very clearly see how the Core 2 Duo spends a lot more time up at that 64fps limit.
Comparing the three-run averages of the two systems, you can visibly see how much more often the Core 2 Duo–based system remains at 64fps. Of course, both systems will give you a really smooth game of WoW. We can't help but wonder how much headroom each CPU has, though. If Blizzard fixed that 64fps limit, what would this cart look like? Would the slight 3% overall average frame rate of the Core 2 Duo be extended, or disappear? We should note that the game can't be identical each time we test, because of the online, player-driven nature of it. Our test of the Core 2 Duo was slightly later in the day than the Athlon 64 X2, and it was a little bit more crowded in the area we spent most of our test time. In other words, if anything, the Core 2 Duo system had a slightly tougher benchmark run.
With the vast majority of new computers coming with dual-core CPUs installed, we hope Blizzard quickly makes dual-core performance a priority. Continued... When we ran our first Real Gaming Challenge almost a year ago, we used CPUs that were about $80 cheaper, and single-core. Though the price of the tested CPUs has gone up, they're still quite affordable, and they're a lot faster. It has quickly become a dual-core world, and Intel's introduction of a whole new processor architecture has made a huge difference. Though the games are more demanding this year, tables have turned completely, with the Core 2 Duo now winning every single test against the comparable Athlon 64 X2—the exact reverse of last year.
Core 2 Duo E6600 Average FPS Athlon 64 X2 5000+ Average FPS Core 2 Duo time beneath threshold Athlon 64 X2 time beneath threshold
Half-Life 2: EP1 113 86 0% 8%
Oblivion 42 40 3% 10%
Battlefield 2 90 76 0% 6%
Rise of Legends 45 37 11% 30%
Titan Quest 72 65 0% 1%
World of Warcraft 63 62 1% 4%
Average 71 61 3% 10%
When you consider all six games, the winner becomes abundantly clear. There's not a single game where the Core 2 Duo didn't deliver a higher average frame rate. Across all games, Intel's new CPU delivered a 16% higher frame rate. What is more important is that it spent a significantly smaller amount of time beneath our arbitrary thresholds for a really smooth and enjoyable gameplay experience. In fact, there are three games—half of our sample group!—where the Core 2 Duo essentially never dipped below the minimum threshold at all.
Both systems are truly fantastic game platforms, and the recent aggressive pricing by both Intel and AMD are very exciting. Compared with last year's tests, the games in our group are more demanding, and our minimum frame-rate thresholds are higher, too. Despite this, both systems delivered higher overall frame rates and, more importantly, spent less time beneath the minimum FPS threshold.
Click here for more Core 2 Benchmarks
Though we proclaim Core 2 Duo the clear winner, we can't be too hard on the Athlon 64 X2. It turned in a great performance and definitely makes a great gaming CPU. We've lauded the game performance of AMD's CPUs for a long time, and they didn't get any slower with the introduction of Intel's Core 2 lineup, just less expensive.
The real winner here is the consumer. You don't need to spend several thousand dollars on hardware to make a great gaming system. At 1280x1024—the most common LCD resolution and 40% higher-res than 720p high definition—you can have a really awesome time gaming with a single sub-$300 graphics card and a CPU just over $300. And you don't need to reduce the game's detail levels to do it; we ran all these tests using each game's "high" setting. With such high-performance dual-core CPUs priced so aggressively, and with Vista coming out in the next year with a strong focus on improving PC gaming, the signs look good for PC game fans.
