CPU benchmark analysis

[watercooled]

I've had some time on my hands lately so I've been running some benchmarks on various systems so I can graph them, to compare Northwood Celerons to Atom, and to my Phenom II for example. I might try to arrange the results properly and post them on this forum if anyone's interested. I've noticed some interesting results so far so I might give a short analysis of the results. For example, here's what I found with SuperPi:

Superpi 1M – Single Threaded
CPU, Time in seconds, lower is better
1055T 21.96
4050e 51.422
N270 94.297
Celeron 2.4 170.484
Celeron 1.5 124.829

That's not a misprint, I even went back and re-run the benchmarks as the 1.5 Celeron being faster than a 2.4 Celeron does seem strange. However, if you look at the specs the mobile 1.5 chip has twice the cache of the 2.4 desktop chip - so it seems SuperPi spends most of its time waiting for data to be pulled from RAM into cache after cache misses so the extra clock speed makes no difference. The same seems to apply to the other chips, most noticeably the 1.6GHz in-order Atom being faster than either. It's hard to compare to the newer chips in the same way as other factors might be coming into play - different caching algorithms, branch prediction, higher ILP cores, but without reading and understanding the compiled code I'd just be guessing. Obviously SuperPi isn't the best way to judge performance, it's not exactly well-optimised but I'm still seeing it used frequently in recent benchmarks. It's also only single-threaded so is likely to favour the fewer, bigger cores from Intel.

Anyway, I thought this might be something interesting to discuss - the nitty-gritty of CPU advancements and such. Any thoughts?

Thanks for reading.

[shaithis]

Like most benchmarks SuperPi isn't indicative of real-world performance....and understandably as it's an extremely specific benchmark.

RAM speed and latency effect SuperPi noticeably as well (while having very little real-world effect in most apps), so it might be worth looking at those as well.

As you say, it is extremely hard to compare CPUs. They differ so much between manufacturer and model.....just keep looking forward instead I wonder when the first 3D transistor chips will arrive and what changes to the architecture that will usher in.

[CAT-THE-FIFTH]

SuperPi seems to do better on Intel processors ATM.

[Tattysnuc]

I've had some time on my hands lately so I've been running some benchmarks on various systems so I can graph them, to compare Northwood Celerons to Atom, and to my Phenom II for example. I might try to arrange the results properly and post them on this forum if anyone's interested. I've noticed some interesting results so far so I might give a short analysis of the results. For example, here's what I found with SuperPi:

Superpi 1M – Single Threaded
CPU, Time in seconds, lower is better
1055T 21.96
4050e 51.422
N270 94.297
Celeron 2.4 170.484
Celeron 1.5 124.829

That's not a misprint, I even went back and re-run the benchmarks as the 1.5 Celeron being faster than a 2.4 Celeron does seem strange. However, if you look at the specs the mobile 1.5 chip has twice the cache of the 2.4 desktop chip - so it seems SuperPi spends most of its time waiting for data to be pulled from RAM into cache after cache misses so the extra clock speed makes no difference. The same seems to apply to the other chips, most noticeably the 1.6GHz in-order Atom being faster than either. It's hard to compare to the newer chips in the same way as other factors might be coming into play - different caching algorithms, branch prediction, higher ILP cores, but without reading and understanding the compiled code I'd just be guessing. Obviously SuperPi isn't the best way to judge performance, it's not exactly well-optimised but I'm still seeing it used frequently in recent benchmarks. It's also only single-threaded so is likely to favour the fewer, bigger cores from Intel.

Anyway, I thought this might be something interesting to discuss - the nitty-gritty of CPU advancements and such. Any thoughts?

Thanks for reading.

I've been using SuperPi for a number of years to test stability, and I've found 2 things:

Superpi likes high CPU frequency
It responds best when using high FSB frequencies.
time to write the data to a file is also significant.

To get the best speed, overclock the nuts off your CPU with the highest possible FSB speed (I know they dont exist any more - RAM "speed" or uncore on S1366 has become more relevent) and run it on from a ramdisk. I don't run a ramdisk, but that is the only way I can think of how I could improve my write speeds from my 2xSSD's in Raid 0

Out of interest, on my i7 2600K @ 4.7, the time for 1m on mine is 7.488S. This is a good 2+ seconds faster than my i7 970 at 4.3Ghz. I'll run the scores for you if you are interested.

[krazy_olie]

Interesting!

CPU architecture is a very interesting topic! I don't know what superpi is coded like but if it's raw maths then yes higher clock speeds is good. However I imagine an atom would be crushed in certain scenarios. Things like cache sizes will play different levels of importance, in fact if i write a program to do things in a peculiar fashion a large cache will be a bad thing!

Those northwood processors have a long pipeline. The benefit of a long pipeline is high clock speeds, which in truth was partly a marketing thing. The problem with the celerons is that for their architecture the clock speed is a bit rubbish. Looking at those two celerons the slower one has double the cache. Most likely the bottleneck on that system is the amount of time it spends fetching data. I can't imagine superpi jumping around to different areas of memory so a larger cache is massively helpful.

as a VERY dumbed down example:
I declare an array of numbers called line this "int a[100];"

So i have a 100 numbers and I need to do something with all of them.

When I go to fetch a[0], the first element, I will likely also fetch a[1], a[2], a[3] etc. The larger my cache the more it will fit in and the less data reads I need, as I already have them in the cache.
The cache is very small in this cpu, if operating on contiguous memory cache will help a lot!

The atom cache is about double of the 1.5 celeron at the top of my head.

In the real world caches won't have as dramatic effect on performance because programs jump all over the place, after a certain point a larger cache stops helping.

[watercooled]

Thanks for the replies, was afraid this would go unnoticed. Feel free to add any of your own benchmarks and/or thoughts.

Are there any real differences between the Pentiums and Celerons besides different L2 cache size and FSB?

wPrime is the next benchmark I ran:

wPrime 32M - Multithreaded
CPU | Time in seconds, lower is better
1055T - 9.649
4050e - 43.187
N270 - 132.844
Celeron 2.4 - 148.749
Celeron 1.5 - 244.261

Multithreading obviously plays a huge role here but the Atom is still ahead of both Celerons, but cache doesn't seem to be as big a bottleneck as before. I'll try the Atom without HT later and see how it compares then - maybe wPrime is heavy on single-issue instructions so the SMT of the Atom really helps? I'm still trying to get hold of the more detailed specs of the above CPUs but they're not all that easy to find.