Results 1 to 10 of 10

Thread: Features - Ian Drew Blog: Benchmarking for Smartphones and Tablets

  1. #1
    HEXUS.admin
    Join Date
    Apr 2005
    Posts
    29,414
    Thanks
    0
    Thanked
    1,924 times in 668 posts

    Features - Ian Drew Blog: Benchmarking for Smartphones and Tablets

    This is a guest blog by Ian Drew, Chief Marketing Officer and Executive Vice President Business Development for ARM. The views expressed in this blog are his and his alone. We invited Ian to share some of his thoughts on a hot topic of late: benchmarking for smartphones and tablets. Do let us know in the forums if you agree or disagree with his take on events.
    Read more.
    Last edited by DR; 13-08-2013 at 04:07 PM.

  2. #2
    Editable... jimbouk's Avatar
    Join Date
    Aug 2005
    Location
    Bristol
    Posts
    2,567
    Thanks
    189
    Thanked
    192 times in 153 posts
    • jimbouk's system
      • Motherboard:
      • Asrock 970M Pro3
      • CPU:
      • AMD FX-6300
      • Memory:
      • 8GB DDR3
      • Storage:
      • 480GB SSD
      • Graphics card(s):
      • AMD Radeon HD 7870
      • PSU:
      • Corsair HX 520W
      • Case:
      • Old Asus mATX case
      • Operating System:
      • Windows 8.1 Pro
      • Monitor(s):
      • Dell U2713HM
      • Internet:
      • Orange...

    Re: Features - Ian Drew Blog: Benchmarking for Smartphones and Tablets

    * removed by myself *
    Last edited by jimbouk; 13-08-2013 at 04:01 PM.

  3. #3
    Senior Member
    Join Date
    Jul 2009
    Location
    Oxford
    Posts
    485
    Thanks
    8
    Thanked
    37 times in 31 posts
    • Roobubba's system
      • Motherboard:
      • MSI P55 GD60
      • CPU:
      • Intel i7 860 @ 3.58GHz (Megahalems + Apache)
      • Memory:
      • 8GB Patriot Viper DDR3
      • Storage:
      • 80GB Intel X25-M + bunch of HDDs
      • Graphics card(s):
      • ATI 5870 + Earplugs.
      • PSU:
      • TAGAN 800W
      • Case:
      • Lian Li V1110 (bit disappointing)
      • Operating System:
      • Windows 7 Pro
      • Monitor(s):
      • Samsung 24" LCD (TN)
      • Internet:
      • Virgin Media 20MBit

    Re: Features - Ian Drew Blog: Benchmarking for Smartphones and Tablets

    Gosh, what a huge insight into ARM's approach to (cough advertising cough) benchmarking.

  4. #4
    Registered+
    Join Date
    Dec 2005
    Posts
    77
    Thanks
    0
    Thanked
    1 time in 1 post

    Re: Features - Ian Drew Blog: Benchmarking for Smartphones and Tablets

    I think it's only right when Intel claims to be faster than all it's other competition in the mobile market, that other competitors get to counter-punch those claims.

    I quite like what this guy has to say, I trust it's not BS and it gives a fair comparison of ARM Vs x86 on android devices.... what we don't know is how much of this is due to the issues of running android on x86 architecture....in addition and in Intel's defence, this ATOM APU is old now, the new stuff will improve on the comparisons here.

  5. #5
    Member
    Join Date
    Jun 2013
    Location
    Leicester
    Posts
    103
    Thanks
    0
    Thanked
    3 times in 3 posts

    Re: Features - Ian Drew Blog: Benchmarking for Smartphones and Tablets

    A really great insight into ARM's benchmarking, though they haven't compared OS's that use the chipsets. As I'm guessing there could be a difference into performance.

  6. #6
    Super Nerd
    Join Date
    Jul 2008
    Location
    Cambridge
    Posts
    1,785
    Thanks
    22
    Thanked
    105 times in 72 posts

    Re: Features - Ian Drew Blog: Benchmarking for Smartphones and Tablets

    I'm sorry but as far as I'm concerned graphs and charts without numbers are also graphs and charts without value, nothing more than marketing guff when we have to take it on trust the bars are accurately proportioned (they too often aren't). If the raw numbers are so amazing then use them and don't assume the public are too stupid to comprehend anything other than brightly coloured bars, frankly it's insulting - especially when other sources have a quite different story to tell.

    I agree that using actual shipping devices is the correct approach to benchmarking but then you also need to use many devices to be fair and ensure you haven't picked a single very good implementation of one SoC and a single very bad implementation of another. You harp on about using a variety of benchmarks but then fail to apply them to a wide variety of devices...

    Obviously using many benchmarks on many devices and publishing the raw numbers would make a much longer article but that's the key problem, it is too heavily dumbed down and summarised to the point of being almost useless for any objective analysis.

    It's also a shame to see so many synthetic benchmarks and known outdated ones such as SunSpider. What about some real game frame rates, playing time (per battery capacity unit) until battery dead or measuring page load performance in a browser? The eternal problem with synthetic benchmarks is that users don't use their phones like that, it's all too often a poor indicator of how it will actually feel to use but this is rarely acknowledged in this type of marketing.

  7. #7
    Senior Member watercooled's Avatar
    Join Date
    Jan 2009
    Posts
    10,928
    Thanks
    1,508
    Thanked
    946 times in 811 posts

    Re: Features - Ian Drew Blog: Benchmarking for Smartphones and Tablets

    What other sources have 'a quite different story to tell'? Anandtech? The test essentially run by Intel with a few best/worst-case examples? I'm not saying something demonstrated by ARM themselves is going to be impartial either, but these results agree with other stuff I've seen around the web e.g. Phoronix, with the A9 a fair bit ahead of even the desktop Atom for example. Some numbers on the graphs would be nice though, to show they start at zero at least. TBH they do look about right in terms of percentage though.

    Game frame-rates would be more dependant on the GPU than CPU a lot of the time, and the current Atom SoCs have fairly poor GPUs even vs entry level ARM SoCs of smaller die size (the Atom core isn't that small vs Bobcat/Jaguar/ARM cores).

    All you have to do is look at the market TBH. Integrators aren't easy to lie to (you won't win them over with a few bars on a presentation ), they will pick what suits their needs best based on cost/performance/power/etc. The Atom SoC platform has been around for plenty long enough now and we have, what, two or three phones? Versus how many ARM based phones? Or tablets? IMO it's even more telling considering this is Intel we're talking about - they've frequently held the lion's share of a market despite being behind on performance e.g. Athlon days.

  8. #8
    Goron goron Kumagoro's Avatar
    Join Date
    Mar 2004
    Posts
    3,009
    Thanks
    20
    Thanked
    141 times in 114 posts

    Re: Features - Ian Drew Blog: Benchmarking for Smartphones and Tablets

    I wonder if many of these companies would prefer not to buy from Intel based on previous history. Having more sourcing options and control its better than Intel gouging then in the future.

  9. #9
    Senior Member watercooled's Avatar
    Join Date
    Jan 2009
    Posts
    10,928
    Thanks
    1,508
    Thanked
    946 times in 811 posts

    Re: Features - Ian Drew Blog: Benchmarking for Smartphones and Tablets

    True, with something like ARM (or MIPS) you're free to go with another SoC MFR without having to change much - the MFRs know that, which encourages competition. However, not all the OEMs will have had previous experience, meaning they'd be looking at things like price, performance, power. Unless you mean just their reputation?

  10. #10
    Super Nerd
    Join Date
    Jul 2008
    Location
    Cambridge
    Posts
    1,785
    Thanks
    22
    Thanked
    105 times in 72 posts

    Re: Features - Ian Drew Blog: Benchmarking for Smartphones and Tablets

    Quote Originally Posted by watercooled View Post
    What other sources have 'a quite different story to tell'? Anandtech? The test essentially run by Intel with a few best/worst-case examples? I'm not saying something demonstrated by ARM themselves is going to be impartial either, but these results agree with other stuff I've seen around the web e.g. Phoronix, with the A9 a fair bit ahead of even the desktop Atom for example. Some numbers on the graphs would be nice though, to show they start at zero at least. TBH they do look about right in terms of percentage though.

    Game frame-rates would be more dependant on the GPU than CPU a lot of the time, and the current Atom SoCs have fairly poor GPUs even vs entry level ARM SoCs of smaller die size (the Atom core isn't that small vs Bobcat/Jaguar/ARM cores).

    All you have to do is look at the market TBH. Integrators aren't easy to lie to (you won't win them over with a few bars on a presentation ), they will pick what suits their needs best based on cost/performance/power/etc. The Atom SoC platform has been around for plenty long enough now and we have, what, two or three phones? Versus how many ARM based phones? Or tablets? IMO it's even more telling considering this is Intel we're talking about - they've frequently held the lion's share of a market despite being behind on performance e.g. Athlon days.
    I essentially meant Intel (and any tech company) who also produce this kind of generalised, number deficient marketing guff (I used the word story quite deliberately with connotations of fiction). If you want to counter marketing guff you have rise above it with actual evidence, numbers and scientific rigor. Guff vs guff is like robot wars, daft creations bashing each other until one slips up by accident and gets caught out or the clock runs out in an anticlimax.

    I'm not here defending Intel, I'm criticising this output from ARM as no better. If they want to show they have the better solution then they need to present better data.

    Besides I agree with Kumagoro, ARM's dominance of the ultra mobile segment is based on more than raw performance. OS support, industry politics etc. Let's also not forget profitable competitors like Qualcomm and Samsung who unlike AMD aren't short of R&D and executive lunch budgets...

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •