Results 1 to 10 of 10

Thread: Features - Ian Drew Blog: Benchmarking for Smartphones and Tablets

  1. #1
    HEXUS.admin
    Join Date
    Apr 2005
    Posts
    31,709
    Thanks
    0
    Thanked
    2,073 times in 719 posts

    Features - Ian Drew Blog: Benchmarking for Smartphones and Tablets

    This is a guest blog by Ian Drew, Chief Marketing Officer and Executive Vice President Business Development for ARM. The views expressed in this blog are his and his alone. We invited Ian to share some of his thoughts on a hot topic of late: benchmarking for smartphones and tablets. Do let us know in the forums if you agree or disagree with his take on events.
    Read more.
    Last edited by DR; 13-08-2013 at 04:07 PM.

  2. #2
    Editable... jimbouk's Avatar
    Join Date
    Aug 2005
    Location
    Bristol
    Posts
    3,069
    Thanks
    319
    Thanked
    278 times in 226 posts
    • jimbouk's system
      • Motherboard:
      • Asrock B450M-HDV R4.0
      • CPU:
      • AMD Ryzen 5 3600
      • Memory:
      • Corsair Vengeance LPX 16 GB (2 x 8 GB) DDR4 3200 MHz C16
      • Storage:
      • Sabrent Rocket Q 1TB NVMe PCIe M.2 2280
      • Graphics card(s):
      • Sapphire Pulse RX 580 8GB
      • PSU:
      • Seasonic Core Gold GC-650
      • Case:
      • Lian-Li PC-V1100 ATX
      • Operating System:
      • Windows 10 Pro
      • Monitor(s):
      • AOC CU34G2/BK 34" Widescreen
      • Internet:
      • EE FTC

    Re: Features - Ian Drew Blog: Benchmarking for Smartphones and Tablets

    * removed by myself *
    Last edited by jimbouk; 13-08-2013 at 04:01 PM.

  3. #3
    Senior Member
    Join Date
    Jul 2009
    Location
    Oxford
    Posts
    510
    Thanks
    8
    Thanked
    45 times in 34 posts
    • Roobubba's system
      • Motherboard:
      • MSI P55 GD60
      • CPU:
      • Intel i7 860 @ 3.58GHz (Megahalems + Apache)
      • Memory:
      • 8GB Patriot Viper DDR3
      • Storage:
      • 80GB Intel X25-M + bunch of HDDs
      • Graphics card(s):
      • ATI 5870 + Earplugs.
      • PSU:
      • TAGAN 800W
      • Case:
      • Lian Li V1110 (bit disappointing)
      • Operating System:
      • Windows 7 Pro
      • Monitor(s):
      • Samsung 24" LCD (TN)
      • Internet:
      • Virgin Media 20MBit

    Re: Features - Ian Drew Blog: Benchmarking for Smartphones and Tablets

    Gosh, what a huge insight into ARM's approach to (cough advertising cough) benchmarking.

  4. #4
    Registered+
    Join Date
    Dec 2005
    Posts
    78
    Thanks
    0
    Thanked
    1 time in 1 post

    Re: Features - Ian Drew Blog: Benchmarking for Smartphones and Tablets

    I think it's only right when Intel claims to be faster than all it's other competition in the mobile market, that other competitors get to counter-punch those claims.

    I quite like what this guy has to say, I trust it's not BS and it gives a fair comparison of ARM Vs x86 on android devices.... what we don't know is how much of this is due to the issues of running android on x86 architecture....in addition and in Intel's defence, this ATOM APU is old now, the new stuff will improve on the comparisons here.

  5. #5
    Member
    Join Date
    Jun 2013
    Location
    Leicester
    Posts
    103
    Thanks
    0
    Thanked
    3 times in 3 posts

    Re: Features - Ian Drew Blog: Benchmarking for Smartphones and Tablets

    A really great insight into ARM's benchmarking, though they haven't compared OS's that use the chipsets. As I'm guessing there could be a difference into performance.

  6. #6
    Super Nerd
    Join Date
    Jul 2008
    Location
    Cambridge
    Posts
    1,785
    Thanks
    22
    Thanked
    105 times in 72 posts

    Re: Features - Ian Drew Blog: Benchmarking for Smartphones and Tablets

    I'm sorry but as far as I'm concerned graphs and charts without numbers are also graphs and charts without value, nothing more than marketing guff when we have to take it on trust the bars are accurately proportioned (they too often aren't). If the raw numbers are so amazing then use them and don't assume the public are too stupid to comprehend anything other than brightly coloured bars, frankly it's insulting - especially when other sources have a quite different story to tell.

    I agree that using actual shipping devices is the correct approach to benchmarking but then you also need to use many devices to be fair and ensure you haven't picked a single very good implementation of one SoC and a single very bad implementation of another. You harp on about using a variety of benchmarks but then fail to apply them to a wide variety of devices...

    Obviously using many benchmarks on many devices and publishing the raw numbers would make a much longer article but that's the key problem, it is too heavily dumbed down and summarised to the point of being almost useless for any objective analysis.

    It's also a shame to see so many synthetic benchmarks and known outdated ones such as SunSpider. What about some real game frame rates, playing time (per battery capacity unit) until battery dead or measuring page load performance in a browser? The eternal problem with synthetic benchmarks is that users don't use their phones like that, it's all too often a poor indicator of how it will actually feel to use but this is rarely acknowledged in this type of marketing.

  7. #7
    Senior Member watercooled's Avatar
    Join Date
    Jan 2009
    Posts
    11,478
    Thanks
    1,541
    Thanked
    1,029 times in 872 posts

    Re: Features - Ian Drew Blog: Benchmarking for Smartphones and Tablets

    What other sources have 'a quite different story to tell'? Anandtech? The test essentially run by Intel with a few best/worst-case examples? I'm not saying something demonstrated by ARM themselves is going to be impartial either, but these results agree with other stuff I've seen around the web e.g. Phoronix, with the A9 a fair bit ahead of even the desktop Atom for example. Some numbers on the graphs would be nice though, to show they start at zero at least. TBH they do look about right in terms of percentage though.

    Game frame-rates would be more dependant on the GPU than CPU a lot of the time, and the current Atom SoCs have fairly poor GPUs even vs entry level ARM SoCs of smaller die size (the Atom core isn't that small vs Bobcat/Jaguar/ARM cores).

    All you have to do is look at the market TBH. Integrators aren't easy to lie to (you won't win them over with a few bars on a presentation ), they will pick what suits their needs best based on cost/performance/power/etc. The Atom SoC platform has been around for plenty long enough now and we have, what, two or three phones? Versus how many ARM based phones? Or tablets? IMO it's even more telling considering this is Intel we're talking about - they've frequently held the lion's share of a market despite being behind on performance e.g. Athlon days.

  8. #8
    Goron goron Kumagoro's Avatar
    Join Date
    Mar 2004
    Posts
    3,147
    Thanks
    37
    Thanked
    170 times in 139 posts

    Re: Features - Ian Drew Blog: Benchmarking for Smartphones and Tablets

    I wonder if many of these companies would prefer not to buy from Intel based on previous history. Having more sourcing options and control its better than Intel gouging then in the future.

  9. #9
    Senior Member watercooled's Avatar
    Join Date
    Jan 2009
    Posts
    11,478
    Thanks
    1,541
    Thanked
    1,029 times in 872 posts

    Re: Features - Ian Drew Blog: Benchmarking for Smartphones and Tablets

    True, with something like ARM (or MIPS) you're free to go with another SoC MFR without having to change much - the MFRs know that, which encourages competition. However, not all the OEMs will have had previous experience, meaning they'd be looking at things like price, performance, power. Unless you mean just their reputation?

  10. #10
    Super Nerd
    Join Date
    Jul 2008
    Location
    Cambridge
    Posts
    1,785
    Thanks
    22
    Thanked
    105 times in 72 posts

    Re: Features - Ian Drew Blog: Benchmarking for Smartphones and Tablets

    Quote Originally Posted by watercooled View Post
    What other sources have 'a quite different story to tell'? Anandtech? The test essentially run by Intel with a few best/worst-case examples? I'm not saying something demonstrated by ARM themselves is going to be impartial either, but these results agree with other stuff I've seen around the web e.g. Phoronix, with the A9 a fair bit ahead of even the desktop Atom for example. Some numbers on the graphs would be nice though, to show they start at zero at least. TBH they do look about right in terms of percentage though.

    Game frame-rates would be more dependant on the GPU than CPU a lot of the time, and the current Atom SoCs have fairly poor GPUs even vs entry level ARM SoCs of smaller die size (the Atom core isn't that small vs Bobcat/Jaguar/ARM cores).

    All you have to do is look at the market TBH. Integrators aren't easy to lie to (you won't win them over with a few bars on a presentation ), they will pick what suits their needs best based on cost/performance/power/etc. The Atom SoC platform has been around for plenty long enough now and we have, what, two or three phones? Versus how many ARM based phones? Or tablets? IMO it's even more telling considering this is Intel we're talking about - they've frequently held the lion's share of a market despite being behind on performance e.g. Athlon days.
    I essentially meant Intel (and any tech company) who also produce this kind of generalised, number deficient marketing guff (I used the word story quite deliberately with connotations of fiction). If you want to counter marketing guff you have rise above it with actual evidence, numbers and scientific rigor. Guff vs guff is like robot wars, daft creations bashing each other until one slips up by accident and gets caught out or the clock runs out in an anticlimax.

    I'm not here defending Intel, I'm criticising this output from ARM as no better. If they want to show they have the better solution then they need to present better data.

    Besides I agree with Kumagoro, ARM's dominance of the ultra mobile segment is based on more than raw performance. OS support, industry politics etc. Let's also not forget profitable competitors like Qualcomm and Samsung who unlike AMD aren't short of R&D and executive lunch budgets...

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •