Page 2 of 4 FirstFirst 1234 LastLast
Results 17 to 32 of 62

Thread: GPU Testing/Reviewing techniques and Runt Frames with Crossfire

  1. #17
    Moosing about! CAT-THE-FIFTH's Avatar
    Join Date
    Aug 2006
    Location
    Not here
    Posts
    32,039
    Thanks
    3,910
    Thanked
    5,224 times in 4,015 posts
    • CAT-THE-FIFTH's system
      • Motherboard:
      • Less E-PEEN
      • CPU:
      • Massive E-PEEN
      • Memory:
      • RGB E-PEEN
      • Storage:
      • Not in any order
      • Graphics card(s):
      • EVEN BIGGER E-PEEN
      • PSU:
      • OVERSIZED
      • Case:
      • UNDERSIZED
      • Operating System:
      • DOS 6.22
      • Monitor(s):
      • NOT USUALLY ON....WHEN I POST
      • Internet:
      • FUNCTIONAL

    Re: GPU Testing/Reviewing techniques and Runt Frames with Crossfire

    Quote Originally Posted by Nvidia
    We’re proud of the work that we’ve put into this – and we think it can help gamers get the experience they’re paying for. So we’re opening up our FCAT solution, making the scripts and software associated with FCAT freely-modifiable and redistributable. The technical press has already dug in, and the results have been dramatic.

    Our hope: that third-party apps can replicate and replace our tools, giving gamers what they need to be sure they’re getting all of the graphics quality they’re paying for.
    That statement does make it pretty evident that PCPer was using at least the FCAT software,or had got technical help from Nvidia. This should have been disclosed and as of today they still have not said anything.

    If Nvidia had said nothing,none of us would have known at all.

    Review websites have to be more transparent about these things,and too many are not.
    Last edited by CAT-THE-FIFTH; 01-04-2013 at 05:14 PM.

  2. #18
    Senior Member
    Join Date
    Jun 2012
    Posts
    2,401
    Thanks
    87
    Thanked
    151 times in 145 posts
    • Willzzz's system
      • Motherboard:
      • Gigabyte
      • CPU:
      • 4670K
      • PSU:
      • FD Newton R3 600W
      • Case:
      • Corsair 350D

    Re: GPU Testing/Reviewing techniques and Runt Frames with Crossfire

    PCPer (and all the rest) did say they got the scripts from Nvidia, I'm not sure what the problem is.

  3. #19
    Oh Crumbs.... Biscuit's Avatar
    Join Date
    Feb 2007
    Location
    N. Yorkshire
    Posts
    11,193
    Thanks
    1,394
    Thanked
    1,091 times in 833 posts
    • Biscuit's system
      • Motherboard:
      • MSI B450M Mortar
      • CPU:
      • AMD 2700X (Be Quiet! Dark Rock 3)
      • Memory:
      • 16GB Patriot Viper 2 @ 3466MHz
      • Storage:
      • 500GB WD Black
      • Graphics card(s):
      • Sapphire R9 290X Vapor-X
      • PSU:
      • Seasonic Focus Gold 750W
      • Case:
      • Lian Li PC-V359
      • Operating System:
      • Windows 10 x64
      • Internet:
      • BT Infinity 80/20

    Re: GPU Testing/Reviewing techniques and Runt Frames with Crossfire

    Well Nvidia tools being used to prove that AMD cards have problems, just smells a bit funky to me and damages the credibility of the reports.

  4. #20
    Senior Member
    Join Date
    Jun 2012
    Posts
    2,401
    Thanks
    87
    Thanked
    151 times in 145 posts
    • Willzzz's system
      • Motherboard:
      • Gigabyte
      • CPU:
      • 4670K
      • PSU:
      • FD Newton R3 600W
      • Case:
      • Corsair 350D

    Re: GPU Testing/Reviewing techniques and Runt Frames with Crossfire

    All the Nvidia tools do is draw the graphs, it's all totally open and transparent.

    At least when people complained about the use of FRAPS data they did at least have a point. If you have issues with FCAT then please explain, but don't just say 'it smells fishy' that isn't an argument.

    What would be nice to see that we don't have yet is how internal game timings tally with what is displayed. That would be pretty interesting, but it wouldn't mean that the runt/dropped frames didn't matter.

  5. #21
    Moosing about! CAT-THE-FIFTH's Avatar
    Join Date
    Aug 2006
    Location
    Not here
    Posts
    32,039
    Thanks
    3,910
    Thanked
    5,224 times in 4,015 posts
    • CAT-THE-FIFTH's system
      • Motherboard:
      • Less E-PEEN
      • CPU:
      • Massive E-PEEN
      • Memory:
      • RGB E-PEEN
      • Storage:
      • Not in any order
      • Graphics card(s):
      • EVEN BIGGER E-PEEN
      • PSU:
      • OVERSIZED
      • Case:
      • UNDERSIZED
      • Operating System:
      • DOS 6.22
      • Monitor(s):
      • NOT USUALLY ON....WHEN I POST
      • Internet:
      • FUNCTIONAL

    Re: GPU Testing/Reviewing techniques and Runt Frames with Crossfire

    Quote Originally Posted by Willzzz View Post
    PCPer (and all the rest) did say they got the scripts from Nvidia, I'm not sure what the problem is.
    No they didn't. Other sites said they were using FCAT specifically. PCPER did not.

    Quote Originally Posted by First article
    Author: Ryan Shrout
    Date: January 3, 2013
    Subject: Graphics Cards
    Manufacturer: PC Perspective
    Tagged: video, stutter, radeon, nvidia, graphics, gpu, geforce, frame rating, fluid, amd

    A change is coming in 2013

    If the new year will bring us anything, it looks like it might be the end of using "FPS" as the primary measuring tool for graphics performance on PCs. A long, long time ago we started with simple "time demos" that recorded rendered frames in a game like Quake and then played them back as quickly as possible on a test system. The lone result was given as time, in seconds, and was then converted to an average frame rate having known the total number of frames recorded to start with.

    More recently we saw a transition to frame rates over time and the advent frame time graphs like the ones we have been using in our graphics reviews on PC Perspective. This expanded the amount of data required to get an accurate picture of graphics and gaming performance but it was indeed more accurate, giving us a more clear image of how GPUs (and CPUs and systems for that matter) performed in games.

    And even though the idea of frame times have been around just a long, not many people were interested in getting into that detail level until this past year. A frame time is the amount of time each frame takes to render, usually listed in milliseconds, and could range from 5ms to 50ms depending on performance. For a reference, 120 FPS equates to an average of 8.3ms, 60 FPS is 16.6ms and 30 FPS is 33.3ms. But rather than average those out by each second of time, what if you looked at each frame individually?

    Scott over at Tech Report started doing that this past year and found some interesting results. I encourage all of our readers to follow up on what he has been doing as I think you'll find it incredibly educational and interesting.

    Through emails and tweets many PC Perspective readers have been asking for our take on it, why we weren't testing graphics cards in the same fashion yet, etc. I've stayed quiet about it simply because we were working on quite a few different angles on our side and I wasn't ready to share results. I am still not ready to share the glut of our information yet but I am ready to start the discussion and I hope our community find its compelling and offers some feedback.

    View Full Size

    At the heart of our unique GPU testing method is this card, a high-end dual-link DVI capture card capable of handling 2560x1600 resolutions at 60 Hz. Essentially this card will act as a monitor to our GPU test bed and allow us to capture the actual display output that reaches the gamer's eyes. This method is the best possible way to measure frame rates, frame times, stutter, runts, smoothness, and any other graphics-related metrics.

    Using that recorded footage, sometimes reaching 400 MB/s of consistent writes at high resolutions, we can then analyze the frames one by one, though with the help of some additional software. There are a lot of details that I am glossing over including the need for perfectly synced frame rates, having absolutely zero dropped frames in the recording, analyzing, etc, but trust me when I say we have been spending a lot of time on this.

    Continue reading our editorial on Frame Rating: A New Graphics Performance Metric.

    The result are multi-GB files (60 seconds of game play produces a 16GB file) that have each frame presented to the monitor of the gamer in perfect fashion.

    View Full Size

    View Full Size

    The second image here was made in Photoshop to show you the three different "frames" of Unigine that make up the single frame that the gamer would actually see. With a 60 Hz display, this equates to three frames being shown in 16ms, though each frame has a variable amount of real estate on the screen.

    We are still finalizing what we can do with all of this data, but it definitely allows us to find some unique cases:

    View Full Size

    This shot from Sleeping Dogs shows three frames on our single 16ms instance to the monitor, but notice how little screen spaces the green frame takes up - is measuring this frame even useful for gamers? Should it be dropped from the performance metrics all together?

    I put together this short video below with a bit more on my thoughts on this topic but I can assure you that January and February are going to bring some major change to the way graphics cards are tested. Please, we want your thoughts! This is an important dialogue for not only us but for all PC gamers going forward. Scott has done some great work "Inside the Second" and we hope that we can offer additional information and clarification on this problem. Rest assured we have not been sitting on our hands here!

    Is this kind of testing methodology going to be useful? Is it overkill? Leave us your thoughts below!
    Quote Originally Posted by second article
    Frame Rating Part 2: Finding and Defining Stutter
    Author: Ryan Shrout
    Date: January 16, 2013
    Subject: Graphics Cards
    Manufacturer: PC Perspective
    Tagged: video, stutter, radeon, nvidia, graphics, gpu, geforce, frame rating, fluid, amd
    Another update

    In our previous article and video, I introduced you to our upcoming testing methodology for evaluating graphics cards based not only frame rates but on frame smoothness and the efficiency of those frame rates. I showed off some of the new hardware we are using for this process and detailed how direct capture of graphics card output allows us to find interesting frame and animation anomalies using some Photoshop still frames.

    View Full Size

    Today we are taking that a step further and looking at a couple of captured videos that demonstrate a "stutter" and walking you through, frame by frame, how we can detect, visualize and even start to measure them.

    View Full Size

    This video takes a couple of examples of stutter in games, DiRT 3 and Dishonored to be exact, and shows what they look like in real time, at 25% speed and then finally in a much more detailed frame-by-frame analysis.





    Obviously this is just a couple instances of what a stutter is and there are often times less apparent in-game stutters that are even harder to see in video playback. Not to worry - this capture method is capable of seeing those issues as well and we plan on diving into the "micro" level as well shortly.

    We aren't going to start talking about whose card and what driver is being used yet and I know that there are still a lot of questions to be answered on this topic. You will be hearing more quite soon from us and I thank you all for your comments, critiques and support.

    Let me know below what you thought of this video and any questions that you might have.
    No mention of Nvidia software or scripts used in any of them.

    If Nvidia had not linked back to the article,all of us would have thought PCPER were using a 100% in-house developed solution.

    The Titan review never mentioned anything either.

    This was MONTHS before FCAT was used by other sites who mentioned they were using it.

    Not even a simple,Nvidia assisted us,with software development line.

    So basically PCPER,were palming off software tools developed by Nvidia as their own,without even mentioning Nvidia was involved,to get a one up on The Tech Report methinks.

    Journalists have to be 100% transparent in their dealings with companies. What is the point of independent journalism then,if they hide their affiliations??

    You should read what DR(owner of Hexus) has said in regards to this.
    Last edited by CAT-THE-FIFTH; 01-04-2013 at 08:11 PM.

  6. #22
    Senior Member
    Join Date
    Jun 2012
    Posts
    2,401
    Thanks
    87
    Thanked
    151 times in 145 posts
    • Willzzz's system
      • Motherboard:
      • Gigabyte
      • CPU:
      • 4670K
      • PSU:
      • FD Newton R3 600W
      • Case:
      • Corsair 350D

    Re: GPU Testing/Reviewing techniques and Runt Frames with Crossfire

    CAT, you aren't making any sense. PCPer didn't have FCAT for the first two articles.

    The Titan article is dated the 30th of March, so AFTER they spoke about FCAT being from Nvidia. Should they mention this in every future review now?

    The graphs in the original Titan review are different to the FCAT output.

  7. #23
    Moosing about! CAT-THE-FIFTH's Avatar
    Join Date
    Aug 2006
    Location
    Not here
    Posts
    32,039
    Thanks
    3,910
    Thanked
    5,224 times in 4,015 posts
    • CAT-THE-FIFTH's system
      • Motherboard:
      • Less E-PEEN
      • CPU:
      • Massive E-PEEN
      • Memory:
      • RGB E-PEEN
      • Storage:
      • Not in any order
      • Graphics card(s):
      • EVEN BIGGER E-PEEN
      • PSU:
      • OVERSIZED
      • Case:
      • UNDERSIZED
      • Operating System:
      • DOS 6.22
      • Monitor(s):
      • NOT USUALLY ON....WHEN I POST
      • Internet:
      • FUNCTIONAL

    Re: GPU Testing/Reviewing techniques and Runt Frames with Crossfire

    Quote Originally Posted by Willzzz View Post
    CAT, you aren't making any sense. PCPer didn't have FCAT for the first two articles.

    The Titan article is dated the 30th of March, so AFTER they spoke about FCAT being from Nvidia. Should they mention this in every future review now?

    The graphs in the original Titan review are different to the FCAT output.
    The Titan article is 21st February.

    Plus I saw this:

    Quote Originally Posted by Ryan Smith,Anandtech GPU editor
    PCPer was beta testing FCAT a few months before its wider release. They already had the equipment, so when NVIDIA was ready to try it outside the company it was a good match for them.
    Plus,Nvidia linked back to them in that article.

    Not a big website like Anandtech,TH or TR,but PCPER who are not that well known TBH.
    Last edited by CAT-THE-FIFTH; 01-04-2013 at 08:27 PM.

  8. #24
    Oh Crumbs.... Biscuit's Avatar
    Join Date
    Feb 2007
    Location
    N. Yorkshire
    Posts
    11,193
    Thanks
    1,394
    Thanked
    1,091 times in 833 posts
    • Biscuit's system
      • Motherboard:
      • MSI B450M Mortar
      • CPU:
      • AMD 2700X (Be Quiet! Dark Rock 3)
      • Memory:
      • 16GB Patriot Viper 2 @ 3466MHz
      • Storage:
      • 500GB WD Black
      • Graphics card(s):
      • Sapphire R9 290X Vapor-X
      • PSU:
      • Seasonic Focus Gold 750W
      • Case:
      • Lian Li PC-V359
      • Operating System:
      • Windows 10 x64
      • Internet:
      • BT Infinity 80/20

    Re: GPU Testing/Reviewing techniques and Runt Frames with Crossfire

    Quote Originally Posted by Willzzz View Post
    All the Nvidia tools do is draw the graphs, it's all totally open and transparent.

    At least when people complained about the use of FRAPS data they did at least have a point. If you have issues with FCAT then please explain, but don't just say 'it smells fishy' that isn't an argument.

    What would be nice to see that we don't have yet is how internal game timings tally with what is displayed. That would be pretty interesting, but it wouldn't mean that the runt/dropped frames didn't matter.
    Since when has anything Nvidia has done been trasparent and open?

    Can you categorically say that there is no chance Nvidia have put anything in their tools that would give preferential results to their own cards?

    I wouldn't put it past them given their attitude in the past.

  9. #25
    Senior Member
    Join Date
    Jun 2012
    Posts
    2,401
    Thanks
    87
    Thanked
    151 times in 145 posts
    • Willzzz's system
      • Motherboard:
      • Gigabyte
      • CPU:
      • 4670K
      • PSU:
      • FD Newton R3 600W
      • Case:
      • Corsair 350D

    Re: GPU Testing/Reviewing techniques and Runt Frames with Crossfire

    Biscuit, if you are so suspicious then you can examine the scripts when they are released.
    Do you really think all these different sites haven't examined the scripts already?

    CAT, oh ok, so that was an early version of FCAT. Is it that unusual that they don't spill the beans before the product is ready?

  10. #26
    Oh Crumbs.... Biscuit's Avatar
    Join Date
    Feb 2007
    Location
    N. Yorkshire
    Posts
    11,193
    Thanks
    1,394
    Thanked
    1,091 times in 833 posts
    • Biscuit's system
      • Motherboard:
      • MSI B450M Mortar
      • CPU:
      • AMD 2700X (Be Quiet! Dark Rock 3)
      • Memory:
      • 16GB Patriot Viper 2 @ 3466MHz
      • Storage:
      • 500GB WD Black
      • Graphics card(s):
      • Sapphire R9 290X Vapor-X
      • PSU:
      • Seasonic Focus Gold 750W
      • Case:
      • Lian Li PC-V359
      • Operating System:
      • Windows 10 x64
      • Internet:
      • BT Infinity 80/20

    Re: GPU Testing/Reviewing techniques and Runt Frames with Crossfire

    I wouldnt know what to look for and im not catagorically saying that they have done anything untoward, i just think it would be naive not to approach the tools and results without considering that they were developed by nvidia, who are a company with a long history of using less that savoury business practices.
    Its in Nvidias interest for AMD cards to look bad, perhaps this time its 100% honest, perhaps its not, i just dont really know. AMD dont seem to up in arms fighting it... yet.

  11. #27
    Senior Member
    Join Date
    Mar 2010
    Posts
    2,567
    Thanks
    39
    Thanked
    179 times in 134 posts

    Re: GPU Testing/Reviewing techniques and Runt Frames with Crossfire

    so PCPer are using NVidia software on AMD (the competition) hardware to prove theres a problem.... all the while hiding there own problems with law suits and `adaptive vsync`....


    a bit like anandtech being impartial about AMD cpu`s , when 2 intel employee`s are MODS on there forum! and another writes software that's used for benchmarking!


    might as well say PCPer are a branch of NVidia.....

  12. #28
    Senior Member
    Join Date
    Jun 2012
    Posts
    2,401
    Thanks
    87
    Thanked
    151 times in 145 posts
    • Willzzz's system
      • Motherboard:
      • Gigabyte
      • CPU:
      • 4670K
      • PSU:
      • FD Newton R3 600W
      • Case:
      • Corsair 350D

    Re: GPU Testing/Reviewing techniques and Runt Frames with Crossfire

    Why is it that some people want to see a conspiracy everywhere they look?

    It was quite clear from the work that PCPer did that there was a problem way before they had the FCAT software to analyse their data.

    I suppose Anandtech, Techreport and Tom's are also being paid off by Nvidia according to you?

  13. #29
    Senior Member watercooled's Avatar
    Join Date
    Jan 2009
    Posts
    11,478
    Thanks
    1,541
    Thanked
    1,029 times in 872 posts

    Re: GPU Testing/Reviewing techniques and Runt Frames with Crossfire

    There's a difference between being paid off and just not being completely open. In the professional world, it's fully expected for trials to be picked apart if there are any areas of uncertainty or methodological flaws. In the PC world, it seems people are just accused of trying to find fault with everything.

    As biscuit said, it may not even be a problem in this case, but if so then where's the harm in being more open about it? Holding back details makes people suspicious about the results - all the details should be presented so people can draw their own conclusions.

  14. #30
    Senior Member
    Join Date
    Jun 2012
    Posts
    2,401
    Thanks
    87
    Thanked
    151 times in 145 posts
    • Willzzz's system
      • Motherboard:
      • Gigabyte
      • CPU:
      • 4670K
      • PSU:
      • FD Newton R3 600W
      • Case:
      • Corsair 350D

    Re: GPU Testing/Reviewing techniques and Runt Frames with Crossfire

    But what are they holding back on? All 4 sites that have published FCAT data have gone into great detail explaining the process, how the data is collected and what the data means. I'd get your point if they just came up with a graph and didn't explain where it came from, but that isn't what has happened at all.

    I'm not sure what more you want them to do?

  15. #31
    Oh Crumbs.... Biscuit's Avatar
    Join Date
    Feb 2007
    Location
    N. Yorkshire
    Posts
    11,193
    Thanks
    1,394
    Thanked
    1,091 times in 833 posts
    • Biscuit's system
      • Motherboard:
      • MSI B450M Mortar
      • CPU:
      • AMD 2700X (Be Quiet! Dark Rock 3)
      • Memory:
      • 16GB Patriot Viper 2 @ 3466MHz
      • Storage:
      • 500GB WD Black
      • Graphics card(s):
      • Sapphire R9 290X Vapor-X
      • PSU:
      • Seasonic Focus Gold 750W
      • Case:
      • Lian Li PC-V359
      • Operating System:
      • Windows 10 x64
      • Internet:
      • BT Infinity 80/20

    Re: GPU Testing/Reviewing techniques and Runt Frames with Crossfire

    Quote Originally Posted by Willzzz View Post
    Why is it that some people want to see a conspiracy everywhere they look?

    It was quite clear from the work that PCPer did that there was a problem way before they had the FCAT software to analyse their data.

    I suppose Anandtech, Techreport and Tom's are also being paid off by Nvidia according to you?
    Why is it so easy to just accept what is being said and not hold back with cautious consideration? I don't think there is a conspiracy, I just don't think we should immediately fall in line behind Nvidia. I actually don't like the word conspiracy, because it is always used to immediately discredit the discussion by making the people look crazy for thinking anything other that the status quo. The status quo for Nvidia is to be shifty and awkward so im not even thinking against it!
    The same caution should be exercised with all big corporations and companies that try and use tools and software that is designed by or sponsored by them to prove that their stuff is the best. Even Intel and yes, AMD have been guilty of it in some form or another in the past. Consider all sides and all possibilities including any bias and ulterior motives.

    I think the inference with this is that PCPerspective actually had help from nvidia in their early work in this area from the get go, why did they not reach out to AMD aswell? Did Nvidia reach out to them?

    AMD have admitted they have problems, I don't think anyone is trying to pretend that there isn't any at all, but are they AS bad as the FCAT tools suggest? I don't know but im going to keep reading the reports and see what happens.

    Quote Originally Posted by Willzzz View Post
    But what are they holding back on? All 4 sites that have published FCAT data have gone into great detail explaining the process, how the data is collected and what the data means. I'd get your point if they just came up with a graph and didn't explain where it came from, but that isn't what has happened at all.

    I'm not sure what more you want them to do?
    The process, science and theory of it all is very good, there is no denying. (wih the exception of PCPerspective) I don't think they are necessarily, knowingly, holding anything back but the tools are Nvidia developed and they are the only ones around at the moment.
    Last edited by Biscuit; 01-04-2013 at 11:12 PM.

  16. #32
    Senior Member
    Join Date
    Jun 2012
    Posts
    2,401
    Thanks
    87
    Thanked
    151 times in 145 posts
    • Willzzz's system
      • Motherboard:
      • Gigabyte
      • CPU:
      • 4670K
      • PSU:
      • FD Newton R3 600W
      • Case:
      • Corsair 350D

    Re: GPU Testing/Reviewing techniques and Runt Frames with Crossfire

    I didn't just blindly accept what is being said, but when it comes with a very plausible technical explanation and is corroborated by multiple independent sources then I become reasonably convinced.

    I have no idea, but I'm imagining that PCPer started off investigating on their own and at some point Nvidia heard about it and offered to help.

    It's not like the FCAT results are particularly damning towards AMD cards. The data generally shows that AMD single card solutions perform very well, but Crossfire has big issues in certain titles. In ties in pretty well with what we had already seen from FRAPS and other sources.

    EDIT: Oh and it's not as if FCAT doesn't have some bad news for Nvidia either, the graph on Anandtech shows that the SLI scaling on Titan is downright awful for AC3.

    The reason I find FCAT convincing is because at it's heart it is tremendously simple. All it does is count how many lines have a certain colour in them, it's so simple it would be hard to cheat without being exceedingly obvious. We've seen stills from before FCAT that show the problem of runt frames etc. without the overlay even being present.
    Last edited by Willzzz; 01-04-2013 at 11:23 PM.

Page 2 of 4 FirstFirst 1234 LastLast

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •