View Poll Results: Can BETA-test sites be trusted to provide 100% independent and objective reviews ?

Voters
27. You may not vote on this poll
  • Yes

    8 29.63%
  • No - they are deluding themselves

    2 7.41%
  • No - they are deluding their readers

    4 14.81%
  • No - they are fooling themselves and their readers

    13 48.15%
Results 1 to 16 of 16

Thread: Can BETA-test sites be trusted to provide 100% independent and objective reviews ?

  1. #1
    DR
    DR is offline
    on ye old ship HEXUS DR's Avatar
    Join Date
    Jul 2003
    Location
    HEXUS HQ, Elstree
    Posts
    13,367
    Thanks
    1,014
    Thanked
    775 times in 352 posts

    Question Can BETA-test sites be trusted to provide 100% independent and objective reviews ?

    The purpose of BETA-testers is to specifically help manufacturers make products better.

    The principle of BETA testing is a good thing, and manufacturers should do more of it.

    However: -

    BETA-testers provide manufacturers with recommendations on how to make a product sell better, make more money and also how a manufacturer can save money.

    Do you entirely trust BETA-testers to provide you with 100% independent and objective comparative buying advice on the products they BETA-test?

    Do you entirely trust BETA-testers to provide you with 100% independent and objective comparative buying advice on the competing alternatives of products they BETA-test?

    Would it make a difference if the primary source of income for a publication came from the manufacturer for which it was BETA-testing?

    If you found out you’d been hoodwinked into believing that buying advice was independent and objective - when it was not - how would you feel?

    How would you feel about the ethics of a manufacturer which encouraged its BETA-testers to publish reviews and buying advice of its products, and its competitor’s products?

    This is a very important subject and your opinion is very important.

    It could well affect the honesty and quality of buying advice and editorial you read in the future.

    Before you post your thoughts, please vote in the poll above

  2. #2
    Administrator Moby-Dick's Avatar
    Join Date
    Jul 2003
    Location
    There's no place like ::1 (IPv6 version)
    Posts
    10,662
    Thanks
    53
    Thanked
    383 times in 313 posts
    good question , but what if tester were testing not only the product , but its competitors as well ,would that give a more objective opinion.

    should testing only be done by independant testers who have nothign to gain by quality of the test ?
    my Virtualisation Blog http://jfvi.co.uk Virtualisation Podcast http://vsoup.net

  3. #3
    Vive le pants! directhex's Avatar
    Join Date
    Jul 2003
    Location
    /dev/urandom
    Posts
    17,074
    Thanks
    228
    Thanked
    1,027 times in 678 posts
    • directhex's system
      • Motherboard:
      • MSI X99A Gaming 7
      • CPU:
      • Intel Core i7 5280k
      • Memory:
      • 32GiB ADATA DDR4
      • Storage:
      • Corsair Neutron XT 960GB
      • Graphics card(s):
      • MSI GTX 980 Gaming 4G Twin Frozr 5
      • PSU:
      • Corsair AX860i
      • Case:
      • NZXT H440
      • Operating System:
      • Ubuntu 17.10, Windows 10
      • Monitor(s):
      • Dell U2713HM
      • Internet:
      • FIOS
    you should never *review* beta hard/software. things are all too subject to change. but i don't see any harm in offering previews, based upon experience with a beta product

    what's important, though, is not to let the previewer also be the reviewer - it's far too easy to get emotive if the beta and final don't match, and slant scores.

  4. #4
    Administrator Moby-Dick's Avatar
    Join Date
    Jul 2003
    Location
    There's no place like ::1 (IPv6 version)
    Posts
    10,662
    Thanks
    53
    Thanked
    383 times in 313 posts
    If you seperate QA testing and public beta testing - then you might have the best of both worlds ?

    If the QA phase was carried out in house , you'd know that any informatino published as a result of that testing would potentially be slewed.
    Even if a beta tester had nothing to gain , I dont think they'd be able to sit on the fence when it came to attempting to improve 2 competing products. So reviews / advice based on that testing should carry informatino to that effect ?
    my Virtualisation Blog http://jfvi.co.uk Virtualisation Podcast http://vsoup.net

  5. #5
    Administrator Moby-Dick's Avatar
    Join Date
    Jul 2003
    Location
    There's no place like ::1 (IPv6 version)
    Posts
    10,662
    Thanks
    53
    Thanked
    383 times in 313 posts
    and would it make a difference if the primary source of income for a publication came from the manufacturer for which it was BETA-testing?
    I dont belive it would automatically discredit anything they produce - but you would have to bear it in mind.
    my Virtualisation Blog http://jfvi.co.uk Virtualisation Podcast http://vsoup.net

  6. #6
    Senior Member
    Join Date
    Jul 2003
    Location
    ZA ✈ UK
    Posts
    622
    Thanks
    0
    Thanked
    0 times in 0 posts
    *notes all the orange and red in this thread*

    It's a rather biased question to ask, from the phrasing used ("masquerade" has negative connetations, for example, as does the "hoodwinked" question) and the 3 no answers vs. 1 yes answer. But maybe that's just me.

    Either way, in my opinion, no, it's not okay. But there's a difference between [p]reviewing a beta product, and being a beta-tester. The former case is entirely allowable, I feel, so long as the reviewer has nothing to gain from doing so, but for giving his audience a look at what's to come.

    Of course, if a reviewer never told anyone about being a beta-tester, no one would ever know, in the same way that if a company pays off a reviewer, no one would ever know. In both cases, there's opportunity for bias and downright corruption (Of course, accepting money for a review is already pretty corrupt). And I'd be happier, as a reader, knowing that the opportunity doesn't exist in the first place.

    (Side note, I voted option 3 by accident, saw it was 3: 3 votes, and 4: 4 votes, clicked back, set 4, and tried again, this time with it being 3: 3 votes and 4: 5 votes. So not too sure if I voted twice, or if it was actually changed.)

  7. #7
    Furry Shorty's Avatar
    Join Date
    Jul 2003
    Location
    Manchester, UK
    Posts
    1,237
    Thanks
    1
    Thanked
    2 times in 2 posts
    I think it's too subjective question to actually select a poll option.

    A beta tester can still be a reviewer as long as they have the integrity and publish an honest review rather being slanted into publishing what makes the manufacturer happier.

    An astounding number of writers and reviewers of Windows 2003 server were beta-testers. That's the very nature of the product.

    The difference between the reviews & reviewers is those that felt they had to give a favourable review because it's Microsoft and they beta tested it.. and those who were completely honest and beta tested it.

    It's more down to the publications integrity & reviewer honesty than a hard & fast rule of yes/no.
    "In a world without walls and fences, who needs Windows and Gates?"

  8. #8
    Senior Member kalniel's Avatar
    Join Date
    Aug 2005
    Posts
    29,196
    Thanks
    1,520
    Thanked
    2,933 times in 2,376 posts
    • kalniel's system
      • Motherboard:
      • Gigabyte X58A UD3R rev 2
      • CPU:
      • Intel Xeon X5680
      • Memory:
      • 12gb DDR3 2000
      • Graphics card(s):
      • nVidia GTX 1060 6GB
      • PSU:
      • Seasonic 600W
      • Case:
      • Cooler Master HAF 912
      • Operating System:
      • Win 10 Pro x64
      • Monitor(s):
      • Dell U2311H
      • Internet:
      • O2 8mbps
    Since when was reviewing all about
    Quote Originally Posted by Hexus
    objective comparative buying advice
    ?

    Beta testers will have access to a product, and presumably know a lot about it, thus may be the most informed to give out information. I'm glad just to get the information in a review.

    Beta testers do not always work for the company of the product they are testing, and there is not always (or even usually in some cases) any incentive to falsely promote a product in a 'review'.

    Finally Hexus itself is happy to give opinions during reviews - why should everyone else have to be objective when even Hexus isn't? I'm glad for subjective reviews sometimes - it's obvious enough if someone isn't trustworthy and then it's just the case that no-one listens to their reviews.

    I'm disappointed hexus would chose to weigh such a discussion by using such language as 'masquerading as reviewers'. Just present to us the content, but also give us (factual) information about the reviewer so we can make our own minds up.
    Last edited by kalniel; 21-11-2005 at 03:06 PM.

  9. #9
    Senior Member
    Join Date
    Jul 2003
    Location
    ZA ✈ UK
    Posts
    622
    Thanks
    0
    Thanked
    0 times in 0 posts
    Shorty - You raise some good points, but the problem is establishing that a reviewer is honest and has integrity in the first place. Everyone represents themselves as being honest and accurate, since that's best for business, so as a reader, it becomes difficult to tell who really is honest, and who isn't.

    As I see it, there could be three categories here:
    1) Reviewers who say they don't/neglect to mention that they beta-test, but do
    2) Reviewers who say they do beta-test, but assure everyone that it won't affect reviews
    3) Reviewers who say they don't beta-test, and don't

    For me, category 2 is potentially susceptible to corruption, which unfortunately puts anyone in that category in a bad light, since I'll never be 100% certain that the review is objective. Of course, I can't really tell the difference between 1 and 3 unless something gives it away (As has been the case with one or two other review sites, in terms of paid reviews).

  10. #10
    Furry Shorty's Avatar
    Join Date
    Jul 2003
    Location
    Manchester, UK
    Posts
    1,237
    Thanks
    1
    Thanked
    2 times in 2 posts
    Eldren,

    I agree with you wholeheartedly. Unless you know the publications "people" personally, there is no real assurance of honesty. That goes for any publication on any subject. They are may seem sincere but.. you know how it goes.

    I still insist that whether someone beta tests a product is not a factor in whether they can fairly review it (im not a beta tester Id just to point out!). It's down to the person reviewing to provide an accurate analysis regardless of whether they tinkered with it in it's early days or now.
    "In a world without walls and fences, who needs Windows and Gates?"

  11. #11
    geo
    geo is offline
    Member
    Join Date
    Sep 2005
    Posts
    69
    Thanks
    0
    Thanked
    0 times in 0 posts
    Gosh, I've got the strongest feeling there is something y'all aren't telling us here. You didn't wake up this morning, stare at your cheerios, and "discover" this question. So, give --why this now?

    Also, "100%" objective seems a bit like putting the thumb on the scale to me to get an answer, and the options weighted as well.

    So, come on fellas, give out where you're coming from on this one. . .

  12. #12
    Senior Member kalniel's Avatar
    Join Date
    Aug 2005
    Posts
    29,196
    Thanks
    1,520
    Thanked
    2,933 times in 2,376 posts
    • kalniel's system
      • Motherboard:
      • Gigabyte X58A UD3R rev 2
      • CPU:
      • Intel Xeon X5680
      • Memory:
      • 12gb DDR3 2000
      • Graphics card(s):
      • nVidia GTX 1060 6GB
      • PSU:
      • Seasonic 600W
      • Case:
      • Cooler Master HAF 912
      • Operating System:
      • Win 10 Pro x64
      • Monitor(s):
      • Dell U2311H
      • Internet:
      • O2 8mbps
    Thanks PD. Appreciate it.

  13. #13
    Agent of the System ikonia's Avatar
    Join Date
    May 2004
    Location
    South West UK (Bath)
    Posts
    3,734
    Thanks
    39
    Thanked
    75 times in 56 posts
    I've found pretty much with every opensource / public beta style application, the beta testers are %100 objective and honest, to the point where I have seen beta's pulled and a whole product revision dropped and re-wrote.

    I would assume the same with companies like microsoft (I use microsoft as an easy example) simpley because by the time they do a "public" beta all the really serious problems should be gone, and kept internally, however if something bad is found I'd expect to read between the lines as companies will only release as positive spin on a major problem.

    I think people are pretty open with beta's though as a general rule of thumb
    It is Inevitable.....


  14. #14
    TiG
    TiG is offline
    Walk a mile in other peoples shoes...
    Join Date
    Jul 2003
    Location
    Questioning it all
    Posts
    6,213
    Thanks
    45
    Thanked
    48 times in 43 posts
    I'd agree that Beta testing is designed to make products better. BETA testing is only worthwhile if it comes EARLY in the process. IT has to be able to influence the product in development and as such the public should pay NO attention to the review, as it should have only a small amount of bearing on the retail product.

    Will it be interesting to read beta testing reviews on the latest ECS motherboard or ATI Graphics card. YES definitely. But if its being published to the world, then i would say it would definitely have a different tact to a direct report to the manufacturer.

    If Hexus did this and got paid for it - i wouldn't personally have a problem with it, but then i doubt if its the community that is most important in this decision.

    A big PRO or big Negative about a Beta review might create a storm?. (Nick's review of Pacific Fighters still makes me laugh)

    TiG
    -- Hexus Meets Rock! --

  15. #15
    Registered User
    Join Date
    Nov 2004
    Location
    Athens, Hellas
    Posts
    8
    Thanks
    0
    Thanked
    0 times in 0 posts
    As most of you know, until recently I was working for a Greek magazine as a hardware reviewer. I am also a beta tester for ATI, so I think I have a saying in the questions Dave presented (sorry for not noticing the thread earlier, but I was looking for something totally irrelevant and kinda stumbled upon this )

    The reasoning is simple really, and it is indifferent whether you beta test or not. It's all about your journalistic integrity. From my point of view, beta testing a product has a very good influence on your perspective review. The reason is plain simple: you have spent a great deal of time with a product, trying hard to break it, to expose its flaws. If you are a capable beta tester, you pretty much know what it can do and what it can't do. Do you get that kind of in-depth knowledge from a (perfectly capable otherwise) reviewer, who will only review a product, most times in a hurry, in order to meet a deadline? Will he have time to do an exhaustive review, and know where to look in order to have a better view of the product? The answer, in my opinion, is a 'No' on both accounts.

    Of course I am not saying that reviewers should beta test every hardware they review, that's not my point at all. But when reviewers review products they also beta test, their job is much easier exposing flaws than people who haven't got this much "breaking experience" with a said product. Now, there's the issue of trust, as Dave said. Well, my answer is pretty obvious. Not all journalists are beta testers, and not all beta testers are journalists. Journalists are judged by the readers with every review they publish - and that is a constant process; we are judged by what we write, what we say and what we do. If you see a journalist being biased, this probably means that he's not a good journalist in the first place, regardless whether he beta tests something he reviewed or not. If you take pride at what you do, and you want to be considered one of the best, you don't let either beta tests, personal friendships with PR Managers and any other relationships influence your judgement. And you try your best to be impartial. That's what journalism is all about really; and I believe our readers can tell impartiality when they see it.

    Just my two eurocents
    Last edited by Kombatant; 28-11-2005 at 12:15 AM.

  16. #16
    geo
    geo is offline
    Member
    Join Date
    Sep 2005
    Posts
    69
    Thanks
    0
    Thanked
    0 times in 0 posts
    Well, personally I wouldn't think the objectivity pressures are any worse than for the smaller sites that have to scuffle for cards and have the ever present fear that they won't see another if they don't give a positive review. So I think it comes down to integrity, buttressed (hopefully) by a strong site in the first place to help keep those pressures at bay.

    Tho I think if I was running the shop I'd have someone else do the review and just use the beta guy as a "resource" if questions come up.

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •