How much does the internet skew product "goodness"?

[Whiternoise]

Wondering what other peoples' opinions on this are, i'm talking about how if you're after a new laptop or a new dvd player, etc as web users, most of you will probably go onto google and search for "------- reviews". A lot of the user reviews up there will discuss the various generic problems that they all seem to have, for instance a laptop might have a problem with a latch not functioning properly. But how much salt should you pinch when reading them?

It is, i think, reasonable to assume that most of the people posting reviews are going to be spurred on either by having an issue with the device or it working perfectly. And as a result, lots of people write reviews saying that the latch is faulty and more people sign up and agree with them.

But i was interested in how many are simply the result of this perverse viral advertising (if you will) and how many are actually true. Say you want to buy the DVD player that everyone says has a screwed laser, should you take the advice of the 200 forum goers that moan about it, or would you assume that they're merely a small user base who are more tech savvy and more likely to complain? And is it simply because all the problems are compacted into one site (and thus appear to apply to every unit) but don't really reflect the hundred thousand products that the company have actually shipped?

[Funkstar]

It's a very interesting point.

people are more likely to complain about something than praise it. the few times I have actually taken interest in reviews of a product on a shopping site was when I was looking for my current laptop from NewEgg. There were some very possitive reviews, some negative points, but over all they were good and surprisingly well written and balanced.

Many glowing, gushy reviews are probably more likely to put me off than a few bad ones as it starts to look very artificial very quickly.

[bigblue]

Personally I try to seek a bit of balance. Anything and everything seems to have flaws, so I look for somewhere that the happy owners hang out! It's usually pretty easy, as most of the gear I buy is PC & AV stuff, so places like Hexus and AV Forums usually have plenty of posts and info from people who are happy with products.

If someone wasn't very net savvy, I imagine that simply Googling "... Review" would probably give them a very skewed perspective. Or even the fact that many of the big e-tailers seem to be putting customer reviews on now, so you get people putting negative reviews up out of sheer childishness (consoles), loyalty to a rival brand (Apple, MS, Sony, etc) or simply because they want to see their own text on the web, and they've probably never owned the product at all.

In fact, I think that the e-tailer reviews are probably the worst thing out there. Before any "hot" product is even released you've got people reviewing them, usually in text talk. Just idiotic.

[moogle]

I try to get website reviews mainly. Take a look at quite a few and compare. But if I see alot of user reviews going the same way ie positive or negative I take them into account too

[Whiternoise]

The obvious example to all this is Apple, take a look at a typical rumours/fan forum and it's a striking example of this.

Take for instance MacBook Pros, if you read the reviews of pretty much any dedicated hardware website, the ratings are generally very good - around 8-9/10. If you look on the subsection of an Apple forum, it will be inundated with users complaining about the whiny DVD drive or the hinge not sitting flush with their book or whatever the problem is. Do you take their word for it that the product is cack? Or do you go with the website reviews, Apples say so and the fact that they've sold half a million of them.

The problem is amplified by the fact that Apple still has a relatively small userbase in comparison to PC, so if a few people have what may well be an isolated problem, they're all likely to note it on the same web forum, thus making it seem like all the users there have it (of course this may be the case as they've signed up to try and sort it out).

Which is another issue, how many users sign up to web boards and review sites only to say the product is bad, purely because they had an issue with it and otherwise wouldn't.

(BTW, i use Apple as an example purely because they're well known for having a fanatic fan base, i personally don't have a problem with them - i don't mean to come across as a Win fanboy )

[TheAnimus]

The problem with a lot of the review sites is the people, often people give a first impressions review, then only ever bother to comment if its negative.

Thats not to say review sites aren't often a only to be a bit trusted. As always its the funding for these things thats the problem, and their desire to have future relationship.

[TooNice]

This is the reason it takes a lot of convincing for me to avoid certain hard disk brands. Loss of data is, rightly, something that gives a very bad impression, enough for most victim to be vocal about. And while hard drives I consider reliability to be the most important factor when it comes to hard drive, it is also the one factor I do not believe can be measured properly. I do use StorageReview's reliability database, but I suspect that even that is prone to sampling issues. When a problem reaches near epidemic proportions (DeathStar, some Maxtor models, the Samsung F1 ECC error) I'll pay close attention and avoid accordingly. I don't do not doubt that some drives may be more reliable than others, but without information on number sold, how many were accidentally dropped en route etc., it's difficult to determine if failure occurs due to inferior design/manufacturing or externalities outside the control of the company.

I also avoid being an early adopter. Products with no mechanical components tend to last for a long time if they survive first couple of months in my experience (not foolproof, but works well enough). I use reviews mainly for features/usability, as I do not think it is normally possible to measure good quality in a short period of time (only bad quality - when something falls apart).

[Stringent]

Nothing is perfect and you will always find people saying this or that isn't working. However if it appears to be a common trend then take note. Remember that people who find a fault are probably searching for it themselves on Google. Some random numbers to make a point, I'd say out of 100 people who bought a Sony KDL-40W4000 HDTV, 10 would find a problem and Google it, 1 would post a question saying "hey I have this problem" the rest would then follow. Perhaps then another 20 would search and find that thread, then go and LOOK hard at their TV to see if they have said problem. They probably wouldn't have noticed but now they look for it they get it and post, while about 10 post up saying theirs is fine. The other 60 would be happy with their purchase and not bother.

For me when I get an HDTV I am not going to go looking for problems unless I spot something!

[DR]

So there are several issues with review sites and user generated reviews - I will start with review sites.

Half of them don't actually know what they are talking about and tend to base their information on what others have written or indeed on what the PR company has told them about the products. You need to understand the product, the tech, the market positioning, and the USPs.

Also a fair few don't have the right kit to test the products with - you should see our labs it is a ridiculous place, we have every component you could possibly dream of needing to test with. Ironically I remember a few years back when dual graphics - SLI/Crossfire was first touted - and 30" panels were the only thing you should really push them with. We were the only publication in the UK to invest in 30" panels and such so whilst others could claim it was tested properly it wasn't.


User reviews are useful (and watch this space!) but there is a challenge, the user can often suffer from personal justification disorder, people don't like to slag off something which they have spent their hard earnt on. Also they don't necessarily have the comparable products in to compare against - i.e in isolation a lot of products can look 'great' but take a 40" TV - take the Sony, Samsung, LG, Panasonic - put them side by side that's the only way to compare etc.

We genuinely break our balls to 'get it right' and that's why we have done so well with growing - we have people engaged fulltime to sit and test - nothing else but build rigs up and test - we work to specific margins of error (very low!) and we make sure we do the right thing by you our readers.

We also try to bring in ways to give you MORE information to do an informed 'buy' so we our take, (often when possible) the manufacturers take a HEXUS.right2reply - your take a readers HEXUS.right2reply....

[Sinizter]

If there are 200 forum goers moaning about a screwed laser of whatever, it might be a small userbase, but its not a risk I am willing to take when handing over my money.

I would rather go for a product which has mostly good reviews, or no major moans.

Rather more people will complain about a product, and less will come and say how good it was. But since I have more or less done my research and decided on a certain product or two, goodness is now moot, but complaints are important as that is where any further problem will lie.

[DR]

If there are 200 forum goers moaning about a screwed laser of whatever, it might be a small userbase, but its not a risk I am willing to take when handing over my money.

I would rather go for a product which has mostly good reviews, or no major moans.

Rather more people will complain about a product, and less will come and say how good it was. But since I have more or less done my research and decided on a certain product or two, goodness is now moot, but complaints are important as that is where any further problem will lie.

I would prefer to see an issue raised and a company fix it rather than just positive throughout - customer service means more to me as a buyer

[Ferral]

I personally feel that when I am reading a review I want to hear about the bad stuff not just the good. Yeah you might have something that is absolutely superb with a few minor niggles. Thing is though, those few minor niggles might not seem so minor to everyone else.

Like David has just said, reviews should tell the manufacturer of faults / problems or whatever else so they can get on with fixing it.

Read an Alone in the Dark review this morning over at Eurogamer. Starts off really singing is praises with set pieces and visuals. Then it actually gets down to the nitty gritty of awkward controls and clunky type movement. No holds barred either, they told it straight like the HEXUS reviews do on what the issues are.

I like the pros and cons on the gamiong reviews here. Short pointers to look at which does help in making a decision on a purchase.

[badass]

We were the only publication in the UK to invest in 30" panels and such so whilst others could claim it was tested properly it wasn't.

My heart bleeds for your poor unfortunate reviewers........

[badass]

I would prefer to see an issue raised and a company fix it rather than just positive throughout - customer service means more to me as a buyer

Exactily. Many things have faults, often beyone the control of the manufacturer. However only a few will actuallt put effort into putting it right.
Take the PS2 for instance with the screwed laser. They lost a class action lawsuit in the US over it, but still charge people in the UK £40 to fix their manufacturing fault
That is of course not isolated so whenever anyone asks me for buying advice, I never recommend anything by Sony. Unless of course people are after a console that can play PS3 games.

[TheAnimus]

User reviews are useful (and watch this space!) but there is a challenge, the user can often suffer from personal justification disorder, people don't like to slag off something which they have spent their hard earnt on. Also they don't necessarily have the comparable products in to compare against - i.e in isolation a lot of products can look 'great' but take a 40" TV - take the Sony, Samsung, LG, Panasonic - put them side by side that's the only way to compare etc.

Very, very true.

The other intresting thing thou about user generated review content is that sometimes, thou all too rarely you get people who are real tried and tested industry authorities on the matter making them, rather than journos who just can't compete with their level of understanding.

I wounder weather user reviews, or professional reviews suffers from trends more thou? I've noticed that with many products that get good reviews lots of sites end up giving very similar reviews, people do tend to run with the bulls even in the day job world.

[Whiternoise]

So there are several issues with review sites and user generated reviews - I will start with review sites.

Half of them don't actually know what they are talking about and tend to base their information on what others have written or indeed on what the PR company has told them about the products. You need to understand the product, the tech, the market positioning, and the USPs.

Also a fair few don't have the right kit to test the products with - you should see our labs it is a ridiculous place, we have every component you could possibly dream of needing to test with. Ironically I remember a few years back when dual graphics - SLI/Crossfire was first touted - and 30" panels were the only thing you should really push them with. We were the only publication in the UK to invest in 30" panels and such so whilst others could claim it was tested properly it wasn't.


User reviews are useful (and watch this space!) but there is a challenge, the user can often suffer from personal justification disorder, people don't like to slag off something which they have spent their hard earnt on. Also they don't necessarily have the comparable products in to compare against - i.e in isolation a lot of products can look 'great' but take a 40" TV - take the Sony, Samsung, LG, Panasonic - put them side by side that's the only way to compare etc.

We genuinely break our balls to 'get it right' and that's why we have done so well with growing - we have people engaged fulltime to sit and test - nothing else but build rigs up and test - we work to specific margins of error (very low!) and we make sure we do the right thing by you our readers.

We also try to bring in ways to give you MORE information to do an informed 'buy' so we our take, (often when possible) the manufacturers take a HEXUS.right2reply - your take a readers HEXUS.right2reply....

Thanks for the reply, please note that i wasn't really aiming this at Hexus at all - more at the internet in general where the culture of user submitted reviews has sprung up. Hexus is good in the "customer service" department anyway, any issues and i can just flag it up on the forum and editors are always quick to respond to comments after the review.

The problem with a lot of the issues that users flag up, however, is that they're only really applicable to long term or as you might say "random" use. Hardware sites do their best to stress test, but - and i think i'm right in saying this - most have to return the product afterwards or do not follow up reviews later especially in cases like laptops where problems are generally a lot more serious than desktops. Hexus has been pretty good with this on the whole, the few "diary" style articles that have been posted were on the whole very good, and were definitely helpful.

Ultimately it's the testing environment which is missed, and too many official reviews concentrate purely on benchmarks - which, to their credit is what the average tech user wants to see - but it's not always the most important thing.

And i completely agree about customer service being the most important thing, take the Shure E2G headphones. I bought them after reading the Hexus review, and after a year ended up with the common problem of the wires wearing thin on the ear. Sent an email to Shure, posted the headphones and details and they sent me a new (and up to date, i might add) set of headphones within a month. That's proper customer service imo.