Technology Journalism

[TheAnimus]

Perhaps this should have been titled why I like Nick Farrel.

So last night I read in the London Lite a head to head review show down of Snow Leapord and Win7.

I was completely discusted by the very low grade reviewing going on.

For a start off the gentleman involved has obviously been at the apple kool aid, but also it seams at the MS kool aid.

It really wasn't worth the paper it was written on.

My favourate bit, was when he came to 'review' the touch capabilities.
He gave windows 7 a 9 out of 10. OSX 8 out of 10.

Now wait a damn minuite. OSX deserves, at most 1, for having gesture touch pads, but windows 7 is in no way a 9.

They have got too many small hit boxes for full finger use, it clearly is needing another revsion (windows 7 touch edition perhaps?) before it is worthy of been 8 or higher, when compared with what is going on in the MID and mobile phone space.

Its utterly discusting to read an article that just throws praze at a company whilst ignoring the fact a lot of them are not new, and exisited in previous versions, and not mentioning any critical points on the mac one, and not nearly enough on the win one.

utter tosh. If i could remeber his name i'd put it here so the google ranking of hexus give some sway to a debate on him not been fit to write for a blog post.

[Phage]

I saw that. But...you know...it's the Lite. Lite on details, lite on facts, lite on quality.

[TheAnimus]

Okay the person not fit to be a technology journalist is Duncan Graham-Rowe
(mod wanna edit that into my title?)

And the dispare is:
http://e-edition.thelondonlite.co.uk/2009/09/10/15.html

you have to have flash installed, or it won't work, which is nice for those of us who are, i don't know on a proper 64bit OS say......

I mean giving a company points on unconfirmed rumours, of an up coming product that is out of the scope of the review............

[GheeTsar]

I've heard both London freesheets are going under soon anyway.

[tiggerai]

TheLondonPaper is *sniff*

[TheAnimus]

oh well, just have to find something else to shoo the prolites away with.

This rubbishrubbishrubbishrubbish also writes for the Grauniad so no surprise there. Luckily thanks to PPP squandering from our prudent mr brown, odds are teachers won't be getting a pay rise for many many years, so perhaps they won't be able to afford that rag and it will go bust.

[Saracen]

Well, that's the technology journalism market for you. There are people that know what they're doing, and people that don't. The trouble is, it used to be a field occupied largely by professionals, a high percentage of whom knew what they were doing. The internet has seen that change, and now everybody that can use a PC and string half a dozen words together thinks they're a technology journalist, and most publishers have people wanting work dropping out of the skies .... and often offering to work for peanuts, or even free ".... to get a foot in the door".

Well, you pay peanuts, you get monkeys.

And it gets worse. Because there's so many people offering to do stuff on the cheap, the perfectly understandable temptation for editors (with pressure from management) is to cut costs, and especially in hard economic times, it's hard to justify paying more for an article when you can get it for less.

So .... be selective what publications (be it print or net) you rely on, because the articles are only as good as the people that wrote them .... and, only as good as even those that do know what they're doing can afford to spend on them.

Of course, the above is only my personal opinion, but it's an opinion based on nearly 20 years as a technology journalist, several thousand articles published in a good proportion of UK computer press, photographic press, national newspapers, major tech websites and international magazines.

My expectation would be that technology "reviews" in free press are about one step ahead of many of those on the internet. And how good are ones on the 'net? Some good, some (like HEXUS of course) very good .... and some are absolutely dire.

But anyone with a bit if technical expertise can set up a website of sorts in a very short period, write "reviews" and publish them. You, the reader/netizen have to work out if the writer knows what he's talking about. I could (and have) set up websites. So, if you were to read my site, how do you know if I'm a precocious 14-year old who might well be technically good but have very little experience of other products to relate to, or if, as is the case, I started my computer experience with IBM mainframes in the 60's, was programming analog computers in the '70s, spent some weeks at the factory of a major US manufacturer in the early-mid 80s learning about these new-fangled PC things, and then been writing about them since the early 90's.

But how would you know that, unless I tell you?

If you're going to trust a review, you have to feel you can trust the competence and integrity of whoever wrote it, and you trust a publication if they generally use writers you can trust. Personally, I'd put about as much faith in a tech review in a free or local rag as I'd put in a surgeon that got his training via a medical 101 correspondence course from Ulan Bator, or getting car tuning advice from Gardener's Weekly.


Note - I have no idea who wrote this specific article, and no idea if he knows his stuff or not, and haven't read the article. And I'm not likely to. You may be right, TheAnimus, or not, and the article may be great or garbage .... I've no idea.

One last point .... I've seen enough garbage written by so-called experts in magazines that really should know better, so I'm not about to start exploring the free press (etc) for specialised material.

[Lucio]

I always wondered at how a review score really helped things. After all what something is worth to a person, might be completely different to another. A proper review should provide quantifiable facts, which can be compared to other, similiar products, along with some feedback on how the product performs it's intended functions.

For example, Windows 7 can be reviewed and compared to OSX10.6, but given the two different design philosophies, how you can say one is quantifiably better than the other escapes me.

[Saracen]

I always wondered at how a review score really helped things. After all what something is worth to a person, might be completely different to another. A proper review should provide quantifiable facts, which can be compared to other, similiar products, along with some feedback on how the product performs it's intended functions.

Perhaps it should provide quantifiable facts, and perhaps it sometimes does, but there's several factors that go into the mix.

For a start, how long is the review? It's not quite as bad on the net, but in print, you get a word count with the commission, and you can't go much over it. If you do go over it (by more than a small amount), all that will happen is you force an editor to trim it to the required word count, because only so much space is available, and you can only fit so many words into it. So, in print, a large part of the art of writing a review is deciding which points, good or bad, are important enough to justify inclusion.

And, obviously, you need to strike a balance. First, you need an introduction, a hook to get people reading. Then you need to cover the major features, new aspects and/or benefits. Then there's any signification limitations or problems. But what feature or problem is important to one reader might not be to another, so clearly, you need to understand your reader demographics, and write accordingly. That might well be part of the problem TheAnimus encountered ...... what type of reader is a tech article in that type of mag aimed at? It won't, typically, be the type of reader that will belong to a site like Hexus. Pretty much by definition, if you're a member here, you're either more technically aware, or at least more technically interested, than the majority of Joe Public.

You can't write a decent review unless you understand who you're writing it for, and an article on a given subject will be written very differently if it's for a national newspaper compared to a technical mag or site. And a technical reader reading the generalised national newspaper review almost certainly is going to find it superficial - if it wasn't, if it was filled with technical detail, non-techies reading it wouldn't understand it and if you do that too often, you don't get any more commissions from that newspaper as you aren't writing what they need.

So what features do you cover? What level of technical detail do you include? You have to try to pitch it at a level targeting the average reader of that publication, and by definition, that means it won't suit all readers. Do you, for instance, cover the basic levels of usability, of mouse gestures, or do you cover the differences in memory mapping or how drivers interacting with low levels does or doesn't enhance security or safety? Answer .... it depends on the reader you're targetting.

I said you need an introduction, and obviously, you need a conclusion. In between, you have to try to give a flavour of everything else. If a typical product review is 600-900 words, you have to be really careful which points you chose to cover, because you need to give a flavour of the whole product, but can't possibly clinically dissect it. You've only got space for, at most, about half a dozen real points, and maybe a problem or two.

Next, style. For a mainstream publication, you need a fairly relaxed, conversational style. It can't be too clipped, and it also can't make too many assumptions about what the reader will understand without you spelling it out. You have to be careful of anything but basic acronyms, too, because you can't assume your reader knows what you're talking about. These days, he probably understands RAM or CPU, but if you talk about (to pick a few random DB terms) Codd's Law, or referential integrity, or primary keys, you're going to lose the average non-tech reader .... and turn them off.

So now, you're trying to whittle a review down to a handful of key points, but to provide a balance between good and bad, while proving an overall flavour of the product, and a conclusion. This is why you have to decide if you trust the judgement, and competence, of the reviewer.

Oh .... and by the way, how much does the reviewer get paid? Well, generally, it depends on word count. Obviously, the longer the article, the bigger the payment. No reviewer, for example, could spend a week researching a product that's 800 words long. If they tried, and were doing this for a living, they'd starve. So word count not only determines how much you can say about the product, but also determines how much time it is practical for you to spend on it. If you get a good reviewer, an experienced one, one with a strong understanding and plenty of background knowledge of previous versions and of the competition, you'll get someone that can pick the salient points out of product testing relatively quickly. If you get an inexperienced newb that thinks he's a tech journalist just because he has a PC and O-Level English, you'll be lucky to get anything much more insightful than a rehashed press release ... and Gawd knows, we see enough of those dressed up as "reviews". And even if you do get a conscientious newb that spends a week writing an 800-word review, he won't do it very often before he gets jaded, and starts to cut corners, because he works out he'd earn more stacking shelves in Tesco that he does writing reviews that way.

If you want good quality technical journalism, you need to be prepared to pay for it. And these days, increasingly, few people are, because there's so much "free" on the web. Sure there is, and some of it is even good. But a lot isn't. The same is true of free and/or restricted circulation papers, and it's even true of some supposedly professional publications.

[Saracen]

I always wondered at how a review score really helped things. After all what something is worth to a person, might be completely different to another. A proper review should provide quantifiable facts, which can be compared to other, similiar products, along with some feedback on how the product performs it's intended functions.

For example, Windows 7 can be reviewed and compared to OSX10.6, but given the two different design philosophies, how you can say one is quantifiably better than the other escapes me.

It depends on the publication, and on the journalist. In my experience, it varies a LOT.

Some "scores" will be little more than finger-in-the-wind gut feel by the reviewer. Again, with someone that knows their stuff, they've got experience to back that up, and might be applying some kind of consistency. If you're very lucky, they'll be doing it relative to products they know, even if the didn't review them, as well.

But at the other end of the spectrum, some publications send out a spreadsheet that needs to be completed for each product, especially for Lab tests. A score might well be based on a mix of a point-by-point feature comparison, and a subjective assessment of certain key features. It might include elements of subjective assessments of usability, points gained or lost for the presence or absence (compared to competing products) of features and an element based on benchmark performance. Clearly, you can compare features on similar products, and some products you can benchmark while others you can't. You clearly can't directly compare benchmarks between different types of product, though, so how do you decide if a hard disk gets 8/10 or 9/10, compared to whether a monitor, or a spreadsheet program, does?

Well, either you rely on the experience of the reviewer, or you have guidelines about philosophy, whereby getting 9/10 probably indicates best of breed, and ought to be pretty rare, and damn hard to achieve. Is ANY product ever a 10? I doubt it, if we're being realistic. Is anything ever perfect?

I mentioned filling in a spreadsheet for some publishers. You'll often see feature tables published, by I've regularly seen articles where the tech details published is a tiny proportion of the contents of that spreadsheet. You'll often see benchmark results, but again, probably not the fine detail, very likely not the full testing methodology (as nobody wants to give their competitors a template), and certainly not the comparative data that may go into final decisions about scores.

If you give a monitor 5-star, Editor's Choice, Best-Of-Breed award, it may well be the result not only of very good marks in all relevant areas for that monitor, but also in comparison to historical data on other products. And then you have to put all that in context of price. A 5-star monitor at £150 might well be 5-star at that price, but is (one hopes) seriously poor compared to one at £1000.

Again, all these factors come back to whether you, as a reader, do or don't trust the expertise of the reviewer, and/or publication, you're reading. And if you don't, why read it? And how can you be sure that anyone else is any better?

If you want good material, trust professionals and don't expect to get it if you pay peanuts.