In my technical reports for school, we always state the issues at hand first. I'll follow that format, since it makes sense for this article too. The big problem is that people who are completely unqualified to make judgements on things are the ones supplying information for the masses. I don't have a problem with someone who isn't an engineer writing and posting his thoughts to the web. But when they start "testing" and getting "results," and using those to determine if you should spend your hard earned money on a certain product – well, that’s what I have a problem with. In too many cases, the results are not acceptable! Either the benchmarks that were used are not indicative of the real performance, they were not set up right to reduce variables, or the writers were not able to make sense of what results came out. I'll give some general examples:
· For testing water blocks, or heatsinks, many "reviewers" use Motherboard Monitor, or other software which reads from a socket diode, or an on chip diode for temperature results. Why is this so bad? Because neither is remotely accurate. You are adding variables to the testing, not removing them.
· Another one is for say, memory being tested, and its performance is decided good or bad based on SiSoftware SANDRA buffered memory tests, or PCMark2002 memory tests, both of which are synthetic, and have absolutely no bearing on the real world. Now, these numbers can show something; however, most reviewers are just as uninformed as a large number of readers, and cannot begin to explain what it is that is causing these results. Just because a processor scores higher in FPU performance, as opposed to INT based calculations, what does that mean?
This brings me to my next point: a complete lack of technical education in many articles and reviews. If you can't explain why something is better or worse to others who might be spending their hard earned cash on what you recommend, you shouldn't be doing so in the first place. Now, I have a degree (disclaimer, I *will* have, in April) in a field that allows me in depth studying of computer architecture, not everyone is interested in pursuing that. However, there is more than enough free information around the internet to educate any willing searcher properly. This goes for those who read the articles as well: if you don't understand something, go somewhere else to find out. Monetary decisions shouldn't be based on what Tom, Dick, or Harry may have to say in their piece of web real estate.
Some of the biggest contributors to this problem are the hardware manufacturers themselves. Why do they insist on sending hardware to those who have no ability to do a conclusive analysis of it? Because it's essentially free advertising. Look at any number of reviews out there. Notice how few “bad” or “not recommended” reviews are out there? Why is this? Those who are getting hardware are afraid to say such things, for fear of losing the goodwill of those who send them stuff for free. I can't say I don't understand the fear, because the idea is a bit scary. However, I find that I have a responsibility to those who read what I have to say, and shouldn't be playing nice based on getting more stuff for free. This means that I have a responsibility, as a reviewer, to both the reader, who’s looking for knowledgeable advice, and to the manufacturer, who’s looking to see how their offering measures up in an unbiased setting. Every piece of hardware should get a fair review, showing its plusses and negatives in an equal light, with technical data and reproducible, reliable results to back that up. If a site can not show it is capable of both of those, then they should be forced to buy their own hardware for review until they can. Many sites do currently, or have done this. It's the transgressions of the other, and those who are giving out product and who do not put the product in the right hands, that set me off on rants like this.
Next: What Testing Methodology Should Be >>
More Opinions Articles
More By DMOS
| Recommended by Dev Hardware |
|---|
|
