Poe Probes > Reviewing Samples, Part 6: Q&A with Michael Crawford

This is the sixth in a series of articles about the practice of toy reviewers being given free samples for review, and whether that represents a problem for readers looking for honest assessments to make purchase decisions. You can find the other articles in the series here.

Today’s Q&A is with Michael Crawford, one of the best-known toy reviewers. His website is www.mwctoys.com.

1.) When you receive free samples, do you find an urge to be kinder to the item than you might be? If so, do you find yourself softening a bit, do you try to simply be fair, or do you think you end up trying to be even more objective than usual?

Michael Crawford: I do my best to be fair. Whether you pay for something or you get it for free, there can be bias. For example, I have found over the years that when someone pays a lot of their own hard-earned cash for a collectible, there is a natural bias toward wanting it to be good – if it isn’t, it implies you just spent a lot of money for crap, and people hate to admit that. I wrote an article on the general subject awhile back, since I find this sort of opinion bias interesting.

Since reviews are opinion, there’s always some bias working in one direction or the other, and I think the best the reviewer can do is to be aware of it. Awareness means you can look for it and understand it, and should make your reviews more fair.

That being said, it’s not just the fairness of the writer that can be effected by the sample, but the perception of that fairness by the readers. Ideally, a reviewer would never take free samples to avoid any perception that the reviews might not be fair, but the reality is that’s simply not feasible. There’s no one bankrolling toy reviews to allow them to afford $200 collectibles on a regular basis, which means that if they go with only the figures they can afford to buy, it will only be figures they like and want to begin with. What you’ve done is simply trade one bias for another. It also means there will be less variety in that reviewers subjects, limiting them to only the items they would buy for themselves.

I try to never ask for anything from anyone – if someone offers, I generally accept, but I don’t go looking. I also tell them up front that they should only be sending something that they really feel strongly about, because I will do my very best to be fair, and they might not like the results. At times I’ve even had the conversation telling them not to send me something in particular because I already am buying it – but that if there’s something else I would normally not pick up that they’re releasing, they should consider that instead.

2.) Do you ever worry when writing a review that being too negative will hurt your chances for more free samples?

No. The advertising on the site allows me to buy the collectibles that I want to review – if I end up with something additional that’s sent by the manufacturer, it’s nice, but not crucial. What the samples tend to afford me is a better variety of collectibles to review.

I have no idea if I’ve ever ended my chances of getting something free from a particular company through a review. I have been threatened with lawsuits, told by companies that I’d never be allowed to review their products again (not sure how they thought they were going to pull that off), and told by others that I had a personal vendetta against them. Generally though that’s been very rare, and companies have told me that they feel I’m fair and straightforward, and to me that reputation is more important than any free samples.

3.) Do you think that, in general, reviews of free sample toys on the Web are generally fair, or do they tend to be biased? Is this a problem for the toy collecting hobby, or a tempest in a teacup?

I don’t think there’s anything special about toys that makes this any different than big screen televisions or digital cameras or any other consumer goods. Toy reviewers get to keep the toy generally and television reviewers don’t (generally), but both could be effected by the concerns that a negative review could have on getting the next item to review.

I think what it comes down to are people have bias, and they come from more than just whether the toy was given to them or not. The best the reviewer can do is be aware of them and do their best to be fair. The best the reader can do is find a reviewer that they either agree with or disagree with most of the time – those are the reviewers that will help them make decisions.