Poe Probes > Reviewing Samples, Part 6: Q&A with Michael Crawford

This is the sixth in a series of articles about the practice of toy reviewers being given free samples for review, and whether that represents a problem for readers looking for honest assessments to make purchase decisions. You can find the other articles in the series here.

Today’s Q&A is with Michael Crawford, one of the best-known toy reviewers. His website is www.mwctoys.com.

1.) When you receive free samples, do you find an urge to be kinder to the item than you might be? If so, do you find yourself softening a bit, do you try to simply be fair, or do you think you end up trying to be even more objective than usual?

Michael Crawford: I do my best to be fair. Whether you pay for something or you get it for free, there can be bias. For example, I have found over the years that when someone pays a lot of their own hard-earned cash for a collectible, there is a natural bias toward wanting it to be good – if it isn’t, it implies you just spent a lot of money for crap, and people hate to admit that. I wrote an article on the general subject awhile back, since I find this sort of opinion bias interesting.

Since reviews are opinion, there’s always some bias working in one direction or the other, and I think the best the reviewer can do is to be aware of it. Awareness means you can look for it and understand it, and should make your reviews more fair.

That being said, it’s not just the fairness of the writer that can be effected by the sample, but the perception of that fairness by the readers. Ideally, a reviewer would never take free samples to avoid any perception that the reviews might not be fair, but the reality is that’s simply not feasible. There’s no one bankrolling toy reviews to allow them to afford $200 collectibles on a regular basis, which means that if they go with only the figures they can afford to buy, it will only be figures they like and want to begin with. What you’ve done is simply trade one bias for another. It also means there will be less variety in that reviewers subjects, limiting them to only the items they would buy for themselves.

I try to never ask for anything from anyone – if someone offers, I generally accept, but I don’t go looking. I also tell them up front that they should only be sending something that they really feel strongly about, because I will do my very best to be fair, and they might not like the results. At times I’ve even had the conversation telling them not to send me something in particular because I already am buying it – but that if there’s something else I would normally not pick up that they’re releasing, they should consider that instead.

2.) Do you ever worry when writing a review that being too negative will hurt your chances for more free samples?

No. The advertising on the site allows me to buy the collectibles that I want to review – if I end up with something additional that’s sent by the manufacturer, it’s nice, but not crucial. What the samples tend to afford me is a better variety of collectibles to review.

I have no idea if I’ve ever ended my chances of getting something free from a particular company through a review. I have been threatened with lawsuits, told by companies that I’d never be allowed to review their products again (not sure how they thought they were going to pull that off), and told by others that I had a personal vendetta against them. Generally though that’s been very rare, and companies have told me that they feel I’m fair and straightforward, and to me that reputation is more important than any free samples.

3.) Do you think that, in general, reviews of free sample toys on the Web are generally fair, or do they tend to be biased? Is this a problem for the toy collecting hobby, or a tempest in a teacup?

I don’t think there’s anything special about toys that makes this any different than big screen televisions or digital cameras or any other consumer goods. Toy reviewers get to keep the toy generally and television reviewers don’t (generally), but both could be effected by the concerns that a negative review could have on getting the next item to review.

I think what it comes down to are people have bias, and they come from more than just whether the toy was given to them or not. The best the reviewer can do is be aware of them and do their best to be fair. The best the reader can do is find a reviewer that they either agree with or disagree with most of the time – those are the reviewers that will help them make decisions.

Comments now closed (15)

  • This was actually the one I was most interested in as a reviewer and a reader. Michael has been the most well known toy review for the last decade, and while I notice that at the bottom of every review he says if they were purchased or given, it's never seemed to impact his review in a way that I noticed.

    But I think the point that all reviewers are biased is an important one. No one wants to read a review that lacks personality and ideally, you want some context on the toy. Personally, I don't collect the 6-inch or 3 3/4 inch scales anymore, so I know my thoughts on something like MOTU or Marvel Legends aren't something that carries as much weight. But when it comes to LEGO and Hasbro Heroes, I consider myself a decent authority on those subjects.

    I don't think anyone is going to be upfront and say that if the item is free, they'll review it better because the toy collecting sector isn't that big. What would be really interesting would be seeing how reviews for parents on toys for their kids turn out across mom blogs.

  • I rarely read MWC reviews, but the ones I have read didn't seem to gush on the toys although I don't remember paying much attention to wether or not they were samples. I agree with his view that all reviews are biased in some ways. How often do you read reviews by someone who isn't interested in the review subject?

    It's also normal for people to embellish how good a toy is when they spent a lot of money on it. I'm . There definitely is a line between something I enjoy and something that is crap, but I find itvery difficult to write about a bad purchase.

    I also agree with Jestergoblin that no reviewer is going to admit they're being kinder to a free sample.

  • Great interview. MWC reviews, along with OAFE is the only site I tend to trust on toy reviews.

  • mike is the gold standard (no offense to any of my peeps, but you all know it's true.) i've known poe and noisy in a virtual sense for many years now (i think we're rolling rapidly towards a decade, aren't we gents?) and you guys both know i love you guys. the fact of the matter is, mike crawford's toy review remains the temple and we're all pilgrims on our way up the mount. that dude is a stand up guy, and even when i've disagreed w/ him, i've respected his PoV, and i know i'm not the only geek that feels that way. he is generally very direct about whether or not he's gotten a sample, and he's generally quite observant and fair minded. you always feel, reading one of his reviews, that you're getting a best faith effort at an objective review… even w/ warts, it beats a "feature" every time out of the gate. 🙂 i like reviews, because they help me make informed buying choices.

    • OAFE gives me great info about the subject, and MWC gives technical details about the toy. I love them both, and my favorite is if both sites review something.

      I actually just recently bought the Silent Screamers Orlok, which I had wanted to pick up for about a decade THANKS TO THE OAFE REVIEW.

  • I 2nd Poe’s notion. Shocker probably flipped their s*** when he named them worst company.

  • I’ve always liked the “this product was purchased for this review” lines, but the fact that he’s told companies “please don’t send that – I have it ordered for myself already!” That’s just amazing. I definitely respect that.