This is the seventh in a series of articles about the practice of toy reviewers being given free samples for review, and whether that represents a problem for readers looking for honest assessments to make purchase decisions. You can find the other articles in the series here.
Today’s Q&A is with Justin of GeneralsJoes.com.
1.) When you receive free samples, do you find an urge to be kinder to the item than you might be? If so, do you find yourself softening a bit, do you try to simply be fair, or do you think you end up trying to be even more objective than usual?
Well, I don’t receive free samples often…generally what a mainstream event happens as the G.I. Joe: The Rise of Cobra film did in 2009, Hasbro will send out a healthy “care package” to fan sites, and I choose to review those items. They’re not necessarily items specifically for review, but that is generally what I do with them. I also have sponsorship deals and BigBadToyStore pretty much sponsors my review page by offering free product for those reviews. I have also done reviews for smaller companies like Marauder, Inc., Yetibrew Design, and other places that specialize in much smaller, more focused product lines.
I really don’t have an urge to be kinder to the item, no. I strongly believe that posting reviews of product is almost an obligation to be as objective as possible. Granted I’m a pretty big fan of the G.I. Joe product line, and that generally shines through in my reviews, but the source of the items I’m reviewing does not even come into mind. I try extremely hard to simply play it fair. But that is an interesting point…I wouldn’t be surprised if subconsciously I was trying hard to be a little too objective, keeping in mind that I don’t want to come across as soft just because I’m being given product, but I don’t that comes through in the reviews.
2.) Do you ever worry when writing a review that being too negative will hurt your chances for more free samples?
No, I really don’t. I think Hasbro and BBTS have a lot more pressing issues than scrutinizing review websites making sure they’re getting favorable reviews. The interesting conundrum actually comes across when I review items from some of the smaller companies that I mentioned. Generally what I will do there is that if I find a glaring issue with a product I’m reviewing, something I don’t like or don’t agree with, I will solicit feedback from the company. For example, if an item I buy from Marauder, Inc. (which specializes in modular weapons for 1:18 scale action figures) doesn’t meet my expectations, I’ll contact the owner and discuss it with him. If there are manufacturing problems or basic functionality problems, those can usually be explained, or I will at least get some feedback as to their thought processes behind a particular decision. I will certainly bring up the negative point in the review, but hopefully by soliciting feedback or explanation from the company, I can offer reasons why that decision was made.
I don’t see that as being negative, more analytical, but I also think the owner appreciates the chance to rationalize their decision and explain reasons for the design choice.
3.) Do you think that, in general, reviews of free sample toys on the Web are generally fair, or do they tend to be biased? Is this a problem for the toy collecting hobby, or a tempest in a teacup?
The whole review landscape is a funny place. I find that a lot of my readers are way more analytical about reviews than I am myself sometimes. I may give something four stars, and someone will comment that a toy I reviewed much more favorably three years ago only received three stars and will want an explanation. Or I’ll be asked to break down detail-by-detail why I decided on a particular choice for a rating number and how that compares to similar ratings to other products. By and large across the landscape, I think many folks who read reviews instantly think they are biased. In the feedback I’ve received and that I’ve seen, many folks (or at least the vocal ones) assume there are ulterior motives at work or that a reviewer is somehow on the payroll of the company. Then I see reviewers coming out and saying that they are absolutely not on the payroll and are totally independent entities that do not give favor to companies based on free samples.
I feel like the truth probably lies somewhere in the middle. I would say for certain that the majority of the high profile toy reviewers like Pixel Dan and certainly Michael Crawford do not give favorable ratings based on free product received. I feel pretty comfortable saying they tell it like they see it regardless if they spent their own money. But you are absolutely correct in assuming most of us toy reviewers review these specific products because we love the toys and we’re buying them for our collection, so by that very nature many reviewers could be seen as biased. I would assume I asked for Wave 1 of G.I. Joe: Retaliation toys because I think they look fun, right? So you would suspect the review would reflect that.
I do think this is an interesting conundrum that faces review sites in general, not just the toy focused ones. It is obviously a perceived concern with the Federal Trade Commission establishing specific guidelines about the practice. By and large I think the complaints are a bit overblown, and I suspect the majority of product reviews, toy-based or otherwise, are being honest with their readers and themselves. I don’t think free product samples really sway the reviewers as much as the audience might suspect.
After all, most of these people built their entire reputation on their objectivity and their balanced opinions, why risk their entire reputation based on getting some free product in the mail? If they are overly positive and ruin any credibility they might have, the site will get a reputation for that, and they would likely not be as relevant and not receive free product in the future anyway. It’s a no win situation.
Free samples are a very nice perk to being a product reviewer, I will not lie. That’s a big reason that I do what I do. But I strongly believe that free product does not go a long way to sway the words written within the review itself, and that overall longevity and relevance of the site is much more important in the long run than getting a few free figures in the mail.