Shooting Around - Groninger Museum by Niels Kim, on Flickr
Quite a while back, I got to wondering how toy collectors felt about reviews of toys that were provided as free review samples to reviewers. It’s a fairly common practice among most other industries such as movies, television, music, electronics, and videogames. However, the reviewers in those industries are almost always employed (or at least paid) by professional journalistic publications with reader bases that expect honest reviews, and therefore have a vested interest in being as objective as possible.
In most such cases, there is little concern that a bad review will prevent them from getting a review sample from the same company in the future, for three reasons. First off, a review publication should never count on review samples – it’s just bad business. Second, often the publication has enough clout that a publisher/manufacturer will want or even feel it has to provide a review sample. I’m thinking of your New York Times, your Rolling Stone, your CNET, your IGN; these places get review samples because companies know it’s the best way to publicize their product (and even a middling or bad review is often better than no one knowing your product exists).
Another advantage for the professional reviewer is the company is going to do the review regardless of whether they get a review sample or not, and if they have to buy the item themselves, the cost is expensed to the company, not the individual reviewer.
Finally, professional reviewers or review publications usually cover a wide range of product types and genres – Rolling Stone doesn’t review only hip hop albums, Andrew O’Hehir doesn’t solely review films about lesbian shepherds from Azerbaijan, and IGN reviews more than just first-person shooters.
The world of toy reviewing, on the other hand, is centered around individual hobbyists. The closest I think you can get to “professional” toy reviewers are high-volume reviewers like Michael Crawford, Pixel Dan and OAFE. But all of them tend to only review toys that interest them (or that they can afford), which is of course the problem with toy reviewing: because providing review samples is not a standardized practice in the toy industry, most reviewers are self-selecting items they want and, therefore, are more likely to like.
Michael Crawford receives samples from Hot Toys, while Pixel-Dan receives free early samples of Masters of the Universe Classics and DC Universe Classics, as does the Fwoosh and Action Figure Insider. (Given that – case in point – I don’t collect Transformers, G.I. Joe, Star Wars, or Marvel, I’m not sure what Hasbro’s practices are regarding review samples.) Unlike the professional situations described above, where there are editors to oversee the writing process and manage the relationship with the company providing the samples, these bloggers deal with the company as individuals.
Common sense suggests this sort of situation is likely to create more pressure for a positive review, because 1.) the reviewer has built a relationship with the person providing the samples (sometimes carefully and over time) and doesn’t want to hurt their feelings or jeopardize their professional relationship, or 2.) the reviewer doesn’t want to risk losing future free samples to a bad review – especially if they’re a big fan of the line they’re collecting, or the toys being provided are particularly expensive.
To get a sense of how the community – reviewers, readers, and toy companies alike – feel about this issue, I interviewed a number of people. The toy reviewers interviewed include Michael Crawford, Pixel Dan, VeeBee of The Fwoosh, Rob Bricken of Topless Robot, and Dan Pickett of Action Figure Insider. Over the course of the next two weeks, I’ll be posting their answers to these questions, and finally, I’ll post a round-up of reader responses to the issue.
You can find the articles in the series here as they’re posted.
SalemCrow
I write reviews on examiner.com for my local area, and have gotten WWE samples from Mattel's PR folks twice. Not all of it were figures I would collect, so I mentioned that in my reviews, but still admitted what I enjoyed about the products, and mentioned issues I saw with them.
It is a fine line, and I generally review what I buy, but I do my best to help people decide for themselves if it's something they feel they need to have in their own collections.
As for rolling in money from the reviews, I make a modest amount based on traffic on a monthly basis, but it's nowhere near enough to make a living. It's usually just enough to buy a new Joe or MU figure.
I look forward to reading the rest of these articles, great start.
Talyn
Ive only occasionally gotten review samples for ActionFigurePics.com, usually from Hasbro just before SDCC, but it happens rarely. But in those instances, there has never been any kind of push to review it one way or the other, I wouldnt think so highly of myself to think Hasbro's marketing department would give two thoughts to actually go out and try finding the review to check over after the fact, so it never really skewed what I was righting, or in my case, photographing.
PrfktTear
I used to write for a web site that relied upon free samples for nearly all of its content. I can recall being flat-out told to change a review and make it sound more positive. BS. No wonder why I don't write for them anymore.
Anyone who receives free product samples you walk a fine line between being a legitimate reviewer with integrity and giving them free PR.
JayC
Well first of all I can tell you their isn't this free gravy train of toys being sent out by companies to have their stuff reviewed. Hasbro's general policy is they don't send out review samples. Mattel sends out review samples to a select few. Other companies will send out stuff on occasion but certainly not on any kind of regular basis. 99% of toy reviews are done by fans who basically do it on their own dollar. Some if they have enough traffic on their site or youtube channel may make deals with etailers to get toys in exchange for advertising and some guys buy production samples from overseas sellers and review those. The real question IMO though is how often does a toy review really effect your decision to buy a toy? Look at MOTUC. Reviews showed Mattel basically put the Stinkor arms on backwards, yet did that stop people from buying it? Didn't seem so. Most these sites cater to a small number of Uber or diehard fans who are likely going to go out and buy the product they are a fan of despite what any reviewer says. I think sometimes if it is an actual advance review (IE Mattel stuff) people like to look at the pictures, but the number of minds that get changed by these reviews as far as if they are going to buy something or not I would say are minimal.
As for this article, Its a good idea in theory, but your interviewing basically all the guys who get the Mattel samples, and what do you expect them to say, oh yeah we have to lie to get free toys? And thats not to say they are necessarily lying, but if they did or felt they had to in order to get the toys, do you think they would say so in public????
Poe Ghostal
Well, first off, I'm interviewing more than just reviewers Mattel works with – I've got Michael Crawford, who doesn't get Mattel samples, and Josh of CollectionDX just agreed to do one as well. And Topless Robot certainly doesn't get free samples of MOTUC.
And as you'll see once the interviews start getting posted, many of the interviewees were quite honest about the difficulties they face when writing reviews of samples.
And I never suggested there was some "gravy train" of free toys – one of the points I was making is that because review samples aren't a common practice, the relationship between the companies and the people who get the review samples is closer (and arguably more precious and/or prestigious) and therefore potentially more problematic than it is for industries where review samples and professional reviewers are standard.
As for whether or not reviews make any difference to collectors' purchase decisions, it's a valid point but outside the scope of this series (and certainly has been discussed at length in the community before).
A.J.
"However, the reviewers in those industries are almost always employed (or at least paid) by professional journalistic publications"
Pay isn't even guaranteed, these days. Even then, you're bound to get more objective reviews for other industries. I really believe there's something about the toy collecting culture that leads to free review samples skewing opinion.
Mecha-Shiva
Poe should probe the Scalpers or toy swappers next,REAL HARD!
Newton Gimmick
This is a slippery topic to me, because nobody is going to admit to you that they are in the back pocket of a company. Why would they? They're riding the free toy gravy train. I know there's at least one reviewer that always gives glowing reviews to freebies. Their entire empire is built off of freebies To be fair, I'm not sure I've ever seen a "bad" review from this person, but you know, it sort of goes back into the same problem. If all you review is freebies, you never say anything bad.
It becomes less of a review and more of a game show, "Tell em about the product" segment from the Price is Right. Even when the figures are known to have problems. Thankfully it seems a lot of folks can see through these acts, but unfortunately, for every one who can see through it, you've got 25 who don't care and just want to look at stuff first.
I find this particularly annoying when the reviewer gets stuff that's not in their wheelhouse, doesn't get the traffic for it because the people that visit their site aren't interested in that and yet still gets it because of the notoriety. But whatever, I'm so far down the rabbit hole that I've got tons of stuff to review. It might not be a super exclusive three month early sneak peek, but thems the breaks.
There is something to me that adds an intrinsic value to a review. See, you can probably overlook a minor flaw when you're sent a figure for free. Didn't have to search for it. Didn't have to pay an inflated second market price. Didn't have to fight the fat guys at Target. But on the opposite side of that, if I buy it, through all the grief, then I'm going to point that flaw out. I spent money on this and that's always going to be the key difference.
I've been lucky enough to get some free samples and the stuff I've gotten has been pretty good, so I haven't had to complain much. But I've also taken some of the same companies to task for other products and been told that I wouldn't get anymore free samples as a result. I try to write a generally positive site, but if I can't complain when there are issues within a company, might as well hang it up.
Whatever it is. I assume the super hero costume I write reviews in.
Poe Ghostal
Great points, NG. I've only had a few free samples myself over the years and always tried to be objective. I can't think of a company that's offered me review samples on more than one occasion except small outfits like Spy Monkey Creations and Minimate Factory. (On the occasions I did receive samples, I often got more than one figure, but I can't recall being offered a second round of samples by the same company aside from the two aforementioned ones.)
One thing about reviewing items one has paid for, though – there's also what Michael Crawford refers to as the Emperor's New Clothes angle, whereby a reviewer who's paid quite a bit for something (or fought the man-child scalpers at Target and Walmart for it, etc.), might be inclined to be more positive about the review because the only alternative is to admit to themselves they wasted a lot of time and money on something that sucks.
I'd say of all the pitfalls of reviewing, that is probably the one I'm most on guard for in my own reviews.
Monkey boy
I feel like people who have to buy things will always see the flaws and potential bias of getting things for free, while people who receive freebies will think it removes the bias of having invested too much in the toy. In the end, what you’re really asking is “should someone spen money on this?” which I tend to think is an easier question to answer if you actually spent the money.
Tribsaint
I like the not in their wheelhouse point I'm a BIG OAFE fan the reviewers seem like the kind of people that shoot straight (Oafe is how I got here). YO's reviews have led me to buy toys I would not have , but I can't think of anything I haven't bought (heart wants what the heart wants). I make a point to to read every review from every reviewer . I'm a very specific collector I only collect 6-8 inch figures mostly movies and comic figures. But I still read the mini mates and the transformer reviews, the same way I read a lot of scientific articles but I teach History and English. Get outside your comfort zone!
Second point, you're talking about Pixel Dan call a spade a spade. Dan seems like a super likable guy no doubt . But I rarely watch his reviews, First off, if free toys always led to positive reviews Dan is the poster child example furry mascot of this theory. Second I'm not a fan of video reviews, no real reason , just a love for the written word. They review I have watch never talk about the issue or point out problems in the line (even when mention a fix) Nothing against Dan. It just seems to be the way it is and if there is a company that would stop the gravy train on even the hint of an ill word it would be the Matty mob. As stated I don't watch a lot of Dan's stuff so maybe I miss critical Pixel D.
ridureyu
This is a really tricky question, though I can't help but think that at least in some situations (example: Michael Crawford and Hot Toys), they don't have to corce to get good reviews, as those toys would be self-selected, anyway. And in at least that particular one, he does point out flaws when he finds them.
But yeah… the issue of objectivity is gonna be a tough one, and relies a lot on each individual reviewer. The folks at OAFE, at least, have consistently impressed me since I was in college.
Monkey boy
It’s true that Crawford does point out glaring flaws in his Hot Toys samples, but I can’t hell but wonder if he’s not a little biased by getting such high end stuff for free. I he does indeed get to do whatever he pleases after the review is written, he could be putting his kids through college by reselling those things. But perhaps he doesn’t get to hold on to the product or needs to agree not to. But it can be speculated that someone who gets so many high end things for free might be averse to criticizing too harshly. And indeed it can actually color our perception…how many people really know how good hot toys figures are beyond reading a review, seeing photos online or seeing the toys in comic shops? Most collectors can’t afford them, but when you start reading the forums of guys who collect the high end stuff, you see hot toys has kind of the opposite reputation you might expect from all the positive reviews. I know my own hot toys experience is mixed. Not that I’m trying to criticize mr Crawford, but it’s just a very interesting topic and I’m glad Poe is pursuing it.
jestergoblin
I think the big reason I love OAFE is the fact that they don't cherry pick a whole lot and review entire lines, faults and all. That's a big reason on my site I make sure I review every single Super Hero Squad set because I think context is very important when it comes to reviews. Toys need to be looked at both in a vacuum and in contrast of other current offerings.
Tribsaint
I think an interesting point is many other products that get reviewed (mostly entertainment), not providing free sample in itself is a bad review. It is often noted when press is not provided screening or demos of a new movie or album. This automatically means the film or music is "bad" and fans know it. The fact is that 99% of reviewers are fans first and reviewers second. Sometime I wish I could get review from a non fanboy, but that has problems too, will non fanboys have enough background knowledge to give my info that is going to sway me one way or another. The other problem for the industry is who do you provide samples to, anyone for 50 bucks a year and a weekend of time can have a decent looking toy site. SO who exactly gets sample could be an issue.
ANYWAY GREAT ARTICAL I CAN'T WAIT FOR THE NEXT ONE!
Kevin
Great idea, Poe! I've often wondered about this, myself. Can't wait to see who you probe next!
Poe Ghostal
Haha…yeah, I knew what I was getting into with that title.
Mario
Can't wait.
@rxchrisg
If this is going to be a continuing series,I think you should contact Josh Bernard from Collection DX. Awhile back there was this troll insisting that CDX gives positive reviews for toys they got free samples of,even though IIRC the toys in question weren't even samples. The one that really sticks out in my mind is a Hot Toys product (I think Batman?) that got a good review because-come on,it's HT,ALL their stuff is top notch-and this kid was flaming them in the comments about being paid off or some BS.
NoisyDvL5
As a delightedly unprofessional toy reviewer, the only thing that occasionally interests me when it comes to preview samples is the notoriety and traffic that having those first looks can provide.
IAT has featured some samples from smaller companies in the past and the articles performed tremendously well, whetting my appetite for samples from the big boys. I’d love to get IAT out to more people and I know that having the reviews before the product is released would help achieve that goal.
That said, I’ve only halfheartedly pursued samples a couple times over the years because I wouldn’t want to be obligated to give a glowing review (admittedly that’s all I mostly do since I only buy/review what I like). I’ve been fortunate enough to really enjoy the samples from the smaller companies and noted that in the reviews, but bigger companies miss the mark much more often and that would quickly get hairy I imagine.
Without getting into specifics, I can think of at least a couple figures where some of the previews seemed undeservedly positive, particularly later when contrasted with the traditional reviews. While I wouldn’t say figure X is awesome when it wasn’t, or downplay a negativity to keep getting a sample, I do have the tendency to gush when I enjoy something and I think that could be a blurry line for some of my readers.
It’s a tricky situation, but one I give a lot of thought and still never really know what to do. I’ve often wondered why our corner of the product review world doesn’t work like others and if it would be better if it did. I’ll be looking forward to all the articles in this series. Great idea for a column, Poe!
Monkey boy
This has always interested me as a toy reviewer. You gotta think with the amount of samples hot toys provides Michael Crawford, he can pay off his mortgage if he just sells off the stuff he reviews. Is there any stipulation about the resale of samples provided for review?
Reis O'Brien
I can only speak from my experience, but I know that About.com wouldn't allow us to accept any samples over $100 unless the sender could provide a return shipping label and we sent it back. They also made us offer to send back any samples under $100, which, if the sender says we can keep, we can keep. But under no circumstances could we use them in a give-away or whatever (I got in trouble for that). And they made a big deal about reviewers not re-selling their samples on eBay or whatever. But, again, that's just the rules I had to work under.
dayraven
i guess this is one of those times when being a pod pays off for me. i wouldn't cave to the offer of free toys. i wouldn't even consider altering my review in any way because i got something for free.
Reis O'Brien
This sounds like an awesome topic, Poe, and I'm greatly looking forward to future posts on the subject.
As a former so-called "professional reviewer" the About.com: Action Figures site (and boy, am I glad that gig is someone else's problem, now), I ran into the review-samples issue often. In many cases, companies would offer samples without comment or coercion. But there were times that I would get hints that positive reviews would result in more samples (one company in particular often said this) and in some cases additional gifts would be offered in return positive reviews, which I never gave in to.
There was another issue from the "editors" of About.com sort of badgering me about contacting toy companies and asking for samples, which I did a couple of times, but felt dirty about it, so I stopped that practice altogether. It didn't help that they often wanted me to review toys that I didn't feel were action figures (like Monster High dolls or Mega Bloks building sets). Another issue for me stemmed from how to review a sample of a particular toy that I just wasn't into, like Transformers. Was I supposed to feign fandom?
I also had personal issues with review samples, such as feeling guilty for not liking a toy after they went through the trouble of sending it to me, or feeling like people will think I'm in the company's pocket if I genuinely did like it.
I'm very interested in hearing what your interviewees have to say on the subject.
Fengschwing
I'm very much looking forwards to reading these. Nice work Poe.
One thing, if I may?
The asterisk in the article doesn't seem to correspond to any footnote. Not trying to nitpick!
Poe Ghostal
Haha, that was a consequence of this piece being rewritten three times. I was originally going to add a note saying that Duke Nukem Forever's bad reviews made me skeptical of Aliens: Colonial Marines, then realized Gearbox still probably sold more DNF units thanks to the reviews making people aware of it, even if the reviews were terrible. So I decided against including the footnote.
Cool and Collected
Excellent topic Poe! I've often wondered why the toy companies don't come knocking nearly as often as the book publishers–it just seems like good business to supply the uber-fan bloggers with free products for review. I can't wait to read the responses from the experts you interviewed as I read each and every one of them regularly.