Poe Probes > Reviewing Samples, Part 1: Introduction

Shooting Around - Groninger Museum

Shooting Around - Groninger Museum by Niels Kim, on Flickr

Quite a while back, I got to wondering how toy collectors felt about reviews of toys that were provided as free review samples to reviewers. It’s a fairly common practice among most other industries such as movies, television, music, electronics, and videogames. However, the reviewers in those industries are almost always employed (or at least paid) by professional journalistic publications with reader bases that expect honest reviews, and therefore have a vested interest in being as objective as possible.

In most such cases, there is little concern that a bad review will prevent them from getting a review sample from the same company in the future, for three reasons. First off, a review publication should never count on review samples – it’s just bad business. Second, often the publication has enough clout that a publisher/manufacturer will want or even feel it has to provide a review sample. I’m thinking of your New York Times, your Rolling Stone, your CNET, your IGN; these places get review samples because companies know it’s the best way to publicize their product (and even a middling or bad review is often better than no one knowing your product exists).

Another advantage for the professional reviewer is the company is going to do the review regardless of whether they get a review sample or not, and if they have to buy the item themselves, the cost is expensed to the company, not the individual reviewer.

Finally, professional reviewers or review publications usually cover a wide range of product types and genres – Rolling Stone doesn’t review only hip hop albums, Andrew O’Hehir doesn’t solely review films about lesbian shepherds from Azerbaijan, and IGN reviews more than just first-person shooters.

The world of toy reviewing, on the other hand, is centered around individual hobbyists. The closest I think you can get to “professional” toy reviewers are high-volume reviewers like Michael CrawfordPixel Dan and OAFE. But all of them tend to only review toys that interest them (or that they can afford), which is of course the problem with toy reviewing: because providing review samples is not a standardized practice in the toy industry, most reviewers are self-selecting items they want and, therefore, are more likely to like.

Michael Crawford receives samples from Hot Toys, while Pixel-Dan receives free early samples of Masters of the Universe Classics and DC Universe Classics, as does the Fwoosh and Action Figure Insider. (Given that  – case in point – I don’t collect Transformers, G.I. Joe, Star Wars, or Marvel, I’m not sure what Hasbro’s practices are regarding review samples.) Unlike the professional situations described above, where there are editors to oversee the writing process and manage the relationship with the company providing the samples, these bloggers deal with the company as individuals.

Common sense suggests this sort of situation is likely to create more pressure for a positive review, because 1.) the reviewer has built a relationship with the person providing the samples (sometimes carefully and over time) and doesn’t want to hurt their feelings or jeopardize their professional relationship, or 2.) the reviewer doesn’t want to risk losing future free samples to a bad review – especially if they’re a big fan of the line they’re collecting, or the toys being provided are particularly expensive.

To get a sense of how the community – reviewers, readers, and toy companies alike – feel about this issue, I interviewed a number of people. The toy reviewers interviewed include Michael Crawford, Pixel Dan, VeeBee of The Fwoosh, Rob Bricken of Topless Robot, and Dan Pickett of Action Figure Insider. Over the course of the next two weeks, I’ll be posting their answers to these questions, and finally, I’ll post a round-up of reader responses to the issue.

You can find the articles in the series here as they’re posted.