It is some time since I started being interesting in evaluating and in improving the “quality” of information that review and rating sites make available to their users. The word “quality” may be an indefinite buzz word, but behind it I imagine some less abstract concept like “usefulness” and “persuasiveness”, and some preliminary research questions as: “how much impact has that information on users when they take a decision?” or “how can I help the user in exploiting at the best the online reviews data?”. Thousands of reviews (despite ordered by some importance criterion) still force the user to a peer reading, and it may not be uncommon that users read what they already expect to read.
A part from my thoughts, I have recently stepped into a paper that brings light to the topic. “Do online reviews matter? An empirical investigation of panel data” by W. Duan, B. Gu and A. B. Whinston.
It is an interesting study that examines the persuasive effect and the awareness effect of online reviews on movie’s box office revenues. Persuasiveness is the quality of making someone believes to do something (by giving him good reasons to do it), in this case, in buying a movie ticket. Awareness is the quality of being informed of something, in this case, of the existence of a movie. The study’s outcome is, in the very synthesis, surprising. On line reviews have no effect on box office revenues; their pervasive efficacy is almost zero. On the contrary the volume of online posting, which makes more users be aware of the existence of a movie, clearly has an impact on box office revenues. This result has also another interesting consequence; the word-of-mouth (which understands the production of online reviews) seems to have more importance than the quality of the reviews themselves in moving users’ decision to purchase a ticket.
It is difficult, for me, to understand whether the efficacy of the word-of-mouth versus the quality of the information is something inherent to the human way of coping with information or it is not. But and limited to the movie context, the paper shows that the information brought by online reviews is currently not useful as we would like. I see here very promising opportunities for future work, and we researcher should try to investigate further into this direction.