Should We Trust the Stars?
- Research
- Studying & Teaching at TU Dortmund University
- Press Releases
Especially at Christmas time, online shopping is a popular way to purchase presents. Before people click to buy, they usually look at the reviews of other customers who have already bought the desired product or tested the desired service. But how meaningful is the number of stars that signify the average customer ratings? Should we blindly trust others’ assessments? Dr. Sarah Köcher and Dr. Sören Köcher from the Marketing Department of the Faculty of Economics at TU Dortmund University investigated these questions in a study. The research results were published in the current issue of the Journal of Marketing Behavior.
In order to establish a measure of “objective” product quality, the author team collected test results from Stiftung Warentest for a total of 2,473 electronic products in more than 300 product categories, which were tested between 2014 and 2017. In a next step, the test results were combined with the average ratings given to the products by Amazon customers. Ultimately, the scientists were able to compare the test results and ratings of 1,322 products in 224 categories – including televisions, smartphones and vacuum cleaners, for example.
The survey shows that average customer ratings are not necessarily a good measure of the “objective” quality of a product, although they are used by customers as a key criterion in product evaluation. “Only in 69 out of the 224 tested categories was the test winner at Stiftung Warentest also the product with the best average rating on Amazon. This means that you do not necessarily buy a good product if you choose a product with a high average rating,“ said Dr. Sarah Köcher. Overall, the online shoppers’ assessments were therefore hardly consistent with the results of Stiftung Warentest. The researchers also found that the longer a product is on the market, the less the correlation between evaluated and objective quality. With older products, there is no longer any demonstrable correlation between test ratings and average customer ratings.
Another result of the marketing study is that better customer ratings obfuscate the view of buyers on the actual quality of products: the higher the average rating, the less the purchasing decisions are influenced by the actual product quality.
“It is not just that average customer ratings are a rather poor indicator of the quality of a product,” said Dr. Sören Köcher. “To a certain extent, high average ratings even seem to obscure the actual quality of a product.”