- Incentivizing online reviews may not give you the types of reviews you want.
- Incentivized reviews don’t improve—and may even hurt—business.
New research focused on whether incentivizing would-be online reviewers to post reviews can make a difference for the seller has shed light on the role those reviews play in sales and revenue outcomes.
Researchers worked with Airbnb to conduct a large-scale and costly experiment to find out just how those incentivized reviews impact day-to-day marketing. They found that although incentivizing reviews does lead to an increased number of reviews, the induced reviews tend to be more negative in nature, and have no impact on the number of nights sold or total revenue.
The study published in the INFORMS journal Marketing Science, “Do Incentives to Review Help the Market? Evidence from a Field Experiment on Airbnb,” is authored by Andrey Fradkin of Boston University and David Holtz of the Haas School of Business, UC Berkeley, and the MIT Initiative on the Digital Economy.
The Airbnb experiment divided participants into two groups: treatment and control. The treatment group was sent an email offering them $25 Airbnb coupons in exchange for a review if they had not already posted a review within the typical eight or nine days after checkout. The control group was sent an email simply reminding them to leave a review with no financial incentive offered.
“We found that reviews induced by incentives were more likely to contain information about lower quality experiences,” Fradkin says. “We also found that when compared to non-incentivized reviews with the same star ratings, incentivized reviews corresponded to worse guest experiences. In other words, a seller may have received the same star rating from both study groups, but the incentivized guest was more likely to post a negative review, while upholding an inflated star rating.”
“While the incentivized review program impacted the number and type of reviews left by guests, it had neutral-to-negative impacts on the actual business outcomes of the platform and its sellers,” says Holtz. “Incentivized reviews did not affect the total quantity (nights) sold for study listings, but they did cause a change in the composition of those transactions. More to the point, incentivized listings led to more transactions, but those transactions lasted fewer nights on average. Thus, a poorer match.”
The study found that as a consequence, the revenue impacts of incentivized reviews did not represent either a positive or negative material change. Incentivized reviews failed to improve transaction quality, and according to some measures, resulted in worse matches. At the same time, the incentivized treatment did not lead to higher complaint rates or reduced likelihood of repeat business.
“Ultimately, we found that while reviews do play a meaningful role in the overall marketing efforts for online sellers, incentivizing those reviews may not deliver the results many would assume,” Fradkin says.
Read the full study.
About INFORMS and Marketing Science
Marketing Science is a premier peer-reviewed scholarly marketing journal focused on research using quantitative approaches to study all aspects of the interface between consumers and firms. It is published by INFORMS, the leading international association for operations research and analytics professionals. More information is available at www.informs.org or @informs.
This news release has been republished from the INFORMS News Room.