Review systems have been covered in novels and television (Black Mirror’s Nosedive covers a take on social media where review scores become a way of applying power).

Uber drivers have spoken about how four stars mean that they may lose money, or worse still their jobs. If you give an Uber driver four stars you are effectively saying to other users: don’t use this driver. Worse, Uber may cut payments to such a driver and ultimately stop them working if their score falls behind a pre-designated average. The point that Uber drivers are in danger is an average of 4.6 out of 5. I’ll just let that sink in for a minute.

If you do not give your Uber driver a 5 out of 5 you are effectively removing their livelihood.

 

With little understanding of how rating systems work, customers often use rating systems to punish providers. An example of that, close to home for me, is when customers have given 2-star reviews for valid, but small complaints that can potentially remove any potential sales from a digital product.

Here the customer seems to have two issues. One is the high price (£5) for the resource, which would have been known in advance of purchasing the item. The other points can be broken down into two issues with the resource:

  1. Not making sense
  2. Not being able to use the resource

On investigation, the task sheet had an extra page attached with two tasks on it which should have been removed (absolutely my fault). However, I am not clear why the consumer feels that they were not able to use the resource as there are 19 tasks (one is affected by an extra sheet attached, and I would claim still usable).

My reply to the customer offers to make things right (as they did not contact me before), but the outcome has been that the resource no longer sells. The income from this is now zero.

It is not down to me to say whether the punishment fits the crime, however two stars results in resources not selling. This is different to the official ratings guidence given by tes. When you give a review for a resource you are given the following diagram:

However, there is evidence that the use of ratings systems often reveals a bimodal distribution (although this is influenced by context, and the customer’s education and IQ). When customers are asked to offer detailed ratings (i.e. a review) the ratings offer a normal distribution.

There are no real answers to these review issues. However five star rating systems rarely work because that are not intuitive to users or meaningful in assessing performance and certainly not without more context. Netflix have retired their 5 point system (which also resulted in more reviews being submitted), and Amazon ask people for detailed reviews.

Time for a revamp from tes?