Rating systems

Author

Ddraig

Frostfire Dragon|Retired lurker|FFF|Loved by RNG
Messages:
7,855
Likes:
22,460
Points:
457
Blog Posts:
33
A lot of the less homely bunch (read: elitist) always tend to have an expectation that someone's average ratings should be at the middle, aka 5/10. That has always bothered me as I cannot understand how can someone make such a dumb claim.

Let us take the example of anime, The average rating of someone should be 5/10 only when they are watching series at random - no rec systems, just plain old random amongst all available animes. Of course this just does not work as people do not watch series at random, people pick series that they expect to like - aka series they would be more likely to vote 7+ / 10.

Normalizing those to average out as 5/10 has problems. Your votes now correspond to how you rank the series only amongst the series you have watched. The un-normalized ratings on the contrary represent how you would rank the series against the whole list of animes - something much more meaningful. Normalized votes would also have to re-evaled every time you watch a new show.

A better rating scheme for an out-of-10 ratings for anyone interested is,
  • 5 and below for when you didn't want to even watch it because you know how garbage it would turn out
  • 6 when it just failed to live up to your expectations that you thought it would when you decided to watch/read it
  • 7 for the average and forgettable
  • 8 for the good-ish
  • 9 for the super great ones
  • 10 for the select few masterpiece
I think the user average under such ratings tends to be between 7 and 8.

But here is the thing, such a voting method requires more distinct levels in the upper half of the rating system. A vote out of 5 like on NU puts the user average at between 3.5 and 4. So now you are left with,
  • 3 mapping to (5) and (6) - aka garbage and a bit disappointing
  • 4 mapping to either [ (7) , (8) , and a subset of (9) ] - Anything average and above that is not masterpiece-tier.
  • 5 mapping to remaining (9) and (10) - Aka great and masterpieces.
There is also a small rant about people who treat rating systems as a binary system with only valid choices being 1 and 10 but let's just ignore those for this blog.

TL;DR - People do not understand that every thing you choose is an educated choice and a rating out-of-5 sucks more than rating out-of-10

Permalink
iampsyx and AMissingLinguist like this.

Comments

    1. Ddraig Apr 8, 2021
      @iampsyx the reason 1-4 usually goes empty is because you are selecting stuff to watch that looks decent at the very least. So usually even the worst disappointments amongst those never really deserve less than 5. They are essentially reserved for the stuff whose existence makes you question your sanity. 1-4 will only be filled if you get tricked into watching it tbh.

      I agree with your out-of-5 mapping but in practice I really dont feel like those (5) ~ (6) shows really deserve a (2). It kinda feels just unjust. Like yeah they were disappointing but they were not so bad that they deserve a 2/5.

      I view (1) as the opposite of (10), aka the literal bottom of the barrel.

      It is this feeling that makes rating harder, as you dont feel like giving them a 2 but if you give them a 3 you need to either bump up the average shows to 4 or say these below average shows are as good as average shows.
      iampsyx and AMissingLinguist like this.
    2. iampsyx Apr 8, 2021
      Your out-of-10 rating is the same as mine! blobmelt_thumbs

      But when we really think about it, in this kind of system, the 3 spots below 5 are kinda just useless, right? Aside from letting you vent out your dissatisfaction/annoyance/disgust/anger/etc.

      So in theory,
      1 should map to anything below (5) - disappointing and/or just awful
      2 mapping to (5) & (6) - forgettable and/or not the worst but still pretty bad
      3 mapping to (7) & (8) - average
      4 mapping to (9) - great but fell short of being a 10
      5 mapping to (10) - masterpiece and will go into your absolute favourite list

      Personally, I'd rather have an out-of-3 system (+ list of favourites and/or your personal top 10 list). 1 for bad, 2 for null, and 3 for good. And then instead of a global average score, you get shown instead how many people who rates like you scored the work.

      I think the point of rating systems is just to be able to tell if you will enjoy something before you spend any time on it. Hence, the only criteria should be "Did people who like the same things I do find this good?"

      Global average scores make you follow the majority instead of your personal ranking. I've found myself giving works a 4 here in NU even thought I actually think they're a 3.
      AMissingLinguist and Ddraig like this.
    3. AMissingLinguist Apr 7, 2021
      I give this blog a 3.5/5! That translates to a 7/10. :blobpopcorn_cool:
      Ddraig likes this.
  1. Sort Comments

    Sort Comments