Novel Updates Ratings are misleading.

Discussion in 'Novel Updates Site Discussion' started by Kourinn, Aug 4, 2019.

Tags:
  1. Kourinn

    Kourinn Active Member

    Joined:
    Apr 15, 2019
    Messages:
    2
    Likes Received:
    9
    Reading List:
    Link
    I find the Novel Updates ratings to be quite misleading, but attempting to fix this may not be "fair" to older novels.

    Although novels are rated on a 5 star scale, the lowest possible rating is 1 star, thus converting stars to a 0%-100% scale, it's effectively a 4 star scale. The formula for converting star-rating to percentage-rating is: (#stars - #lowest-rating) / (#highest-rating - #lowest-rating) = (#stars - 1) / 4

    Thus giving the following breakdown:
    1 star: 0%
    2 stars: 25%
    3 stars: 50%
    4 stars: 75%
    5 stars: 100%

    Personally, I find this range too narrow to accurately rate novels I've read. In my opinion, 75% and 100% are huge differences. However, just allowing 0 star reviews and converting star count while keeping the same % ignores human bias. From a glance, the ratings appear to be a 5 star (20%) scale, thus someone with a 90% review might mistakenly rate 4 stars, even though 90% is closer to 100% than 75%. Additionally, some reviewers who hate a genre or trope will simply give a 1 star review without reading, so some rating bloat (pretending 4 star scale is 5 stars) is arguably beneficial, but I think it's currently too much.

    From analyzing the 25 all-time most popular novels (42k reviews), the breakdown is:
    1 star: 8.42% (3512)
    2 star: 4.97% (2075)
    3 star: 8.23% (3431)
    4 star: 13.14% (5482)
    5 star: 65.24% (27214)

    Assuming 75% avg bell curve (used in US academics), 4 stars and below should make up 50% of ratings, with 5 stars making up other 50%. The skewed result may be due to sample bias (only analyzing the 25 most popular novels), but the score lowering biases written in the previous paragraph may have only small effect.

    Possible solutions are:

    1. Switch to a true 5 star rating system adjusting with old reviews being scaled by percentage. Optionally, reduce 0 star troll reviews by requiring written review if giving 0 stars.

    2. Keep 1 star lowest, but double scale from 5 stars (effectively 4) to 10 stars (effectively 9), to improve score accuracy.

    3. Do not change scale but display average percentage next to average star score (ie: 3.9 (73%), 4.4 (85%), 3.5 (63%)).

    Anyways that's my thoughts on the current novel rating system. Oh and if we could sort by average rating, not just popularity/activity, that'd be great. Thanks for reading. o/
     
  2. oblueknighto

    oblueknighto Blue Person

    Joined:
    Oct 20, 2015
    Messages:
    3,197
    Likes Received:
    2,237
    Reading List:
    Link
    There's no good solution. This has already been brought up many many times on NU and on pretty much any system where people rate anything.
     
  3. otaku31

    otaku31 Well-Known Member

    Joined:
    Nov 26, 2015
    Messages:
    6,603
    Likes Received:
    26,150
    Reading List:
    Link
    OP's thread title I somewhat agree with. Whatever OP said afterward, I didn't fully read or understand (and don't really care to, TBH). :blobdizzy:
     
    Last edited: Aug 4, 2019
  4. Freezy

    Freezy Well-Known Member

    Joined:
    Jan 26, 2017
    Messages:
    253
    Likes Received:
    346
    Reading List:
    Link
    Ughh, trying to create a perfect rating system is like trying to create a new colour, wishful thinking and nothing more. Not to mention the ranking system can be filtered by activity and popularity, both of which moot the rating system not being totally accurate.
     
  5. StuffedDuck

    StuffedDuck Stuffed with Fluff

    Joined:
    Nov 20, 2016
    Messages:
    1,775
    Likes Received:
    1,256
    Reading List:
    Link
    ... or

    ... maybe there are just 27000 amazing novels featured on NU

    ... maybe we (the readers) are so brain dead by now that even a shopping receipt gets a 5 star rating
     
    whitespade and Devil of Boredom like this.
  6. Robbini

    Robbini Logical? Illogical? Random? Or Just Unique?

    Joined:
    Oct 20, 2015
    Messages:
    2,886
    Likes Received:
    1,749
    Reading List:
    Link
    To be fair, there's also a decent chance of people purposefully downvoting something they've never read, whereas there likely are fewer that would upvote something they've never read unless for fun.

    So that would maybe need to be taken into consideration as well.
     
  7. onnyfp

    onnyfp Well-Known Member

    Joined:
    Mar 10, 2016
    Messages:
    554
    Likes Received:
    607
    Reading List:
    Link
    Oh damn. Math. You work hard play hard aren't you?
     
  8. Ddraig

    Ddraig Frostfire Dragon|Retired lurker|FFF|Loved by RNG

    Joined:
    Apr 6, 2016
    Messages:
    7,853
    Likes Received:
    22,455
    Reading List:
    Link
    The best solution is to pretend that ratings dont exist. Seriously. Ratings and ranking list on NU are trash and you will be better of ignoring them.
     
    Toralk likes this.
  9. Warlock Of The Void

    Warlock Of The Void Well-Known Member

    Joined:
    Apr 29, 2017
    Messages:
    717
    Likes Received:
    506
    Reading List:
    Link
    Well , while I do agree with u , rating at it's core is a SUBJECTIVE opinion which means that it will always be eschewed no matter what . U can't expect people to offer a rational opinion for anything really , there will always be a bias present that they themselves might not be aware of . On paper it seems possible but that's the thing On Paper , it doesn't mean it's possible. Maybe u will get 10% of actual unbiased review but even that is an optimistic hope . Hell , I can even prove it , for eg. Me , I don't really put reviews much and even when I do I try to stay to the core of the story and not let my bias for any tags , incidents , tropes present in the story blind my opinion , I am not the best reviewer but I still try to give an honest review , but still there r Novels that I would be heavily biased against in my case those r WMW , Experimental Log of the Crazy Lich and many more . Does that mean my review shouldn't be considered or taken with a grain of salt , maybe or maybe not , there isn't a real answer to that because again the question wasn't clear in the first place , the answer is what u make of it.
     
    AliceShiki, runsing and otaku31 like this.
  10. Devil of Boredom

    Devil of Boredom Exists.

    Joined:
    Jan 25, 2017
    Messages:
    503
    Likes Received:
    398
    Reading List:
    Link
    MTL is to blame for that
     
  11. Vemmy

    Vemmy Well-Known Member

    Joined:
    Sep 8, 2017
    Messages:
    253
    Likes Received:
    332
    Reading List:
    Link
    How about making it so that there must be a reason for voting something 5 or 1? Like, in order to rate something on either side of the extremes, there must be something written in the review? That will deter those who rate without reading to some extent and also allow prospective readers to see exactly why there are so many low ratings on a novel other than sheer spite or malice and vice versa?
     
    Toralk likes this.
  12. Pell

    Pell Active Member

    Joined:
    Apr 25, 2016
    Messages:
    6
    Likes Received:
    1
    Reading List:
    Link
    There are so many people trash-talk about novels in reviews that it should be demotivating for translators and we get so many dropped novels. It would be hard to expand the community if there is not enough positivity left.
     
  13. Feng Tian

    Feng Tian Well-Known Member

    Joined:
    Mar 9, 2018
    Messages:
    966
    Likes Received:
    779
    Reading List:
    Link
    And you know whats even worse that the system? The ones dropping the reviews. I have seen 1s and 2s on CoS, which is one of the best novels on NU, while MGA has multiple 5s (lul), and has a fairly high average vote while being vile trash. And if you see the comments under some of these novels you flat out lose faith into humanity. People want the MC to kill his half-sister who did until now nothing wrong for a minor act of treason to which she was lead with deceit? WTF???? Meanwhile they hype when the next boring ass massacre in ED, ATG or MGA comes, in the later two with plot armour of doom and in all three cases with zero tension? Dafuq?
     
  14. Toralk

    Toralk Certified Grouch

    Joined:
    Mar 23, 2018
    Messages:
    380
    Likes Received:
    655
    Reading List:
    Link
    I agree that NU ratings are pretty useless, but I think the problem is actually too many choices. Giving a 1*-5* choice for ratings makes 3* ratings pretty much worthless, and thoughtfully giving a 2* 4* just makes your rating worth less than people that impulsively give 1* or 5*. Worse, lots of people (as admitted in their own reviews) lie in rating 5* just because they feel a story has been underrated, and clearly their personal opinion should count for more than multiple people that gave 2*-4* ratings.

    I think it would probably be best to go with a simple thumbs-up or thumbs-down rating. It would devalue all the careless ratings and ratings bombers, and make the people who actually give a **** matter again. Sure it might make people uncomfortable when they rate a novel they have mixed feelings about, but people could always not vote if they're torn.
     
    PoneyLach, StuffedDuck and Kourinn like this.
  15. SakuraKoi

    SakuraKoi Active Member

    Joined:
    May 20, 2018
    Messages:
    28
    Likes Received:
    22
    Reading List:
    Link
    Any rating system with numbers or stars is trash. Either "Recommended" or "Not Recommended". Not "my vote has more weight than yours because I am a radical believer in whatever doctrine".

    "Do you recommend a work based on its description and price (the readily apparent) matching the contents, quality and target audience (the not readily apparent)?"

    This can be asked and simply answered for any work made for entertainment. That is what most want to know.

    And if one has mixed feelings one could visibly abstain, not affecting the positive-negative ratio (i.e not taken into consideration since either side would lose and gain %) which is a fine replacement for those arbitrary numbers but still telling viewers how many or few have mixed feelings.
     
    Toralk likes this.
  16. MangoGuy

    MangoGuy Rambling Mango

    Joined:
    Apr 15, 2016
    Messages:
    7,625
    Likes Received:
    8,697
    Reading List:
    Link
    can you explain this? This seems like a feasible option
     
  17. Kourinn

    Kourinn Active Member

    Joined:
    Apr 15, 2019
    Messages:
    2
    Likes Received:
    9
    Reading List:
    Link
    Something close to this could be implemented by switching to a 3 star system (effectively 2 stars, with 1 still the lowest). By collapsing 1-2 star(s) = negative, 3 stars = neutral, 4-5 stars = positive.

    This would just add the non-inflated percentage next to star values, so from my example, someone that wants to rate something 90%, but has to choose between 4 stars or 5 stars, realizes that 4 stars = 75% and 5 stars = 100%, not 4 stars = 80%.
     
  18. tsundere_taichou

    tsundere_taichou [Cute Demon Sect]

    Joined:
    Mar 17, 2016
    Messages:
    1,435
    Likes Received:
    1,132
    Reading List:
    Link
    a really easy solution would be implementing half star votes so you can be like "yep this is 4.5 stars" that way older series wont be as affected
     
    Kourinn likes this.
  19. MangoGuy

    MangoGuy Rambling Mango

    Joined:
    Apr 15, 2016
    Messages:
    7,625
    Likes Received:
    8,697
    Reading List:
    Link
    So, if for a series the rating is 4.0 by average, we will also show 75% just to let people know that the decimal is projecting the score a bit higher?
     
  20. draw2much

    draw2much Well-Known Member

    Joined:
    Feb 23, 2018
    Messages:
    47
    Likes Received:
    98
    Reading List:
    Link
    Just have a rating system based on emoticons. Call it Reader Feels. Rather than it being about objective quality (which is a pipe dream anyway) it's a system based on how the story made the reader feel. A good tragedy should have a lot of sad emotes :blobsob:, a romance a lot of happy ones :blobmelt:, and so on. Even an angry emote :blobangery: would work well for revenge stories. If people leave a written summery of their feelings, than can leave two emotes as a reward.

    You can't abuse a rating system designed to be subjective to begin with, no matter how people vote it doesn't speak to the quality of the story. Only how the reader felt about it. That's how rating systems are used by the majority anyway, why not make a system that properly reflects how they're used? :blobpopcorn:
     
    Kourinn likes this.