Feature Request Complete Overhaul of the Rating System

Discussion in 'Suggestions & Bug Reports' started by ShasuX, Dec 3, 2019.

  1. ShasuX

    ShasuX Well-Known Member

    Joined:
    Mar 29, 2016
    Messages:
    108
    Likes Received:
    160
    Reading List:
    Link
    Let me state first, this is a rough list of suggestions that describes various issues that should be addressed. I'm not claiming to fully cover all details nor that I'm giving a perfect solution to all problems. Personally I think that it would make sense to discuss it internally among the NU staff and to also ask the NUF members about their opinion.

    I mentioned it at another place already, but the rating system on NU is horrible. That's not solely owed to the system itself, but of course the system enhances this through its lack of regulation.
    To make a very simple example, I can go to any random novel and write a review stating, "McDonald's sucks."
    And that comment will be accepted as valid review while it's obviously completely unrelated to anything.

    But instead of simply bashing the system itself and being done with it, I want to offer a few suggestions how to actually set up a proper review system that allows new readers to get a better judgment about a novel.

    First off, a "No-Review-Lock.
    For example, reviewing a novel with 400 chapters immediately after the Prologue makes no sense. No matter what anyone says, it's bullshit. After all there has to be a reason why the novel is going on for 400 chapters.
    As such I'd suggest putting a lock on posting reviews and ratings on new novels up to a certain chapter count.
    Of course I know that giving a hard lock doesn't really work well in reality with exceptions like one-shots etc. At the same time, there's the wonderful value called "Status in COO" which lists the current total count of chapters, allowing you to use that value for a flexible review lock (Status in COO could be overworked to reflect only the total count of the current chapters).
    Thus I suggest a review lock at following limits (initial chapter count, disregarding any changes in the future):
    Review/Rating for novels with 1-10 chapters, starting with release 1.
    Review/Rating for novels with 10-300 chapters, starting with 10% of the total.
    Review/Rating for novels with 301-600 chapters, starting with 5% of the total.
    Review/Rating for novels with 601-1000 chapters, starting with 3% of the total.
    Review/Rating for novels with >1000 chapters, starting with 2% of the total.
    Review/Rating for novels with >3000 chapters, starting with 1% of the total.

    This should be a matter of coding, and thus possible to realize.


    Next, an extended Rating System.
    Currently readers rate from 1 to 5 stars. But those stars don't stand for anything. Anyone can add any meaning to those stars. Or in other words, I can rate a novel with 5 stars because the illustrations are full of boobs, and that would be fine. That also makes no sense.
    I suggest making the rating more specific, which means: let the readers rate different attributes of a novel. Let me suggest a few that seem important to me:

    - Grammar/Spelling (English)
    - Theme(s) of the story
    - Progression of the story
    - Story Flow/Pace
    - Characterization/Character Development
    - World Building
    - Dialogues

    There might be more that I missed, but I think those would at least cover a wider range of aspects of a novel. A comprehensive rating can be drawn through coding from all the various ratings for these points.


    Next, "Review Rules"
    Well, in any system, human input is the most "error-prone" component. The two points above won't hinder anyone posting "McDonald's suck." Thus it's necessary to set some some rules for a proper review.
    I suggest a form with some subjects such as:
    - Summary
    - Pro
    - Contra
    - Personal Opinion

    I know that these would need to be checked by humans to properly verify the validity of a review in regards to its content, but I think NU has several moderators and there's also the option of reporting a review, which would naturally lead to a reassessment of what's written.


    These are the three core suggestions for a Rating/Review System Overhaul. I'm sure that someone smarter than I am will find several other parts that might need to be addressed, but that's all I can come up with for the moment.
    At the moment NU basically acts as simple index site listing all the chapters released by translation groups. At the same time NUF serves as central meeting place for readers following those novels.
    I think a transformation of NU towards a proper rating/review site would also help the translation groups, whose chapters are posted in the index, and it would likewise add further quality to the site as a whole.

    I hope you can consider my suggestions favorably.


    Cheers,
    Shasu
     
  2. Nom de Plume

    Nom de Plume [Shio’s Disciple] [True Villain] [Equip: Gunblade] Novel Updates Staff

    Joined:
    Oct 20, 2015
    Messages:
    2,693
    Likes Received:
    13,032
    Reading List:
    Link
    Small note: Every single review is read. If you leave a review, only you will see it at first. It will be gone if you log out. It can take up to 24 hours for a review to be approved, since hundreds are submitted a day. Occasionally more if our admin is on vacation. This isn’t perfect since he doesn’t read the novels, but it does filter out most troll and foreign language submissions.

    Tl;dr McDonald’s sucks would never be an accepted review.

    As for the rest, there are many suggestions for improvements to the review system. I agree with some of your points, but making the system complicated will not fix people aiming to submit odd reviews like one star across the board. In fact, it may limit legitimate readers who find the rating system too complicated.

    Another thing to keep in mind are whether people are reviewing the book or the translation? We don’t want people reviewing translators, but many people do. This adds more outliers of somewhat legitimate reviews that can skew results.

    To use status in CoO as a variable, it would need changed from a string to a usable numerical value. On top of this, a system that reviews multiple aspects of a novel will have final scores that are average across the board, unless people are able to sort and filter by those individual aspects. That does sound neat, but it's more time and development for something that is going to still be an arbitrary result in the end.

    Summary: Reviews will always remain flawed, because the people giving them will always be giving an opinion someone else might not agree with. It may still be a valid opinion. Numbered reviews (1.2/5) are best at filtering out novels that everyone agrees are bad. They rarely help people find the greatest novels, since everyone has different tastes. Instead, NU has systems such as recommendation lists to help find well liked novels, and a mix of genres, tags, and written reviews to help discern if it might be a novel you are interested in as well as the quality of the novel. Users can and do explain why they rate a novel a certain way.

    I’ve fought for a better review system in the past, but simple isn’t always bad especially considering that the site has other methods for finding novels that are comprehensive. Even series finder has alternate ways to find popular novels by factors such as “number of readers.”
     
  3. kenar

    kenar ヽ(`・ω・´)ゝ

    Joined:
    Nov 2, 2016
    Messages:
    2,887
    Likes Received:
    4,568
    Reading List:
    Link
    NU lack the manpower to implement that afaik.
     
    Nom de Plume likes this.
  4. Havisu

    Havisu Night Lurker

    Joined:
    Nov 13, 2015
    Messages:
    754
    Likes Received:
    767
    Reading List:
    Link
    While i understand this op views, i wish op also knows just how much of a hassle it is to overhaul a feature, no matter how easy it seems

    While all said above are true, about either it wss possible to be done or not, the consequences in changing something so deeply rooted in the system will surely not always be a good one. while it is possible to do code-wise, one person doing the entirety of this whole system is already ameiizing. but i bet even tony would feels tired. I means, look, we aren't even paying him. Most people in this website probably been using adblocker. In my opinion the review system is not as bad as we all think. Someone trolling or bashing your fav novels?!write back another review invalidating that bashing kid. As a reader who always check the review first, whenever i saw something like a clash of opinion in reviews happens, i would get really curious to read the novel.
     
    solivagantsoul, Nom de Plume and Shio like this.
  5. ShasuX

    ShasuX Well-Known Member

    Joined:
    Mar 29, 2016
    Messages:
    108
    Likes Received:
    160
    Reading List:
    Link

    Well, I didn't realize that there's someone actually reading all the reviews, or at least skimming through them. To me that sounds like a lot of work being already done, and narrowing down the review/rating system would actually lower the work needed for the mods checking all those daily-posted reviews.



    Actually I think it's the other way around. If the rating/review system becomes more complicated, people, who just want to post some "venting review" or whatever, won't bother as it'll be too much of a hassle. Meanwhile those who actually want to write a proper review already account for the time it will take any way. Meaning, making it more complicated would be positive as limitation towards legitimate readers.



    That's simply how it is. Though I find it legit to review on the grammar/spelling used in the target language, and that's definitely something depending on the translation team and not so much on the author or the story itself.


    Maybe it's possible to make a new numerical variable in addition to CoO? I'm not familiar with the coding, so I won't comment on the technical aspects, but I see that "Year" is used as variable, too, so my simple thinking believed it to be possible to adjust CoO.
    Also having average scores across the board is nothing bad? It would actually reflect reality more than some of the current values, where novels are downvoted heavily just because they have a NTR tag and similar.

    You're right in many points here. I'm not suggesting to "censor" reviews in any way. My idea is to qualify reviews by adding guidelines. Those wanting to write reviews, will always do unless you make it technically impossible for them. For those wanting to get a good idea about a novel it should be a big help though.
    Well, in the end, those are only suggestions. It is up to NU staff to decide whether it's a topic worth to be worked on. :)

    I don't know how deeply rooted in the system it is. Tony and the NU team can certainly judge it better whether it would be a time-consuming, difficult work or not. It's their decision whether it's feasible to implement or not, my post is just a suggestion to add more guidelining to reviews/ratings.
    Also, assuming there was a review I'd hate in one of my favorite novels, I have no real option to react to it. You can post only one review per account per novel, iirc, and if it was really my favorite novel, I wouldn't waste that one to bash one person smack talking the novel. In this case it'd be better to be allowed to comment on reviews. But since we are on the topic already, it's kind of funny how you can "Like" a review, but not "Dislike." Another suggestion.


    Cheers,
    Shasu
     
    Havisu likes this.
  6. imK

    imK Artful Dodger

    Joined:
    Oct 20, 2018
    Messages:
    1,221
    Likes Received:
    2,630
    Reading List:
    Link
    I would appreciate the ability to rate the translation work separately from the source.

    Sometimes I've read work that I haven't liked for whatever reason, yet appreciated the hard work of the translator(s). It would be nice to be able to acknowledge their efforts. I have felt bad about giving a low rating for a work I consider sub-par despite the translation quality, but since there's no way to separate the two I don't want to give it a high rating and possibly mislead people who only look at star-ratings/skim read other reader's reviews.

    I loathe people who 'review' other people's reviews by leaving snarky, back-handed, passive-aggressive insults. If you do that you are both a bully and a coward. There is a report button there for troll reviews and genuinely derogatory reviews. Use it. There's no excuse for trying to bully someone simply because you don't like what they said. Don't do it. I've seen a fair amount of this and report this behaviour whenever I have the time. The review section is meant to review the work, not shame and coerce other people into agreeing with you. I would 100% support harsher penalties for repeat offenders including removing the offender's current reviews, removing their ability to post reviews in future, temporary bans and permanent bans.
     
    Last edited: Dec 23, 2019
  7. frostcrystal

    frostcrystal Well-Known Member

    Joined:
    Jan 31, 2018
    Messages:
    41
    Likes Received:
    446
    Reading List:
    Link
    A bit of a thread zombification, but I'd like to second that it would be nice to be able to review the translation and the story separately. Perhaps a decent separation might be to assign ratings to translator groups? Although that might lead to drama, unfortunately.

    I'd also like to call out a hole in the review lock concept, which is that people do read novels via MTL or in the original language, and often post reviews afterward. This are very useful reviews because they let you know if the story is worth sticking to or not, regardless of how many chapters are out at the moment, and would be prevented by the review lock.
     
  8. applebeans

    applebeans Well-Known Member

    Joined:
    Apr 17, 2018
    Messages:
    284
    Likes Received:
    589
    Reading List:
    Link
    review lock can be used for people who say they've only read one chapter, maybe
    I like the idea of separating stars and what they represent
    maybe some ideas can be implemented but not all at once? I was looking for threads like this to suggest 1-10 ratings or 0,5 stars addition
    this is ambitious idea.... I'd also suggest ratings for reviewers, so problematic reviewers may be moderated more, and reviewers of good reputation can have their reviews posted immediately? that may help with time issue
     
  9. ZeroXOXO

    ZeroXOXO Active Member

    Joined:
    Jun 14, 2020
    Messages:
    5
    Likes Received:
    24
    Reading List:
    Link
    I guess I have the opposite? opinion. While I would never condone bullying other reviews, I really hate how unreliable some reviews can be. As an avid reader, I know the caveats of jumping to conclusion... especially in the prologue or first half of the story. Yet, the most popular and “top” liked reviews on most novels are often whoever jumps the gun and posts first. So misleading to newcomers. Oftentimes, these reviews aren’t even reflective of the final product... otherwise why else would the author write a whole novel after the first 2 chapters? Am I going to “report” or scroll through and ignore every single early review? No, that’s way too unrealistic. No other novel site I have come across, whether that is goodreads or amazon, promotes this type of reviewing behavior. Maybe it is because you can buy the entire book outright rather than wait for translations to trickle in. Even if people are free to DNF because of personal distaste, the large portion of “top” reviews on those sites are from people who did finish the novels, no matter how much they loved/hated it.

    I feel like the finished reviews are more useful reviews. I guess this applies more to controversial books with sensitive topics... where people will often DNF early and discourage newcomers because of it... now the readers who did actually make it until the end leave reviews to argue otherwise... only to be buried and never seen. What’s the point of the review system anymore? To know if a novel is worth it after the first 10%? Is that how fast a story has to captivate someone to be worthwhile?

    The finished readers are often times not even arguing to be “snarky” (well some...), but more often just trying to be like “hey, there is more to this story after the first 2 chapters.” The snarkiest comments on those types of novels are the unfinished reviews and are like “only an idiot would make it to the end.” More than abrupt/unwarranted snarkiness, I find the back-and-forth between those who finished (and thus know more) against those who brandish their limited knowledge much more common on those types of novel reviews. This reviewing behavior is something I see almost exclusively to NU's system because of how episodic the releases are. Unlike other sites, there isn't even a comment section for reviews... and that's why it perpetuates this endless back-and-forth in snarky reviews because people who progressed further get frustrated with some misleading reviews on here.

    Why is it that the top reviews on these types of novels all have 1 or 2-star ratings and the “chapter completed” is always like less than 20%? Jeez. I think this unreliability is the most frustrating thing about the review system on this site.

    Either way, everyone should be free to review. Every opinion and review is valid. They just aren’t of the same quality. If someone made it to the end and told me “yeah, this is trash”... I trust that a lot more than someone who gave me that opinion in chapter 2.

    There should be at least a way to filter for completed reviews or filter by descending % progress, rather than promoting whichever review is the “most liked” (often earliest placed...). That should be the default on a site that reviews episodic releases, rather than by “likes” or date or longevity of reviews (since rarely are the longest lasting reviews the furtherest in progress).
     
    Last edited: Jul 17, 2020
    applebeans, ShasuX and kenar like this.