Question Novel Updates' bot and Analytics

Discussion in 'Translator's Corner' started by Weslley, May 2, 2021.

  1. Weslley

    Weslley Member

    Joined:
    Nov 16, 2020
    Messages:
    15
    Likes Received:
    5
    Reading List:
    Link
    First of all, hi everyone, My name is Weslley and I'm new to the forums.

    To quickly brief my situation. I created a group called Pawprint translations about 2 weeks ago and started translating Tearmoon Empire Story. According to my analytics, I've been receiving 500 to 800 views per chapter, which I consider very good for a not so popular series.

    Now comes the problem and my question. When looking at traffic's details, most of it comes from Novel Updates. That is expected. But while some come from expected pages, like NU's homepage, the series main page and the reading page, some comes from weird places, like the [REDACTED] address.

    I know the address is related to Novel Updates, but the extnu part got me concerned. When I learned about the NU bot, I thought it simply consumed RSS feeds, but maybe it is an actual crawler doing HTTP requests?

    The short version of the question is: Does NU's bot influence the traffic information I'm getting from my analytics?

    Thanks a lot for the attention.
     
    Last edited by a moderator: May 2, 2021
  2. MangoGuy

    MangoGuy Rambling Mango

    Joined:
    Apr 15, 2016
    Messages:
    7,625
    Likes Received:
    8,697
    Reading List:
    Link
    @runsing @Tony
     
    Last edited by a moderator: May 2, 2021
  3. runsing

    runsing status : bleeding, health -10/s Novel Updates Staff

    Joined:
    Nov 4, 2015
    Messages:
    3,383
    Likes Received:
    6,846
    Reading List:
    Link
    Sutad Aatma likes this.
  4. LaDyViL

    LaDyViL New Member Staff Member

    Joined:
    Nov 9, 2015
    Messages:
    10,050
    Likes Received:
    23,555
    Reading List:
    Link
    :blobsnow: don't mind me, I'm just here to learn more about NU.
     
  5. Weslley

    Weslley Member

    Joined:
    Nov 16, 2020
    Messages:
    15
    Likes Received:
    5
    Reading List:
    Link
    Got it. So, it is legit traffic, but the origin is being masked. Fair enough. Thanks for answering so fast.
     
  6. MangoGuy

    MangoGuy Rambling Mango

    Joined:
    Apr 15, 2016
    Messages:
    7,625
    Likes Received:
    8,697
    Reading List:
    Link
  7. runsing

    runsing status : bleeding, health -10/s Novel Updates Staff

    Joined:
    Nov 4, 2015
    Messages:
    3,383
    Likes Received:
    6,846
    Reading List:
    Link
    in the first place, NU bot only reads the rss feed, in other words, whatever data contains in it. and it is not configured to crawl over content pages (chapters) on the trasnlator sites, except for a few sites, which are known to use some kind of password protection (read: adwall)

    what i think happened was external bots arriving to your site through NU's redirector links and crawling around. so there were bots, just not NU's. internet are infested with bots afterall. and some can be quite aggressive.
    https://forum.novelupdates.com/online/?type=robot

    whatever you see on a novel page, basically. covers, description, chapters links, all that stuff. that's how you see when a novel page on NU uses wrong cover, a pirate site will also use that wrong cover. even when it was fixed on NU, sometimes they still continue to use the wrong ones. (example : try googling Apotheosis of a Demon – A Monster Evolution Story and choose any aggregator you found. compare it to NU)
     
  8. MangoGuy

    MangoGuy Rambling Mango

    Joined:
    Apr 15, 2016
    Messages:
    7,625
    Likes Received:
    8,697
    Reading List:
    Link
    Damn... I knew aggregators existed, but I didn't think they would target NU itself!
     
  9. runsing

    runsing status : bleeding, health -10/s Novel Updates Staff

    Joined:
    Nov 4, 2015
    Messages:
    3,383
    Likes Received:
    6,846
    Reading List:
    Link
    why wouldn't they? pleathora of pages with ready-made infos and links, no need to go find each of translator sites etc.

    not only did they piggyback NU, they even piggyback each other.

    look at what happened when mangadex's down. all of the aggregators now has missing updates, slower or outright failed loading, everything.
     
    MangoGuy likes this.