I was practicing my python internet interfacing skill when I realized I'd been writing a scraper bot for NU subconsciously (I just used the first URL I looked at and you know...). Essentially I can just set it to watch the site and log information about what's uploaded. For example, I can just have it run through checking the popularity of various new novels and log the popularity so I can make a better decision on what to read. But I also know that the site has had trouble with high bot populations in the past, and I don't want to get IP banned or anything so I figured I'd just ask and get a few nuffies in the process. Thanks.
First things first, you would have to run through the logs for at least a week or a month or more to even get proper results and as for using it or not, this is @Tony's cue here~
*** silently shuts down serverless function instance running in Gcloud thats scraping NU for data analysis practice project that they have done years ago **
You should get @Tony's permission first, of course, but I have to ask -- why? Novel Updates tracks novel popularity based on reading list additions, bookmark updates, reviews and ratings. It will rank novels for you by week, month, all time.. and even let you see graphs charting a novel's rise and/or fall in popularity between any two given dates. You can even compare and contrast that rise and/or fall across multiple ranking categories. I'm not sure what more you could possibly need in terms of a novel's popularity, or how you would even get that information from scraping the site. You're not going to have much luck tracking, say, when users update their reading list bookmarks for a particular novel. Internal statistics like that allow Novel Updates to present a better popularity ranking than you'll get without the same information.
It's mostly just practice so I doubt any of the information I get will be terribly useful. I might use it for a comp sci project later this year. So all in all, before using a bot on the site run it by Tony?