CounterContent Scraper Group

Discussion in 'Tech Discussion' started by TamaSaga, Mar 17, 2018.

  1. chencking

    chencking [Daolord Grammar Nazi]

    Joined:
    Aug 1, 2016
    Messages:
    6,085
    Likes Received:
    4,162
    Reading List:
    Link
    China can't control the rest of the world, but they can censor Chinese sites
     
  2. DocB

    DocB "I see you, little mouse! Run along"

    Joined:
    Nov 10, 2015
    Messages:
    3,573
    Likes Received:
    8,110
    Reading List:
    Link
    the firewall is to keep the rest of the world in check, for chinese sites they can just destroy them
     
  3. delf20k

    delf20k Active Member

    Joined:
    Apr 29, 2016
    Messages:
    47
    Likes Received:
    12
    Reading List:
    Link
    If these tools get made available the original website of the webnovel will allso use them and most of the translators will go away due to machine translation and then editing becoming almost imposible.
     
  4. noisypixy

    noisypixy Sacatunn que pen, que summum que tun.

    Joined:
    Jun 25, 2016
    Messages:
    716
    Likes Received:
    950
    Reading List:
    Link
    I'm in. How much's the pay?
     
  5. chencking

    chencking [Daolord Grammar Nazi]

    Joined:
    Aug 1, 2016
    Messages:
    6,085
    Likes Received:
    4,162
    Reading List:
    Link
    Internet cookies
     
  6. AliceShiki

    AliceShiki 『Ms. Tree』『Magical Girl of Love and Justice』

    Joined:
    Apr 27, 2016
    Messages:
    24,650
    Likes Received:
    98,372
    Reading List:
    Link
    Hmmmmm... 2 things for you to think about when doing this.

    1) Will this hamper users of Reader Mode?

    2) Will this hamper users of text-to-speech?

    I don't use any of them, but I think you'll lose readers if you affect those.
     
    TamaSaga likes this.
  7. TamaSaga

    TamaSaga Well-Known Member

    Joined:
    Oct 11, 2016
    Messages:
    1,726
    Likes Received:
    2,173
    Reading List:
    Link
    1. After a brief search, I haven't found any documentation on the developer side for Reader Mode. As a result, I'm inclined to believe that it just works. So we'd need to reverse engineer the secret sauce to figure out whether or not any protections would mess with this feature.

    2. It may, especially if we start adding in image-based protections. And if it didn't, then we may need to consider a way to prevent content thieves from exploiting a scrape with audio. Although, to be fair, just because they can listen to it doesn't mean the periods and quotation marks get included properly.
     
  8. noisypixy

    noisypixy Sacatunn que pen, que summum que tun.

    Joined:
    Jun 25, 2016
    Messages:
    716
    Likes Received:
    950
    Reading List:
    Link
    Audio! Hell yeah!

    I've always wanted a robotic voice to read me these:
    Code:
    General Information
    
    Name: Goldalfa
    Age: 44
    Race: Beastkin (White Rhino Tribe / Black Steel Rhino)
    Job: Sharpaxer
    Status Level: 72/99
    HP: 1256
    MP: 422
    STR: 654
    VIT: 582
    AGI: 267
    INT: 173
    MGC: 247
    DEX: 299
    
    Skills
    Intimidation: Lv 8
    Super Herculean Strength: Lv 8
    Fist Techniques: Lv 5
    Fist Arts: Lv 5
    Presence Detection: Lv 3
    High Speed Regeneration: Lv 8
    Herculean Strength: MAX
    Club Techniques: Lv 6
    Club Arts: Lv 6
    Mining: Lv 8
    Regeneration: MAX
    Resistance to Abnormal Status Conditions: Lv 7
    Blink: Lv 3
    Resistance to Mental Abnormalities: Lv 7
    Elemental Blade: Lv 8
    Greater Earth Resistance: Lv 4
    Charge: Lv 7
    Axe Techniques: MAX
    Axe Arts: MAX
    Divine Axe Techniques: Lv 6
    Divine Axe Arts: Lv 7
    Magic Perception: Lv 3
    Vigour Mastery
    Muscles of Steel
    Goblin Killer
    Super Fast Reflexes
    Dampened Sense of Pain
    Dragon Killer
    Strengthened Skin
    Hardened Skin
    
    Innate Skills
    Awakening
    Shockwave
    
    I'd love to be unable to skim over these!

    Looking forward to this program.
     
  9. Mitch528

    Mitch528 Active Member

    Joined:
    Oct 20, 2015
    Messages:
    13
    Likes Received:
    22
    Reading List:
    Link
    While the documentation on reader mode is basically non-existent, both Apple Safari and Firefox use Readability as the base for their own reader mode. [Source]

    Not sure if anyone would want to use this, but I once made a WordPress plugin that sort of does what the OP was thinking except that it just uses Recaptcha instead of encrypting the content. I never ended up publishing it, but for those that are interested in such a thing you can get the plugin here.
     
    Last edited by a moderator: Mar 17, 2018
    noisypixy and TamaSaga like this.
  10. LysUltima

    LysUltima Riichi! Tsumo! Toitoi! Suuankou!?

    Joined:
    Aug 6, 2017
    Messages:
    2,144
    Likes Received:
    5,554
    Reading List:
    Link
    The problem with what the OP's trying to do is that readers get mad when the chapter text is messed up.
    It also doesn't stop idiots from manually copy + pasting chapters into the f-ing comment sections of pirate sites, which is what happens to me.

    The solution I'd recommend is to use a captcha or to add text to chapter announcement posts (what I use and what has worked so far)
    I made a plugin to automatically do that, so you can PM me if interested.
     
  11. HnM_Pete

    HnM_Pete Well-Known Member

    Joined:
    Sep 13, 2016
    Messages:
    120
    Likes Received:
    212
    Reading List:
    Link
    FYI, I tried a captcha briefly in 2017 ( Google's reCaptcha v2 ). It doesn't work. One of the aggregators pulled some weird shit with iframes and spoofing the captcha domain check. I'm still not exactly sure how that worked, but the captcha was broken within 72 hours and other aggregators just copied from the one that broke me.
     
  12. chencking

    chencking [Daolord Grammar Nazi]

    Joined:
    Aug 1, 2016
    Messages:
    6,085
    Likes Received:
    4,162
    Reading List:
    Link
    Well, if they earn like manga aggregators they earn big bucks from breaking captchas and the like. Tbh I've never seen a novel aggregator though, unless xianxiaworld's thing counted
     
  13. AliceShiki

    AliceShiki 『Ms. Tree』『Magical Girl of Love and Justice』

    Joined:
    Apr 27, 2016
    Messages:
    24,650
    Likes Received:
    98,372
    Reading List:
    Link
    Just google read light novels online free or something, you should find some.