preventing web reapers from accessing site

Discussion in 'OT Technology' started by Benson, Jan 27, 2004.

  1. Benson

    Benson Guest

    what can i do to prevent web reapers from downloading all my pics ? i want to host a huge amount of wallpapers and don't mind people saving them but if a lot of people reap them it will eat up all my allocated bandwidth for the month in a few days - how can I do this ?

    note - i'm not asking how to diable right click - it's just the reapers I don't want
     
  2. SLED

    SLED custom title

    Joined:
    Sep 20, 2001
    Messages:
    28,123
    Likes Received:
    12
    password protect your folders with an .htaccess file?
     
  3. carlin

    carlin Guest

    you can either block certain useragents or code it so that there is a limit on how many things can be downloaded in a specific amount time. (i.e 1 image every 20 secs). Having random filename or a simple signup form doesn't hurt either...

    but remember most reapers probably give false useragent info...and there are threaded reaper programs...
     
  4. SLED

    SLED custom title

    Joined:
    Sep 20, 2001
    Messages:
    28,123
    Likes Received:
    12
    and what web server are you running? apache? IIS?
     
  5. DatacomGuy

    DatacomGuy is moving to Canada

    Joined:
    Oct 14, 2002
    Messages:
    16,546
    Likes Received:
    0
    Location:
    Tampa, FL
    Robot file could help a tad as well.
     
  6. Penguin Man

    Penguin Man Protect Your Digital Liberties

    Joined:
    Apr 27, 2002
    Messages:
    21,696
    Likes Received:
    0
    Location:
    Edmonton, AB
    I believe that using .htaccess you can limit the referrers that have access to a specific directory. Thus, you could make it so that you can only download the images from a link on your site. Or, as posted above, password protect.
     
  7. col_panic

    col_panic calm like a bomb Moderator

    Joined:
    Sep 19, 2003
    Messages:
    188,152
    Likes Received:
    219
    Location:
    winter haven, fl

Share This Page