Hi Jan
Sorry to hear about your problems. Check out eserviceinfo. If you can get something like theirs running, it'll prevent someone from downloading your files en masse.
Regards
Mike
Sorry to hear about your problems. Check out eserviceinfo. If you can get something like theirs running, it'll prevent someone from downloading your files en masse.
Regards
Mike
I have come across some sites offering the exact same schematics and service manuals as on my site. The funny part is if I change the name of a schematic (removing a letter or correct a spell error), shortly after the other sites have the renamed schematic (and the old one as well). So they just continiusly download everything on my site and publish it on their own.....
Hi Jan,
Now that is very interesting. Probably being downloaded from their server IP, or perhaps they think they are smartypants for setting up a distrubuted downloading bot network that downloads manuals from all over the web (probably not just your website) and assembles them on their own websites in order to make money from google traffic and advertising.
Depending on how they do it, you could use random and constantly changing urls to confuse their bots (takes some programming) or also construct your urls using javascript to piece them together so a "stupid" bot can't just scrape your website for plain text and download the new files. This approach is done commonly to protect email addresses. As the bots get smarter they have learned to actually run the javascript on a page though...
Another angle might be to say something like this:
That way, there are no urls to scrape from your website, it requires a small amount of intelligence to "create" the url required to download but it's simple and obvious enough for a real person to work it out. Only real downside I can see - Google won't find or archive the manuals so less traffic for you.
If these guys are determined to "mine" your website in particular, there's not much you can do. What amount of bandwidth are we talking about? Bandwidth is cheap these days if you are with the right host. I'm a fan of dreamhost, and they usually have unlimited bandwidth offers every now and again..
Now that is very interesting. Probably being downloaded from their server IP, or perhaps they think they are smartypants for setting up a distrubuted downloading bot network that downloads manuals from all over the web (probably not just your website) and assembles them on their own websites in order to make money from google traffic and advertising.
Depending on how they do it, you could use random and constantly changing urls to confuse their bots (takes some programming) or also construct your urls using javascript to piece them together so a "stupid" bot can't just scrape your website for plain text and download the new files. This approach is done commonly to protect email addresses. As the bots get smarter they have learned to actually run the javascript on a page though...
Another angle might be to say something like this:
That way, there are no urls to scrape from your website, it requires a small amount of intelligence to "create" the url required to download but it's simple and obvious enough for a real person to work it out. Only real downside I can see - Google won't find or archive the manuals so less traffic for you.
If these guys are determined to "mine" your website in particular, there's not much you can do. What amount of bandwidth are we talking about? Bandwidth is cheap these days if you are with the right host. I'm a fan of dreamhost, and they usually have unlimited bandwidth offers every now and again..
Thanks Jason,
Many solutions are in play, and sakis (and some of his employed) are testing some to find the most friendly solution for both users and me. Thanks sakis
Yes they are gathering schematics from all over the net, to present on their own. Each time I deny a group of IP addresses from that area where they downloads from, they manage to alter the IP to a new area address
From the stats I can see that they use multiple downloads at the same time, using all bandwidth 24 hours a day = tons of GB each hour...........
Many solutions are in play, and sakis (and some of his employed) are testing some to find the most friendly solution for both users and me. Thanks sakis
Yes they are gathering schematics from all over the net, to present on their own. Each time I deny a group of IP addresses from that area where they downloads from, they manage to alter the IP to a new area address
From the stats I can see that they use multiple downloads at the same time, using all bandwidth 24 hours a day = tons of GB each hour...........
Perhaps use mod_evasive in that directory, and when you get more than a few hits in a minute from any IP address it will go into a ban list, and instead of serving an error page, serve a lovely PDF containing an advertisement for your website.
Thanks. That could be fun;
"I'm just a F?=)(* A$$hole ripping other peoples websites!
Please go to Welcome to Audio Circuit Denmark to download the schematic"
A solution is on it's way
Thanks to my Greek friend diyaudio member sakis, I expect to have my download site up and running again tommorow
Wednesday with Captchas
Thanks to all for your kind support
I hope I will be able to provide free Schematics and Service Manuals to the DIY community for many years to come......
criss661;
Yes! That could have been a real laught
Thanks to my Greek friend diyaudio member sakis, I expect to have my download site up and running again tommorow
Wednesday with Captchas
Thanks to all for your kind support
I hope I will be able to provide free Schematics and Service Manuals to the DIY community for many years to come......
criss661;
Yes! That could have been a real laught
- Status
- This old topic is closed. If you want to reopen this topic, contact a moderator using the "Report Post" button.