PDA

View Full Version : Added an anti-spider



ErosOlmi
14-05-2013, 22:44
http://www.vbulletin.org/forum/showthread.php?t=268208

Recently, most of the forum bandwidth was killed by Baidu Spider and others.
At every single time there were more than 100 spiders scanning thinBasic forum.
I've installed an anti-spider module in order to reduce this situation to a normal one rejecting scanning from spiders having too much process running at the same time.

Hope this will release web resources to humans.

ErosOlmi
09-06-2013, 15:38
Banned BingBot spider (Microsoft Engine Spider) from thinBasic web site.
They had more than 30 simultaneous spiders scanning thinBasic web site sending web server CPU to 100% for hours.