WorkTheWeb Forums > Blocking search Bots question

Loki

Mar 11 2005, 10:35 PM

I am posting this here since I guess it would fall under a security topic and there is no forum for it. My question is how can I block search bots from combing through my site. I seem to have them on it continually day and night. I have tried .htaccess files and the IP Deny Manager to block the IPs, but it has not stopped them. Is there a way to block them completely? I have seen them even in my private forums. The main area I need to secure is my forum which is a Vbulletin board.

Thanks for any ideas you might have.

phpfreak

Mar 11 2005, 10:49 PM

QUOTE(Loki @ Mar 11 2005, 05:35 PM)

I am posting this here since I guess it would fall under a security topic and there is no forum for it. My question is how can I block search bots from combing through my site. I seem to have them on it continually day and night. I have tried .htaccess files and the IP Deny Manager to block the IPs, but it has not stopped them. Is there a way to block them completely? I have seen them even in my private forums. The main area I need to secure is my forum which is a Vbulletin board.

Thanks for any ideas you might have.

One method to handle these punks is to do a auto_prepend_file in your .htaccess. I've done this many times.

Here's kind of how you do it.

In your .htaccess file add the code:

CODE

php_value auto_prepend_file "/path/to/blocker.php"

Now, in this "blocker.php" script, you can drop their IP's or even drop the USER Agent sort of like this:

CODE

<?php
if($_SERVER['REMOTE_ADDR'] == '111.111.111.111')
{
echo "Get Lost";
exit();
}
?>

Now, if you're good you can loop through Arrays and block multiple IPs and etc.

Also... take note that this script will be called before any other PHP code on your site. It automatically prepends the file to the top of the code. If this user triggers the code inside the IF statement, it will block them out and stop execution of all subsequent PHP code.

Good luck!

Loki

Mar 11 2005, 11:41 PM

QUOTE(phpfreak @ Mar 11 2005, 10:49 PM)

One method to handle these punks is to do a auto_prepend_file in your .htaccess. I've done this many times.

Here's kind of how you do it.

In your .htaccess file add the code:

CODE

php_value auto_prepend_file "/path/to/blocker.php"

Now, in this "blocker.php" script, you can drop their IP's or even drop the USER Agent sort of like this:

CODE

<?php
if($_SERVER['REMOTE_ADDR'] == '111.111.111.111')
{
echo "Get Lost";
exit();
}
?>

Now, if you're good you can loop through Arrays and block multiple IPs and etc.

Also... take note that this script will be called before any other PHP code on your site. It automatically prepends the file to the top of the code. If this user triggers the code inside the IF statement, it will block them out and stop execution of all subsequent PHP code.

Good luck!

Thanks for the quick reply. ok now will placing these in the root work on all areas of my site or will the htaccess files need to be placed in each folder that I want them blocked from. Also in the htaccess code where you have "/path/to/blocker.php" does this change in reference to where it is located on the server? for instance "/home2/nukepla/public_html/blocker.php" ? or do I leave it as is?

phpfreak

Mar 12 2005, 12:11 AM

Hi, you can put it in your root directory and link to it. And yes, the path you link to is the full filesystem path such as:

/home2/nukepla/public_html/blocker.php

Sincerely,

Loki

Mar 12 2005, 12:18 AM

ok Great Thanks Again.