Notifications
Clear all
create the robot.txt File?
Topic starter
Posted : 27/10/2010 4:58 am
Re: How we create robot.txt File
Well most of the time robots.txt is much easier and less time. With the meta tag, you can say "follow" or "nofollow" and you can not specify directories without complicated rules. robots.txt is very simple and quick.
Posted : 27/10/2010 9:33 am
Re: How we create robot.txt File
It depends on how it is spent. For example - if you have a Perl script that you want to call, you can not call without typing the full directory com / cgi-bin.
Posted : 29/10/2010 5:01 am
Re: How we create robot.txt File
user-agent: *
disallow: /
Posted : 01/11/2010 9:51 am
Forum Statistics
6
Forums
5,497
Topics
35.8 K
Posts
10
Online
22.1 K
Members
Latest Post: why does salesforce best CRM ? Our newest member: prpfurniture Recent Posts Unread Posts
