Select Page
Notifications
Clear all

Define Robotxt file and tell me the use?

optimizenewyork
(@optimizenewyork)
Estimable Member

Define Robotxt file and tell me the use?

Quote
Topic starter Posted : 13/06/2012 12:54 pm
lray
 lray
(@lray)
Eminent Member

Re: Define Robotxt file and tell me the use?

it's a small text file that tells search engine spiders/bots where to go on your site and where not to go. for example you wouldn't want a spider crawling on your checkout page if you have an e-commerce site

ReplyQuote
Posted : 14/06/2012 8:46 pm
alexanderfrost
(@alexanderfrost)
New Member

Re: Define Robotxt file and tell me the use?

Web Robots are also known as Web Wanderers, Crawlers are programs that traverse the Web automatically. Web site owners most oftenly use the /robots.txt file to give instructions about their site to web robots. This file tells search engine where to go on your site and where not.

ReplyQuote
Posted : 15/06/2012 12:49 pm
bryanbell
(@bryanbell)
Eminent Member

Re: Define Robotxt file and tell me the use?

Robots.txt file used by webmasters to instruct the search engine bots for which page they have to crawl and which are not.

Spinx INC - Creative Web Design Company in Florida http://www.spinxwebdesignflorida.com

ReplyQuote
Posted : 18/06/2012 8:52 am
hani
 hani
(@hani)
New Member

Re: Define Robotxt file and tell me the use?

It tells the search engine bots which pages of your website is important or need to be indexed.

ReplyQuote
Posted : 25/06/2012 8:30 am
simplyelectricals
(@simplyelectricals)
Eminent Member

Re: Define Robotxt file and tell me the use?

Robots.txt is a text based file for spider to allow which page of this website is indexing/ crawling or which one ignored.

online shopping sites in India - Environmental Courses

ReplyQuote
Posted : 25/06/2012 8:48 am
satinderbal
(@satinderbal)
Eminent Member

Re: Define Robotxt file and tell me the use?

robot.txt is the file that is uploaded in the root directory. Which i used to stop the google to crawl the page that you dont want to google reach.

Metro taxi Detroit | Govt Jobs India

ReplyQuote
Posted : 28/06/2012 11:13 am
Share: