Hiding a page from the search engines????

Re: Hiding a page from the search engines????

On Thu, 2 Jun 2005 22:08:43 +0100, Johny wrote:
> Is there a way to stop the search engines listing a particular page?

yes, see www.robotstxt.org.

i

> Thanks
> J
>
>

--

Re: Hiding a page from the search engines????

On Thu, 02 Jun 2005 23:08:43 +0200, Johny
wrote:

> Is there a way to stop the search engines listing a particular page?

robots.txt or .
But better check details here

http://www.google.com/intl/en/webmasters/3.html#B3

and here

http://www.robotstxt.org/wc/exclusion.html#meta

Best,
Borek
--
http://www.chembuddy.com - chemical calculators for labs and education
BATE - Base Acid Titration and Equilibria
program for pH calculations
CASC - Concentration and Solution Calculator
program for solution preparation and concentration conversions

Re: Hiding a page from the search engines????

Johny wrote:
> Is there a way to stop the search engines listing a particular page?
>

No, there isn't, the least you could do is stop the searchengines that
obey the robots.txt.

There are however enough that don't obey them, or even take the effort
to load it befor they start reading your files.

Hidden for a searchengine, or on the internet is only that file, that
isn't on it.

--
Glas geschiedenis http://vision2form.nl/glashistorie.html
Passie en talent http://vision2form.nl/young_talent.html
Webontwerp http://vision2form.nl/webontwerp/index.html
SEO gaat zo http://vision2form.nl/webontwerp/gevonden-worden.html

Re: Hiding a page from the search engines????

Hi Johny !

"Johny" skrev i en meddelelse
news:d7nsgp$ivl$1@newsg3.svr.pol.co.uk...

> Is there a way to stop the search engines listing a particular page?

A password protected page will not be seen by the searchengines, and
therefore is not indexed and shown in searhresults.

Kind Regards
--
Med venlig hilsen

Ren? L?weneck
www.lowesite.dk

Re: Hiding a page from the search engines????

"Ren? L?weneck" skrev i en meddelelse
news:429fe73b$0$755$edfadb0f@dread16.news.tele.dk...
>
> A password protected page will not be seen by the searchengines, and
> therefore is not indexed and shown in searhresults.

Or if you do not link to the page?

Or if you put them in a /temp/ directory?

Mikkel
--
Guide til Guatemala p? nettet:
http://lakjer.dk/mikkel/
rejser, kultur og politik

Re: Hiding a page from the search engines????

> Is there a way to stop the search engines listing a particular page?
>
> Thanks
> J

Use dirty tricks, such as hidden text, which looks like an attempt to
fool the SE. Then they'll penalize the site and more or less remove it.

Re: Hiding a page from the search engines????

Hi Mikkel !

"Mikkel M?ldrup-Lakjer" skrev i en meddelelse
news:42a006c9$0$18648$14726298@news.sunsite.dk...

> Or if you do not link to the page?

That would do the trick too, I think.

> Or if you put them in a /temp/ directory?

I'm not sure about this, though. Because, if you host your site yourself,
you could strenghten the accesslevel for one or more directories on the
server.

But then again, this would cause the server to prompt you for an username
and a password ;-))

If you, on the other hand, use an outside webhotel as a host for your site,
I don't think that this procedure is an useable option for you.

Kind Regards

--
Med venlig hilsen

Ren? L?weneck
www.loweneck.dk

Re: Hiding a page from the search engines????

"Mikkel M?ldrup-Lakjer" wrote in news:42a006c9$0$18648
$14726298@news.sunsite.dk:

> "Ren? L?weneck" skrev i en meddelelse
> news:429fe73b$0$755$edfadb0f@dread16.news.tele.dk...[color=green]
>>
>> A password protected page will not be seen by the searchengines, and
>> therefore is not indexed and shown in searhresults.
>
> Or if you do not link to the page?[/color]

I'm not sure if this is still happening, but it was at one point definitely
the case that even if a page was not linked to from anywhere, it could
still be indexed if it was visited by someone using the Google toolbar. I
have tested this.

Content in a /temp/ directory can certainly be indexed, (which is when you
realise too late you had meant to stick a password on the dratted thing but
forgot).

The only completely reliable way to prevent a page being indexed is to put
it behind a password (and have the password set by the server, not by
client side coding such as Javascript in the page*).

Victoria

* actually you could put it behind a clientside password if you did it
right, but it would be more fiddly and I can't see why you would.

Victoria
--
Clare Associates Ltd
http://www.clareassoc.co.uk/
--

Re: Hiding a page from the search engines????

On Fri, 03 Jun 2005 10:57:57 +0100, Victoria Clare
wrote:

>I'm not sure if this is still happening, but it was at one point definitely
>the case that even if a page was not linked to from anywhere, it could
>still be indexed if it was visited by someone using the Google toolbar. I
>have tested this.

I've been working on some new domains recently and have been accessing
them for testing purposes many times with the toolbar installed and
with Google Adsense ads on the sites.

So far not one page has been indexed until there are links. Some have
had test sites online for weeks like this with no links. This site
education-news.info has a bunch of tests on it and none are indexed.
If indexed it would be thousands of pages.

This doesn't prove conclusively that Google doesn't index via the
toolbar since it could take longer than I've waited, but does suggest
it. I'll leave some of it up for a few months to see if it ever gets
indexed.

David
--
Free Search Engine Optimization Tutorial
http://www.seo-gold.com/tutorial/

Re: Hiding a page from the search engines????

SEO Dave wrote in
news:nsa0a1l7qonlrprntngck43ell8urm8bg3@4ax.com:

> On Fri, 03 Jun 2005 10:57:57 +0100, Victoria Clare
> wrote:
>[color=green]
>>I'm not sure if this is still happening, but it was at one point
>>definitely the case that even if a page was not linked to from
>>anywhere, it could still be indexed if it was visited by someone using
>>the Google toolbar. I have tested this.
>
> I've been working on some new domains recently and have been accessing
> them for testing purposes many times with the toolbar installed and
> with Google Adsense ads on the sites.
>
> So far not one page has been indexed until there are links. Some have
> had test sites online for weeks like this with no links. This site
> education-news.info has a bunch of tests on it and none are indexed.
> If indexed it would be thousands of pages.
>
> This doesn't prove conclusively that Google doesn't index via the
> toolbar since it could take longer than I've waited, but does suggest
> it. I'll leave some of it up for a few months to see if it ever gets
> indexed.[/color]

It was doing it within a few days when I tested it, but that was a year or
more ago. My guess is that they have now changed it (good, as it was very
annoying).

Still worth bearing in mind though, as there is no knowing if they will
turn it back on at some point.

Victoria
--
Clare Associates Ltd
http://www.clareassoc.co.uk/
--

Re: Hiding a page from the search engines????

On 2 Jun 2005 21:10:59 GMT, Ignoramus12247
wrote:

>On Thu, 2 Jun 2005 22:08:43 +0100, Johny wrote:[color=green]
>> Is there a way to stop the search engines listing a particular page?
>
>yes, see www.robotstxt.org.[/color]

Bad Guy Robots will simply stroll past robots.txt instructions,
tilting their fedoras and tapping cigar-ash on them as they go by.
This is like making laws without enforcement, the only people who
respect them are the law-abiding. They have no effect on criminals.

BB

--
www.kruse.co.uk/ [email]seo@kruse.demon.co.uk[/email]
seo that watches the river flow...
--

Re: Hiding a page from the search engines????

On Thu, 02 Jun 2005 23:14:45 +0200, Borek
wrote:

>On Thu, 02 Jun 2005 23:08:43 +0200, Johny
> wrote:
>[color=green]
>> Is there a way to stop the search engines listing a particular page?
>
>robots.txt or .
>But better check details here
>
> http://www.google.com/intl/en/webmasters/3.html#B3
>
>and here
>
> http://www.robotstxt.org/wc/exclusion.html#meta
>
>Best,
>Borek[/color]

No need, Borek, they won't work either. I think Googlebot will observe
the noindex bit but if no other robots do then sooner or later it'll
get indexed through somewhere else.

BB
--
www.kruse.co.uk/ [email]seo@kruse.demon.co.uk[/email]
seo that watches the river flow...
--

Re: Hiding a page from the search engines????

On Thu, 2 Jun 2005 22:08:43 +0100, "Johny"
wrote:

>Is there a way to stop the search engines listing a particular page?
>
>Thanks
>J

Don't put it on the web would be the obvious answer.
Password-protecting it used to be a good idea but of course Google's
catch-all proxy may well decide to scoop up your protected page and
link to it for all the world to see. So there goes that idea. No, on
balance, I'd say don't upload it.

BB
--
www.kruse.co.uk/ [email]seo@kruse.demon.co.uk[/email]
seo that watches the river flow...
--

Re: Hiding a page from the search engines????

On Fri, 3 Jun 2005 09:28:41 +0200, "Mikkel M?ldrup-Lakjer"
wrote:

>"Ren? L?weneck" skrev i en meddelelse
>news:429fe73b$0$755$edfadb0f@dread16.news.tele.dk...[color=green]
>>
>> A password protected page will not be seen by the searchengines, and
>> therefore is not indexed and shown in searhresults.
>
>Or if you do not link to the page?[/color]

But if you did this and someone ran a link checker, say Xenu, over
your site and published (for whatever insane reason) the results?

>Or if you put them in a /temp/ directory?

I'm stumped here, Mikkel, I dunno what you mean.

BB
--
www.kruse.co.uk/ [email]seo@kruse.demon.co.uk[/email]
seo that watches the river flow...
--