[Typo3] Robots.txt and indexed search
Cleveland at winnefox.org
Fri Jun 3 23:28:21 CEST 2005
I'll try to clarify what it is I'm trying to do. I've got a site that
has a few hundred pages, and a few hundred pdf files. I've got a few
pages that link to those pdf files, and I've got typo set to index
external files. I've got a handful of those pdf files that I don't want
indexed. So, to satisfy google, and other search engines, I use a
robots.txt file so they aren't indexed.
My question is, how do I get typo to honor that robots.txt file, and
skip indexing those handful of pdf files?
> -----Original Message-----
> From: typo3-english-bounces at lists.netfielders.de
> [mailto:typo3-english-bounces at lists.netfielders.de] On Behalf
> Of Dmitry Dulepov
> Sent: Friday, June 03, 2005 3:29 AM
> To: typo3-english at lists.netfielders.de
> Subject: Re: [Typo3] Robots.txt and indexed search
> I did not try but you can: add
> <!--TYPO3SEARCH_begin--><!--TYPO3SEARCH_end--> on those pages.
> This will cause indeved search to "index" empty text between this tag.
> Jody Cleveland wrote:
> > Hello,
> > I'm running typo 3.7 on a redhat 3.0AS server.
> > I've got the Indexed Search Engine version 1.2.8 running in
> there too.
> > It's setup to also index external files, which is does
> well. There are a
> > few files I don't want indexed, which I've got listed in a
> > file. How do I get typo's indexed search to honor robots.txt for
> > searching external files?
> > --
> > Jody Cleveland
> > Computer Support Specialist
> > cleveland at winnefox.org
> Typo3-english mailing list
> Typo3-english at lists.netfielders.de
More information about the TYPO3-english