[TYPO3] Google spider problem

[Piotr Burda]

Piotr Burda napisał(a):
> hi
> 
> I have made www.nowinek.pl (rss agragator)
> But i have big problem with google spider who don`t want to index my 
> site. In my stats i see that google bot is visiting my site.
> 
> I`ve made some investigations about how google see my site, and the 
> result was a big bad surprise:
> 
> http://www.gritechnologies.com/tools/spider.go
> 
> and similar services, said:
> 
> Error 503
> 
> service temporarily unavailable.
> 
> in same time service was perfectly visible for browsers.
> 
> linki.nowinek.pl works ok and are "visible" for spider.
> 
> 
> Why my typo3 responds to spider that service is unavailable, or mayby 
> the problem is not typo3 related... ?
> 
> 

It was a problem caused by select pro extension, this extension makes 
that the pages are not cached so the response to spider is timed out.

after deleting this line $GLOBALS['TSFE']->set_no_cache();  from 
class.tx_gstselectpro_pi1.php all seems to work.

Piotr Burda