PhotoPost Community - View Single Post - Google and Yahoo Slurp spiders. Robot text

niteflyer32 · October 12th, 2011, 09:29 AM

Running vb 4.15

Integrated Photopost 7.1

Use Spider-friendly URLs? = Yes

The Google and Yahoo Slurp spiders seem to be hanging up trying to index our Photopost and Reviewpost. What directories and files can we block them from? Right now I've got /photopost/ and /reviewpost/ disallowed in our robot .txt file and that doesn't seem to be blocking them.

Bing, Lycos, Google FeedFetcher Spider and other spiders seem to be indexing our forum okay.

Thanks

October 12th, 2011, 09:29 AM	#1 (permalink)
niteflyer32 Junior Member Verified Customer Join Date: Mar 2009 Posts: 3	Google and Yahoo Slurp spiders. Robot text Running vb 4.15 Integrated Photopost 7.1 Use Spider-friendly URLs? = Yes The Google and Yahoo Slurp spiders seem to be hanging up trying to index our Photopost and Reviewpost. What directories and files can we block them from? Right now I've got /photopost/ and /reviewpost/ disallowed in our robot .txt file and that doesn't seem to be blocking them. Bing, Lycos, Google FeedFetcher Spider and other spiders seem to be indexing our forum okay. Thanks