View Full Version : Disallow the data directory in my robots-txt file?
September 12th, 2012, 04:23 PM
I'm curious why my data/ directory would be disallowed (as it is in a default installation of PPPro) in my robots-txt file. Wouldn't this directory being crawled be good for SEO?
September 12th, 2012, 05:30 PM
There are no scripts in the data directory for bots to follow in fact there is a blank index fille so nothing is viewable so nothing to crawl. It is to proect your images. The necessary scripts for google etc to crawl are not blocked.
September 12th, 2012, 06:53 PM
Ahhh. OK. Thank you, Chuck, for the quick reply.
September 12th, 2012, 07:06 PM
Not a problem glad to assist.
September 13th, 2012, 01:24 PM
Here is a good example on the basics its kind of long but is information.
HowStuffWorks "How Internet Search Engines Work" (http://computer.howstuffworks.com/internet/basics/search-engine1.htm)
September 13th, 2012, 08:59 PM
That's great! Thank you, Chuck. :)
September 13th, 2012, 09:14 PM
Yeah thought that might help you understand how some things work with SEO. You do not want to have spiders crawl some stuff.
September 13th, 2012, 09:17 PM
Absolutely. For some reason I thought having them crawl the names of the files in that directory would be a good thing.
Fortunately developers like you know what you're doing. :)
September 13th, 2012, 09:40 PM
Yes does not work like that they crawl pages on a site and grab data. ;)
September 14th, 2012, 02:43 PM
Here are the top ten things a webmaster can do to help there site for better seo overall
Definately in our program you want to make sure our HTACCESS file is renamed to .htaccess and turn on the SEO feature.
This tutorial should assist you in determining the best meta tags to do and other things with the rest of your site to promote it.
vBulletin® v3.8.1, Copyright ©2000-2014, Jelsoft Enterprises Ltd.