Spider Friendly URLs are not 100% spider-friendly.

[SaN-DeeP]

I am sure devs must be aware of this as well ?

Let me take an example of PhotoPost Gallery I will add list of places which needs to be spider friendly as well.
Browsing gallery as a guest user.

Homepage - http://www.photopost.com/photopost/index.php
1. Recent Photos 1 day 7 days 14 days all
2. Calendar - http://www.photopost.com/photopost/s...ate=1144823887
3. Search results

ShowGallery (Category Pages) - http://www.photopost.com/photopost/s...ry.php/cat/507
1. Category - Recent Photos 1 day 7 days 14 days all
2. View Menu Selection for Gallery, Thumbnail, with Titles, and group by member
3. User Profile links on showgallery/category pages.

ShowPhoto Pages - http://www.photopost.com/photopost/s....php/photo/237
1. SlideShow Link
2. Photo Poster Profile Link
3. Photo Poster Gallery Link


User Profile Page - http://www.photopost.com/photopost/member.php?uid=279
1. Last Photo Uploaded Link
2. Last Comment Posted Link

this needs needs immediate attention to avoid duplicate URLs problem and better spider visibility ?
If someone else can trace out additional URLs we can help making a better future release

[Michael P]

Duplicate URLs is not a problem for spiders - in fact, there is no reason not to have duplicate URLs from the articles I have read (take vB's archive urls as well as regular forum URLs, for example).

[SaN-DeeP]

Michael, why not make the gallery more spider-friendly, I am sure the above changes wont be a big hurdle to implement ?

[Michael P]

I guess I'm not following you on the "better" part....

[Swanny]

I'd also prefer to see just one URL for each page. I've noticed that in other PP products, the link on the photo is different from the URL on the text. Both point to the same web page.

I'm sure Google can handle this, but why wouldn't you just point to one "master" URL? It would mean Google would spider half the URLs and get all the same content. That means less bandwidth, less crawling, etc. I guess that's how it's "better" ;-)

[Michael P]

The URL on the showphoto page could be changed to reflect the spider friendly URL instead of the php formatted URL. Again, I think that this is a nonissue; but am willing to listen if you can show me differently.

[Swanny]

Michael, here's the point:
In spidering PhotoPost installations, search engines come across 4 URLs (mabye more?) that all point to the same page:
1. In the "Most Recent Additions" area, the URL looks like this - http://www.fordf150.net/photos/showp...4/limit/recent
2. In the showcategory page, the photo URL is this - http://www.fordf150.net/photos/showp...19684/cat/3790
3. On the member's page, the "Last Photo Uploaded" URL looks like this - http://www.fordf150.net/photos/showp...hp?photo=19684
4. When searching on the photo title, the results page has this URL - http://www.fordf150.net/photos/showp...ics/perpage/18

Sure, it all goes the same place. But Google will index it FOUR times and then have to figure out what the "real/right" URL is. That means 4 pageviews instead of 1, and potentially 4x the bandwidth. Make sense?

[Johnny Doomo]

You have no idea. How about making it use modrewrite and put the category and even the sub category into the url as well?

So if I have a this category structure

Action
-D
--Doom 3
---PC

The url would at least have Doom 3 and PC in the url. I would get soo many more hits. Plus in todays day-in-age of others trying to outrank others by simply making better SEO I think a lot of PP users would benifit from it.

[Garrynz]

What I have done is make the URL's like /photos/showphoto/photo/778.php on the categories and left it like /photos/showphoto.php/photo/778/limit/recent etc for the rest of the url's and then added Disallow: /photos/showphoto.php to my robots.txt it is alot of work to figure it all out and you need to add a bunch of mod_rewrite rules.

[Michael P]

Swanny, you can't have the same URL to the photo pages every time and still maintain all the functionality of PhotoPost.

Take #1 and #2 for example. In #1 you went to the photo via the Recent Photo Uploads feature box, clicking on the NEXT button will take you to the next most recent photo uploaded. In #2, you went straight to the photo within a category, so pressing NEXT on that page will take you to the next photo in that category.

Passing parameters is required if you want to maintain the continuity of what you are doing; otherwise regardless of how you get to a photo, the NEXT and PREV buttons will only work one way and not the way people expect them to.

[Garrynz]

No you can't Michael but it would be good if it was like /photos/showphoto.php/photo/limit/recent/778 instead of /photos/showphoto.php/photo/778/limit/recent as then you could add Disallow: /photos/showphoto.php/limit/ to robots.txt I don't know if this could be done though?

[SaN-DeeP]

Originally Posted by Swanny

Michael, here's the point:
In spidering PhotoPost installations, search engines come across 4 URLs (mabye more?) that all point to the same page:
1. In the "Most Recent Additions" area, the URL looks like this - http://www.fordf150.net/photos/showp...4/limit/recent
2. In the showcategory page, the photo URL is this - http://www.fordf150.net/photos/showp...19684/cat/3790
3. On the member's page, the "Last Photo Uploaded" URL looks like this - http://www.fordf150.net/photos/showp...hp?photo=19684
4. When searching on the photo title, the results page has this URL - http://www.fordf150.net/photos/showp...ics/perpage/18

Sure, it all goes the same place. But Google will index it FOUR times and then have to figure out what the "real/right" URL is. That means 4 pageviews instead of 1, and potentially 4x the bandwidth. Make sense?

while serps visit the same URL from 4 places(different URLs) its very difficult to gauge which page shall be taking the PR value..
Note: There are atleast 4 different URLs for same page... that means google PR is shred at 4 different places.

Can we have some ability like permanent links (in wordpress ?)
Morever, we should definetly make the gallery 100% spider-friendly if this wont be a big hurdle ?

Regards,

[Michael P]

Originally Posted by Garrynz

No you can't Michael but it would be good if it was like /photos/showphoto.php/photo/limit/recent/778 instead of /photos/showphoto.php/photo/778/limit/recent as then you could add Disallow: /photos/showphoto.php/limit/ to robots.txt I don't know if this could be done though?

No, that wouldn't work because of how parameters are passed and expected; you have to have variables and values paired up (i.e. next to one another) for them to work.

[SaN-DeeP]

http://www.photopost.com/photopost/s...rs.php/cat/500
caught another URL - none of the member gallery pages are spider friendly

[Michael P]

Once again, I see no reason that it isn't 'spider friendly'; PhotoPost spider friendly URLs are extremely effective with search engines. Can you provide me with current reference material which would suggest otherwise?

Keep in mind, once again, if you modify these URLs you will lose some functionality.

[gloggs]

I suspect that people are more concerned here with duplicate links going to the same places, rather than spider friendly URLs? Spiders have no problem with even quite complex dynamic URLs these days. The issue is duplicate content, or the appearance of duplicate content.

Contrary to popular belief there is no penalty for duplicate content. All that will happen is that Google and the SEs will chose the URL they feel is most appropriate. But this is where the issue becomes a potential problem. More for user friendliness than for search engine friendliness, IMHO.

From a user friendly point of view, it is best if the same url is shown for each unique page of content. That way when users email their friends the url of their gallery, (for example) it is the same URL each time - everyone is 'on the same page'.

As a byproduct of this, a single URL for each unique viewable piece of content will also help the search engines to spider a site and assign the correct level of 'ranking' for that content - which is also a usability issue, rather than a search engine 'promotional' issue IMHO.

As for the technical issues involved to fix this, I haven't got a clue. I can't even hand code HTML.

My 2 cents.

[greenflag]

T2DMan here again.

For pp, it is essential that there be functionality. But for SEO, as very ably mentioned, there needs to be just one url per page of content.

Why

• one url means that you get the most internal links "voting" for that one url rather than the internal links being split between multiple url's, therefore being a less powerful url.
• as mentioned, duplicate content in its own right wont necessarily stop you getting the url spidered, but google will pick one url to show (if in fact they are that similar).
• one url means that any inbound links will be to only one, and so that one will be that much more powerful

When it is not possible to have one link, you then make those links as rel=nofollow. vBulletin started using rel=nofollow on links as a result of my personal insistence. So I do have quite a bit of experience in this regard.

Content visible to verified customers only.

The concept is that any one page has only so much Google PR that it can give to any other page. So if you reduce the number of links on that page by rel=nofollow on the junk links, then the remaining links are that much more powerful.


How to have your cake, and eat it too.

There is such a thing as cookies.

If for the first time you click on a link for say "recent photos" or last "7 days photos", or photos by user

• that first url is correct to have as a distinct url since it is valid for each of those categories
• so on clicking of that category, set a cookie equal to the category name, and then refer to that cookie for slideshows, and prev/next photos through that category.
• therefore, you have only one url per photo, therefore that photo page will get all the inbound links to only it, rather than the links being split between all the different url's for the same photo.

Simple.

Just a pain for me to hack the code to achieve that, since there are so many lines that would need to be changed, and changed every time there was a code upgrade.

Yes, it would be possible to write a vbseo equivalent for pp. But so much more efficient it it were hard coded into the package itself.

Onpage SEO

Its one thing having unique url's. Its quite another thing to have cnotent that is relevant to google. I have created a hack that enables each page to have a better title, unique meta description, h1, and opening paragraph for each page. Using the method, each page becomes relevant on Google for the title of its page. With enough Google PR into the pages, the page are able to be found on Google. It is one thing to be able to be cached/spidered by Google. Its quite another thing to actually rank high.

The methods I talk about have got many vBulletin sites in the top 10 on Google for their search phrases, in conjunction with vBSEO.

I rank top ten for each of the following threads on my site:
Photopost SEO

vBulletin SEO

vBSEO

There are many on this forum that have asked for better SEO. They see the need, I know the need. Please don't just brush this aside. It is well worth it to SEO better. You benefit the whole community.

[kisii]

talking about spider frinedly urls, why is it that I have to use the non spider friendly urls for my gallery to work. If I change it to spider friendly, I cannot navigate on my site at all. HOw Can I fix this?

[Tim L. Walker]

Originally Posted by greenflag

T2DMan here again.

For pp, it is essential that there be functionality. But for SEO, as very ably mentioned, there needs to be just one url per page of content.

Why

• one url means that you get the most internal links "voting" for that one url rather than the internal links being split between multiple url's, therefore being a less powerful url.
• as mentioned, duplicate content in its own right wont necessarily stop you getting the url spidered, but google will pick one url to show (if in fact they are that similar).
• one url means that any inbound links will be to only one, and so that one will be that much more powerful

When it is not possible to have one link, you then make those links as rel=nofollow. vBulletin started using rel=nofollow on links as a result of my personal insistence. So I do have quite a bit of experience in this regard.

Content visible to verified customers only.

The concept is that any one page has only so much Google PR that it can give to any other page. So if you reduce the number of links on that page by rel=nofollow on the junk links, then the remaining links are that much more powerful.


How to have your cake, and eat it too.

There is such a thing as cookies.

If for the first time you click on a link for say "recent photos" or last "7 days photos", or photos by user

• that first url is correct to have as a distinct url since it is valid for each of those categories
• so on clicking of that category, set a cookie equal to the category name, and then refer to that cookie for slideshows, and prev/next photos through that category.
• therefore, you have only one url per photo, therefore that photo page will get all the inbound links to only it, rather than the links being split between all the different url's for the same photo.

Simple.

Just a pain for me to hack the code to achieve that, since there are so many lines that would need to be changed, and changed every time there was a code upgrade.

Yes, it would be possible to write a vbseo equivalent for pp. But so much more efficient it it were hard coded into the package itself.

Onpage SEO

Its one thing having unique url's. Its quite another thing to have cnotent that is relevant to google. I have created a hack that enables each page to have a better title, unique meta description, h1, and opening paragraph for each page. Using the method, each page becomes relevant on Google for the title of its page. With enough Google PR into the pages, the page are able to be found on Google. It is one thing to be able to be cached/spidered by Google. Its quite another thing to actually rank high.

The methods I talk about have got many vBulletin sites in the top 10 on Google for their search phrases, in conjunction with vBSEO.

I rank top ten for each of the following threads on my site:
Photopost SEO

vBulletin SEO

vBSEO

There are many on this forum that have asked for better SEO. They see the need, I know the need. Please don't just brush this aside. It is well worth it to SEO better. You benefit the whole community.

AMEN!

[Michael P]

I think we can look at adding rel="nofollow"s and possible adding a noindex meta tag to pages that are not the "original" image (or category).