millions of weird urls being indexed and errored by photo post

[dlpcorp]

have a decent sized photopost gallery, and starting in dec when we upgraded to the latest version, we are getting millions (literally) of pages being indexed by google that function, but link to nothing. the urls dont make sense. for example:

/mygallery/showgallery.php?cat=wdsbwgjgr&page=11

there is no category named "wdsbwgjgr". photopost loads up the entire contents of the gallery though.

the problem is - every thumbnail on the page links to a broken page:
/mygallery//my-image-name.html

which doesnt work, as it seems to be missing the cat name.


i have scoured my site, and there are no links to weird urls anywhere i can find.


my biggest issue now: google has just notified us that they penalized us doe to excessive missing content and broken links. we have PLUMMETED in the rankings across the board, and have had 2.25 million (good) pages, de-indexed.

please let me know if you have any ideas on what could be causing this. im out of ideas.

im using the standard htaccess file, but have just added a 404 redirect just in case.

Content visible to verified customers only.

(i'll pm the url if needed)

[Chuck S]

The problem is this probally has nothing to do with your site or our program.

First off your talking two different things which is causing your issue because your urls are bad.

Your bad links are probally from some spam bot or bad google bot I would have no idea because quite frankly from our programs operation the cat variable is always NUMERIC.

Now given that constant here in our programs normal operation your now electing to use SEO urls which is fine. However your url is bad so your seo url is not going to be correct.

/mygallery/showgallery.php?cat=wdsbwgjgr&page=11

The proper url is probally something like

/mygallery/showgallery.php?cat=501&page=11

A url like that would be converted into a category seo url something like this

/mygallery/g501-my-cat-name-page11.html

You can see that as an example here

Toys that Go - PhotoPost Photo Gallery

A proper photopost url to a photo would be say this

/mygallery/showphoto.php?photo=31&cat=501&title=sometitle

That would translate to say this

/mygallery/some-cat-name/p31-some-title.html

see an example of that can be seen here

2006_Fab_Fords_Mustangs_243 - PhotoPost Photo Gallery

So really whatever is going on with your google links or how they got there has nothing particularly to do with our program or its operation.

I can not say how those links got in google since our program has nothing to do with putting links in google.

You can add 404 redirects as a backup but your issue has to do with google and its operation. Google is the one who puts links in there site. They are not getting those links from your site or our program and you need to tell them that. Ask them where they are getting those bad urls because they do not come from your site.

You can turn off seo urls but personally I have google flush your sites links and do a complete reindex of your site because the issue is from them.

[dlpcorp]

right... not saying this is photoposts fault obviously. however, something needs to be done about 1 thing:

if someone hits a url that doesnt exist in photopost (ie: /mygallery/showgallery.php?cat=wdsbwgjgr&page=11) then it should be stopped cold... instead, the ENTIRE gallery (all categories) are piled into one, and displayed... meaning MILLIONS of pages (depending upon the size of your gallery) could be instantly grabbed and indexed... all pointing to things that dont exist.

re-index is out of the question. this is a site with 22.5 million pages... it has taken way too much time to get it all indexed.. and google, at its peak crawl rate, would still take months to get it all back.... all the while, our business is suffering horribly.



i dont like hacking the core of a software package.. but there needs to be something done to handle this issue. ie: if (!cat) {echo ' no category found'; }else {.... list the cat }

[dlpcorp]

can you point me to where i need to make that change to save me some time?

[Chuck S]

The point I am making is our program doesn't have any bad urls.

If there is really a problem with the url you are entering in our program then you get a diewell message which is the proper action. So you get cat can not be found or photo can not be found. That already exists in the program.

You need to find out where google is indexing bad urls from. They are not coming from your application.

I merely said if you have bad urls indexed from google then you can minimize any damage by turning off SEO URLS in global options since the information your using to convert to seo is wrong.

There is nothing I can tell you to do about bad information that is being generated from an outside source that is not our application. If the photo is not found you get a message. The point i have beem making is it is google that is making the mistake if they are indexing pages with wrong data.

Our application has nothing to do with google. I know you want more of an answer but there is literally nothing I can do about something outside our application. I can not fix some bad url in google.

[Chuck S]

If you want to hack your program and kill the ability to show all photos the only other suggestion I have is maybe in showgallery.php

Content visible to verified customers only.

You could change to maybe

Content visible to verified customers only.

There is not support for such an action and do so at your own risk. I can offer no further discussion on code mods.

[dlpcorp]

thanks, but that didnt work.




as for the issue... i dont think you understand what im saying here.

i know your program is not responsible for generating these weird urls. the creation of these urls has nothing to do with you or photopost. it is most likely a bot or spammer.

the problem however, is that your program does WORSE than nothing with it. if a random string is presented as a cat, one which simply doesnt exist... rather than handling it as an error, it presents all of the galleries content.

here is why this is a problem (and a huge problem):
1 or two pages of duplicate content is not that big of a deal. but, with anyone that has a decent sized gallery (we have roughly 50k images)... it leads to the potential to create HUGE number of errors. google will annihilate your site if it sees thousands of pages of duplicate content, much less millions. this is what happened to us (and this is not speculation... this is in conversations with google over this matter... we have just been reindexed). each one of these random string urls that was most likely pinged by a bot or server looking for exploits, was picked up by google. each one of those random string creates en entire gallery worth of empty links, and millions of pages of duplicate content.


i hacked around a bit in the code, but they were messy and created other issues. for us, using mod rewrite, this is probably the most elegant solution i could quickly come up with (had to come up with something yesterday so google would reindex us)

$qqqcurrent = $_SERVER['REQUEST_URI'];
$qqqcheck = 'showgallery.php?cat=';
$pos = strpos($qqqcurrent, $qqqcheck);
if ($pos === true) {
header("HTTP/1.1 301 Moved Permanently");
header("Location: /mygallery/");
}

[Chuck S]

the problem however, is that your program does WORSE than nothing with it. if a random string is presented as a cat, one which simply doesnt exist... rather than handling it as an error, it presents all of the galleries content.

That is because it is suppose to as there is an option to show all photos which is why as I initially said cat=all if it is not numeric.

What I said does indeed work

Message - PhotoPost Photo Gallery

There is one more thing you might want to modify up at the top of the file on the typecast line where cat is STRING change that to INT

Content visible to verified customers only.

I understand exactly what your saying but you need to understand it is not our place to fix google issues. The problem is as I see it is why is google indexing spam bot urls in the first place and where. They are not getting the urls from our application.