Well maybe it’s me.

After a week of tearing my hair out and exchanging a lot of email with SEO specialist Danny Sullivan, Seth Finkelstein, and Phillip Lenssen, and coming up with a lot of theories why ComstockFilms.com fell of the face of the earth on some of our most important search terms, this morning, while chewing over it some more I remembered three different, vaguely connected things and maybe, just maybe I found a reason that this has all happened.

Last week, against the worry that some of our feeds might have been looking like duplicate content, we made changes to our robots.txt file. We checked our robots.txt file in using Google’s webmaster tools, and everything seemed fine. It looked like it was doing what it was supposed to be doing.

But this morning when I tried to feed our robots.txt file to Google’s URL removal form, it spat it back out and accused us of bad syntax.

Really? I went back and checked it with Google’s webmaster tools. All a-okay.

But just for fun, I re-wrote the (alledgedly) offending lines, and bingo, the URL remover ate the file like orange sherbert.

A few hours later there are still some goofy things on the first page of our Google site index, but at least our index.html is there too.

My theory is the bad syntax opened a door into all those files we didn’t want the Googlebot messing around in, and that’s how all those nasty, outdated URLs ended up in our site index, and pushed out all our nice new Relevant URLs. Hopefully by tomorrow morning it will look better, not worse, and maybe by the end of next week things will be back to where they were at the begining of last week.

Or not. Time will tell.

In the mean while, we make a lot of progress on ASHLEY & KISHA and I cold-called like a mad man and picked up a few new retailers. I remember reading about the North Korean human wave tactics and how they would simply flow around restance points and envelop the enemy. This week Peggy and I were our own little human wave. The Google issue resisted everything we could come up with, so we flowed around it and claimed victories in production and sales. I’m exhausted, but proud of us. Whether or not this robots.txt thing is the fix, tomorrow morning’s lox and bagels will feel well-earned!

This entry was posted on Saturday, February 10th, 2007 at 6:29 pm and is filed under Blogging, google. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.