An avalanche of traffic (second only to Google) from Spain yesterday, which it turns out traces back to an oblique two word link on the Spainish launguage blog Vicisitud y Sordidez.

The linking words were “tiempo real”, and linked back to a post titled Memo to Sienna Miller: Real sex does not have jump cuts that was half google-bait, and half a meandering riff on the way the passage of time is depicted in love-scenes.

Judging from the number of people who visited via that one little link, Vicisitud y Sordidez has a readership that would makes the world of sex-blogs, even top tier sex blogs, look pretty small. But my command of the Spanish launguage is barely sufficient to allow me to travel in Mexico and usually get on the bus to the right destination, or actually get what I want in a restarunt. It’s no where near sufficient to make heads or tails of what sort of blog Vicisitud y Sordidez is. Even with the help of the Google translator, I really don’t know either the post in particular, or the blog in general is about.

Whatever the case, I think it’s pretty nifty that six months after I made the “jump cuts” post, such a well-trafficed blog found a reason to link to it. But it still leave the question: Quien es Vicisitud y Sordidez? (Sorry, I can’t find the right key combination to put the upside-down question mark at the front of the sentence.)

Well maybe it’s me.

After a week of tearing my hair out and exchanging a lot of email with SEO specialist Danny Sullivan, Seth Finkelstein, and Phillip Lenssen, and coming up with a lot of theories why ComstockFilms.com fell of the face of the earth on some of our most important search terms, this morning, while chewing over it some more I remembered three different, vaguely connected things and maybe, just maybe I found a reason that this has all happened.

Last week, against the worry that some of our feeds might have been looking like duplicate content, we made changes to our robots.txt file. We checked our robots.txt file in using Google’s webmaster tools, and everything seemed fine. It looked like it was doing what it was supposed to be doing.

But this morning when I tried to feed our robots.txt file to Google’s URL removal form, it spat it back out and accused us of bad syntax.

Really? I went back and checked it with Google’s webmaster tools. All a-okay.

But just for fun, I re-wrote the (alledgedly) offending lines, and bingo, the URL remover ate the file like orange sherbert.

A few hours later there are still some goofy things on the first page of our Google site index, but at least our index.html is there too.

My theory is the bad syntax opened a door into all those files we didn’t want the Googlebot messing around in, and that’s how all those nasty, outdated URLs ended up in our site index, and pushed out all our nice new Relevant URLs. Hopefully by tomorrow morning it will look better, not worse, and maybe by the end of next week things will be back to where they were at the begining of last week.

Or not. Time will tell.

In the mean while, we make a lot of progress on ASHLEY & KISHA and I cold-called like a mad man and picked up a few new retailers. I remember reading about the North Korean human wave tactics and how they would simply flow around restance points and envelop the enemy. This week Peggy and I were our own little human wave. The Google issue resisted everything we could come up with, so we flowed around it and claimed victories in production and sales. I’m exhausted, but proud of us. Whether or not this robots.txt thing is the fix, tomorrow morning’s lox and bagels will feel well-earned!

Oops. I missed it!

The very first post.

The first blogaversary post.

Since starting this blog, we have released XANA & DAX, DAMON & HUNTER, and MATT & KHYM.

Along the way we’ve had some interesting adventures, and a few successes.

In not too many more weeks we’ll release a new film, which from the looks of it will be as sweet and lovely and sexy as anything we’ve made so far.

So so far, so good. We’ll see that the next year holds!

So with a new title out (MATT AND KHYM) and DAMON AND HUNTER up for a GayVN award, I thought I should head over to technorati and program some new blog mention searches as feeds. (Another aspect of the always endearing Media Whore/Trafic Whore part of being an independent filmmaker.)

While I’m doing a few test searches to see what’s going to give me good results without overwelming me I notice that now there’s a drop down box. I can ask to see results from blogs with “a lot of authority”, “some authority”, “a little authority”, or “any authority”. It’s certainly interesting to search Comstock Films with varying levels of authority and see who Technorati holds in high esteem.

One blog utterly absent from any of these searches is Violet Blue/TinyNibbles. Considering the phrases I’m searching for (ie things I know Violet has written about recently,) the result make me more inclined to question Technorati’s authority than to question Violet’s.

I’m sure it’s just a bug, just an algorithym tweak or two and the good folks at Technorati will have it sorted in no time. Right?

A few days after the Boing Boing post about Google’s sexblog snafu, I was exchanging e-mail with Mark Glaser, who writes the PBS blog MEDIASHIFT. Mark also talked to Violet Blue, Matt Cutts, and Danny Sullivan, and the result is a very thoughtful piece about how the power wrapped up in Google’s search algorithyms can hurt the little guy.

The piece opens with a long quote from Chelsea Girl’s Love Letter to Google. It quotes both me and especially Violet quite a bit. It quote’s from my blog, and even refers readers to it. But the piece does not link to my blog. Nor does it link to Comstock Films. And though it mentions them, it does not link to Pretty Dumb Things or Tiny Nibbles either.

It does link to Search Engine Land, it links to Matt Cutts, and it links to Boing Boing.

Perhaps it’s just an oversight. I hope so, but ten years’ experience of trying to bring the best of what I have, as a filmmaker and as a human being, to the depiction of sex makes me doubt that’s the reason that the PBS article doesn’t link to Comstock Films.

We’ve had printers refuse to print our inserts and posters because they were “pornographic”. I had my words used without attribution, let alone a link. Hell, I’ve even had a government ban one of my films from an entire continent.

So when PBS runs a story that started with a post that I made on my blog, and then doesn’t even link to my blog, it doesn’t surprise me. Am I disappointed? Yes. But surprised? No.

Fortunately, our situation with Google seems stable. If anyone goes looking for us as a result of the PBS piece, they should be able to find us!

P.S. Should anyone care to contact PBS ombudsman Michael Getler, you can do so here. Tell ‘im Tony says “hi”.

P.S.S. I knew something wasn’t quite right. The editors at MEDIASHIFT had no problem linking to TinyNibbles.com in the January 4 Top 5 Stories post. What do you supposed happened between this week and last?

So last night I ended up having an hour-long phone conversation with David Cay Johnston, the New York Times reporter whose byline appears on yesterdays article Indications of a Slowdown in Sex Entertainment Trade.

Mr. Johnston pointed out that 1) the article’s headline was “slowdown”, and 2) the article mentioned that independently generated financial data about the “adult entertainment industry” is all but impossible to come by. Fair enough. I think the article might have made the second point more forcefully, and perhaps sited financial speculation about the size of “the industry” that doesn’t come from people with either an pro-porn or anti-porn agenda. (If he were a regular reader of this blog, he’d know where to look.

We also talked about the book Innumeracy. It turns out it was written by a friend of Mr. Johnston’s, and aside from his reporting duties, he often lectures to journalism students about the importance of understanding numbers.

Everyone’s got an agenda, even me. In my essay The Porn Monster, I laid out my case for the stake that pro-industry people, anti-porn people, journos, and academics have in the continued propegation of (what I feel are) figures for the size of the “adult industry” that simply aren’t supported by evidence or common sense. In short, if you go looking for money in the adult industry, you can’t find it.

Now I’ll cop to my agenda.

Hysterionic reporting on the porn industry, especially the grossly inflated size of the porn industry has given rise to the popular notion that the industry is a beheomuth that nearly perfectly serves the erotic entertainment needs of the public. A phrase you’ll hear time and time again in “the bizz” is, “With 10,000 titles a year, there’s something for everyone!”

Looking from the outside, the existence of title like DIRTPIPE MILKSHAKES (volumes 1 and 2) lends credence to this idea. After all, if a $13B/year industry is making dozens, perhaps even hundreds titles a year devoted to such exotic sexual interests as women eating semen out of other women’s anuses, then certainly there must be something out there for people with more pedestrian tastes – things like convincing, well-crafted depictions of mutually pleasurable sex.

But while there’s no shortage of anal creampie themed videos, gaping anus themed videos, and other things to unsavory to mention on this blog, finding well-crafted sexually explicit films that convincingly depict mutual pleasure is all but impossible. As I said to Stacy Grenrock Woods in Esquire a couple of years ago, it’s easier to find a well-made fishing show than a well-made sex film.

Porn’s supposed to be this multi-billion dollar a year business, so big and dangerous there’s an entire department at the DOJ devoted to it; and it churns out thousands and thousands of titles each year, seeming to serve every nitch fetish, but it can’t seem to serve the wide-spread and basic desire that many people have to see a well-crafted depiction of two people who really seem to be enjoying having sex with each other.

People know in their gut something’s not right. People know there’s a disconnect. People know that what they want to see isn’t some specialized nitch, it’s a basic human desire. Yet it goes unserved. Why? To me the answer is quite simple.

The restrictions on the distribution of erotic images (as in you won’t be able to find MATT AND KHYM at Walmart, Blockbuster, etc.) has restricted the business to making money in a very few, and not especially lucrative ways. Porn margins are razor thin, and the result is that “the industry” vastly overserves niche sexual interest markets, where issues of production quality, or even simple honesty in packaging will be overlooked, while it vastly underserves sexual interest with broader appeal, but much higher expectations.

The combination of the digital video revolution and the internet has removed virtually all barriers to entering the market. These days, any idiot with a BestBuy credit card can make and market porn, and that’s just what’s happened. And anyone who’s taken a highschool economics class knows what happens when too much supply chases too little demand. Certainly Greg Zobary does (from LukeIsBack.com)

Luke: “Do you think there are any millionaires in the industry who are solely employees?”
Greg: “No.”
Luke: “Do you think there are any billionaires in the industry?”
Greg: “No way.”
Luke: “Maybe this isn’t a $12 billion a year industry.”
Greg: “It’s a $400 million [DVD] industry, maybe $500 million. The industry went out and promoted these figures that included strip club revenues, hotel revenues, etc and came up with this [$12 billion] figure, hoping it would lead to the legitimization of the industry. What it has really led to is a bunch of idiots who watch this stuff and think that porn is the new gold rush. They jump in and produce a few movies and think they’re going to get rich. Everyone I’ve seen who’s done that has walked away with no money. We no longer insure these people. They don’t stick around.

But even if you take the well-publicized porn figures at face value, porn just isn’t that big a business when you compare it to other things American’s spent money on. For example:

According to the American Sport Fishing Association who take their data from the US Fish and Wildlife Service, in 2003 Americans spent over $5 billion on equipment, nearly $15 billion on fishing trips, and some $20 billion more on boats, trucks, licenses and other fishing-related products and services. Anglers paid $290 million on ice alone and the total annual economic impact of the sport fishing industry for 2003 was estimated at $116 billion.

According to PackagedFacts.com in 2005 the market for feminine hygene products was over $3 billion. (As with porn in the NYT article, apparently the tampon industry is looking for ways to counter the effects of an aging population on their bottom line.)

Do people spend money on porn? Obviously yes. Is porn “big business”. That depends on whose numbers you use and what you compare it to, and what you think “big business” is. It might be bigger than tampons, it’s definately not as big as fishing. Do we spend to much time thinking about porn? Maybe. Do I? Definitely!

I’m probably late to the party on this (busy making sex films, y’know) but I had a thought: Is the priority that Google places on link quality (and the resulting rankings and revenues) an example of an effort to monetize social capital in action?

Conversely, is the social capital that Comstock Films enjoys in its little corner of the internet devalued by the larger company it keeps? Our outbound links are all either: a) People who have said nice things about us and our work, or b) People, places, or things we think are nifty. Isn’t that what the googlebot likes? Shouldn’t ComstockFilms.com be a “clean and trusted” site? (Sound sort of like talking about VD in the Forties, doesn’ it?)

I used to think we were “clean and trusted”, but in light of my recent sex-related searches, I’m not so sure. Maybe we’re bad company.Does our linking to Tramp Lamps help them or hurt them?

In fact, just now I’ve just revisited them and see they’ve added this bit on their link page, “Thanks for stopping by our little web site. If you would like to link up, and your content would not embarrass my mother, contact us about an exchange.”

Would Momma Tramp Lamp be embarrassed by Comstock Films? Hard to know, isn’t it?

P.S. Maybe this why Andrew Sullivan didn’t link to me when he quoted me. Maybe he’s afraid his clean and trusted blog might catch whatever I have. Is there such a thing as Valtrex for blogs? Blogtrex?

In her follow-up post, Google is Still Pretty Broken, Violet Blue writes:

It’s clear by reading this post that Google absolutely does not have the tools — or current knowledge — to evaluate sex on the web. And possibly a lot of other things as well. This is how they do it? Really? They need a community liaison for each of the types of spam they’re expected to deal with, because it’s crystal clear they are in the dark.

Over at Threadwatch.org, Mr. Turner writes:

To understand why the results are poor in adult, you have to understand how adult sites work. You essentially have under 1% of the sites that actually sell something, and over 99% of the sites pushing traffic to those sites as affiliates. There is very little middle ground here. You have very few sites that put up porn for the fun of it. It’s essentially like taking a mainstream industry and dropping every site in it that doesn’t sell something or is an affiliate for someone.

**I’d also like to add that it looks like they have been targeting blogs of late in Google. Not older established blogs, but new ones that generate feeds, trade links with other blogs, and never obtain links from good neighborhoods. I think the fact that sex blogs don’t rank as well also has to do with the fact that newer and smaller blogs are having a much more difficult time than before.

In a comment at Matt Cutts blog I wrote:

Last week when I needed some rubber tipped flu-flu arrows, I had no trouble using Google to find what I needed, compare products and prices, and make a purchase. But then you don’t see a lot of archery spam, do you?…are sex-related searches important enough in the grand scheme Google and the internet as whole to warrent an ombudsmen devoted having a person or even small department devoted to making sure sex related search results as relevent, useful, and informative as archery querries?

I don’t really encounter that much porn related spam in my websearching because for the most part, I already know where to look and how to look. So I have been mulling over this notion of good neighborhoods, trusted sites, and the rest with fresh eyes, and I spent sometime this morning trying to imagine I was someone who was trying to use Google to find sexual information and entertainment, but who hadn’t spent the last 15 years of his life learning where and how to find the good stuff. In this case, we’ll assume that the searcher in question has an interest in women’s sexual pleasure that is similar to mine. Some Google searches:

videos showing real female pleasure

videos showing authentic female orgasms

is female ejaculation real

real female exhibitionist videos

And just one for TMI

hardcore pictures of redheads with freckles

I wouldn’t go as far as saying the above results are useless, but then I’m looking at them as someone whose journey began nearly 20 years ago when I sent Xandria five bucks for their catalog (which turned out to be same flesh-colored rubber dildos as everywhere else, but with inoffensive names and illustrated with demure line drawings.)

Now some archery searches:

flu flu arrows

archery supplies

Notice a difference? I sure do, and I don’t think it’s all Google’s fault.

While I do think the name-search thing was a grade-a fuckup, I think it’s a ligitimate question to ask how much effort Google (or anyone else) should put into figuring out how to separate the wheat from the chaff when it comes to online sex.

The sex industry is mostly shabby, and nowhere is the shabbiness to sexy ratio higher than online. Even if porn is a $52 billion/year business (a highly dubious figure,) it’s still not as big as the recreational fishing industry. How much time do you think Google spends worrying about fishing spam?

That doesn’t mean I’m letting Google off the hook. (Robots don’t groan at puns!) Google’s business is Relevency, and no matter how you slice it, beastiality and anal rape sites should never rank ahead of Comstock Films when someone searchs ‘couples porn’.

Meanwhile, it’s is a good reminder that what we do, or what Violet does, what Njoy does or what Chelsea Girl does, we do against an overwhelming tide of badness and indiffernce (Doubt it? take a look at the Digg comments when the story broke.) Safely ensconsed in our community of like-minded people, it’s easy to think that the diffence between Comstock Films and a TGP link farm is as self-evident as the diffence between an Njoy buttplug and a jelly rubber buttplug.

It’s not. Not to everyone. Not yet!

In an e-mail to Matt Cutts, I wrote:

It’s hard for me to image what sort of algorithm would be able to distinguish the highly entertaining, very intelligent, but often utterly filthy Pretty Dumb Things (apparently still in the Google penalty box) from run-of-the-mill sites that use similar language in similar quantities, and even in similar, but tremendously less artful ways.

Right now Peggy and I are re-jiggering page titles and writing meta-tags. In other words, we’re writing for robots, and it’s the most hateful writing assignment I’ve ever had.

If what I’ve learned from Matt and Danny and others is correct, robots are decidedly unimpresses by nearly every aspect of language I adore. Robots don’t care about alliteration, simile or metaphor. Robots don’t subvocalize when they read.

Robots don’t cringe at puns. Robots can’t appreciate a well turned phrase. Robots might write poetry, but they don’t read it. They don’t appreciate music, and they don’t appreciate the musicality of words lined up just so.

I am writing for robots, and I hate it.

Thank God I don’t have to make movies for them.

In all that discussion of machine vs. human search, this sort of got buried in the preveious post, so I’m pulling it out for a post of it’s own. Google’s Matt Cutts askes:

I think it would be a great question to open up to your blog. Maybe a good way to do it would be to ask people what their favorite three sex-industry sites are?

Here’s my answer:

Violet Blue’s Tiny Nibbles
As far as I’m concerned, Violet is the internet’s most passionate voice for sex being treated in an intelligent, adult and fun-loving way.

Adult DVD Talk
ADT is giant repository of viewer-written reviews of all sorts of porn, erotica, and adult films (pick the name you like). I rarely read them. I am, however, addicted to hang out in the ADT Discusion Forums. Purile enough to make things fun, smart enough to keep things interesting.

Blowfish.com
It’s where I shop when I want something new to shove up my ass.

Here’s an answer from Drew Black at ADT:

Here’s three recommendations for quality businesses in the adult DVD retail sector: adultdvdempire.com, xrentdvd.com and bushdvd.com. These three companies are class acts and constantly improve their sites to the benefit of their customers.

We’re still waiting on Matt’s list, but I’m not sure how comfortable he is with sharing his sexual proclivities with the world. But I know lots of you delight in sort of sharing (just as much as we delight in reading it!), so please do – either here, on on a blog if you have one!