On most days my morning routine goes something like this: The alarm goes off about 6:30. I wonder down the hall toward the kitchen. Along the way I knock on Older Daughter’s door and call, “It’s time to wake up.” Once in the kitchen I put water on to boil and wake up my laptop. The next hour is spent multitasking between making coffee for me and Peggy, making breakfast for our girls, making lunch for Older Daughter, and checking overnight e-mail and the previous day’s stats for our website.

The triage for checking stats goes something like this:

First I look for evidence that something’s wrong. Has there been a massive drop in traffic, in general, or from a particular source? If there has, that could mean our server’s down, or been hacked, or that Google’s lost control of the Googlebot again.

Next I look for evidence that something good has happened. Maybe there’s a big bump in search traffic from Russia, and that’s how I find out we got a nice mention in Pravda.

Last I look for things that don’t quite fit. An increase in visitors on uncommon search strings, or referrals from an unexpected page or site, or an increase in page views on some long dormant blog post. Some of our best marketing ideas have come from trolling through the depths of our keyword logs, or looking at a post that have gone 0 pages-views to 7 page-views overnight, and trying to figure out why. This morning brings two notable items.

The first is that Peggy’s blog post from yesterday, the one where she wrote having to explain E.D. to our daughter, seems to have provoked interest among parents. Deep in last night’s logs are a half-dozen visitors on the string [comstockfilms.com talking to my children about sex]

The other is this touching note we got from a customer:

After having bought both Ashley and Kisha and Matt & Khym I must write you this short message in praise of your work. Those films are further away from pornography than even the average PG 13 high-school comedy, not to mention adult pornography that I’ve either seen or heard of.

I have never, ever, seen physical love portrayed in such a loving way, regardless of genre, fictional or non-fictional.

When my young boys grow a little older and start getting bombarded with (and seeking out) the products of the porn industry, and if I find out about it, these films are what I will give them to watch: “This is what loving sex is about, this is what you can hope for in life. The rest has little to do with any of us.”

Again, thanks.

Google’s Banned Words and  Google No Fly-List really have me bent out of shape. I think the banned words and the No-Fly list together this quote from Matt Cutts Blog give a pretty good window into just what kind of site Google thinks ComstockFilms.com is, and what our place is in Google’s worldview:

“As the head of Google’s webspam team, I prowl around some pretty hairy places on the internet. Almost every day I encounter hacked pages, malware , porn, and generally scuzzy pages. The security model in Google Chrome is much stronger than most other browsers I’ve used. I’ve surfed through hundreds of seedy back alleys of the Internet over the last several months, and Google Chrome has safely kept me from being infected or affected by the junky web pages I encounter.”

When I get asked what my films are about, my stock answer is that if I could put the answer to that into words, I wouldn’t have to be bothered with the time and expense of making films about sex and love, I could just write about them. But there are things about what it feels like to be in love and what it feels like to make love that I just can’t get at with words alone; at a certain point words fail and I have to take up the camera.

But I also have to write about these films. It’s a good way to work through (some) ideas without the expense of actually being in production, and it’s a necessary part of marketing and promoting the work. We need to make a living or we can’t make these films, so I have to write.

But in light of the discoveries of the last few days – what words Google keys off of, and the ways Google uses these words to filter their search result – it’s become increasingly clear that the way I write about these films is hurting us. That in using language that is as down-to-earth and real as the imagery is in our films, our home in the internet has ended up being categorized as one of the “hairy places on the internet” a “seedy back alley”, a “scuzzy” place that decent people will want to avoid.

I know in my heart that this in not true. I know in my heart that the father who sent us e-mail last night is right, and that the Googlebot is wrong. I know that Jenn P and Spaced Cowgirl are right, and the Googlebot is wrong.

The Googlebot is wrong because the Googlebot is a machine. A cunning machine no doubt, with its “advanced proprietary technology that checks keywords and phrases, URLs and Open Directory categories”, but a machine none the less. It’s crude approximation of context might be useful for parsing facts, but not for parsing feelings. The Googlebot can’t parse urgency, passion, desire, need. And it certainly can’t parse love.

No, the Googlebot can’t parse love, and what’s more, it doesn’t even try. It is calibrated to find hate and hurt and harm, and using its crude machine logic, its algorithm, it’s found me wanting. The Googlebot can’t see my films, so it’s looked at the language I use to talk about them, it’s found those words hateful, hurtful, harmful.

So I’m thinking about the words I use to talk about sex with my children vs the words I use to talk about sex with my beloved. And I’m thinking that maybe I’m going to have to start talking about my films as if I were talking to children instead of talking to grown-ups. I’m thinking that I’m going to have to start talking about these films as if I was talking to a doctor a medical problem instead of talking to a lover about desire.

That clinical, muted, euphemized view of sex seems at odds with the joyous carnality I try to convey in my films. It’s not how I talk about sex; not with my wife, not with my friends. To me, this dulled, detached language seems feels as hollow as pornography; each having little to do with my experience of love and sex. I chafe at the thought of self-censorship.

But making films is fraught with compromise. We bend so we do not break.

Okay, so now you’ve read about it here, at Being Amber Rhea, and at Bacchus’s Eros Blog. Google Suggest will autofill somethings, like [stormfront] or [comstock films podcast]; but not other things, like [being amber rhea] or [violet blue]. Let’s call it the Google Suggest No Fly List.

This is a fact. A lot of what people say about Google is rumor-mongering and tinfoil-hate conspiracy paranoia. But the quirks in Google Suggest’s autofill, that [peggy comstock] will autofill at the ‘m’, but [tony comstock] will not – that’s not something somebody said might be so on some underground blackhat SEO bulletin board. It is a fact. You can call up Google.com right now and try it for yourself.

How and why Google does this gets more into Renolds Wrap territory, and unless Google decides to tell us how and why they constrain the Google Suggest autofill all we can do is speculate. But since speculating about Google is even more popular than downloading dirty pictures, here’s a little of mine.

Given what’s been observed by Amber Rhea and Bacchus, it would seem a fairly reasonable inference that it has something to do with sexuality. It would also seem to have something to do with Google’s concept of “safety”; that is to say, protecting Google from returning search results that people might find offensive. Since as far as we know, Google can only deal with sexuality algorythmically, it would also seem reasonable to investigate at Google’s construct of “safety.”

(If you’d like to open another browser tab and switch your Google “SafeSearch” preferences to “strict filtering” you can play along at home.)

Open up google.com. Enter [nude] into the search. You’ll have to enter the entire string, because Google Suggest won’t autofill [nude]. Hit the “Google Search” button. The results? No returns. Under Google’s “strict filtering” there are no returns for the search [nude]. Now toggle your preferences to “moderate filtering”. What do you see now in the top results? nude.hu, paradisenudes.com, freshnudes.com, etc; sites that are not appropriate or even legal to display to minors.

Now try search [sex] with under Google’s “strict filtering” setting. Again, you’ll have to enter the entire string because Google Suggest will not auto fill [sex].

Unlike the “strict filtering” search for [nude], a search for [sex] under the “strict filtering” setting does deliver returns; wikipedia.org, webmd.com, amazon.com, others. Perhaps some people might be upset by having their children wonder into WebMD’s “6 Mistakes Men Make Having Sex”, but the important thing is that WebMD uses only proper medical terminology and euphemisms. And “nude”. We know that WebMD doesn’t use the word “nude” in the article, otherwise it wouldn’t show up in a Google “strict filtering” return. 

Now toggle to “moderate filtering” and search [sex] again. The number 1 return? Pornhub.com. My best guess is that’s why, although there are “strict filtering” returns for [sex], Google Suggest does not autofill for [sex]. If there are high ranking returns for a string that will be filtered by Google’s “strict filtering” settings, then Google Suggest will not autofill the string, regardless of how the user has the set Google SafeSearch filter settings on their browser.

Let’s test the hypothosis.

Toggle back to “strict filtering” and search [peggy comstock]. Google Suggest will autofill this sting at [peggy com   ]. Now look at the returns. Since we’ve done a “strict filtering” search, there’s nothing in the returns that has words like [nude] or any of the other words that deliver no returns under “strict filtering ([fuck], [tits], [bastard] to name a few strings.)

Now toggle to “moderate filtering and search [peggy comstock] again.

Like before Google Suggest autofills the string at [peggy com   ]. And like before, the results are “strict filtering” compatible.

Another thing that is odd is that for some strings, Google Suggest will auto fill right pass the “root” string, a string that has returns that won’t pass SafeSearch “strict filtering” and auto fill longer dirivitve strings that have returns that will pass Google’s “strict filtering”. So Google will suggest [comstock films podcast] butnot [comstock films]. Google will suggest [video video download], but not [video video]; [sex offender] but not [sex]. 

But wait, before you go off an tell everyone what a genius I am for figuring out how and why Google Suggest does or does not autofill some search strings and not other, try this one. Toogle your settings back to “strict filtering” and enter [bastard]. Google suggest will autofill the string at [bast   ]. Now hit search. 

Now isn’t that weird. Google Suggest autofills the string [bastard], but Googles SafeSearch “strict filtering” doesn’t deliver any returns. Hmmm. Let’s toggle back to “moderate filtering” and search [bastard again].

Double hmmmm. wikipedia.org, reference.com, and bastards.org, an adult adoptee rights advocacy organization. Google Suggest auto fills [bastard], but Google’s “strict filtering” doesn’t return results, and I can’t see the how or why of that, at least not algorythmically. What is it about these results that pass the  ”moderate filtering” setting that gets them trapped in the “strict filtering” setting. And with no “strict filtering” result, why does Google Suggest autofill the string [bastard]? Maybe the Google algorythm that controls these things is more sophisticated than I first thought!

Let’s try another!

Back to “strict filtering”. Now type in n i g g e r. Yep, I’m dropping the n-bomb. Let’s do a Google SafeSearch “strict filtering search for [nigger]. You’ll have to type the whole thing because Google Suggest will not autofill the string.

Now hit search.

Hmmm, ummm, okay. Top result? Wikipedia’s entry for nigger. Next result? Niggermania.com, which bills itself as “the best site for nigger jokes, rants, and racist humor.” Well as long as they don’t use the word “nude”, I guess it’s okay…  Let’s toggle back to “moderate filtering” and try it again. Pretty much the same results. This is kind of the reverse of [bastad]. No on the Google Suggest autofill, but yes on the SafeSearch “strict filtering” returns.  There’s no way I can arrange overlapping circles that explains this!

So here’s what we have so far:

Sometimes it appears that Google suggests won’t autofill a search string because their are no “strict filtering” search returns for the string. [nude] and [fuck] are two such strings.

Sometimes it appears that Google suggests won’t autofill a search string because the top returns for the string under “moderate filtering” are results that “strict filtering” will not allow to pass. The way that Google handles the search string [sex] is an example of this.

(Doing a venn diagram in my head, I don’t see a situation where these two conditions would be incompatible, but perhaps a reader with an information science degree and access to a white board can can either confirm this or correct me.)

But then we have [bastard], which Google Suggest will autofill at [bast  ], but has no “strict filtering” returns. And further more, when we look at the Google SafeSearch “moderate filtering” results, it’s not clear why any of them would fail to pass the “strict filtering” SafeSearch settings. On [bastard] the apparent link between Google Suggest autofill and search returns breaks.

Adding to a sense that things are more complicated than they first appear is Google’s treatment of the search [nigger]. Google Suggest will not autofill the string, but Googles SafeSearch “strict filtering” delivers returns. Someone Google Suggest has identified [nigger] as a potentially offensive with something more sophisticated than simply looking at whether or not the search results for the string pass the “strict filtering” setting. My venn diagram approach is starting to look like it’s no sufficient to explaining what’s going on at Google.

Now I suppose if a Google engineer, with her Stanford/MIT/Cal Tech PhD is reading this she’s chuckling at my simplistic attempt to explain the behavior of the Googlebot.  That’s fine. I’m not a computer scientist. I’m a filmmaker and a businessman, and if my understanding Google and search and the internet is simplistic, or skewed by my own self-interest, or just dead wrong, it’s not something I’m ashamed of. If Google or anyone else wants to lay it all out for me,  I’m a quick study. I’ll sort it out and adjust accordingly. Or not. My parents played “Man of La Mancha” over and over again when I was a toddler, and I’m pretty sure that gave my brain a quixotic bent.

Now let’s look at one more.

Toogle back to “strict filtering” and enter [sex education] in the search box. Again, you’ll have to type the whole thing because Google Suggest won’t do it for you. Now search.

Not surprisingly, the results are pretty benign looking. Nothing on the first page of the “strict filtering” results that raises a “won’t someone please think of the children!” flag. But that was (mostly) true about [sex]. It wasn’t until we look at the “moderate filtering” results that we got sites that were not appropriate for minors. Go ahead, toggle to “moderate filtering”.

Hmmm. I don’t know what to say. Granted, I come from a pretty liberal point of view about children and sex information, but if ”6 Mistakes Men Make Having Sex” passes Google’s “strict filtering” algorythm, I’m not seeing anything here that raises an obvious red flag. Just for kicks, let’s toggle our SafeSearch settings to “do not filter my results” and see what happens. I know, I know, “do not filter my results” isn’t supposed to be about text, just image. But let’s give it a try anyway.

Well hmmm and hummm and hrrrm. Again, using ”6 Mistakes Men Make Having Sex” as a benchmark (remember, that was a page 1 “strict filtering” return for [sex]) nothing on a “do not filter my results” search for [sex education] that waves a big “won’t someone please think of the children” flag. So why doesn’t Google Suggest autofill for [sex education]?  Who would be against sex education?

Oops.

I think we just arrived back at Google’s concept of safety, and it’s starting to look like it’s not really about protecting children, is it?

So what I was talking about when I said I saw some trends I didn’t understand is that about two years ago we started getting Google visitors on the search [real sex] and that was very good for our business. Visitors arriving on the search [real sex] were just as likely to return and ultimately just as likely to buy one of our DVDs as visitors arriving on the searches [comstock films] or [tony comstock].

What that means is we were catching and keeping customers who didn’t already know our name or our films. They were wandering the internet, looking for “real sex”, and when they wandered into our little corner they found something that was real enough to spend $28 on a DVD.

Then last December, it all went to shit. We and many other sexually oriented sites got caught in the gears Google’s ongoing fight against spam. Not only did we lose returns on [real sex], we lost returns on [comstock films] and nearly everything else. 

(more…)