If you’re reading this, it’s fairly likely that you’ve found it from Google. Of course, you could also have come here from many other places, but statistically, Google is the most likely place. Actually, there’s a 46.3% chance you’re from Google, according to the Neilson ratings for November of 2005. (There’s also a 23.4% chance you’re here from Yahoo, and a 11.4% chance you’re here from MSN)

To be realistic, however, the REAL chance that you’re here from Google isn’t very big. Why? Because Duskrider.com isn’t listed very prominently in Google. I’ve done all I can, and I’m hoping and praying (as most webmasters do) that the big G will begin to pick my site up and list it higher in the rankings than it does now. I’m not terribly worried since my site is pretty new (only a month old as of this writing), but no matter who you are or how old your site is… Google is still the most frustrating thing about being a webmaster.

Since Google does almost half of the world’s searches, it is IMPERITIVE that a site be listed on Google, and listed well, in order for that site to generate any substantial traffic. As I write this, I get about 300 hits a day to my site. Not terrible, but nowhere near where I would like to be. The real frustration comes with Google and the way it ranks sites. On MSN, even after being online for less than a month, I’m ranked #11 for the search ‘free desktop backgrounds’ which is a very targeted keyword for my site. Why won’t Google pick me up the same way? It all has to do with what Google considers to be ‘relevant’.

To Google, my site is relevant when it has many other sites linking to it and it doesn’t violate any of Google’s rules (of which there are many). These other sites use a link to cast a sort of online vote for my site. Google only counts that link, however, if it’s from a highly relevant site. Google measures relevancy in something it calls ‘PageRank’ (known as PR) and PR is listed for every site Google indexes on a scale of 1 to 10. Furthermore, this scale isn’t linear, it’s more like the Richter scale where numbers higher on the scale are several orders of magnitude better than numbers lower on the scale. So, as far as Google is concerned, a site with PR of 5 is many many many times more relevant than a site with PR of 4, which is many many many times more relevant than a site with PR of 3 and so on. Duskrider.com has a PR of 2 - which is a holdover from when it was my personal site. I must admit that I’m a bit worried that Google won’t update my PR from 2 when it figures out I’m no longer a personal site and now a site giving away free wallpaper.

The top site in Google for my search terms, which happens to be Digital Blasphemy, has a PR of 5. I know all this because I have a plug-in for FireFox that tells me what the PR of any site I’m visiting is. Digital Blasphemy also has over 1200 inbound links from other sites, at least according to Google… so until I get that many links to my site I have no chance of ever getting the #1 spot on Google. That’s just how Google works. (This is, of course, very simplified - I’d rather have 1 PR 6 site link to me than 20 PR 1 sites) Now, I’m not saying I should be where Digital Blasphemy is, but I should certainly be listed before an eBay auction for bathroom wallpaper… right?

Well, according to Google, no. It just doesn’t work like that… and nobody knows why because Google keeps all their listing methods a deep dark secret. I understand why… If everyone knew exactly how to get #1 on Google for any search term then it would be a disaster for any REAL relevant site because the #1 spot would belong to a few money hungry spammers.

However, it is my opinion that Google is currently broken.

Ever notice how, when you do a search on Google, you don’t always get the top page from a site, but you probably get just the page you’re looking for? This is because Google indexes ALL the pages on a site and not just the top page. That way Google can give you what you’re looking for right away, and you don’t need to wade through several links to get there from the first page of a site. This was good for Google, because it got to list more results, which was good for the searching public because they got more results for their search, which was good for webmasters because they got more results listed for their site… thus more chances for John Q. Public to see their product.

Notice… I said was.

Google recently pushed an update called Big Daddy. One of the things this update did was to change the way Google indexed sites. Now, you only get your entire site indexed if you have a high PR. So, since I have a PR of 2, I have a total of two pages indexed in Google - my main page and this blog. Even though I have closer to 50 or 100 unique pages on this site (one for each image, album, category in my gallery, one for the details on each image, etc..), Google only sees fit to list two. Since those two also have a low PR, they aren’t very high in the results (called SERPS for Search Engine Result Pages). So, I have yet to get a single hit from Google EVEN THOUGH almost half of all the world’s searches are done via Google. If my site had a PR of 4 or so, I might get 10-15 pages indexed.

Messed up huh?

The problem really is that the only way to get good inbound links is to get traffic. Other people see a page and think “Hey, that’s a cool page… I think I’ll link to it.” So, there’s an inbound link and a vote for my page. If nobody can find my page, however, how am I supposed to generate links to it? I’ve done what I can… submitted Duskrider.com to every directory, engine, free site, and spider I can think of. The rest is up to the public.

It’s a vicious circle. Once you’re in, then you’re golden, but getting in is a real chore sometimes. I can be fun, but it can also be scary.

The REAL downside to this Big Daddy update goes like this: Imagine you have a website with 10,000 pages. That may sound like a bunch, but an e-commerce site with a thousand products for sale can generate that number of pages pretty easy. Now let’s say your site has a PR of 5, which is pretty respectable. Pre Big Daddy, Google has 9000 of your pages in its index. All of a sudden Big Daddy hits and the next day you see that you have 25 pages in the index. Since you only have a PR of 5, Google feels it only needs to go one or two links deep in your site. You’ve just lost over 99% of all your search results from 50% of all the searches done in the Internet. Ouch. That’s BIG money, and it can even go so far as to drown a business that makes its living from the Internet. Even though those 25 pages may be the most relevant pages you have, you’ve still lost the majority of your site listings. Those 25 pages might supply you with 50% of your revenue… great, so you’ve only lost the other 50%. No big deal right?

The sad thing is that this is no hypothetical situation. It JUST HAPPEND. Don’t ever let anyone tell you that Google is just a search engine. Google’s actions have implications reaching farther than almost anyone knows… perhaps even Google itself.

Hence the title: The Google Monster.

As for me? Big Daddy is an inconvenience. It may mean the difference between Duskrider.com being successful enough to pay for the hosting and successful enough to pay for college. My livelihood isn’t made from the Internet… but it would be nice.

-Ed