Logfile analysis - too much data!

[pyesetz]

Recently I restarted my visitor-sniffing program.  In 20 days it recorded 3091 hits, of which 80% (2458) were from search-engines. This is way too much data!  There were only 183 hits from actual search-engine users, plus 71 from CYD readers (via this thread).

I've added some more code to my journal.php program that will hopefully cut down on all those hits from search-engines:

• Miva is now banned.  Their database is for private corporate use, of no value to my potential friends.  I'm thinking about banning Yahoo! Slurp, since they keep scanning my images but never show them in their search-results.
• The Expires: header is now 1 month instead of 1 week, which should cut back on the rereading of unchanged pages (but new entries might not be noticed for a month).
• The tags and the Link and Parent links for comments are now marked with rel="nofollow", which is supposed to cause search-engines to ignore them.  We'll have to see whether that really works.
• The Leave a comment links now use rel="nofollow" instead of being trashed by my program.  Under the old system, anyone who found my journal through a search-engine could never comment on anything!

* * * * *
I tried an ego-surf at Yahoo.  It came up with this and this, both of which seem spurious (my story mentions both Cherokee totems and hot tubs, but only as props) and the current versions of these pages don't mention me.

Even more boring news: I've now updated all my PHP files at Furtopia to have month-long expirations.  Previously only the journal had any expiration at all.  Reduced traffic, here I come!