Question: If the robots.txt file disallows access to /cgi-bin, but the
actual file is in ~user/cgi-bin, do we still disallow?
I'm going with yes, for the moment. But semantically, this is slightly
different. The user is punished, not us. :>
_____________________________________________________________________________
"The human mind is a 400,000-year-old legacy application...and you expected
to find structred programming?" -- Randall Davis, 1996 AAAI Pres. Address