Skip to main content.
home | support | download

Back to List Archive

Re: More on IndexOnly and NoContents

From: David Norris <dave(at)>
Date: Mon Jun 12 2000 - 10:45:59 GMT
Jose Manuel Ruiz wrote:
> IndexOnly only works with File-System.
> When trying with HTTP (-S http option) I get:
> Bad directive on line #148: IndexOnly .html

Yes, IndexOnly is only valid is FS mode.  The theory has been that only
Content Type headers should be used instead of file extensions.  I think
a number of us might not totally agree with that.  My opinion is that
IndexOnly would be beneficial in both HTTP and FS mode.

> NoContents only works with HTTP. With FS the
> directive simply is ignored. I think this is
> solved by Andrew's patch.

This is a serious bug which did not exist in versions prior to the
addition of HTTP indexing.  It must have appeared in version 1.2 or
1.3.  If anything, NoContents should only work in FS mode.  However, I
think this option should also work in both HTTP and FS modes.  I am not
sure how the spider handles NoContent or if it bothers to handle it.

> Perhaps it is time to patch these options. I think
> both of them may be used in both FS and HTTP. 

I agree that both options should be available in HTTP and FS modes. 
There doesn't seem to be a technical reason against this.  And, it could
be useful in some cases.

> But, since they are mutually exclusive, one must be more
> important than the other.

They should not be mutually exclusive.  Both options are equally
important.  Both options would be used together very often.  Making them
mutally exclusive breaks backwards compatibility with older versions of

,David Norris
  Dave's Web -
  Dave's Weather -
  ICQ Universal Internet Number - 412039
  E-Mail -
Received on Mon Jun 12 03:44:06 2000