Hello all,

I am curious on how the max_depth setting works in spider.pl and sub 
domains. For example if i index the url www.somesite.com/sub/ and set the 
max_depth to 2 will the spider stay within the sub folder for links or will 
it look inside somesite.com?

I've done a few tests and it seems to go back up into root folders at 
certain times, i assume when it needs more links? Can anyone explain how it 
traverses the pages and if it is possible to limit the spider to only take 
links from the sub domain?

thanks
andy

_________________________________________________________________
MSN Hotmail is evolving – check out the new Windows Live Mail 
http://ideas.live.com