Skip to main content.
home | support | download

Back to List Archive

Re: [swish-e] -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64

From: David L Norris <dave(at)not-real.webaugur.com>
Date: Wed Nov 28 2007 - 16:44:27 GMT
On Wed, 2007-11-28 at 10:02 -0600, Peter Karman wrote:
> I don't use the feature myself. I was basing my fs-detection suggestion on
> these threads:
> 
>  http://swish-e.org/archive/2003-09/6207.html

The information there is maybe outdated to some extent.

>  http://swish-e.org/archive/2005-05/9428.html

I don't think there are any problems with enabling 64-bit on UNIX except
for index binary compatibility with Windows.  As he said, it will
segfault either way when the index hits 2GB if the filesystem doesn't
support large files.


Sadly (or not), the Windows large file support remains broken.  I had
it mostly working by implementing the UNIX Large File API as part of 
our libreplace.  But I ran into a buffer overflow or something I just 
didn't have the time to debug and resolve.  (prop file overflows 
at 2 GB I think)  It took a day or more of indexing just to get to the 
point where it fails.  One of the users had fixed the problem but never 
sent a patch or explained what was wrong.  And I have very little 
incentive to bother with it, really.  I haven't used Windows in a decade 
and its a pain to even get my hands on a (working) Windows system to test.

Honestly, I can't believe someone hasn't written a simple UNIX Large
File Support library for Windows.  It wasn't very difficult to
implement.  Just a pain to test.  Windows has large file support in the
"POSIX" API but its a non-standard Microsoft API that never caught on.
Just a matter of wrapping that API with LFS function calls and faking a
64-bit int using a 32-bit signed and unsigned int in an array.  The code
is 99% written.  I see that its not in SVN.  Probably in a ZIP file on
my web server somewhere.  Hrm.  Maybe I should find it and put the code
in src/replace even if we aren't using it.

And how does BDB effect this?  Is Swish-e still writing to the
filesystem itself with the BDB backend?  Or is that abstracted through
libdb?

-- 
 David L Norris
  http://webaugur.com/
  ICQ - 412039



_______________________________________________
Users mailing list
Users@lists.swish-e.org
http://lists.swish-e.org/listinfo/users
Received on Wed Nov 28 11:44:28 2007