headers
Olly Betts | 2 Sep 04:17
Favicon
Gravatar

Re: Proposed changes to omindex

On Tue, Aug 29, 2006 at 04:22:29PM +0100, Olly Betts wrote:
> I've made some and I'm in the process of working through the rest.

OK, I've done fettling.

After some research, I went for the public domain MD5 implementation
written by Colin Plumb.  It's used widely (including in the Linux
kernel), compiles as C++, and doesn't add further relicensing obstacles.

I couldn't find a guarantee that std::string::c_str() will have the
correct alignment for access as a 32 bit integer (though I can believe
it typically will be), so I've used memcpy() instead of
reinterpret_cast<>.

I've been wondering if there should be a command line option to
enable/disable the MD5 checksumming.  If you don't want to collapse
indentical documents, it's just overhead (slower indexing, bigger
database, and probably some slowdown when sorting by lastmod with
the current way we store values).

So I did some simple benchmarking by indexing /usr/share/doc on my
Ubuntu box:

Without MD5:

real    1m56.279s
user    1m44.573s
sys     0m7.358s
58536   usrsharedoc
(Continue reading)

Permalink | Reply | 0 comments |
headers
Olly Betts | 2 Sep 05:57
Favicon
Gravatar

Re: omindex patch

On Sun, Aug 20, 2006 at 08:33:56PM +0200, Reini Urban wrote:
> The patch is not yet complete.

I've had a quick look through - I think there are some useful things in
here.  Let me know when you've finished tweaking it.

> It needs autoreconf to update configure and the Makefiles.
> Note that unrar is not patent infected, only rar, the compressor.
> I've put some AC_PATH_PROG checks into configure for all helpers.

This assumes that the filters installed at configure time are the same
as those installed at run time, which isn't necessarily the case (for
binary packaged versions, it's probably rarely true).

I'd prefer to just run the filter anyway and check if it fails.  I've
just added some code to remove the ext->mime-type mapping when the
filter fails because it couldn't be found, so we now effectively lazily
probe the filters we want to use at run-time.

> +AM_LDFLAGS = -no-undefined

Sadly adding this unconditionally causes problems on some platforms (I
forget which off the top of my head).  Do you need it for cygwin?

> +#define SAMPLE_WORDS  500

This is actually the number of *CHARACTERS*, not words.

> +#ifdef HAVE_TEXTCAT
> +    char * lang;
(Continue reading)

Permalink | Reply | 4 comments |
headers
Charlie Hull | 29 Sep 16:10

Problems building PHP extension for Xapian bindings on Windows

Hi all,

I've been attempting to build the PHP bindings on Windows using Visual 
C++ and have run into some serious problems. I thought I'd let everyone 
know in case someone can suggest a solution. I'm using the Visual C++ 
2003 toolkit, the Platform SDK, PHP 4.4.3 sources and the PHP 4.4.3 
Windows binaries, and the attached make file.

Xapian bindings are to be built as a PHP extension, necessitating 
linking with the PHP libraries (PHP4TS.LIB or PHP5TS.LIB) supplied with 
the binary version of PHP. However PHP extensions must be built to use a 
'shared DLL' version of the Windows runtime libraries, using the /MD or 
/MDd compiler options, as this is how PHP itself is built.

However attempting to build the SWIG-generated bindings (xapian_wrap.cc 
etc.) using Visual C++ using /MD causes various problems:

1. Errors are produced while compiling around line 1110 of xapian_wrap.cc:
#include <string>
   which leads inside the Microsoft header <ostream>, at line 921:
error C2491: 'std::endl' : definition of dllimport function not allowed
   Moving this #include to the beginning of the file fixes this error 
for now.

2. Linking (to a version of the Xapian core also built with /MD, which 
causes other problems but we'll gloss over those for now) causes many 
errors of the form:

liblanguages.lib(api.obj) : error LNK2001: unresolved external symbol 
"__declspec(dllimport) public: __thiscall std::basic_string<char,struct
(Continue reading)

Permalink | Reply | 3 comments |
headers
richard | 29 Sep 16:27

Re: Problems building PHP extension for Xapian bindings on Windows

Just to add a note to Charlie's post, in order to get this far we had to
re-generate xapian_wrap.cc with a recent development version of SWIG: the
xapian_wrap.cc file included in release 0.9.6 of Xapian doesn't compile
with microsoft's compilers (due to declarations of the form
"zval **args[0];").  The current HEAD version of SWIG avoids generating
this.  If we can get the linking fixed, it might be worth generating the
PHP xapian_wrap.cc for the next Xapian release with the development version
of SWIG (or with 1.3.30 if it's been released by then).

--

-- 
Richard
Permalink | Reply | 1 comment |
headers
Olly Betts | 29 Sep 16:36
Favicon
Gravatar

Re: Problems building PHP extension for Xapian bindings on Windows

On Fri, Sep 29, 2006 at 03:27:02PM +0100, richard <at> lemurconsulting.com wrote:
> If we can get the linking fixed, it might be worth generating the
> PHP xapian_wrap.cc for the next Xapian release with the development version
> of SWIG (or with 1.3.30 if it's been released by then).

I'm planning to anyway, since SWIG 1.3.29 doesn't have the PHP5 OO
support in.  It also fixes some bugs reported to the Xapian bugzilla.

Cheers,
    Olly
Permalink | Reply | 0 comments |
headers
Olly Betts | 30 Sep 18:23
Favicon
Gravatar

Re: Problems building PHP extension for Xapian bindings on Windows

On Fri, Sep 29, 2006 at 03:10:13PM +0100, Charlie Hull wrote:
> I've been attempting to build the PHP bindings on Windows using Visual 
> C++ and have run into some serious problems. I thought I'd let everyone 
> know in case someone can suggest a solution. I'm using the Visual C++ 
> 2003 toolkit, the Platform SDK, PHP 4.4.3 sources and the PHP 4.4.3 
> Windows binaries, and the attached make file.

I'd suggest attacking a simpler problem first, to try to narrow down
where the problem(s) lie.

Can you build a simple SWIG generated PHP extension using the same set
of tools?  There are some examples in the SWIG source tree which would
probably serve as a useful test.

> 3. After some research I found various information suggesting that it's 
> difficult to export bits of the STL in a DLL, although Microsoft 
> suggests it's possible in a very messy way:
> http://support.microsoft.com/default.aspx?scid=kb%3ben-us%3b168958

I don't see where we try to export an STL class from xapian-core.
Methods accept/return std::string, etc but that's not the same as
exporting std::string, etc unless I misunderstand what's meant by
"exporting".  Perhaps I'm missing something here.

> Someone else said 'The STL is a way to share code, so is a DLL - but 
> they don't play nice together'.

I'm not sure I really believe that, since I've not heard of problems
using Xapian in a DLL in other cases.
(Continue reading)

Permalink | Reply | 0 comments |

Gmane