Re: PHP4 support in Xapian 1.1

Le mercredi 28 novembre 2007 à 20:55 +0000, Olly Betts a écrit :

> We're currently tentatively looking at releasing Xapian 1.1.0 in March
> 2008, so I'm proposing we drop PHP4 support at that point.

+1

Yannick

errors with flint database, xapian 1.0.4

Hello everyone!
I have met a problem with installation of xapian 1.0.4 on RHEL4 x86_64 ( 
kernel is 2.6.9).
I attached test log for xapian-core.
It seems all errors occur with flint database and remotetcp tests. I 
used flint database in xapian 0.9.10 and they worked fine.
Please advise,
Thanks.

Vaso.

make  check-TESTS
make[3]: Entering directory `/root/xapian-core-1.0.4/tests'
Running test './btreetest' under valgrind
./btreetest completed test run: All 15 tests passed.
PASS: btreetest
Running test './quartztest' under valgrind
./quartztest completed test run: All 10 tests passed.
PASS: quartztest
Running test './apitest' under valgrind
Running nodb tests with none backend...
./apitest completed test run: All 19 tests passed.
Running unicode tests with none backend...
./apitest completed test run: All 3 tests passed.
Running anydb tests with inmemory backend...
./apitest completed test run: All 63 tests passed.
Running specchar tests with inmemory backend...
./apitest completed test run: All 1 tests passed.

Re: errors with flint database, xapian 1.0.4

Vasiliy Sergeev wrote:
> Hello everyone!
> I have met a problem with installation of xapian 1.0.4 on RHEL4 x86_64 ( 
> kernel is 2.6.9).
> I attached test log for xapian-core.
> It seems all errors occur with flint database and remotetcp tests. I 
> used flint database in xapian 0.9.10 and they worked fine.
> Please advise,
> Thanks.

Could you test with a recent SVN snapshot?  (From 
http://www.oligarchy.co.uk/xapian/trunk/ )

There have been various fixes since 1.0.4 - in particular, we're using 
an updated valgrind suppressions file to remove some incorrect error 
reports from valgrind on 64 bit hosts.  If the errors still occur with a 
snapshot, though, we'd like to know so that we can fix them before the 
1.0.5 release (which should be pretty soon now).

--

-- 
Richard

Re: errors with flint database, xapian 1.0.4

Richard Boulton wrote:
> Could you test with a recent SVN snapshot?  (From 
> http://www.oligarchy.co.uk/xapian/trunk/ )
>
> There have been various fixes since 1.0.4 - in particular, we're using 
> an updated valgrind suppressions file to remove some incorrect error 
> reports from valgrind on 64 bit hosts.  If the errors still occur with 
> a snapshot, though, we'd like to know so that we can fix them before 
> the 1.0.5 release (which should be pretty soon now).
>
Hello Richard!
I attach log-file from svn version checking.
I doubt that problem is in valgrind incorrect reports. I also was unable 
to make a simple test on php5 after installing xapian-core and xapian 
php-binding.
php simple code that add one document to created Db.
<?php
include "xapian.php";
$_xapianDB = new XapianWritableDatabase("./database", 
Xapian::DB_CREATE_OR_OPEN);
$xapianDoc = new XapianDocument();
$xapianDoc->add_posting('1000',1);//, 100000);
$_xapianDB->add_document($xapianDoc);
?>

Thanks for your help,
Vaso.

make  check-TESTS

Re: errors with flint database, xapian 1.0.4

On Thu, Dec 06, 2007 at 12:20:11PM +0600, Vasiliy Sergeev wrote:
> I attach log-file from svn version checking.

Can you run:

cd tests
./runtest ./apitest -b flint -v zerodocid1 puncterms1 checkatleast3 valuerange1 sortrel1 adddoc4 emptyterm2
./runtest ./apitest -b multi -v zerodocid1 puncterms1 checkatleast3 valuerange1 sortrel1

> I doubt that problem is in valgrind incorrect reports.

Perhaps "incorrect" is the wrong word - the issue that Richard refers to
is that zlib deliberately performs multi-byte reads beyond the end of
the passed buffer in some cases.  Technically this is wrong, but it's
actually safe in the cases where it does it, and it is done for
performance reasons.

Valgrind doesn't normally report this as it has suppressions for such
cases, but those included don't match the errors reported on some newer
64 bit platforms (Ubuntu gutsy on x86-64 for example)

> I also was unable 
> to make a simple test on php5 after installing xapian-core and xapian 
> php-binding.
> php simple code that add one document to created Db.
> <?php
> include "xapian.php";
> $_xapianDB = new XapianWritableDatabase("./database", 
> Xapian::DB_CREATE_OR_OPEN);
> $xapianDoc = new XapianDocument();

Re: errors with flint database, xapian 1.0.4

I prefer not be be cc:-ed on list replies - thanks.

On Thu, Dec 06, 2007 at 06:36:30PM +0600, Vasiliy Sergeev wrote:
> I attached logs of mentioned commands for freshest svn version, jsut 
> installed. 111 is for first command and 222 is for the 2nd one.

Yes, that's just yet another variant of the zlib issue.  I've added
a suppression for this variant too.  The "possible memory leaks" aren't
genuine problems either, just std::string stuff (it holds onto blocks
by a pointer to somewhere other than the start).

> >So what does this actually do wrong?
>
> It cause segmentation fault on the command of adding document to xapian DB.

I'm unable to reproduce this by using your example with PHP 5.2.3 on
x86-64 Ubuntu gutsy.  What PHP version are you using?

Cheers,
    Olly

Re: errors with flint database, xapian 1.0.4

Olly Betts wrote:
> Yes, that's just yet another variant of the zlib issue.  I've added
> a suppression for this variant too.  The "possible memory leaks" aren't
> genuine problems either, just std::string stuff (it holds onto blocks
> by a pointer to somewhere other than the start).
>
>   
I see, thanks for this note.
> I'm unable to reproduce this by using your example with PHP 5.2.3 on
> x86-64 Ubuntu gutsy.  What PHP version are you using?
>
>   
My php configuration
PHP Version => 5.2.2
System => Linux 2.6.9-55.ELsmp #1 SMP Fri Apr 20 16:36:54 EDT 2007 x86_64
Build Date => Aug  7 2007 03:04:23
Configure Command =>  './configure' '--with-apxs2=/usr/sbin/apxs' 
'--with-mysql=/usr/local/lib/mysql' '--with-pdo-mysql' '--with-pgsql' 
'--with-pdo-pgsql' '--with-tidy'
Server API => Command Line Interface
Virtual Directory Support => disabled
PHP API => 20041225
PHP Extension => 20060613
Zend Extension => 220060519

I was using xapian 0.9.10 before and it worked well. But all 1.0.x 
versions are causing segmentation fault on the server.
I use fc7 on my local machine and example works fine on it too. It seems 
RHEL has some specific behavior exactly with xapians 1.0.x

Flax prerelease - search engine based on Xapian and Python

We (Lemur Consulting) thought members of this list might be interested 
in "Flax": the open source enterprise search engine we've been building, 
based on Xapian and Python.  We've just released a pre-release version 
of it: the announcement of this pre-release follows.

(There's a flax announcements mailing list at 
http://groups.google.com/group/flax-announce - I'll post future 
announcements there to avoid pulling this list off-topic.)

--

We're very happy to announce that a pre-release version of our free, 
open source enterprise search system is now available from 
http://www.flax.co.uk .

Flax Basic is a simple packaged version of our Flax search suite, 
suitable for building searchable indexes of various common types of 
files on a Windows system (it will also work to some degree on Linux and 
Macs but you'll have to download the source code and do some configuration).

Flax is based on Xapian and Python and includes features such as 
indexing of Microsoft Office, PDF and HTML files, automatic spelling 
correction and a web-based administration system.

We're hoping to launch Flax Basic "officially" early in the New Year, 
but we're letting a small number of people know about it now in the hope 
that we'll get some feedback - so *please* let us know what you think!

You can email us at info <at> lemurconsulting.com - if you encounter a 
problem please let us know:

Newbie ?? re: case-sensitive file extensions

All is well and we are using Xapian / Omega but I am unsure how to modify
the extensions so I can avoid this when I omindex due to the case of the
file extension for PDF.

[Entering directory /somedirectory]
Unknown extension: "/somefile.PDF" - skipping

Lowercase .pdf is fine. I have searched the list archives and haven't found
anything. Any help appreciated -

Jonathan
---------------------------
Jonathan Mergy <jmergy <at> powis.com>
Director of IT, Powis Parker Inc.
775 Heinz Avenue
Berkeley, CA  94710
Tel: 510-848-2463 x216
Fax: 510-295-2489

Re: Newbie ?? re: case-sensitive file extensions

On Fri, Dec 07, 2007 at 11:52:50AM -0800, Jonathan Mergy wrote:
> All is well and we are using Xapian / Omega but I am unsure how to modify
> the extensions so I can avoid this when I omindex due to the case of the
> file extension for PDF.
> 
> [Entering directory /somedirectory]
> Unknown extension: "/somefile.PDF" - skipping
> 
> Lowercase .pdf is fine. I have searched the list archives and haven't found
> anything. Any help appreciated -

The mapping from extension to mime-type should probably be case
insensitive, but for now you can just use the --mime-type option to
omindex to specify the uppercase versions of extensions:

http://www.xapian.org/docs/omega/overview.html

Cheers,
    Olly