headers
Olly Betts | 2 Feb 2008 04:49
Favicon
Gravatar

Re: Exception: Couldn't read enough (EOF)

On Thu, Jan 31, 2008 at 05:39:54PM +0100, Andreas Marienborg wrote:
> 
> On 31. jan.. 2008, at 17.28, Olly Betts wrote:
> 
> >Are you on a multicore/multiprocessor machine?  That might make this
> >more likely to manifest.
> 
> yes, 4-way system, but single proc writing (indexer-daemon)

I suspect that means it's more likely that a reader can run while the
base file is being rewritten.

> >I think I need to write a patch and get you to test it.  As a
> >workaround, catching the exception and simply retrying reopen() should
> >do the trick.
> 
> I can try, but I might have a hard time reproducing it in anything but  
> prod, which is a bit hard to test.

I guess all you can really do is apply the patch and run as normal.  If
the problem hasn't manifested after a reasonable period of time, then we
will just have to assumed that we've nailed it.

Here's the patch I've applied to SVN HEAD, which should apply to recent
releases as I don't believe we've touched this code for a while:

http://svn.xapian.org/trunk/xapian-core/backends/flint/flint_table.cc?r1=10081&r2=10080&view=patch&pathrev=10081

Cheers,
    Olly
(Continue reading)

Permalink | Reply | 0 comments |
headers
Noah Hoffman | 6 Feb 2008 02:09
Picon

problem compiling static python bindings

Hello -

I'm attempting to compile xapian bindings for python for use on a
shared (university-based) web server, so I've had to compile
everything (python, xapian-core) in my home directory using
"--prefix=$HOME/public_html/local". Here's some system info:

uname -a
... i686 i686 i386 GNU/Linux

gcc --version
gcc (GCC) 3.2.3 20030502 (Red Hat Linux 3.2.3-59)

$HOME/public_html/local/bin/python2.5
Python 2.5.1 (r251:54863, Feb  5 2008, 13:48:55)
[GCC 3.2.3 20030502 (Red Hat Linux 3.2.3-59)] on linux2

Compiling normally:

wget http://www.oligarchy.co.uk/xapian/1.0.5/xapian-bindings-1.0.5.tar.gz
tar -xzvf xapian-bindings-1.0.5.tar.gz
cd xapian-bindings-1.0.5
export XAPIAN_CONFIG=$HOME/public_html/local/bin/xapian-config
export PYTHON=$HOME/public_html/local/bin/python2.5
./configure --prefix=$HOME/public_html/local --with-python && make &&
make install

This completes without errors and results in the following:

ldd $HOME/public_html/local/lib/python2.5/site-packages/_xapian.so
(Continue reading)

Permalink | Reply | 7 comments |
headers
Homer | 6 Feb 2008 10:52
Picon
Favicon

omindex character sets

Hi there folks :)

I compiled xapian / omega on a windows box.
the omindex did not work for me, because the indexing seemed to hang, while
indexing text / html. 
the workaround was to use the cygwin omindex ports.

Now the problem:
i was indexing some documents with german special characters inside the document
and inside the path to the document.
when i use the omega search cgi, some special characters in the path to the
document are screwed up.
i think the path is encoded using iso-8859-1 and the main content is endoded
using utf-8.

is this true, or am i just doing some beginners mistakes?
would be nice if someone can tell me how to fix this.

thanks in advance

greets
Homer
Permalink | Reply | 1 comment |
headers
James Aylett | 6 Feb 2008 14:47

Re: problem compiling static python bindings

On Tue, Feb 05, 2008 at 05:09:44PM -0800, Noah Hoffman wrote:

> I'm attempting to compile xapian bindings for python for use on a
> shared (university-based) web server, so I've had to compile
> everything (python, xapian-core) in my home directory using
> "--prefix=$HOME/public_html/local".

It's a bad idea to put this kind of stuff inside your web
root. $HOME/local would be a better choice (unless your CGIs run
inside a chroot or jail, which is unlikely).

> However, I believe that I need to compile the python bindings
> statically, because attempts to import xapian from the cgi script (I'm
> using MoinMoin - moin.cgi, for what it's worth) result in an error:
> 
> libxapian.so.15: cannot open shared object file: No such file or directory

Hmm. Where is libxapian.so.15? Is there any documentation on strange
restrictions provided by the CGI mechanism?

> When I try to compile statically, I get an error:
> 
> make distclean
> ./configure --prefix=$HOME/public_html/local --with-python
> --enable-shared=no --enable-static=yes && make && make install

I'm not convinced this is the right approach. It's not that you don't
want to build static libraries, it's that you want to link against the
static *xapian* library. Try building xapian-core with
--enable-shared=no --enable-static=yes and then build the bindings
(Continue reading)

Permalink | Reply | 1 comment |
headers
Bruno Rezende | 6 Feb 2008 14:56
Picon

problem when compiling xapian in windows

Hi,

I'm trying to compile the xapian version that I get when execute the
get_xapian.py script, from xappy.

I'm using Microsoft Visual Studio 8

First error:

.\tcpclient.cc(82) : error C3861: 'close': identifier not found
.\tcpclient.cc(94) : error C3861: 'close': identifier not found
.\tcpclient.cc(109) : error C3861: 'close': identifier not found
.\tcpclient.cc(125) : error C3861: 'close': identifier not found
.\tcpclient.cc(139) : error C3861: 'close': identifier not found
.\tcpclient.cc(143) : error C3861: 'close': identifier not found

If I apply the first attached patch the error stops.

Then, comes the second error:

remoteconnection.cc
.\remoteconnection.cc(374) : error C3861: 'msvc_posix_open': identifier not
found
.\remoteconnection.cc(393) : warning C4244: 'argument' : conversion from
'__int64' to 'size_t', possible loss of data
.\remoteconnection.cc(402) : error C2065: 'header' : undeclared identifier
.\remoteconnection.cc(571) : warning C4244: 'argument' : conversion from
'__int64' to 'size_t', possible loss of data
.\remoteconnection.cc(604) : warning C4244: '=' : conversion from '__int64'
to 'size_t', possible loss of data
(Continue reading)

Permalink | Reply | 501 comments |
headers
Charlie Hull | 6 Feb 2008 15:09

Re: problem when compiling xapian in windows

Bruno Rezende wrote:
> Hi,
> 
> I'm trying to compile the xapian version that I get when execute the
> get_xapian.py script, from xappy.
> 
> 
> Can someone help me on this?

Hi Bruno,

Xappy (www.xappy.org) isn't part of core Xapian - it's best to address 
questions about it to the xappy-discuss mailing list, as I don't think 
many people here will know anything about it!

Cheers

Charlie
Permalink | Reply | 501 comments |
headers
Yannick Warnier | 6 Feb 2008 16:34
Gravatar

Setting self-defined doc ID with PHP bindings

Hello,

I've finally started playing with Xapian in a small test application
designed with CakePHP, so I've also started running into small practical
problems.

I would like to have a one-to-one relationship between a document
indexed in Xapian and a database table containing the full list of
documents (and details about them to access them from inside my web
application).

I've found some comments about that dating from some time ago, saying
that one should use the XapianWritableDatabase::replace_document()
method rather than the XapianWritableDatabase::add_document(), so that I
can add the document ID myself (being numeric and auto-incremented, I
don't suppose that would create a problem, would it?)

However, if that seems to work well (I get no error message), I have no
idea how to check if my document was indexed using the right ID (is
there some command-line tool to check the list of documents in my
database?).

Also, there seems to be no way (as far as the PHP bindings are
concerned) to get this document ID back from search results. I actually
do get search results, but the PHP print_r() function isn't very helpful
with the contents of an mset or one document out of an mset, as it
always give me a "Resource" output.

I have tried a 
  $mset->get_document()->get_data();
(Continue reading)

Permalink | Reply | 4 comments |
headers
James Aylett | 6 Feb 2008 16:43

Re: Setting self-defined doc ID with PHP bindings

On Wed, Feb 06, 2008 at 10:34:38AM -0500, Yannick Warnier wrote:

> I've found some comments about that dating from some time ago, saying
> that one should use the XapianWritableDatabase::replace_document()
> method rather than the XapianWritableDatabase::add_document(), so that I
> can add the document ID myself (being numeric and auto-incremented, I
> don't suppose that would create a problem, would it?)

It shouldn't do, no. You'll start getting issues if you need to move
to multiple databases, or something else more complex, but for now you
should be fine.

> However, if that seems to work well (I get no error message), I have no
> idea how to check if my document was indexed using the right ID (is
> there some command-line tool to check the list of documents in my
> database?).

If you use replace_document(), it will be indexed with the right
ID. To get a list of docids, probably the quickest thing is to write a
quick PHP script to check $db->get_document(docid) for all docids you
care about. $db->get_lastdocid() and $db->get_doccount() will help here.

> Also, there seems to be no way (as far as the PHP bindings are
> concerned) to get this document ID back from search results. I actually
> do get search results, but the PHP print_r() function isn't very helpful
> with the contents of an mset or one document out of an mset, as it
> always give me a "Resource" output.
> 
> I have tried a 
>   $mset->get_document()->get_data();
(Continue reading)

Permalink | Reply | 3 comments |
headers
Olly Betts | 6 Feb 2008 18:00
Favicon
Gravatar

Re: problem when compiling xapian in windows

On Wed, Feb 06, 2008 at 02:09:29PM +0000, Charlie Hull wrote:
> Xappy (www.xappy.org) isn't part of core Xapian - it's best to address 
> questions about it to the xappy-discuss mailing list, as I don't think 
> many people here will know anything about it!

The errors reported seem to be in compiling Xapian code, so I'm guessing
this is actually a bug in the MSVC makefiles...

Cheers,
    Olly
Permalink | Reply | 0 comments |
headers
Olly Betts | 6 Feb 2008 18:11
Favicon
Gravatar

Re: problem compiling static python bindings

On Tue, Feb 05, 2008 at 05:09:44PM -0800, Noah Hoffman wrote:
> ldd $HOME/public_html/local/lib/python2.5/site-packages/_xapian.so
>         libxapian.so.15 =>
> /rc21/d75/labmed1/public_html/local/lib/libxapian.so.15 (0x00cf4000)

That's good - libtool has set an rpath so libxapian.so.15 can be found.

> However, I believe that I need to compile the python bindings
> statically, because attempts to import xapian from the cgi script (I'm
> using MoinMoin - moin.cgi, for what it's worth) result in an error:

We don't support building xapian-bindings with a static xapian-core.
For more information see:

http://xapian.org/cgi-bin/bugzilla/show_bug.cgi?id=187

> I've tried to set LD_LIBRARY_PATH from within the script (using
> os.environ['LD_LIBRARY_PATH'] = some_val) to every possible absolute
> or relative path that I can think of, but I get the same error. But in
> any case, I don't think that the cgi script has access to the
> filesystem where libxapian.so.15 is located.

You shouldn't need to set LD_LIBRARY_PATH - the rpath should do the
trick, and has for me in the past on Linux.  It would be interesting
to work out why this doesn't seem to work.

But if you do set it, you may need to set it before invoking the python
script using a shell script wrapper.

For example:
(Continue reading)

Permalink | Reply | 4 comments |

Gmane