headers
Charles | 15 May 2013 08:35

How to omindex some sub-directories?

Given a directory tree like ...

/foo
|
+-- A
|
+-- B
|
+-- C

... what is the best way to index A and C into a single Xapian database?

AFAIK the alternatives are:

omindex --db /my_db --no-delete /foo /foo/A
omindex --db /my_db --no-delete /foo /foo/B

or

omindex --db /my_A_db /foo /foo/A
omindex --db /my_B_db /foo /foo/B
xapian-compact /my_A_db /my_B_db /my_db

The first alternative does not delete files deleted from the file system
from the database.  Is there any way around this except by emptying the
database and starting over?

The second alternative increases storage and processing requirements. 
OK if needs must but would prefer to avoid.
(Continue reading)

Permalink | Reply | 2 comments |
headers
Cséri Tamás | 15 May 2013 04:11
Picon
Gravatar

Match positions of a queryresult

Hello,

I've just started learning Xapian and I'm facing the following problem.

I've indexed many text files (using a TermGenerator from std::string), each
document in my database is a single file on the disk.
The search works pretty well and finds the files that match the query
string, but I can't figure out how I can determine the location of the
actual matched terms. I want to show the user the row and column number of
the match (to somehow highlight the match).

So far I haven't found a solution. The closest I've got is the
Enquire::get_matching_terms_* functions but this does not really work for
phases and I'm still far from character positions.

I hope someone can give me some hints where to look to begin with.

Thanks,
- Tamás Cséri -
_______________________________________________
Xapian-discuss mailing list
Xapian-discuss <at> lists.xapian.org
http://lists.xapian.org/mailman/listinfo/xapian-discuss
Permalink | Reply | 0 comments |
headers
Olly Betts | 4 May 2013 06:40
Favicon
Gravatar

Xapian 1.3.1 development snapshot released

After rather longer than expected, Xapian 1.3.1 is now available.

Please note that 1.3.x releases are development releases - they are made
to encourage earlier and wider use and testing of new and changed code
so that 1.4.0 can both happen sooner and be better than otherwise.

Our record with 1.1.x was very good - all the bugs I am aware of were
either in new features, or were also present in the corresponding 1.0.x
release.  But if you main concern is minimising risk of accidental
breakage, sticking with 1.2.x would be prudent, at least for deployment.

The 1.3.x development series will lead to a stable 1.4.x release series.
We don't have a date set, but towards the end of this year seems plausible.

If you make packages of this release, please make sure that they are very
clearly labelled as not being a stable version, and ensure that they can be
installed in parallel with the stable version (the default paths and program
suffix are set to help make this easier).  If they are binary packages you
should also be aware that 1.3.x comes with no guarantee of ABI stability
(code built against 1.3.0 will need rebuilding for 1.3.1).

We've not yet put together release notes or a download page for 1.3.x, but
you can read the full lists of user-visible changes here:

  http://svn.xapian.org/*checkout*/tags/1.3.1/xapian-core/NEWS

  http://svn.xapian.org/*checkout*/tags/1.3.1/xapian-applications/omega/NEWS

  http://svn.xapian.org/*checkout*/tags/1.3.1/xapian-bindings/NEWS
(Continue reading)

Permalink | Reply | 0 comments |
headers
Tae | 1 May 2013 20:42
Gravatar

Tutorial on OS X

Hi, guys.

Is there any example Cocoa project which uses Xapian?

Best,
Tae
Permalink | Reply | 1 comment |
headers
Charles | 30 Apr 2013 05:54

What is the significance of 22 in the Debian libxapian22 packages

What is the significance of 22 in the Debian libxapian22 and
libxapian22-dbg packages?  According to
http://packages.debian.org/search?keywords=libxapian22 they are built
from upstream versions 1.2.3 to 1.2.12 (of xapian-core?)

The reason for asking is that we need xapian-omega (and hence
xapian-core) at 1.2.8 or later.  Previously we have built and installed
directly from source but a package would be more convenient.  So I am
studying the existing Debian packages for guidance while creating 1.2.15
packages..

Charles
Permalink | Reply | 7 comments |
headers
Michael Lewis | 26 Apr 2013 16:42

remote backend

So, given what I've read in the documentation I would create a text file named document_database.txt that
might have the following:

remote 192.168.1.10:30000

chert /var/lib/xapian_database/segment1

remote 192.168.1.10:30000 chert /var/lib/xapian_database/segment2

remote 192.168.1.10:30000 chert /var/lib/xapian_database/segment3

etc.

I would then in my PHP program open document_database.txt as the database and then perform normal Xapian
calls. The same with Omega, I would, in my template, change the database name to document_database.txt.

Given the above, it appears that I do not have to refer to each segment (segment1, segment2, etc.) for my
calls, as it appears they are aggregated and treated as a single database and Xapian takes care of which
actual database segment to search. IE, searching all the segments for my search terms.

Michael

-----Original Message-----

From: simon.roe <at> gmail.com<mailto:simon.roe <at> gmail.com> [mailto:simon.roe <at> gmail.com] On Behalf Of
Sym Roe

Sent: Friday, April 26, 2013 9:26 AM

To: James Aylett
(Continue reading)

Permalink | Reply | 2 comments |
headers
Tae Won Ha | 26 Apr 2013 08:45
Gravatar

Compiling Xapian within a Cocoa project

Hi, folks.

I don't know much about C++, so please excuse my newbie question: I just tried to include Xapian in a dummy
Cocoa project, ie created a new project and added the libxapian.a file installed via MacPorts. When I
include xapian.h in an Objective-C++ file (either mm or h), the compilation fails with the following message:

=================
In file included from /opt/local/include/xapian.h:34:
In file included from /opt/local/include/xapian/database.h:32:
In file included from /opt/local/include/xapian/document.h:33:
/opt/local/include/xapian/valueiterator.h:143:10: error: expected member name or ';' after
declaration specifiers
     bool check(Xapian::docid docid);
     ~~~~
^
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.8.sdk/usr/include/AssertMacros.h:1291:28:
note: expanded from macro 'check'
         #define check(assertion)  __Check(assertion)

^
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.8.sdk/usr/include/AssertMacros.h:280:5:
note: expanded from macro '__Check'
                   do                                                                      \
                   ^
In file included from /Users/hat/Projects/OCXapian/OCXapian/OXAppDelegate.mm:9:
In file included from /Users/hat/Projects/OCXapian/OCXapian/OXAppDelegate.h:10:
In file included from /opt/local/include/xapian.h:50:
/opt/local/include/xapian/postingsource.h:223:43: error: too many arguments provided to
function-like macro invocation
     virtual bool check(Xapian::docid did, Xapian::weight min_wt);
(Continue reading)

Permalink | Reply | 3 comments |
headers
Michael Lewis | 25 Apr 2013 23:56

Converting MySQL database to Xapian

I am looking for some guidance on converting a large MySQL database to Xapian. The current structure is that
the database is broken up into 160 "sub-databases". There are 50,000 or so records in each stub database.
Each record has content that I am full-text indexing. The average size of the text is about 59k characters.
The database is broken up into sub-databases because the MySQL full-text handling is so abysmal. I use
MySQL-Proxy and some Lua modules to make the searching and inserting transparent to the client programs.
This data is written once and never (or once in a blue moon a record is deleted) modified. What I am looking to
do is:

1. Create a xapian database on a remote server where each record's indexed text content would be a document
(storing the document ID in the MySQL data record). I will then load the DB from the old MySQL server into the
xapian server (eventually retiring the old server). It must remain in operation during the change-over.

2. Be able to access the xapian database via PHP and Omega from multiple clients (read only) on remote systems.

3. Insert new content on the server containing the xapian database. This insertion will take place on the
server containing the xapian database.

Looking over the docs and googling suggests that I may need multiple xapian databases since the update time
appears to be appreciable and I can't knock the db offline while updating (24/7 operation).

Omega will be run from a remote system.

Part of my problem is that I am not sure of how the remote backends are set up and do I need local stub DBs, which
back-end to use, etc.

I would appreciate any help

Thanks,

Michael Lewis
(Continue reading)

Permalink | Reply | 5 comments |
headers
Tim Brody | 23 Apr 2013 09:51
Picon
Favicon

Re: warning: dl() [function.dl]: xapian: Unable to initialize module Module compiled with module API=20050922

I haven't tried building on RHEL 5 but this is how you would do it:

yum -y install rpm-build yum-utils

Download the source tarballs from here:
http://xapian.org/download

rpmbuild -ta xapian-core-*.tar.gz
(install any missing dependencies)
rpm -ivh <path to core-RPMs>
rpmbuild -ta --without csharp --without perl --without python
xapian-bindings-*.tar.gz

/Tim.

On Mon, 2013-04-22 at 12:21 -0500, Ron Parker wrote:
> Can you tell me how I rebuild the bindings as suggested for my version of PHP?
> 
> Or, how to install the packages you built?
> 
> PHP 5.2.16 (cli) (built: Dec 17 2010 14:09:03)
> Copyright (c) 1997-2010 The PHP Group
> Zend Engine v2.2.0, Copyright (c) 1998-2010 Zend Technologies
> 
> CentOS release 5.3 (Final)
> Tikanga - added for OMSA install
> 
> Thanks so much!
> 
> -ron
(Continue reading)

Permalink | Reply | 0 comments |
headers
Ron Parker | 22 Apr 2013 19:21

Re: warning: dl() [function.dl]: xapian: Unable to initialize module Module compiled with module API=20050922

Can you tell me how I rebuild the bindings as suggested for my version of PHP?

Or, how to install the packages you built?

PHP 5.2.16 (cli) (built: Dec 17 2010 14:09:03)
Copyright (c) 1997-2010 The PHP Group
Zend Engine v2.2.0, Copyright (c) 1998-2010 Zend Technologies

CentOS release 5.3 (Final)
Tikanga - added for OMSA install

Thanks so much!

-ron

----- Original Message -----
From: "Tim Brody" <tdb2 <at> ecs.soton.ac.uk>
To: "Ron Parker" <sysop <at> scbbs.com>
Cc: "Xapian Discussion" <xapian-discuss <at> lists.xapian.org>
Sent: Monday, April 22, 2013 4:52:07 AM
Subject: Re: [Xapian-discuss] warning: dl() [function.dl]: xapian: Unable	to initialize module
Module compiled with module API=20050922

Hi,

I've built packages for 1.2.15 against the latest RHEL 6. Let me know if
you have any further problems.

Thanks,
Tim.
(Continue reading)

Permalink | Reply | 0 comments |
headers
Tim Brody | 22 Apr 2013 13:52
Picon
Favicon

Re: warning: dl() [function.dl]: xapian: Unable to initialize module Module compiled with module API=20050922

Hi,

I've built packages for 1.2.15 against the latest RHEL 6. Let me know if
you have any further problems.

Thanks,
Tim.

On Sat, 2013-04-20 at 03:20 -0500, Ron Parker wrote:
> Thank you for the response.  How do I rebuild the bindings against the version of PHP I have installed?  I
installed Xapian using yum:
> 
> # rpm -ivh http://rpm.eprints.org/rpm-eprints-org-key-1-1.noarch.rpm
> # rpm -ivh http://rpm.eprints.org/xapian/6/noarch/rpm-eprints-org-xapian-6-1.noarch.rpm
> # yum update
> # yum install xapian-omega xapian-bindings-{php,python,tcl8,perl,ruby}
> 
> I thought this would have done that, but apparently not.  How do I rebuild the bindings?
> 
> Thanks!
> 
> -ron
> 
> ----- Original Message -----
> From: "Olly Betts" <olly <at> survex.com>
> To: "Ron Parker" <sysop <at> scbbs.com>
> Cc: xapian-discuss <at> lists.xapian.org
> Sent: Friday, April 19, 2013 11:45:55 PM
> Subject: Re: [Xapian-discuss] warning: dl() [function.dl]: xapian: Unable	to  initialize module
Module compiled with module API=20050922
(Continue reading)

Permalink | Reply | 0 comments |

Gmane