headers
Peter Karman | 1 Dec 05:17
Favicon
Gravatar

SWISH::Prog::Xapian

Early[0] implementations of Swish3 for Perl are now available on CPAN.

 http://search.cpan.org/dist/SWISH-Prog-KSx/
 http://search.cpan.org/dist/SWISH-Prog-Xapian/

These Perl implementations offer KinoSearch[1] or Xapian[2] as alternate backend
libraries.

SWISH::Prog::Xapian should build indexes compatible with the swish_xapian tool
that comes with libswish3.

Feedback welcome.

[0] They are "early" because libswish3 1.0 has not yet been released.
[1] http://search.cpan.org/dist/KinoSearch-0.30_07/
[2] http://search.cpan.org/dist/Search-Xapian/

--

-- 
Peter Karman  .  http://peknet.com/  .  peter <at> peknet.com
Permalink | Reply | 0 comments |
headers
Olly Betts | 1 Dec 05:43
Favicon
Gravatar

Re: Can Omega be used as an OpenSearch engine for Windows Federated Search?

On Tue, Dec 01, 2009 at 12:56:25AM +0100, Rune Kock wrote:
> Now, Omega has an OpenSearch 1.0 template.  But the following comment
> isn't encouraging: "The opensearch template isn't really finished - it
> was more a quick hack to see how fiddly it would be."
> (http://lists.tartarus.org/pipermail/xapian-devel/2006-January/000288.html).
>  That mail is three years old, and nothing much seems to have happened
> since then.

Nothing much has happened since then because no problems with the
OpenSearch template have been reported since those I fixed back then.  It's
hard for me to know if that's because nobody uses it or it is free of
obvious problems.

If OpenSearch 1.1 is incompatible, it would be good to know what needs
changing for it.

Cheers,
    Olly
Permalink | Reply | 0 comments |
headers
Jim | 1 Dec 20:33
Favicon

Re: How does remote server work?

Olly Betts wrote:
> On Sat, Nov 28, 2009 at 07:23:22AM -0500, Jim wrote:
>   
>> I'm concerned that there will be a performance hit if there is a large
>> amount of data transferred over a slow link.  Does the actual search
>> occur on the client or the server side?  If on the client side, then how
>> much data is transferred?  If it's on the server side then can I assume
>> that a small amount of data is pushed to the server and just the results
>> returned?
>>     
>
> A search is performed on the server and the resulting MSet serialised and
> returned to the client.  If there are other sub-databases (local or
> remote), the results are merged to give the final MSet.  So it should
> behave fairly well on slower connections.  We also try to minimise the
> number of exchanges required for the benefit of high latency connections.
>
> If you find it is slower than you'd like, all the messages are sent and
> received by the same mechanism, so you could easily dump out the message
> type and size to see what is going on.
>
> Cheers,
>     Olly
>
>   
Thanks,

I am sure that will work fine.  I was just concerned that it might want 
to transfer large blocks of the index files.  The connection isn't that 
slow but gigabytes of data over even a fairly fast connection takes a
(Continue reading)

Permalink | Reply | 0 comments |
headers
Andy | 1 Dec 23:25
Picon
Favicon

Does xapian support attribute search & faceted search?

Can I use xapian for a search like:

'title' has the phrase "dinner jacket" AND 'color' == BLACK AND 'state' == MA AND 'size' == XL

What about faceted search?

Thanks
Permalink | Reply | 2 comments |
headers
Niels Boldt | 2 Dec 00:08
Picon

Re: in memory databases

Hi Olly

Thanks for your answer. We ended up copying the files back to a db, because
the time spend implementing something in the thread you mentioned would be
to much work compared to the scope of the project.

Best Regards
Niels

On Mon, Nov 23, 2009 at 1:46 PM, Olly Betts <olly <at> survex.com> wrote:

> On Wed, Oct 28, 2009 at 03:58:32PM +0100, Niels Boldt wrote:
> > However, we would like to store the resulting index in a database in a
> blob
> > instead of storing the index in a file on the filesystem. However we have
> > problems implementing that solution. The only way we have found so far is
> to
> > write the index to a file and then copy this file into the database which
> > seems a bit akward.
> >
> > Isn't there any way to make this a bit smarter. We haven't found any
> > serializeable functions etc on WriteableDatabase
>
> The isn't currently a way to tell the disk-based backends to store stuff
> other than in files in a specified directory.  The "InMemory" backend isn't
> what you're after for sure - it was originally written for development and
> testing, and while it can be useful, it has performance issues in some
> cases if
> fed a lot of data. At some point it's likely to get replaced with a version
> of the disk-based backend adapted to use memory instead of disk.
(Continue reading)

Permalink | Reply | 0 comments |
headers
Olly Betts | 2 Dec 00:44
Favicon
Gravatar

Re: Does xapian support attribute search & faceted search?

On Tue, Dec 01, 2009 at 02:25:10PM -0800, Andy wrote:
> Can I use xapian for a search like:
> 
> 'title' has the phrase "dinner jacket" AND 'color' == BLACK AND 'state' == MA
> AND 'size' == XL

Yes.

> What about faceted search?

Yes.  It's possible in 1.0.x (at least in languages where you can subclass
MatchDecider), but recent 1.1.x provides built-in features to support it
with less work on your part (ValueCountMatchSpy, etc).

Cheers,
    Olly
Permalink | Reply | 1 comment |
headers
Andy | 2 Dec 00:55
Picon
Favicon

Re: Does xapian support attribute search & faceted search?

Great.

Could you please point me to the relevant parts of the documentation?

Thanks.

--- On Tue, 12/1/09, Olly Betts <olly <at> survex.com> wrote:

> From: Olly Betts <olly <at> survex.com>
> Subject: Re: [Xapian-discuss] Does xapian support attribute search & faceted  search?
> To: "Andy" <angelflow <at> yahoo.com>
> Cc: xapian-discuss <at> lists.xapian.org
> Date: Tuesday, December 1, 2009, 6:44 PM
> On Tue, Dec 01, 2009 at 02:25:10PM
> -0800, Andy wrote:
> > Can I use xapian for a search like:
> > 
> > 'title' has the phrase "dinner jacket" AND 'color' ==
> BLACK AND 'state' == MA
> > AND 'size' == XL
> 
> Yes.
> 
> > What about faceted search?
> 
> Yes.  It's possible in 1.0.x (at least in languages
> where you can subclass
> MatchDecider), but recent 1.1.x provides built-in features
> to support it
> with less work on your part (ValueCountMatchSpy, etc).
(Continue reading)

Permalink | Reply | 0 comments |
headers
Richard Heycock | 2 Dec 03:11
Picon

"Rolling" Database

I need a database where the last three months documents need to be
searchable, anything beyond that can be archived. So I'm thinking about
implementing a "rolling" database where I have a database per month and
combine them into one. The latest database would be writable and the
previous two being read-only. When the month ends I would close all the
existing databases and reopen them to include the new month.

For example: December would look like this:

    200912 -> writable
    200911 -> read-only
    200910 -> read-only

And January like this:

    201001 -> writable
    200912 -> read-only
    200911 -> read-only

I'm hoping to use this scheme for a number of reasons: the latest database
is read *significantly* more often than any of the earlier databases;
to be able to manage the ever growing size of the database and to be
able to compact the read only databases.

I'm using the ruby bindings and I've got a couple of questions.

1) is it possible to close a database? I can flush the database and set
   the database object to nil but I can't force the database destructor
   to be called even if I run the garbage collector.
(Continue reading)

Permalink | Reply | 1 comment |
headers
Niels Boldt | 2 Dec 10:27
Picon

Filelocks not released

Hi,

We are using xapian in .Net with the provided bindings.

Currently we need to move the files forming an index into a database, so an
index can be shared by several users of the application. It is a desktop
application.

So we have a Xapian.WritableDatabase which we do arbitrary updates on and at
some time we call dispose on the index and then we want to copy the files.

But this procedure keeps giving us errors because xapian does not release
the locks it holds on the files, so we cannot get access to copy them. Is it
not sufficient to call 'Dispose'. Should the object also be finalized before
the locks on the files are release

Best Regards
Niels

--

-- 
BinaryConstructors ApS
Vestergade 10a, 4th
1456 Kbh K
Denmark
phone: +4528138757
web: http://www.binaryconstructors.dk
mail: nb <at> binaryconstructors.dk
skype: nielsboldt
Permalink | Reply | 3 comments |
headers
Charlie Hull | 2 Dec 10:30

Re: Filelocks not released

Niels Boldt wrote:
> Hi,
> 
> We are using xapian in .Net with the provided bindings.

Can you confirm you're using the C# bindings?
> 
> Currently we need to move the files forming an index into a database, so an
> index can be shared by several users of the application. It is a desktop
> application.

> So we have a Xapian.WritableDatabase which we do arbitrary updates on and at
> some time we call dispose on the index and then we want to copy the files.
> 
> But this procedure keeps giving us errors because xapian does not release
> the locks it holds on the files, so we cannot get access to copy them. Is it
> not sufficient to call 'Dispose'. Should the object also be finalized before
> the locks on the files are release

If you can post source code that would be very helpful.

Regards

Charlie
> 
> Best Regards
> Niels
> 
> 
>
(Continue reading)

Permalink | Reply | 0 comments |

Gmane