Sungsoo Kim | 3 Mar 2006 16:20
Picon

xapian-0.9.4 queryparser build errors after applying utf-8 patch

I have encountered the following build errors after applying xapian-qp-utf8-0.9.2.patch.
There is no build error before the patch.

- CentOS 4.2 64 bit version (RHEL 4.0.2 compatible)
- AMD Opteron dual processor

Can anybody tell me what I am missing?

Thanks!


Sungsoo Kim


--------------------------------------------------


[root <at> saturn queryparser]# make
make  all-am
make[1]: Entering directory `/usr/src/redhat/BUILD/xapian-core-0.9.4/queryparser'
if /bin/sh ../libtool --tag=CXX --mode=compile g++ -DHAVE_CONFIG_H -I. -I. -I.. -I../include
-I../include -I../common -I../api -I/usr/include/glib-2.0 -I/usr/lib/glib-2.0/include   -Wall -W
-Wredundant-decls -Wundef -Wpointer-arith -Wcast-qual -Wcast-align -Wno-multichar
-Wno-long-long -fno-gnu-keywords -O2 -g -pipe -m64 -MT queryparser_internal.lo -MD -MP -MF
".deps/queryparser_internal.Tpo" -c -o queryparser_internal.lo queryparser_internal.cc; \
then mv -f ".deps/queryparser_internal.Tpo" ".deps/queryparser_internal.Plo"; else rm -f
".deps/queryparser_internal.Tpo"; exit 1; fi
 g++ -DHAVE_CONFIG_H -I. -I. -I.. -I../include -I../include -I../common -I../api
-I/usr/include/glib-2.0 -I/usr/lib/glib-2.0/include -Wall -W -Wredundant-decls -Wundef
-Wpointer-arith -Wcast-qual -Wcast-align -Wno-multichar -Wno-long-long -fno-gnu-keywords -O2 -g
(Continue reading)

Sungsoo Kim | 3 Mar 2006 16:56
Picon

Re: xapian-0.9.4 queryparser build errors after applying utf-8 patch

I have solved the problem by changing the path in Makefile.in from /usr/lib/* to /usr/lib64/*.
Sorry for any inconvenience!

Thanks!


Sungsoo Kim
_______________________________________________
Xapian-devel mailing list
Xapian-devel <at> lists.xapian.org
http://lists.xapian.org/mailman/listinfo/xapian-devel
Sungsoo Kim | 7 Mar 2006 10:39
Picon

FLAG_BOOLEAN_ANY_CASE does not work

I have found that lower cased boolean operators such as "and" or "or"
does not work. Of course I never forget setting FLAG_BOOLEAN_ANY_CASE flag.
QP seems to treat them as terms.
 
Just look at the following tests regardless of search results!
 
    $ python search.py -v woman AND man
    Performing query 'Xapian::Query((woman:(pos=1) AND man:(pos=2)))'
    0 results found
    $ python search.py -v woman and man
    Performing query 'Xapian::Query((woman:(pos=1) AND and:(pos=2) AND man:(pos=3)))'
    0 results found
In the following code in queryparser.lemony, I think every "term" within if () condition 
should be changed to "lcterm". I attach a patch file to fix this.
 
 
                    } else if (flags & FLAG_BOOLEAN_ANY_CASE) {
                        string lcterm = downcase_term(term);
                        if (term == "and") {
                            Parse(pParser, AND, NULL, &state);
                            continue;
                        } else if (term == "or") {
                            Parse(pParser, OR, NULL, &state);
                            continue;
                        } else if (term == "not") {
                            Parse(pParser, NOT, NULL, &state);
                            continue;
                        } else if (term == "near") {
                            Parse(pParser, NEAR, NULL, &state);
                            continue;
                        } else if (term == "xor") {
                            Parse(pParser, XOR, NULL, &state);
                            continue;
                        }
                    }
 
 
 
For better Xapian,
 
 
Sungsoo Kim
 
Attachment (queryparser-flag_boolean_any_case.patch): application/octet-stream, 872 bytes
_______________________________________________
Xapian-devel mailing list
Xapian-devel <at> lists.xapian.org
http://lists.xapian.org/mailman/listinfo/xapian-devel
Community Networks | 7 Mar 2006 12:14

AIR wants to move from CD-ROM based products to Web

Hello,

 I am the Software Engineer to CommNet (www.community-networks.net).
 All India Reporter is the largest publisher of law
 journals in India and publishing their content on
 CD-ROMs using Folio for last 7 years. Now they would
 like to also publish using the web.

 Please give me the list of packaged search engine ,
 options they have using for fastsearch. 
 Please give complete range from
 low end to high end along with advantages and
 disadvantages. Also include initial pricing and
 recurring license fees.

 I have promised that we will give them a proposal in
 next couple of days since they would like to decide
 and make investments before March 31, 2006 (before
 end of Financial Year in India). Hence your early
 response will be appreciated.

 Can you also give me idea of the underlying platform
 requirements and bandwidth considerations. I am
 meeting few hosting service providers to get their
 quotations. Also is their way to account for
 bandwidth being used by individual users? This will
 help in billing.

Eagerly waiting for your reply.

 Regards
 Ms.Noorjahan
Olly Betts | 7 Mar 2006 14:15
Favicon
Gravatar

Re: FLAG_BOOLEAN_ANY_CASE does not work

On Tue, Mar 07, 2006 at 06:39:29PM +0900, Sungsoo Kim wrote:
> In the following code in queryparser.lemony, I think every "term"
> within if () condition should be changed to "lcterm". I attach a patch
> file to fix this.
> 
>                     } else if (flags & FLAG_BOOLEAN_ANY_CASE) {
>                         string lcterm = downcase_term(term);
>                         if (term == "and") {
>                             Parse(pParser, AND, NULL, &state);
>                             continue;
>                         } else if (term == "or") {

As you say, it ought to be using lcterm (at least if we want 'woman And
man' to work.  But I can't see why your example ('woman and man')
doesn't already work since we check '(term == "and")'.

How are you setting FLAG_BOOLEAN_ANY_CASE?

Cheers,
    Olly
Sungsoo Kim | 7 Mar 2006 15:15
Picon

Re: FLAG_BOOLEAN_ANY_CASE does not work

> As you say, it ought to be using lcterm (at least if we want 'woman And
> man' to work.  But I can't see why your example ('woman and man')
> doesn't already work since we check '(term == "and")'.

Yes, you are right. I missed the fact that "and" was already
lowercase. So it does not need to be converted to lowercase when I
search by "woman and man".

> How are you setting FLAG_BOOLEAN_ANY_CASE?

I have set the flag as shown below.

 query = qp.parse_query(input,
        xapian.QueryParser.FLAG_BOOLEAN |
        xapian.QueryParser.FLAG_BOOLEAN_ANY_CASE |
        xapian.QueryParser.FLAG_PHRASE |
        xapian.QueryParser.FLAG_LOVEHATE |
        xapian.QueryParser.FLAG_WILDCARD)

I've figured out why QP does not accept "and" as operator.
It is resulted from xapian-qp-utf8-0.9.2.patch.
U_isupper(term[0]) should be changed back before the patch.

original xapian-0.9.4

                if (prefix.empty() && !term.empty() && C_isalpha(term[0])) {
                    if (C_isupper(term[0])) {
                        ...
                    } else if (flags & FLAG_BOOLEAN_ANY_CASE) {
                        ...
                    }
                }

after xapian-qp-utf8-0.9.2.patch

                if (prefix.empty() && !term.empty() && U_isupper(term[0])) {
                    if (C_isupper(term[0])) {
                        ...
                    } else if (flags & FLAG_BOOLEAN_ANY_CASE) {
                        ...
                    }
                }

For better Xapian,

Sungsoo Kim
Sungsoo Kim | 7 Mar 2006 16:05
Picon

How to rebuild python-bindings using SWIG

Dear xapian developers,
 
I've added a function named "set_sort_by_relevance_then_value()".
Now I want to rebuild python-bindings to include the new function
and test if it works or not.
 
I guess there is a way to rebuild all of the binding modules include
the new function.
 
I am not used to SWIG. I want to which files I need to change and
how to build it.
 
 
Thanks!
 
 
Sungsoo Kim
 
_______________________________________________
Xapian-devel mailing list
Xapian-devel <at> lists.xapian.org
http://lists.xapian.org/mailman/listinfo/xapian-devel
Olly Betts | 8 Mar 2006 03:43
Favicon
Gravatar

Re: Re: FLAG_BOOLEAN_ANY_CASE does not work

On Tue, Mar 07, 2006 at 11:15:46PM +0900, Sungsoo Kim wrote:
> I've figured out why QP does not accept "and" as operator.
> It is resulted from xapian-qp-utf8-0.9.2.patch.

I should update the patch, but I was going to do it when I upgraded
gmane to 0.9.4 (which requires a little care because I'm using a patched
version of Omega, but added a better solution to the issue one of the
patches addresses in 0.9.4, so I need to update gmane's setup to use the
approach in 0.9.4).

Anyhow, I've applied a fix so that FLAG_BOOLEAN_ANY_CASE allows "And",
etc to be used as a boolean operator.

Cheers,
    Olly
Olly Betts | 8 Mar 2006 06:37
Favicon
Gravatar

Re: AIR wants to move from CD-ROM based products to Web

On Tue, Mar 07, 2006 at 04:44:50PM +0530, Community Networks wrote:
>  Please give complete range from
>  low end to high end along with advantages and
>  disadvantages. Also include initial pricing and
>  recurring license fees.

Xapian is Free Software.  There's no charge for use at all, though you
have to follow the licensing conditions.

However, if you're looking for someone to help you develop a bespoke
solution, then they'll probably want payment for their work.  There are
several people on this list who may be able to help (including myself)
but you'd need to discuss your requirements, and negotiate a mutually
agreeable payment structure for the work.

>  Can you also give me idea of the underlying platform
>  requirements and bandwidth considerations.

Xapian runs on any modern Unix-like platform (including Linux and
MacOS X), and also on Windows.

I'd imagine that the bandwith usage for a web front-end searching
an archive of legal journals would be pretty low, but it's rather
hard to say without knowing more about your application.

Cheers,
    Olly
Olly Betts | 9 Mar 2006 01:26
Favicon
Gravatar

Re: How to rebuild python-bindings using SWIG

On Wed, Mar 08, 2006 at 12:05:12AM +0900, Sungsoo Kim wrote:
> I've added a function named "set_sort_by_relevance_then_value()".
> Now I want to rebuild python-bindings to include the new function
> and test if it works or not.
> 
> I guess there is a way to rebuild all of the binding modules include
> the new function.
> 
> I am not used to SWIG. I want to which files I need to change and
> how to build it.

I sent a patch by private mail, but for the record I'm in the process of
writing a "HACKING" file to add to the bindings which can serve as a
repository for information like this.

Cheers,
    Olly

Gmane