headers
Wolfgang Meier | 5 Jun 2003 14:25
Picon

Re: Fulltext indexing and selective indexation ( WAS Re: store arrays of numbers efficiently)

> I appreciate that this is part of a remedy for a previously unresolved
> issue, namely that if
> (a) selective indexing had been configured for a given doctype and
> (b) a fulltext index operator was used in a query,
> that query would fail if the search terms had been excluded by the
> selective indexing. I can see that this behaviour was unintended and in the
> general case undesirable, and it needed altering to the current CVS
> behaviour, which in such cases now falls back on searching the actual nodes
> if the fulltext index search fails.

There seems to be some misunderstanding here: the CVS changes do not apply to 
any of the fulltext operators. They will still fail if an element has been 
excluded from the fulltext index. You should thus see the same behaviour as 
before - unless there's a new bug in the code.

The changes just apply to exact string comparisons, i.e. in equality 
expressions (=), and functions contains(), starts-with(), ends-with(). The 
implementation will first check the fulltext index, then fall back to a 
sequential scan.

Wolfgang

-------------------------------------------------------
This SF.net email is sponsored by:  Etnus, makers of TotalView, The best
thread debugger on the planet. Designed with thread debugging features
you've never dreamed of, try TotalView 6 free at www.etnus.com.
Permalink | Reply | 2 comments |
headers
Wolfgang Meier | 5 Jun 2003 14:29
Picon

Re: maintenance release

I would like to submit a new release within a week or two. There are still 
some issues to solve ...

Wolfgang

On Wednesday 04 June 2003 20:10, Jon Williams wrote:
> not to be greedy, but it seems like there've been
> a lot of bug fixes recently - are we very far away
> from the next maintenance release?
>
> (I don't really feel comfortable pulling the CVS versions
> as there's less guarantee of compatibility and reliability,
> but I upgrade pretty regularly when new releases come out)
>
> Anyway, my DB is corrupted again ... and while the backup
> facilities have made it much easier to fix ... 

-------------------------------------------------------
This SF.net email is sponsored by:  Etnus, makers of TotalView, The best
thread debugger on the planet. Designed with thread debugging features
you've never dreamed of, try TotalView 6 free at www.etnus.com.
Permalink | Reply | 0 comments |
headers
Wolfgang Meier | 5 Jun 2003 14:52
Picon

Sending large files through xmlrpc

I nearly forgot to mention another change available in the CVS version: when 
using the xmldb driver to store a larger file into the database, previous 
releases used to either interrupt the transmission of the file or to end up 
with an out-of-memory exception. This is now fixed: files larger than 500K 
are split into chunks and uploaded to the server through the upload() method 
of the xmlrpc interface. The server stores the chunks into a temporary file, 
and the client can call parseLocal() to make the server parse it.

Wolfgang

-------------------------------------------------------
This SF.net email is sponsored by:  Etnus, makers of TotalView, The best
thread debugger on the planet. Designed with thread debugging features
you've never dreamed of, try TotalView 6 free at www.etnus.com.
Permalink | Reply | 0 comments |
headers
Michael Beddow | 5 Jun 2003 13:06

Re: Fulltext indexing and selective indexation ( WAS Re: store arrays of numbers efficiently)

> There seems to be some misunderstanding here: the CVS changes do not apply
to
> any of the fulltext operators. They will still fail if an element has been
> excluded from the fulltext index. You should thus see the same behaviour
as
> before - unless there's a new bug in the code.

Sorry, yes I had misunderstood the changes and misinterpreted the reasons
why I was seeing additional hits from queries run under the CVS version.

So is this right as a summary of the new behaviour:

Case 1: Query uses eXist-specific extended  match operators or functions
      Hits returned only when match on fulltext index
Case 2: Query uses standard XPath equality operator or string-match
functions
      Match attempted first against fulltext index, with fallback to node
scan in case of failure

where only Case 2: is a change of behaviour, and one that should be
transparent to the caller (except
that response times should be faster where the fulltext index lookup
succeeds)?

Michael Beddow

-------------------------------------------------------
This SF.net email is sponsored by:  Etnus, makers of TotalView, The best
thread debugger on the planet. Designed with thread debugging features
you've never dreamed of, try TotalView 6 free at www.etnus.com.
(Continue reading)

Permalink | Reply | 1 comment |
headers
Wolfgang Meier | 5 Jun 2003 18:15
Picon

mailing list archive on gmame?

I just received a message telling me that someone has added the eXist mailing 
list to Gmame. Great idea! I have to admit that I have not heard of Gmame 
before, but I guess it solves our problems with the mailing list archives.

Gmame is a mail-to-news gateway with archiving functions. You can find our 
mailing-list at:

http://news.gmane.org/thread.php?group=gmane.text.xml.exist

Whoever added it: Thanks a lot !!!

Wolfgang

-------------------------------------------------------
This SF.net email is sponsored by:  Etnus, makers of TotalView, The best
thread debugger on the planet. Designed with thread debugging features
you've never dreamed of, try TotalView 6 free at www.etnus.com.
Permalink | Reply | 1 comment |
headers
Wolfgang Meier | 5 Jun 2003 18:19
Picon

Re: Fulltext indexing and selective indexation ( WAS Re: store arrays of numbers efficiently)

> So is this right as a summary of the new behaviour:
>
> Case 1: Query uses eXist-specific extended  match operators or functions
>       Hits returned only when match on fulltext index
> Case 2: Query uses standard XPath equality operator or string-match
> functions
>       Match attempted first against fulltext index, with fallback to node
> scan in case of failure
>
> where only Case 2: is a change of behaviour, and one that should be
> transparent to the caller (except
> that response times should be faster where the fulltext index lookup
> succeeds)?

Yes, this is a correct.

Best Regards,

Wolfgang

-------------------------------------------------------
This SF.net email is sponsored by:  Etnus, makers of TotalView, The best
thread debugger on the planet. Designed with thread debugging features
you've never dreamed of, try TotalView 6 free at www.etnus.com.
Permalink | Reply | 0 comments |
headers
Jim Stauffer | 5 Jun 2003 19:35

Re: Ready for Prime Time?

Mike Zimmerman's "Ready for Prime Time?", 
2003-04-18, asked questions I too need answered, 
but I find no replies in its thread.

In particular, has anybody done benchmarks to 
get hard performance data compared to SQL DBs?

Below is a description of my application.

Info on relevant investigations already done 
would help me considerably, since designing, 
equipping and configuring valid industrial 
strength comparison studies is a big job.

Thanks,
Jim

Target application:

The docs to be housed will be 
XML versions of what currently come as dumps 
from a dozen large relational tables.  
Our legacy processes slice, dice 
and load a subset of the data into a SQL DB, 
in a complex rearrangement of there relations. 

Only one or two fields out of many dozens in 
each new table get indexed as the 
relation's search key.  Updates are at most
one a day.  Queries can be many per second.
(Continue reading)

Permalink | Reply | 0 comments |
headers
Alex Greif | 6 Jun 2003 00:05
Picon

createResource unique ID question

Hi,

Is there a possibility that exist creates the unique id itself when I use
Collection.createResource?

background:
I want to store many datasets of the same type on the fly where all relevant
information is in the xml file itself. I dont want to bother with the
unique doc id.

thanks
alex.

-------------------------------------------------------
This SF.net email is sponsored by:  Etnus, makers of TotalView, The best
thread debugger on the planet. Designed with thread debugging features
you've never dreamed of, try TotalView 6 free at www.etnus.com.
Permalink | Reply | 1 comment |
headers
Alex Greif | 6 Jun 2003 00:16
Picon

Re: createResource unique ID question

hi,

Though the javadoc of Collection.createResource () tells the following:

"If id is null or its value is empty then an id is generated by    
calling createId() ."

in both cases (id=null and id="") I get the following exceptions:

java.lang.ArrayIndexOutOfBoundsException: -1
	at org.exist.util.ByteConversion.byteToInt(ByteConversion.java:92)
	at org.exist.storage.BFile.append(BFile.java:181)
	at org.exist.storage.NativeTextEngine$InvertedIndex.flushWord(NativeTextEngine.java:1031)
	at org.exist.storage.NativeTextEngine$InvertedIndex.flush(NativeTextEngine.java:1010)
	at org.exist.storage.NativeTextEngine.flush(NativeTextEngine.java:251)
	at org.exist.storage.NativeBroker.flush(NativeBroker.java:450)
	at org.exist.Parser.store(Parser.java:959)
	at org.exist.Parser.parse(Parser.java:446)
	at org.exist.xmldb.LocalCollection.storeResource(LocalCollection.java:417)
	at Embed.main(Embed.java:41)
Exception in thread "main" org.xmldb.api.base.XMLDBException: -1
	at org.exist.xmldb.LocalCollection.storeResource(LocalCollection.java:425)
	at Embed.main(Embed.java:41)
Caused by: java.lang.ArrayIndexOutOfBoundsException: -1
	at org.exist.util.ByteConversion.byteToInt(ByteConversion.java:92)
	at org.exist.storage.BFile.append(BFile.java:181)
	at org.exist.storage.NativeTextEngine$InvertedIndex.flushWord(NativeTextEngine.java:1031)
	at org.exist.storage.NativeTextEngine$InvertedIndex.flush(NativeTextEngine.java:1010)
	at org.exist.storage.NativeTextEngine.flush(NativeTextEngine.java:251)
	at org.exist.storage.NativeBroker.flush(NativeBroker.java:450)
(Continue reading)

Permalink | Reply | 0 comments |
headers
Kevin O'Neill | 6 Jun 2003 00:50
Picon

Re: mailing list archive on gmame?

On Thu, 05 Jun 2003 18:15:57 +0200, Wolfgang Meier wrote:

> Whoever added it: Thanks a lot !!!

:)

I'm a gmane addict.

-k.

--

-- 
If you don't test then your code is only a collection of bugs which 
apparently behave like a working program. 

Website: http://www.rocketred.com.au/blogs/kevin/

-------------------------------------------------------
This SF.net email is sponsored by:  Etnus, makers of TotalView, The best
thread debugger on the planet. Designed with thread debugging features
you've never dreamed of, try TotalView 6 free at www.etnus.com.
Permalink | Reply | 0 comments |

Gmane