Mailing list for the Sedna XML Database ()

Sojma, Zdenek | 22 May 2013 12:48

Sedna space requirements

Hello,

I have a question about memory management. I have performed some space evaluation when I periodically stored one document into a collection (document size was approximately 3MB). When I looked at a statistical information about the document, total size of the document in the database was roughly 1MB, so I suppose that compression was in ratio 3:1. Weird thing was, that size of a *.sedata file raised, while the document was periodically stored into the database, in ratio more than 2:1. When 1000 documents (3GB) were successfully stored, size of the *.sedata file was approximately 7.5GB.

I want to ask you if this is a normal behaviour or if it is possible that I have some wrong settings? Or is there a some way how to decrease these space requirements? I have to admit that this is not so critical issue in these days but it would be nice to know that it can be decreased somehow.

Thank you for your answer.

Best regards,

Zdenek Sojma

Software Application Engineer I - Test Systems

Engineering Test Services

_______________________Honeywell

Honeywell International s.r.o.

Turanka 100/1387

Brno 627 00, Czech Republic

Phone: +420 532 115 446

Cell Phone: +420 739 626 842

E-mail: zdenek.sojma-TF0kVB3jB5US+FvcfC7Uqw@public.gmane.org

------------------------------------------------------------------------------
Try New Relic Now & We'll Send You this Cool Shirt
New Relic is the only SaaS-based application performance monitoring service 
that delivers powerful full stack analytics. Optimize and monitor your
browser, app, & servers with just a few lines of code. Try New Relic
and get this awesome Nerd Life shirt! http://p.sf.net/sfu/newrelic_d2d_may

_______________________________________________
Sedna-discussion mailing list
Sedna-discussion@...
https://lists.sourceforge.net/lists/listinfo/sedna-discussion

headers

iman ghali | 6 May 2013 01:16

Help me please

hello

please i need some help

i am using sedna-php-api-ffa

and there is some missing file

i need "error_codes.h" file and i couldn't find it ..so i couldn't debug the program

Can you help me please ?

------------------------------------------------------------------------------
Get 100% visibility into Java/.NET code with AppDynamics Lite
It's a free troubleshooting tool designed for production
Get down to code-level detail for bottlenecks, with <2% overhead.
Download for free and get started troubleshooting in minutes.
http://p.sf.net/sfu/appdyn_d2d_ap2

_______________________________________________
Sedna-discussion mailing list
Sedna-discussion@...
https://lists.sourceforge.net/lists/listinfo/sedna-discussion

headers

Matteo Faralli | 15 Apr 2013 16:59

Problem using SEDNA ruby client

Hi,
i'm working on WIN7 64bit, i've installed Sedna Ruby Client (http://sedna.rubyforge.org/) with external c drvier using binary downloaded from Sedna official site.

I've installed the gem in this way:

% gem install sedna -- --with-sedna-dir=c:/sedna
gem has been installed correctly (no errors found)

now i'm trying on IRB to load the gem doing:
require "sedna"

the result is bad:

irb(main):008:0> require "sedna"
LoadError: 126: The specified module could not be found. - C:/Ruby193/lib/ruby
/gems/1.9.1/gems/sedna-0.6.0/lib/sedna.so
from C:/Ruby193/lib/ruby/site_ruby/1.9.1/rubygems/custom_require.rb:36:i
n `require'
from C:/Ruby193/lib/ruby/site_ruby/1.9.1/rubygems/custom_require.rb:36:i
n `require'
from (irb):8
from C:/Ruby193/bin/irb:12:in `<main>'

if i open the path location i can find sedna.so file, so i can't understand the problem.

Coould you help me please?

Thank you,
Matteo

------------------------------------------------------------------------------
Precog is a next-generation analytics platform capable of advanced
analytics on semi-structured data. The platform includes APIs for building
apps and a phenomenal toolset for data science. Developers can use
our toolset for easy data analysis & visualization. Get a free account!
http://www2.precog.com/precogplatform/slashdotnewsletter

_______________________________________________
Sedna-discussion mailing list
Sedna-discussion@...
https://lists.sourceforge.net/lists/listinfo/sedna-discussion

headers

Ivan Lagunov | 15 Apr 2013 12:43

Is read-only supported by XQJ

Hi Charles,

I couldn’t find the answer on my own so decided to ask. Does XQJ implementation allow running read-only transactions on Sedna? I mean this:

http://www.sedna.org/javadoc/ru/ispras/sedna/driver/SednaConnection.html#setReadonlyMode(boolean)

http://www.mail-archive.com/sedna-discussion-5NWGOfrQmnfLDRD5uJR0wg@public.gmane.orgeforge.net/msg01364.html

Best regards,

Ivan Lagunov

------------------------------------------------------------------------------
Precog is a next-generation analytics platform capable of advanced
analytics on semi-structured data. The platform includes APIs for building
apps and a phenomenal toolset for data science. Developers can use
our toolset for easy data analysis & visualization. Get a free account!
http://www2.precog.com/precogplatform/slashdotnewsletter

_______________________________________________
Sedna-discussion mailing list
Sedna-discussion@...
https://lists.sourceforge.net/lists/listinfo/sedna-discussion

headers

Ivan Shcheklein | 13 Apr 2013 11:17

Re: Sedna-discussion Digest, Vol 80, Issue 6

Hi Jocelyn, Jose

Release won't be available very soon. For now, you can download and use the latest development build:

http://modis.ispras.ru/FTPContent/sedna/development/

It's stable enough to be used in production. Still, I recommend to try it first on your workload in a sandbox. And (as you always should do) backup your data.

This version contains fn:doc-available with two arguments. You should use it instead of doc("$documents") to check if the document is in the specified collection or not. Note! This function call puts read lock on the collection.

Note, use se_exp to move data to that build.

Ivan Shcheklein,

Sedna Team

On Fri, Apr 12, 2013 at 11:48 PM, Jocelyn Raymond <jraymond-yfeSBMgouQgsA/PxXw9srA@public.gmane.org> wrote:

Is it possible to have the fn:doc-available() function made as a new sedna stable release (e.g. sedna-3.5.162-bin-linux-x64.sh currently it is: sedna-3.5.161-bin-linux-x64.sh)? If so, when can we expect to have it?
Thank you,

Jocelyn Raymond

--
Team Lead

Information Systems

Office of the Registrar

University of Alberta

780.492.3874

------------------------------------------------------------------------------
Precog is a next-generation analytics platform capable of advanced
analytics on semi-structured data. The platform includes APIs for building
apps and a phenomenal toolset for data science. Developers can use
our toolset for easy data analysis & visualization. Get a free account!
http://www2.precog.com/precogplatform/slashdotnewsletter
_______________________________________________
Sedna-discussion mailing list
Sedna-discussion <at> lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/sedna-discussion

------------------------------------------------------------------------------
Precog is a next-generation analytics platform capable of advanced
analytics on semi-structured data. The platform includes APIs for building
apps and a phenomenal toolset for data science. Developers can use
our toolset for easy data analysis & visualization. Get a free account!
http://www2.precog.com/precogplatform/slashdotnewsletter

_______________________________________________
Sedna-discussion mailing list
Sedna-discussion@...
https://lists.sourceforge.net/lists/listinfo/sedna-discussion

headers

Jocelyn Raymond | 12 Apr 2013 21:48

Re: Sedna-discussion Digest, Vol 80, Issue 6

Is it possible to have the fn:doc-available() function made as a new sedna stable release (e.g. sedna-3.5.162-bin-linux-x64.sh currently it is: sedna-3.5.161-bin-linux-x64.sh)? If so, when can we expect to have it?

Thank you,

Jocelyn Raymond

--

Team Lead

Information Systems

Office of the Registrar

University of Alberta

780.492.3874

------------------------------------------------------------------------------
Precog is a next-generation analytics platform capable of advanced
analytics on semi-structured data. The platform includes APIs for building
apps and a phenomenal toolset for data science. Developers can use
our toolset for easy data analysis & visualization. Get a free account!
http://www2.precog.com/precogplatform/slashdotnewsletter

_______________________________________________
Sedna-discussion mailing list
Sedna-discussion@...
https://lists.sourceforge.net/lists/listinfo/sedna-discussion

headers

Ivan Lagunov | 12 Apr 2013 19:23

How to calculate hash in XQuery

Hello,

Is there a way to calculate hash in XQuery for Sedna? Ideally I’d like to have something like this module for BaseX:

http://docs.basex.org/wiki/Hashing_Module

Best regards,

Ivan Lagunov

------------------------------------------------------------------------------
Precog is a next-generation analytics platform capable of advanced
analytics on semi-structured data. The platform includes APIs for building
apps and a phenomenal toolset for data science. Developers can use
our toolset for easy data analysis & visualization. Get a free account!
http://www2.precog.com/precogplatform/slashdotnewsletter

_______________________________________________
Sedna-discussion mailing list
Sedna-discussion@...
https://lists.sourceforge.net/lists/listinfo/sedna-discussion

headers

Ivan Lagunov | 12 Apr 2013 18:10

How to locate XQuery errors

Hello,

We extensively use XQuery modules in Sedna. When an error happens, what we see is something like this:

ERROR 12/04/2013 17:53:48 (TRN nxp pid=32721 sid=45 trid=37) [PPSeqChecker.cpp:do_next:75]: (XPTY0019) It is a type error if the result of a step (other than the last step) in a path expression contains an atomic value. Details: at (45:25), filter step contains atomic in position 1

This is a good message showing (row:column) information that helps a lot. However, it misses the module name that is really uncomfortable. It’d be really nice to see the module name and preferably the function name if it’s available. Is it something easy to implement?

Best regards,

Ivan Lagunov

------------------------------------------------------------------------------
Precog is a next-generation analytics platform capable of advanced
analytics on semi-structured data. The platform includes APIs for building
apps and a phenomenal toolset for data science. Developers can use
our toolset for easy data analysis & visualization. Get a free account!
http://www2.precog.com/precogplatform/slashdotnewsletter

_______________________________________________
Sedna-discussion mailing list
Sedna-discussion@...
https://lists.sourceforge.net/lists/listinfo/sedna-discussion

headers

Ivan Lagunov | 12 Apr 2013 17:29

XML:DB API performance issue - fn:doc-available

Hi Charles,

We’re having serious performance issues, probably reached the critical number of documents in the database. According to Ivan Shcheklein, it is likely to be caused by using doc(“documents”) function that actually locks all collections in the database, thus, drastically decreasing performance. Your implementation of XML:DB API works great for small datasets but according to event.log it uses doc(“documents”) quite a lot. So it may indeed result in bad performance I’m facing. As an improvement, there have been implemented a function fn:doc-available that checks if a document is available in the given collection. It locks only one collection. This is an example of usage:

doc-available(“collection”, “document”)

Could you please update the Sedna XML:DB API implementation using this function? I understand it’s recommended to migrate to XQJ implementation but it’s really hard to rewrite all the code we have. Nevertheless, I’ve planned to investigate required efforts for this migration but we need some solution on short notice. Thanks a lot!

Best regards,

Ivan Lagunov

------------------------------------------------------------------------------
Precog is a next-generation analytics platform capable of advanced
analytics on semi-structured data. The platform includes APIs for building
apps and a phenomenal toolset for data science. Developers can use
our toolset for easy data analysis & visualization. Get a free account!
http://www2.precog.com/precogplatform/slashdotnewsletter

_______________________________________________
Sedna-discussion mailing list
Sedna-discussion@...
https://lists.sourceforge.net/lists/listinfo/sedna-discussion

headers

Jocelyn Raymond | 22 Mar 2013 16:20

Deadlock resolved

Hi Sedna Team,

Just few weeks ago I sent the sedna Team an issue regarding deadlocks occurrence in sedna when we were having high volume. We were able to resolve the deadlock issue and was able to get some interesting result which will lead to a question at the end.

First, here is the code that caused the deadlock:

...

this.cn.begin();

SednaStatement st1 = this.cn.createStatement();

for (XmlDocument4Sedna xmlDocument : xmlDocuments) {

try{

if (this.exists(xmlDocument)) { // THIS CAUSE DEADLOCK

st1.execute("DROP DOCUMENT '" + xmlDocument.getDocumentname() + "' IN COLLECTION '" + xmlDocument.getCollectionName() + "'");

}

InputStream in = new ByteArrayInputStream(xmlDocument.getXmlDocument().getBytes("UTF-8"));

st1.loadDocument(in,xmlDocument.getDocumentname(), xmlDocument.getCollectionName());

}catch (Exception es){

...

}

this.cn.commit();

this.cn.close();

...

As shown in the java comment above, when we have the line to check if a document exist, it caused the deadlock. If we removed the exist line (and we knew that during our test all document existed), it did not cause the deadlock. Our test simulated 50 concurrent users. Our architecture is simply that we drop the document if it exist and insert the full new document with the new data.

SOLUTION:

Our solution was to make the "save" method static synchronized such that only one user would be able to "drop"/"load" document. Surprisingly, this serialization was faster than if we remove the line that caused the deadlock. I believe (but I may be wrong) that there are more lock handling during concurrency than serialization and hence making it slower. Our result show:

Serialization: ~4.1 document saved per second.

Concurrent: ~2.8 document saved per second.

Note: 50 users simulation saved a total of 402 documents.

Some of my questions are (I am asking just in case you know an efficient way of doing this):

Is it a good idea/solution to "DROP" and "LOAD" the new document to replace the old one each time it need to be persisted?

Should we use "UPDATE replace ..." statement instead i.e. replacing the entire document from the root of the document? (still require to see if document exist).

Should the app keep track if a document is new or not (hence would not have to check if exists on sedna)?

Which way do you usually use?

Thanks you for your support; always appreciated.

Jocelyn Raymond

FYI: our exists(xmlDocument) call above does the following:

private boolean exists(String documentName, String collectionName) throws Exception {

boolean doesExists = false;

SednaStatement st1 = this.cn.createStatement();

boolean hasResult = st1.execute("exists(doc('$documents')/documents/collection[ <at> name='" + collectionName + "']/document[ <at> name='" + documentName + "'])");

if (hasResult) {

SednaSerializedResult result = st1.getSerializedResult();

String exists = result.next();

doesExists = (exists != null && exists.equals("true")) ? true : false;

}

return doesExists;

}

--

Team Lead

Information Systems

Office of the Registrar

University of Alberta

780.492.3874

------------------------------------------------------------------------------
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_d2d_mar

_______________________________________________
Sedna-discussion mailing list
Sedna-discussion@...
https://lists.sourceforge.net/lists/listinfo/sedna-discussion

headers

Anju | 21 Mar 2013 18:26

document-uri() & doc-available()

Does the current version of Sedna support document-uri() and doc-available() for
documents in a collection?

I ran a test like this:

for $i in collection('dashboard')
 let $uri := document-uri($i)
 let $avail := doc-available($uri)
return <data><uri>{$uri}</uri><available>{$avail}</available></data>

And the output shows the uri as the document ID of the documents with
doc-available() returning false for every document. Is there a way to get
doc-available() working with collections? Thank you!

Anju
________________________________________________________
Anju Tai
Systems Analyst | Office of the Registrar | University of Alberta
1-26 South Academic Building (SAB), Edmonton, AB T6G 2M7
Tel: 780.492.6096 | anju.tai@...

------------------------------------------------------------------------------
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_d2d_mar

4	May 2013
27	April 2013
24	March 2013
34	February 2013
36	January 2013
6	December 2012
15	November 2012
12	October 2012
29	September 2012
16	August 2012
21	July 2012
13	June 2012
7	May 2012
19	April 2012
36	March 2012
26	February 2012
22	January 2012
28	December 2011
21	November 2011
20	October 2011
9	September 2011
26	August 2011
24	July 2011
41	June 2011
33	May 2011
46	April 2011
41	March 2011
32	February 2011
5	January 2011
44	December 2010
12	November 2010
60	October 2010
13	September 2010
13	August 2010
16	July 2010
29	June 2010
33	May 2010
18	April 2010
21	March 2010
16	February 2010
24	January 2010
18	December 2009
28	November 2009
18	October 2009
29	September 2009
67	August 2009
39	July 2009
37	June 2009
8	May 2009
18	April 2009
49	March 2009
51	February 2009
15	January 2009
54	December 2008
55	November 2008
68	October 2008
79	September 2008
33	August 2008
20	July 2008
34	June 2008
23	May 2008
48	April 2008
41	March 2008
24	February 2008
71	January 2008
16	December 2007
25	November 2007
4	October 2007
8	September 2007
15	August 2007
34	July 2007
62	June 2007
12	May 2007
35	April 2007
13	March 2007
49	February 2007
9	January 2007
2	December 2006
6	November 2006
18	October 2006
73	September 2006
30	August 2006
28	July 2006
28	June 2006
19	May 2006
30	April 2006
2	March 2006
22	February 2006
54	January 2006
35	December 2005
36	November 2005
27	October 2005
10	September 2005
7	August 2005
20	July 2005
52	June 2005
73	May 2005
38	April 2005
2	February 2005

Mon	Tue	Wed	Thu	Fri	Sat	Sun

		1	2	3	4	5
6	7	8	9	10	11	12
13	14	15	16	17	18	19
20	21	22	23	24	25	26
27	28	29	30	31