Jackrabbit-RMI performance problems

Re: Jackrabbit-RMI performance problems

Hi,

On Thu, Sep 1, 2011 at 9:14 AM, Kevin Pfaff <kevin.pfaff <at> byteconsult.de> wrote:
> I have a little problem with the performance with jcr over rmi on different
> pcs. [...] Anybody some suggestions?

Sounds like differences in RMI or network configuration. Have you
traced the network traffic between the RMI client and the different
servers?

BR,

Jukka Zitting

[Created] (JCR-3064) Concurrent access performance drop

Concurrent access performance drop
----------------------------------

                 Key: JCR-3064
                 URL: https://issues.apache.org/jira/browse/JCR-3064
             Project: Jackrabbit Content Repository
          Issue Type: Bug
            Reporter: Jukka Zitting
            Priority: Blocker
             Fix For: 2.3.0
         Attachments: ConcurrentReadTest.png, ConcurrentReadWriteTest.png

Our performance tests show a pretty bad drop in concurrent access performance (both read and write) in the
latest trunk when compared to Jackrabbit 2.2. We need to track down the cause and fix it before the 2.3 release.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[Updated] (JCR-3064) Concurrent access performance drop

     [
https://issues.apache.org/jira/browse/JCR-3064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jukka Zitting updated JCR-3064:
-------------------------------

    Attachment: ConcurrentReadWriteTest.png
                ConcurrentReadTest.png

Attached plots of the concurrent performance tests.

> Concurrent access performance drop
> ----------------------------------
>
>                 Key: JCR-3064
>                 URL: https://issues.apache.org/jira/browse/JCR-3064
>             Project: Jackrabbit Content Repository
>          Issue Type: Bug
>            Reporter: Jukka Zitting
>            Priority: Blocker
>             Fix For: 2.3.0
>
>         Attachments: ConcurrentReadTest.png, ConcurrentReadWriteTest.png
>
>
> Our performance tests show a pretty bad drop in concurrent access performance (both read and write) in the
latest trunk when compared to Jackrabbit 2.2. We need to track down the cause and fix it before the 2.3 release.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Re: Latest Jackrabbit performance test results

Hi,

On Thu, Aug 25, 2011 at 11:42 AM, Jukka Zitting <jukka.zitting <at> gmail.com> wrote:
> Results from a performance test run from last night:
>
> http://people.apache.org/~jukka/jackrabbit/report-2011-08-25/report.html

Here's an updated report:

http://people.apache.org/~jukka/jackrabbit/report-2011-09-01/report.html

> * The BigFileReadTest results are pretty fascinating. It looks like
> the set of ten 100MB test files fits into the OS buffer cache (the
> test computer has 4GB RAM) and unlike in previous Jackrabbit versions
> we can now for some reason avoid invalidating that cache.

That was indeed the case. I increased the test set from ten to a
hundred 100MB files, which prevented all of them from fitting into
memory at a time, and now there's much less variance in performance.

Note that there was a massive performance problem in writing large
files in the default configuration of all of Jackrabbit 1.4, 1.5 and
1.6. I had to disable the big file tests on those versions as
otherwise writing the 10GB of test data would have taken way too long
to be practical.

> * We have a notable regression in concurrent read and read/write
> performance. I'll file a blocker issue for that, to be fixed before
> Jackrabbit 2.3.

See https://issues.apache.org/jira/browse/JCR-3064

> * Massive performance improvements for SetProperty and
> UpdateManyChildNodes tests. I'm not sure which of the recent changes
> is giving this performance boost.

The performance boost can not be seen in the latest results anymore,
so it might have been just a fluke.

BR,

Jukka Zitting

Update on j3

Hi,

The level of activity on the j3 sandbox has been pretty nice in the
past few weeks. Would someone working on the code care to give a short
update on the latest developments? The commit messages have been
rather terse for now...

BR,

Jukka Zitting

Re: Update on j3

hi jukka

On Thu, Sep 1, 2011 at 2:11 PM, Jukka Zitting <jukka.zitting <at> gmail.com> wrote:
> Hi,
>
> The level of activity on the j3 sandbox has been pretty nice in the
> past few weeks. Would someone working on the code care to give a short
> update on the latest developments? The commit messages have been
> rather terse for now...

we're still at a very early stage and the code base is still far from
stabilizing.
we're experimenting with different ideas/approaches which sometimes get
discarded soon after (much like in a proper sandbox ;). that's why commit
msgs might be terse at times.

the goal is to have a working spi-based stack with the microkernel underneath
(jcr2spi->spi2mk). such a stack would serve us as a POC for the technical
viability of a MVCC-based microkernel approach. furthermore it would allow
to benchmark the mk-based stack against jackrabbit-core and jcr2spi->spi2jcr.

one thing we've learned so far is that the spi abstraction layer probably needs
to be largely re-written/re-designed in order to fully take advantage of a
MVCC-based microkernel.

cheers
stefan

>
> BR,
>
> Jukka Zitting
>

[Commented] (JCR-2272) Errors during concurrent session import of nodes with same UUIDs

    [
https://issues.apache.org/jira/browse/JCR-2272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13095286#comment-13095286
] 

Julian Reschke commented on JCR-2272:
-------------------------------------

The last patch applies against trunk if 

  public static final int STATUS_STALE_MODIFIED = 5;

gets re-added (not sure about the rational for removing it).

Confirming that testUpdate fails.

I also get four failures in the ShareableNodesTests.

> Errors during concurrent session import of nodes with same UUIDs
> ----------------------------------------------------------------
>
>                 Key: JCR-2272
>                 URL: https://issues.apache.org/jira/browse/JCR-2272
>             Project: Jackrabbit Content Repository
>          Issue Type: Bug
>          Components: jackrabbit-core, xml
>    Affects Versions: 2.0-alpha8
>            Reporter: Tobias Bocanegra
>         Attachments: JCR-2272.patch, JCR-2272_NPE.patch,
JCR-2272__Errors_during_concurrent_session_import_of_nodes_with_same_UUIDs.patch, JCR-2272_revised.patch
>
>
> 21.08.2009 16:22:14 *ERROR* [Executor 0] ConnectionRecoveryManager: could not execute statement,
reason: The statement was aborted because it would have caused a duplicate key value in a unique or primary
key constraint or unique index identified by 'SQL090821042140130' defined on 'DEFAULT_BUNDLE'.,
state/code: 23505/20000 (ConnectionRecoveryManager.java, line 453)
> 21.08.2009 16:22:14 *ERROR* [Executor 0] BundleDbPersistenceManager: failed to write bundle:
6c292772-349e-42b3-8255-7729615c67de (BundleDbPersistenceManager.java, line 1212)
> ERROR 23505: The statement was aborted because it would have caused a duplicate key value in a unique or
primary key constraint or unique index identified by 'SQL090821042140130' defined on 'DEFAULT_BUNDLE'.
> 	at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
> 	at org.apache.derby.impl.sql.execute.IndexChanger.insertAndCheckDups(Unknown Source)
> 	at org.apache.derby.impl.sql.execute.IndexChanger.doInsert(Unknown Source)
> 	at org.apache.derby.impl.sql.execute.IndexChanger.insert(Unknown Source)
> 	at org.apache.derby.impl.sql.execute.IndexSetChanger.insert(Unknown Source)
> 	at org.apache.derby.impl.sql.execute.RowChangerImpl.insertRow(Unknown Source)
> 	at org.apache.derby.impl.sql.execute.InsertResultSet.normalInsertCore(Unknown Source)
> 	at org.apache.derby.impl.sql.execute.InsertResultSet.open(Unknown Source)
> 	at org.apache.derby.impl.sql.GenericPreparedStatement.execute(Unknown Source)
> 	at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown Source)
> 	at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeStatement(Unknown Source)
> 	at org.apache.derby.impl.jdbc.EmbedPreparedStatement.execute(Unknown Source)
> 	at org.apache.jackrabbit.core.persistence.bundle.util.ConnectionRecoveryManager.executeStmtInternal(ConnectionRecoveryManager.java:371)
> 	at org.apache.jackrabbit.core.persistence.bundle.util.ConnectionRecoveryManager.executeStmtInternal(ConnectionRecoveryManager.java:298)
> 	at org.apache.jackrabbit.core.persistence.bundle.util.ConnectionRecoveryManager.executeStmt(ConnectionRecoveryManager.java:261)
> 	at org.apache.jackrabbit.core.persistence.bundle.util.ConnectionRecoveryManager.executeStmt(ConnectionRecoveryManager.java:239)
> 	at org.apache.jackrabbit.core.persistence.bundle.BundleDbPersistenceManager.storeBundle(BundleDbPersistenceManager.java:1209)
> 	at org.apache.jackrabbit.core.persistence.bundle.AbstractBundlePersistenceManager.putBundle(AbstractBundlePersistenceManager.java:709)
> 	at org.apache.jackrabbit.core.persistence.bundle.AbstractBundlePersistenceManager.storeInternal(AbstractBundlePersistenceManager.java:651)
> 	at org.apache.jackrabbit.core.persistence.bundle.AbstractBundlePersistenceManager.store(AbstractBundlePersistenceManager.java:527)
> 	at org.apache.jackrabbit.core.persistence.bundle.BundleDbPersistenceManager.store(BundleDbPersistenceManager.java:563)
> 	at org.apache.jackrabbit.core.state.SharedItemStateManager$Update.end(SharedItemStateManager.java:724)
> 	at org.apache.jackrabbit.core.state.SharedItemStateManager.update(SharedItemStateManager.java:1101)
> 	at org.apache.jackrabbit.core.state.LocalItemStateManager.update(LocalItemStateManager.java:351)
> 	at org.apache.jackrabbit.core.state.XAItemStateManager.update(XAItemStateManager.java:354)
> 	at org.apache.jackrabbit.core.state.LocalItemStateManager.update(LocalItemStateManager.java:326)
> 	at org.apache.jackrabbit.core.state.SessionItemStateManager.update(SessionItemStateManager.java:326)
> 	at org.apache.jackrabbit.core.ItemImpl.save(ItemImpl.java:1098)
> 	at org.apache.jackrabbit.core.SessionImpl.save(SessionImpl.java:925)
> 	at org.apache.jackrabbit.core.ConcurrentImportTest$1.execute(ConcurrentImportTest.java:73)
> 	at org.apache.jackrabbit.core.AbstractConcurrencyTest$Executor.run(AbstractConcurrencyTest.java:209)
> 	at java.lang.Thread.run(Thread.java:637)

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Re: Update on j3

Hi,

>> The level of activity on the j3 sandbox has been pretty nice in the
>> past few weeks. Would someone working on the code care to give a short
>> update on the latest developments? The commit messages have been
>> rather terse for now...
>
> the goal is to have a working spi-based stack with the microkernel underneath
> (jcr2spi->spi2mk). such a stack would serve us as a POC for the technical
> viability of a MVCC-based microkernel approach. furthermore it would allow
> to benchmark the mk-based stack against jackrabbit-core and jcr2spi->spi2jcr.

To get an idea on how far this has been progressed, have a look at the 
pom [1]. I just enabled the TCK test suite for the default build. The 
pom lists the known issues and their reasons. All known issues which 
have a fixme tag in the comment should be fixed. The other ones are 
currently 'as designed' i.e. limitations which we deem acceptable for 
the prototype at its current state.

> one thing we've learned so far is that the spi abstraction layer probably needs
> to be largely re-written/re-designed in order to fully take advantage of a
> MVCC-based microkernel.

That's why I branched jcr2spi, spi and spi-commons. The idea is to 
evolve these into the new architecture where possible and rewrite where 
necessary. There has no real work been done in this area. My large 
commit [2] from today was just syntactical cleanup.

Michael

[1] 
http://svn.apache.org/viewvc/jackrabbit/sandbox/jackrabbit-mk/jackrabbit-spi2microkernel/pom.xml?view=markup
[2] http://svn.apache.org/viewvc?view=revision&revision=1164026

>
> cheers
> stefan
>
>>
>> BR,
>>
>> Jukka Zitting
>>

[Commented] (JCR-2272) Errors during concurrent session import of nodes with same UUIDs

    [
https://issues.apache.org/jira/browse/JCR-2272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13095291#comment-13095291
] 

Stefan Guggisberg commented on JCR-2272:
----------------------------------------

as of JCR-2650 STATUS_STALE_MODIFIED is not applicable anymore 

> Errors during concurrent session import of nodes with same UUIDs
> ----------------------------------------------------------------
>
>                 Key: JCR-2272
>                 URL: https://issues.apache.org/jira/browse/JCR-2272
>             Project: Jackrabbit Content Repository
>          Issue Type: Bug
>          Components: jackrabbit-core, xml
>    Affects Versions: 2.0-alpha8
>            Reporter: Tobias Bocanegra
>         Attachments: JCR-2272.patch, JCR-2272_NPE.patch,
JCR-2272__Errors_during_concurrent_session_import_of_nodes_with_same_UUIDs.patch, JCR-2272_revised.patch
>
>
> 21.08.2009 16:22:14 *ERROR* [Executor 0] ConnectionRecoveryManager: could not execute statement,
reason: The statement was aborted because it would have caused a duplicate key value in a unique or primary
key constraint or unique index identified by 'SQL090821042140130' defined on 'DEFAULT_BUNDLE'.,
state/code: 23505/20000 (ConnectionRecoveryManager.java, line 453)
> 21.08.2009 16:22:14 *ERROR* [Executor 0] BundleDbPersistenceManager: failed to write bundle:
6c292772-349e-42b3-8255-7729615c67de (BundleDbPersistenceManager.java, line 1212)
> ERROR 23505: The statement was aborted because it would have caused a duplicate key value in a unique or
primary key constraint or unique index identified by 'SQL090821042140130' defined on 'DEFAULT_BUNDLE'.
> 	at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
> 	at org.apache.derby.impl.sql.execute.IndexChanger.insertAndCheckDups(Unknown Source)
> 	at org.apache.derby.impl.sql.execute.IndexChanger.doInsert(Unknown Source)
> 	at org.apache.derby.impl.sql.execute.IndexChanger.insert(Unknown Source)
> 	at org.apache.derby.impl.sql.execute.IndexSetChanger.insert(Unknown Source)
> 	at org.apache.derby.impl.sql.execute.RowChangerImpl.insertRow(Unknown Source)
> 	at org.apache.derby.impl.sql.execute.InsertResultSet.normalInsertCore(Unknown Source)
> 	at org.apache.derby.impl.sql.execute.InsertResultSet.open(Unknown Source)
> 	at org.apache.derby.impl.sql.GenericPreparedStatement.execute(Unknown Source)
> 	at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown Source)
> 	at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeStatement(Unknown Source)
> 	at org.apache.derby.impl.jdbc.EmbedPreparedStatement.execute(Unknown Source)
> 	at org.apache.jackrabbit.core.persistence.bundle.util.ConnectionRecoveryManager.executeStmtInternal(ConnectionRecoveryManager.java:371)
> 	at org.apache.jackrabbit.core.persistence.bundle.util.ConnectionRecoveryManager.executeStmtInternal(ConnectionRecoveryManager.java:298)
> 	at org.apache.jackrabbit.core.persistence.bundle.util.ConnectionRecoveryManager.executeStmt(ConnectionRecoveryManager.java:261)
> 	at org.apache.jackrabbit.core.persistence.bundle.util.ConnectionRecoveryManager.executeStmt(ConnectionRecoveryManager.java:239)
> 	at org.apache.jackrabbit.core.persistence.bundle.BundleDbPersistenceManager.storeBundle(BundleDbPersistenceManager.java:1209)
> 	at org.apache.jackrabbit.core.persistence.bundle.AbstractBundlePersistenceManager.putBundle(AbstractBundlePersistenceManager.java:709)
> 	at org.apache.jackrabbit.core.persistence.bundle.AbstractBundlePersistenceManager.storeInternal(AbstractBundlePersistenceManager.java:651)
> 	at org.apache.jackrabbit.core.persistence.bundle.AbstractBundlePersistenceManager.store(AbstractBundlePersistenceManager.java:527)
> 	at org.apache.jackrabbit.core.persistence.bundle.BundleDbPersistenceManager.store(BundleDbPersistenceManager.java:563)
> 	at org.apache.jackrabbit.core.state.SharedItemStateManager$Update.end(SharedItemStateManager.java:724)
> 	at org.apache.jackrabbit.core.state.SharedItemStateManager.update(SharedItemStateManager.java:1101)
> 	at org.apache.jackrabbit.core.state.LocalItemStateManager.update(LocalItemStateManager.java:351)
> 	at org.apache.jackrabbit.core.state.XAItemStateManager.update(XAItemStateManager.java:354)
> 	at org.apache.jackrabbit.core.state.LocalItemStateManager.update(LocalItemStateManager.java:326)
> 	at org.apache.jackrabbit.core.state.SessionItemStateManager.update(SessionItemStateManager.java:326)
> 	at org.apache.jackrabbit.core.ItemImpl.save(ItemImpl.java:1098)
> 	at org.apache.jackrabbit.core.SessionImpl.save(SessionImpl.java:925)
> 	at org.apache.jackrabbit.core.ConcurrentImportTest$1.execute(ConcurrentImportTest.java:73)
> 	at org.apache.jackrabbit.core.AbstractConcurrencyTest$Executor.run(AbstractConcurrencyTest.java:209)
> 	at java.lang.Thread.run(Thread.java:637)

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira