[jira] [Commented] (LUCENE-4161) Make PackedInts usable by codecs

    [
https://issues.apache.org/jira/browse/LUCENE-4161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13404656#comment-13404656
] 

Michael McCandless commented on LUCENE-4161:
--------------------------------------------

bq. The meaning of n is actually a bit complicated.

Thank you for the explanation!  That makes sense.  I think "iterations" is good?  Or ... maybe we simply leave
it as n and then put this nice explanation in there as a comment?

Naming is the hardest part :)

bq. What additional methods do you think we need?

I'm not sure off-hand yet ... we've been iterating in LUCENE-3892 to find the least-cost way to decode from
the underlying byte based storage from the IndexInput, but with no real clear fastest solution yet. 
Logically we are currently storing an int[] and decoding into int[], so I guess encode/decode to/from
int[]?  We should probably try long[] as the backing too ... but, I think we should explore this (adding
int[] based methods) under a new issue?  This patch is already great progress.

> Make PackedInts usable by codecs
> --------------------------------
>
>                 Key: LUCENE-4161
>                 URL: https://issues.apache.org/jira/browse/LUCENE-4161
>             Project: Lucene - Java
>          Issue Type: Improvement

Re: Low pause GC for java 1.6

Nothing?

Bill Bell
Sent from mobile

On Jun 29, 2012, at 9:09 PM, Bill Bell <billnbell <at> gmail.com> wrote:

> We are getting large Solr pauses on Java garbage collection in 1.6 Java.
> 
> We have tried CMS. But we still have. 4 second wait on GC.
> 
> What works well for Solr when using 16 GB of RAM?
> 
> I have read lots of articles and now just looking for practical advise and examples.
> 
> Sent from my Mobile device
> 720-256-8076

[jira] [Commented] (SOLR-1770) move default example core config/data into a collection1 folder

    [
https://issues.apache.org/jira/browse/SOLR-1770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13404666#comment-13404666
] 

David Smiley commented on SOLR-1770:
------------------------------------

+1 to remove the multicore example.  Furthermore, I think integrating example-DIH into the same
configuration would simply things even more.

> move default example core config/data into a collection1 folder
> ---------------------------------------------------------------
>
>                 Key: SOLR-1770
>                 URL: https://issues.apache.org/jira/browse/SOLR-1770
>             Project: Solr
>          Issue Type: Improvement
>    Affects Versions: 1.4
>            Reporter: Mark Miller
>            Assignee: Mark Miller
>            Priority: Critical
>             Fix For: 4.0, 5.0
>
>         Attachments: SOLR-1770.patch
>
>
> This is a better starting point for adding more cores - perhaps we can also get rid of multi-core example

--

RE: Low pause GC for java 1.6

Bill,

As you know, it really depends on the size of your index combined with which features you're using.  There is
really no substitute for having a good load test and monitoring tool and to run multiple tests while trying
different settings.

My guess is that you're experiencing "full" gc's, even with CMS enabled.  This means either your tenured
("old") generation is too small or you have the "-XX:CMSInitiatingOccupancyFraction" set too high (it
starts the CMS too late and runs out of memory before it can finish).  We've found that some of the defaults
the JVM picks and/or the general advice out there doesn't apply to an app like Solr, which is just a
different kind of animal than the typical web frontend you might run in a J2ee container.

Below are the settings i am using as a starting point for our development Solr 4.0 app.  These may or may not
work for you but at least should give you a basic idea of how 1 other installation is configured.

Also, if you're using older grouping patches (I remember you worked on some of these), perhaps you're
hitting some of the scalibility problems that were predicted for some of these?  I'm pretty sure the GA
grouping features in 3.x solved these problems though.

Finally, you probably will get better responses on the users list than the dev list.  Also, other users might
benefit from other answers you get, so perhaps you could cross-post your question.

James Dyer
E-Commerce Systems
Ingram Content Group
(615) 213-4311

# Basic JVM settings.  I thing the 3g new generation size is bigger than you'd normally have with a typical web
app but for us it makes the old gen fill up slower and have fewer CMS gc's.  Minor ("parnew") gc's are still
fast enough for us, even with a biggish new gen.

Re: Low pause GC for java 1.6

Re: Low pause GC for java 1.6

Out of curiosity, how long does your Solr instance run before you hit a GC 
issue? Minutes, hours, days? If reasonably long, maybe you simply need to 
periodically take each Solr instance out of rotation and shut it down and 
restart the JVM before any of them hit the GC issue.

The first question is whether the GC delay is being caused because your JVM 
heap is too small (thrashing) or too large (long GC elapsed time.)

-- Jack Krupansky

-----Original Message----- 
From: Bill Bell
Sent: Saturday, June 30, 2012 8:48 PM
To: Bill Bell
Cc: dev <at> lucene.apache.org
Subject: Re: Low pause GC for java 1.6

Nothing?

Bill Bell
Sent from mobile

On Jun 29, 2012, at 9:09 PM, Bill Bell <billnbell <at> gmail.com> wrote:

> We are getting large Solr pauses on Java garbage collection in 1.6 Java.
>
> We have tried CMS. But we still have. 4 second wait on GC.
>
> What works well for Solr when using 16 GB of RAM?
>
> I have read lots of articles and now just looking for practical advise and 
> examples.
>
> Sent from my Mobile device
> 720-256-8076

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe <at> lucene.apache.org
For additional commands, e-mail: dev-help <at> lucene.apache.org 

[JENKINS] Lucene-Solr-4.x-Linux-Java7-64 - Build # 288 - Failure!

Build: http://jenkins.sd-datasolutions.de/job/Lucene-Solr-4.x-Linux-Java7-64/288/

All tests passed

Build Log:
[...truncated 1434 lines...]
   [junit4] 2012-07-01 06:02:26
   [junit4] Full thread dump Java HotSpot(TM) 64-Bit Server VM (23.0-b21 mixed mode):
   [junit4] 
   [junit4]
"TEST-TestScope-org.apache.lucene.index.TestDocumentsWriterStallControl.testAccquireReleaseRace-seed#[9BF985E4B18F38E1]"
prio=10 tid=0x000000000255a800 nid=0x1c5 waiting on condition [0x00007f1243578000]
   [junit4]    java.lang.Thread.State: WAITING (parking)
   [junit4] 	at sun.misc.Unsafe.park(Native Method)
   [junit4] 	- parking to wait for  <0x00000000db9581f8> (a java.util.concurrent.CountDownLatch$Sync)
   [junit4] 	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4] 	at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:834)
   [junit4] 	at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:994)
   [junit4] 	at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303)
   [junit4] 	at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:236)
   [junit4] 	at org.apache.lucene.index.TestDocumentsWriterStallControl.testAccquireReleaseRace(TestDocumentsWriterStallControl.java:147)
   [junit4] 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4] 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4] 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4] 	at java.lang.reflect.Method.invoke(Method.java:601)
   [junit4] 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1969)
   [junit4] 	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$1100(RandomizedRunner.java:132)
   [junit4] 	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:814)
   [junit4] 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:875)
   [junit4] 	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:889)
   [junit4] 	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4] 	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:32)
   [junit4] 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
   [junit4] 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
   [junit4] 	at org.apache.lucene.util.TestRuleReportUncaughtExceptions$1.evaluate(TestRuleReportUncaughtExceptions.java:68)
   [junit4] 	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
   [junit4] 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
   [junit4] 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4] 	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:821)
   [junit4] 	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$700(RandomizedRunner.java:132)
   [junit4] 	at com.carrotsearch.randomizedtesting.RandomizedRunner$3$1.run(RandomizedRunner.java:669)
   [junit4] 	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:695)
   [junit4] 	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:734)
   [junit4] 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:745)
   [junit4] 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
   [junit4] 	at org.apache.lucene.util.TestRuleReportUncaughtExceptions$1.evaluate(TestRuleReportUncaughtExceptions.java:68)
   [junit4] 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
   [junit4] 	at org.apache.lucene.util.TestRuleIcuHack$1.evaluate(TestRuleIcuHack.java:51)
   [junit4] 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
   [junit4] 	at org.apache.lucene.util.TestRuleNoInstanceHooksOverrides$1.evaluate(TestRuleNoInstanceHooksOverrides.java:53)
   [junit4] 	at org.apache.lucene.util.TestRuleNoStaticHooksShadowing$1.evaluate(TestRuleNoStaticHooksShadowing.java:52)
   [junit4] 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:36)
   [junit4] 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4] 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
   [junit4] 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4] 	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:605)
   [junit4] 	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$400(RandomizedRunner.java:132)
   [junit4] 	at com.carrotsearch.randomizedtesting.RandomizedRunner$2.run(RandomizedRunner.java:551)
   [junit4] 
   [junit4] "Service Thread" daemon prio=10 tid=0x00007f125019b800 nid=0x76f6 runnable [0x0000000000000000]
   [junit4]    java.lang.Thread.State: RUNNABLE
   [junit4] 
   [junit4] "C2 CompilerThread1" daemon prio=10 tid=0x00007f1250199000 nid=0x76f4 waiting on condition [0x0000000000000000]
   [junit4]    java.lang.Thread.State: RUNNABLE
   [junit4] 
   [junit4] "C2 CompilerThread0" daemon prio=10 tid=0x00007f1250197000 nid=0x76f3 waiting on condition [0x0000000000000000]
   [junit4]    java.lang.Thread.State: RUNNABLE
   [junit4] 
   [junit4] "Signal Dispatcher" daemon prio=10 tid=0x00007f1250194800 nid=0x76f1 waiting on condition [0x0000000000000000]
   [junit4]    java.lang.Thread.State: RUNNABLE
   [junit4] 
   [junit4] "Surrogate Locker Thread (Concurrent GC)" daemon prio=10 tid=0x00007f1250192800 nid=0x76ef
waiting on condition [0x0000000000000000]
   [junit4]    java.lang.Thread.State: RUNNABLE
   [junit4] 
   [junit4] "Finalizer" daemon prio=10 tid=0x00007f125013c000 nid=0x76ea in Object.wait() [0x00007f1244311000]
   [junit4]    java.lang.Thread.State: WAITING (on object monitor)
   [junit4] 	at java.lang.Object.wait(Native Method)
   [junit4] 	at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:135)
   [junit4] 	- locked <0x00000000daea6df0> (a java.lang.ref.ReferenceQueue$Lock)
   [junit4] 	at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:151)
   [junit4] 	at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:177)
   [junit4] 
   [junit4] "Reference Handler" daemon prio=10 tid=0x00007f125013a000 nid=0x76e9 in Object.wait() [0x00007f1244412000]
   [junit4]    java.lang.Thread.State: WAITING (on object monitor)
   [junit4] 	at java.lang.Object.wait(Native Method)
   [junit4] 	at java.lang.Object.wait(Object.java:503)
   [junit4] 	at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:133)
   [junit4] 	- locked <0x00000000daea66a0> (a java.lang.ref.Reference$Lock)
   [junit4] 
   [junit4] "main" prio=10 tid=0x0000000001f1f000 nid=0x76b8 in Object.wait() [0x00007f125ae09000]
   [junit4]    java.lang.Thread.State: WAITING (on object monitor)
   [junit4] 	at java.lang.Object.wait(Native Method)
   [junit4] 	at java.lang.Thread.join(Thread.java:1258)
   [junit4] 	- locked <0x00000000e00569e8> (a com.carrotsearch.randomizedtesting.RandomizedRunner$2)
   [junit4] 	at java.lang.Thread.join(Thread.java:1332)
   [junit4] 	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:561)
   [junit4] 	at com.carrotsearch.randomizedtesting.RandomizedRunner.run(RandomizedRunner.java:521)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.execute(SlaveMain.java:145)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.main(SlaveMain.java:238)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe.main(SlaveMainSafe.java:12)
   [junit4] 
   [junit4] "VM Thread" prio=10 tid=0x00007f1250132800 nid=0x76e7 runnable 
   [junit4] 
   [junit4] "Gang worker#0 (Parallel GC Threads)" prio=10 tid=0x0000000001f2b800 nid=0x76ba runnable 
   [junit4] 
   [junit4] "Gang worker#1 (Parallel GC Threads)" prio=10 tid=0x0000000001f2d800 nid=0x76bd runnable 
   [junit4] 
   [junit4] "Gang worker#2 (Parallel GC Threads)" prio=10 tid=0x0000000001f2f800 nid=0x76c0 runnable 
   [junit4] 
   [junit4] "Gang worker#3 (Parallel GC Threads)" prio=10 tid=0x0000000001f31800 nid=0x76c4 runnable 
   [junit4] 
   [junit4] "Gang worker#4 (Parallel GC Threads)" prio=10 tid=0x0000000001f33000 nid=0x76c5 runnable 
   [junit4] 
   [junit4] "Gang worker#5 (Parallel GC Threads)" prio=10 tid=0x0000000001f35000 nid=0x76ca runnable 
   [junit4] 
   [junit4] "Gang worker#6 (Parallel GC Threads)" prio=10 tid=0x0000000001f37000 nid=0x76cb runnable 
   [junit4] 
   [junit4] "Gang worker#7 (Parallel GC Threads)" prio=10 tid=0x0000000001f38800 nid=0x76cd runnable 
   [junit4] 
   [junit4] "G1 Main Concurrent Mark GC Thread" prio=10 tid=0x00007f1250011000 nid=0x76e0 runnable 
   [junit4] 
   [junit4] "Gang worker#0 (G1 Parallel Marking Threads)" prio=10 tid=0x00007f125001c000 nid=0x76e2
runnable 
   [junit4] 
   [junit4] "Gang worker#1 (G1 Parallel Marking Threads)" prio=10 tid=0x00007f125001e000 nid=0x76e4
runnable 
   [junit4] 
   [junit4] "G1 Concurrent Refinement Thread#0" prio=10 tid=0x00007f125000c000 nid=0x76de runnable 
   [junit4] 
   [junit4] "G1 Concurrent Refinement Thread#1" prio=10 tid=0x00007f125000a800 nid=0x76dc runnable 
   [junit4] 
   [junit4] "G1 Concurrent Refinement Thread#2" prio=10 tid=0x00007f1250008800 nid=0x76da runnable 
   [junit4] 
   [junit4] "G1 Concurrent Refinement Thread#3" prio=10 tid=0x00007f1250006800 nid=0x76d8 runnable 
   [junit4] 
   [junit4] "G1 Concurrent Refinement Thread#4" prio=10 tid=0x00007f1250004800 nid=0x76d5 runnable 
   [junit4] 
   [junit4] "G1 Concurrent Refinement Thread#5" prio=10 tid=0x00007f1250003000 nid=0x76d3 runnable 
   [junit4] 
   [junit4] "G1 Concurrent Refinement Thread#6" prio=10 tid=0x00007f1250001000 nid=0x76d1 runnable 
   [junit4] 
   [junit4] "G1 Concurrent Refinement Thread#7" prio=10 tid=0x0000000001f3d000 nid=0x76d0 runnable 
   [junit4] 
   [junit4] "G1 Concurrent Refinement Thread#8" prio=10 tid=0x0000000001f3b000 nid=0x76cf runnable 
   [junit4] 
   [junit4] 
   [junit4] "VM Periodic Task Thread" prio=10 tid=0x00007f12501a6800 nid=0x76f8 waiting on condition 
   [junit4] 
   [junit4] JNI global references: 262
   [junit4] 
   [junit4] Heap
   [junit4]  garbage-first heap   total 484352K, used 249723K [0x00000000dae00000, 0x00000000f8700000, 0x00000000fae00000)
   [junit4]   region size 1024K, 155 young (158720K), 3 survivors (3072K)
   [junit4]  compacting perm gen  total 21504K, used 20946K [0x00000000fae00000, 0x00000000fc300000, 0x0000000100000000)
   [junit4]    the space 21504K,  97% used [0x00000000fae00000, 0x00000000fc2749e8, 0x00000000fc274a00, 0x00000000fc300000)
   [junit4] No shared spaces configured.
   [junit4] <<< JVM J1: EOF ----
   [junit4] Execution time total: 5 hours 54 minutes 7 seconds
   [junit4] ERROR: Forked JVM execution exception: 0, execution line:
/mnt/ssd/jenkins/tools/java/jdk1.7.0_04/jre/bin/java -XX:+HeapDumpOnOutOfMemoryError
-XX:HeapDumpPath=/var/lib/jenkins/workspace/Lucene-Solr-4.x-Linux-Java7-64/heapdumps/
-XX:+UseG1GC -Dtests.prefix=tests -Dtests.seed=9BF985E4B18F38E1 -Xmx512M -Dtests.iters=
-Dtests.verbose=false -Dtests.infostream=false
-Dtests.lockdir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux-Java7-64/checkout/lucene/build
-Dtests.codec=random -Dtests.postingsformat=random -Dtests.locale=random
-Dtests.timezone=random -Dtests.directory=random -Dtests.linedocsfile=europarl.lines.txt.gz
-Dtests.luceneMatchVersion=4.0 -Dtests.cleanthreads=perMethod
-Djava.util.logging.config.file=/dev/null -Dtests.nightly=false -Dtests.weekly=false
-Dtests.slow=true -Dtests.asserts.gracious=false -Dtests.multiplier=3 -DtempDir=.
-Dlucene.version=4.0-SNAPSHOT -Djetty.testMode=1 -Djetty.insecurerandom=1
-Dsolr.directoryFactory=org.apache.solr.core.MockDirectoryFactory -classpath
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux-Java7-64/checkout/lucene/build/test-framework/classes/java:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux-Java7-64/checkout/lucene/test-framework/lib/junit-4.10.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux-Java7-64/checkout/lucene/test-framework/lib/randomizedtesting-runner-1.5.0.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux-Java7-64/checkout/lucene/build/core/classes/java:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux-Java7-64/checkout/lucene/build/core/classes/test:/var/lib/jenkins/tools/ant/supported18/lib/ant-launcher.jar:/var/lib/jenkins/.ant/lib/ivy-2.2.0.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-jsch.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-jai.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-testutil.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-commons-net.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-apache-log4j.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-jmf.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-antlr.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-junit.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-junit4.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-commons-logging.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-swing.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-apache-oro.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-netrexx.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-apache-resolver.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-apache-xalan2.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-apache-regexp.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-jdepend.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-apache-bcel.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-apache-bsf.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-javamail.jar:/mnt/ssd/jenkins/tools/java/jdk1.7.0_04/lib/tools.jar:/var/lib/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-1.5.0.jar
-ea:org.apache.lucene... -ea:org.apache.solr...
com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux-Java7-64/checkout/lucene/build/core/test/junit4-J0-000827383edf838c6.events  <at> /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux-Java7-64/checkout/lucene/build/core/test/junit4-J0-000827383edf838c6.suites
   [junit4] ERROR: Forked JVM execution exception: 1, execution line:
/mnt/ssd/jenkins/tools/java/jdk1.7.0_04/jre/bin/java -XX:+HeapDumpOnOutOfMemoryError
-XX:HeapDumpPath=/var/lib/jenkins/workspace/Lucene-Solr-4.x-Linux-Java7-64/heapdumps/
-XX:+UseG1GC -Dtests.prefix=tests -Dtests.seed=9BF985E4B18F38E1 -Xmx512M -Dtests.iters=
-Dtests.verbose=false -Dtests.infostream=false
-Dtests.lockdir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux-Java7-64/checkout/lucene/build
-Dtests.codec=random -Dtests.postingsformat=random -Dtests.locale=random
-Dtests.timezone=random -Dtests.directory=random -Dtests.linedocsfile=europarl.lines.txt.gz
-Dtests.luceneMatchVersion=4.0 -Dtests.cleanthreads=perMethod
-Djava.util.logging.config.file=/dev/null -Dtests.nightly=false -Dtests.weekly=false
-Dtests.slow=true -Dtests.asserts.gracious=false -Dtests.multiplier=3 -DtempDir=.
-Dlucene.version=4.0-SNAPSHOT -Djetty.testMode=1 -Djetty.insecurerandom=1
-Dsolr.directoryFactory=org.apache.solr.core.MockDirectoryFactory -classpath
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux-Java7-64/checkout/lucene/build/test-framework/classes/java:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux-Java7-64/checkout/lucene/test-framework/lib/junit-4.10.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux-Java7-64/checkout/lucene/test-framework/lib/randomizedtesting-runner-1.5.0.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux-Java7-64/checkout/lucene/build/core/classes/java:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux-Java7-64/checkout/lucene/build/core/classes/test:/var/lib/jenkins/tools/ant/supported18/lib/ant-launcher.jar:/var/lib/jenkins/.ant/lib/ivy-2.2.0.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-jsch.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-jai.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-testutil.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-commons-net.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-apache-log4j.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-jmf.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-antlr.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-junit.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-junit4.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-commons-logging.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-swing.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-apache-oro.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-netrexx.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-apache-resolver.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-apache-xalan2.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-apache-regexp.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-jdepend.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-apache-bcel.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-apache-bsf.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-javamail.jar:/mnt/ssd/jenkins/tools/java/jdk1.7.0_04/lib/tools.jar:/var/lib/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-1.5.0.jar
-ea:org.apache.lucene... -ea:org.apache.solr...
com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux-Java7-64/checkout/lucene/build/core/test/junit4-J1-000827384742956a9.events  <at> /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux-Java7-64/checkout/lucene/build/core/test/junit4-J1-000827384742956a9.suites

BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux-Java7-64/checkout/build.xml:29: The
following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux-Java7-64/checkout/lucene/build.xml:50: The
following error occurred while executing this
line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux-Java7-64/checkout/lucene/common-build.xml:1046:
The following error occurred while executing this
line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux-Java7-64/checkout/lucene/common-build.xml:759:
At least one slave process threw an unexpected exception, first: Quit event not received from a slave
process? This may indicate JVM crash or runner bugs. Inspect full output file(s):
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux-Java7-64/checkout/lucene/build/core/test/junit4-J0-000827383edf838c6.sysout,
and: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux-Java7-64/checkout/lucene/build/core/test/junit4-J0-000827383edf838c6.syserr

Total time: 355 minutes 23 seconds
Build step 'Execute shell' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe <at> lucene.apache.org
For additional commands, e-mail: dev-help <at> lucene.apache.org

Re: Low pause GC for java 1.6

It GC after about 10 minutes.

Bill Bell
Sent from mobile

On Jun 30, 2012, at 10:48 PM, "Jack Krupansky" <jack <at> basetechnology.com> wrote:

> Out of curiosity, how long does your Solr instance run before you hit a GC issue? Minutes, hours, days? If
reasonably long, maybe you simply need to periodically take each Solr instance out of rotation and shut it
down and restart the JVM before any of them hit the GC issue.
> 
> The first question is whether the GC delay is being caused because your JVM heap is too small (thrashing) or
too large (long GC elapsed time.)
> 
> -- Jack Krupansky
> 
> -----Original Message----- From: Bill Bell
> Sent: Saturday, June 30, 2012 8:48 PM
> To: Bill Bell
> Cc: dev <at> lucene.apache.org
> Subject: Re: Low pause GC for java 1.6
> 
> Nothing?
> 
> Bill Bell
> Sent from mobile
> 
> 
> On Jun 29, 2012, at 9:09 PM, Bill Bell <billnbell <at> gmail.com> wrote:
> 
>> We are getting large Solr pauses on Java garbage collection in 1.6 Java.
>> 
>> We have tried CMS. But we still have. 4 second wait on GC.
>> 
>> What works well for Solr when using 16 GB of RAM?
>> 
>> I have read lots of articles and now just looking for practical advise and examples.
>> 
>> Sent from my Mobile device
>> 720-256-8076
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe <at> lucene.apache.org
> For additional commands, e-mail: dev-help <at> lucene.apache.org 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe <at> lucene.apache.org
> For additional commands, e-mail: dev-help <at> lucene.apache.org
> 

[JENKINS] Lucene-Solr-tests-only-4.x-java7 - Build # 76 - Failure

Build: https://builds.apache.org/job/Lucene-Solr-tests-only-4.x-java7/76/

All tests passed

Build Log:
[...truncated 1548 lines...]

[...truncated 1548 lines...]

[...truncated 1548 lines...]

[...truncated 1548 lines...]

[...truncated 1534 lines...]
   [junit4] ERROR: Forked JVM execution exception: 0, execution line: /usr/local/openjdk7/jre/bin/java
-XX:+HeapDumpOnOutOfMemoryError
-XX:HeapDumpPath=/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-4.x-java7/heapdumps/
-XX:+UseG1GC -Dtests.prefix=tests -Dtests.seed=4A73CBC36312E1C8 -Xmx512M -Dtests.iters=
-Dtests.verbose=false -Dtests.infostream=false
-Dtests.lockdir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-4.x-java7/checkout/lucene/build
-Dtests.codec=random -Dtests.postingsformat=random -Dtests.locale=random
-Dtests.timezone=random -Dtests.directory=random -Dtests.linedocsfile=europarl.lines.txt.gz
-Dtests.luceneMatchVersion=4.0 -Dtests.cleanthreads=perMethod
-Djava.util.logging.config.file=/dev/null -Dtests.nightly=false -Dtests.weekly=false
-Dtests.slow=true -Dtests.asserts.gracious=false -Dtests.multiplier=3 -DtempDir=.
-Dlucene.version=4.0-SNAPSHOT -Djetty.testMode=1 -Djetty.insecurerandom=1
-Dsolr.directoryFactory=org.apache.solr.core.MockDirectoryFactory -classpath
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-4.x-java7/checkout/lucene/build/test-framework/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-4.x-java7/checkout/lucene/test-framework/lib/junit-4.10.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-4.x-java7/checkout/lucene/test-framework/lib/randomizedtesting-runner-1.5.0.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-4.x-java7/checkout/lucene/build/core/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-4.x-java7/checkout/lucene/build/core/classes/test:/home/hudson/tools/ant/supported18/lib/ant-launcher.jar:/home/hudson/.ant/lib/ivy-2.2.0.jar:/home/hudson/.ant/lib/js.jar:/home/hudson/.ant/lib/script-js.jar:/home/hudson/tools/ant/supported18/lib/ant-jsch.jar:/home/hudson/tools/ant/supported18/lib/ant.jar:/home/hudson/tools/ant/supported18/lib/ant-apache-oro.jar:/home/hudson/tools/ant/supported18/lib/ant-commons-logging.jar:/home/hudson/tools/ant/supported18/lib/ant-apache-regexp.jar:/home/hudson/tools/ant/supported18/lib/ant-jmf.jar:/home/hudson/tools/ant/supported18/lib/ant-apache-bcel.jar:/home/hudson/tools/ant/supported18/lib/ant-netrexx.jar:/home/hudson/tools/ant/supported18/lib/ant-apache-bsf.jar:/home/hudson/tools/ant/supported18/lib/ant-apache-log4j.jar:/home/hudson/tools/ant/supported18/lib/ant-apache-xalan2.jar:/home/hudson/tools/ant/supported18/lib/ant-jai.jar:/home/hudson/tools/ant/supported18/lib/ant-swing.jar:/home/hudson/tools/ant/supported18/lib/ant-jdepend.jar:/home/hudson/tools/ant/supported18/lib/ant-apache-resolver.jar:/home/hudson/tools/ant/supported18/lib/ant-testutil.jar:/home/hudson/tools/ant/supported18/lib/ant-commons-net.jar:/home/hudson/tools/ant/supported18/lib/ant-antlr.jar:/home/hudson/tools/ant/supported18/lib/ant-javamail.jar:/home/hudson/tools/ant/supported18/lib/ant-junit.jar:/home/hudson/tools/ant/supported18/lib/ant-junit4.jar:/usr/local/openjdk7/lib/tools.jar:/home/hudson/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-1.5.0.jar
-ea:org.apache.lucene... -ea:org.apache.solr...
com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-4.x-java7/checkout/lucene/build/core/test/junit4-J0-0644456099a1424f6.events  <at> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-4.x-java7/checkout/lucene/build/core/test/junit4-J0-0644456099a1424f6.suites

BUILD
FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-4.x-java7/checkout/build.xml:29:
The following error occurred while executing this
line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-4.x-java7/checkout/lucene/build.xml:50:
The following error occurred while executing this
line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-4.x-java7/checkout/lucene/common-build.xml:1046:
The following error occurred while executing this
line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-4.x-java7/checkout/lucene/common-build.xml:759:
At least one slave process threw an unexpected exception, first: Forked process exited with an error
code: 143

Total time: 1697 minutes 7 seconds
Build step 'Execute shell' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure

[...truncated 1548 lines...]

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe <at> lucene.apache.org
For additional commands, e-mail: dev-help <at> lucene.apache.org

RE: [JENKINS] Lucene-Solr-4.x-Linux-Java7-64 - Build # 288 - Failure!

Hangs again in Stall control (this time linux, load of server was horrible: 24 !!!). Something completely
strange, had to reboot (kernel announced in dmesg that some cpu cores were completely overloaded and no
longer responding).

-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: uwe <at> thetaphi.de

> -----Original Message-----
> From: Policeman Jenkins Server [mailto:jenkins <at> sd-datasolutions.de]
> Sent: Sunday, July 01, 2012 8:03 AM
> To: dev <at> lucene.apache.org
> Subject: [JENKINS] Lucene-Solr-4.x-Linux-Java7-64 - Build # 288 - Failure!
> 
> Build: http://jenkins.sd-datasolutions.de/job/Lucene-Solr-4.x-Linux-Java7-
> 64/288/
> 
> All tests passed
> 
> Build Log:
> [...truncated 1434 lines...]
>    [junit4] 2012-07-01 06:02:26
>    [junit4] Full thread dump Java HotSpot(TM) 64-Bit Server VM (23.0-b21 mixed
> mode):
>    [junit4]
>    [junit4] "TEST-TestScope-
> org.apache.lucene.index.TestDocumentsWriterStallControl.testAccquireRelease
> Race-seed#[9BF985E4B18F38E1]" prio=10 tid=0x000000000255a800 nid=0x1c5
> waiting on condition [0x00007f1243578000]
>    [junit4]    java.lang.Thread.State: WAITING (parking)
>    [junit4] 	at sun.misc.Unsafe.park(Native Method)
>    [junit4] 	- parking to wait for  <0x00000000db9581f8> (a
> java.util.concurrent.CountDownLatch$Sync)
>    [junit4] 	at
> java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
>    [junit4] 	at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(
> AbstractQueuedSynchronizer.java:834)
>    [junit4] 	at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterru
> ptibly(AbstractQueuedSynchronizer.java:994)
>    [junit4] 	at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterrupti
> bly(AbstractQueuedSynchronizer.java:1303)
>    [junit4] 	at
> java.util.concurrent.CountDownLatch.await(CountDownLatch.java:236)
>    [junit4] 	at
> org.apache.lucene.index.TestDocumentsWriterStallControl.testAccquireRelease
> Race(TestDocumentsWriterStallControl.java:147)
>    [junit4] 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
>    [junit4] 	at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:
> 57)
>    [junit4] 	at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorI
> mpl.java:43)
>    [junit4] 	at java.lang.reflect.Method.invoke(Method.java:601)
>    [junit4] 	at
> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRu
> nner.java:1969)
>    [junit4] 	at
> com.carrotsearch.randomizedtesting.RandomizedRunner.access$1100(Random
> izedRunner.java:132)
>    [junit4] 	at
> com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(Randomiz
> edRunner.java:814)
>    [junit4] 	at
> com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(Randomiz
> edRunner.java:875)
>    [junit4] 	at
> com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(Randomiz
> edRunner.java:889)
>    [junit4] 	at
> org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSet
> upTeardownChained.java:50)
>    [junit4] 	at
> org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCach
> eSanity.java:32)
>    [junit4] 	at
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfte
> rRule.java:45)
>    [junit4] 	at
> com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.ev
> aluate(SystemPropertiesInvariantRule.java:55)
>    [junit4] 	at
> org.apache.lucene.util.TestRuleReportUncaughtExceptions$1.evaluate(TestRule
> ReportUncaughtExceptions.java:68)
>    [junit4] 	at
> org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThrea
> dAndTestName.java:48)
>    [junit4] 	at
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgn
> oreAfterMaxFailures.java:70)
>    [junit4] 	at
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.ja
> va:48)
>    [junit4] 	at
> com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(Rando
> mizedRunner.java:821)
>    [junit4] 	at
> com.carrotsearch.randomizedtesting.RandomizedRunner.access$700(Randomiz
> edRunner.java:132)
>    [junit4] 	at
> com.carrotsearch.randomizedtesting.RandomizedRunner$3$1.run(Randomized
> Runner.java:669)
>    [junit4] 	at
> com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(Randomiz
> edRunner.java:695)
>    [junit4] 	at
> com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(Randomiz
> edRunner.java:734)
>    [junit4] 	at
> com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(Randomiz
> edRunner.java:745)
>    [junit4] 	at
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfte
> rRule.java:45)
>    [junit4] 	at
> org.apache.lucene.util.TestRuleReportUncaughtExceptions$1.evaluate(TestRule
> ReportUncaughtExceptions.java:68)
>    [junit4] 	at
> org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClass
> Name.java:38)
>    [junit4] 	at
> org.apache.lucene.util.TestRuleIcuHack$1.evaluate(TestRuleIcuHack.java:51)
>    [junit4] 	at
> com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.ev
> aluate(SystemPropertiesInvariantRule.java:55)
>    [junit4] 	at
> org.apache.lucene.util.TestRuleNoInstanceHooksOverrides$1.evaluate(TestRule
> NoInstanceHooksOverrides.java:53)
>    [junit4] 	at
> org.apache.lucene.util.TestRuleNoStaticHooksShadowing$1.evaluate(TestRuleN
> oStaticHooksShadowing.java:52)
>    [junit4] 	at
> org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertio
> nsRequired.java:36)
>    [junit4] 	at
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.ja
> va:48)
>    [junit4] 	at
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgn
> oreAfterMaxFailures.java:70)
>    [junit4] 	at
> org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTest
> Suites.java:55)
>    [junit4] 	at
> com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(Randomized
> Runner.java:605)
>    [junit4] 	at
> com.carrotsearch.randomizedtesting.RandomizedRunner.access$400(Randomiz
> edRunner.java:132)
>    [junit4] 	at
> com.carrotsearch.randomizedtesting.RandomizedRunner$2.run(RandomizedRu
> nner.java:551)
>    [junit4]
>    [junit4] "Service Thread" daemon prio=10 tid=0x00007f125019b800
> nid=0x76f6 runnable [0x0000000000000000]
>    [junit4]    java.lang.Thread.State: RUNNABLE
>    [junit4]
>    [junit4] "C2 CompilerThread1" daemon prio=10 tid=0x00007f1250199000
> nid=0x76f4 waiting on condition [0x0000000000000000]
>    [junit4]    java.lang.Thread.State: RUNNABLE
>    [junit4]
>    [junit4] "C2 CompilerThread0" daemon prio=10 tid=0x00007f1250197000
> nid=0x76f3 waiting on condition [0x0000000000000000]
>    [junit4]    java.lang.Thread.State: RUNNABLE
>    [junit4]
>    [junit4] "Signal Dispatcher" daemon prio=10 tid=0x00007f1250194800
> nid=0x76f1 waiting on condition [0x0000000000000000]
>    [junit4]    java.lang.Thread.State: RUNNABLE
>    [junit4]
>    [junit4] "Surrogate Locker Thread (Concurrent GC)" daemon prio=10
> tid=0x00007f1250192800 nid=0x76ef waiting on condition
> [0x0000000000000000]
>    [junit4]    java.lang.Thread.State: RUNNABLE
>    [junit4]
>    [junit4] "Finalizer" daemon prio=10 tid=0x00007f125013c000 nid=0x76ea in
> Object.wait() [0x00007f1244311000]
>    [junit4]    java.lang.Thread.State: WAITING (on object monitor)
>    [junit4] 	at java.lang.Object.wait(Native Method)
>    [junit4] 	at
> java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:135)
>    [junit4] 	- locked <0x00000000daea6df0> (a
> java.lang.ref.ReferenceQueue$Lock)
>    [junit4] 	at
> java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:151)
>    [junit4] 	at
> java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:177)
>    [junit4]
>    [junit4] "Reference Handler" daemon prio=10 tid=0x00007f125013a000
> nid=0x76e9 in Object.wait() [0x00007f1244412000]
>    [junit4]    java.lang.Thread.State: WAITING (on object monitor)
>    [junit4] 	at java.lang.Object.wait(Native Method)
>    [junit4] 	at java.lang.Object.wait(Object.java:503)
>    [junit4] 	at
> java.lang.ref.Reference$ReferenceHandler.run(Reference.java:133)
>    [junit4] 	- locked <0x00000000daea66a0> (a
> java.lang.ref.Reference$Lock)
>    [junit4]
>    [junit4] "main" prio=10 tid=0x0000000001f1f000 nid=0x76b8 in Object.wait()
> [0x00007f125ae09000]
>    [junit4]    java.lang.Thread.State: WAITING (on object monitor)
>    [junit4] 	at java.lang.Object.wait(Native Method)
>    [junit4] 	at java.lang.Thread.join(Thread.java:1258)
>    [junit4] 	- locked <0x00000000e00569e8> (a
> com.carrotsearch.randomizedtesting.RandomizedRunner$2)
>    [junit4] 	at java.lang.Thread.join(Thread.java:1332)
>    [junit4] 	at
> com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(Randomized
> Runner.java:561)
>    [junit4] 	at
> com.carrotsearch.randomizedtesting.RandomizedRunner.run(RandomizedRunn
> er.java:521)
>    [junit4] 	at
> com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.execute(SlaveMain.java:145)
>    [junit4] 	at
> com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.main(SlaveMain.java:238)
>    [junit4] 	at
> com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe.main(SlaveMainSafe.jav
> a:12)
>    [junit4]
>    [junit4] "VM Thread" prio=10 tid=0x00007f1250132800 nid=0x76e7 runnable
>    [junit4]
>    [junit4] "Gang worker#0 (Parallel GC Threads)" prio=10
> tid=0x0000000001f2b800 nid=0x76ba runnable
>    [junit4]
>    [junit4] "Gang worker#1 (Parallel GC Threads)" prio=10
> tid=0x0000000001f2d800 nid=0x76bd runnable
>    [junit4]
>    [junit4] "Gang worker#2 (Parallel GC Threads)" prio=10
> tid=0x0000000001f2f800 nid=0x76c0 runnable
>    [junit4]
>    [junit4] "Gang worker#3 (Parallel GC Threads)" prio=10
> tid=0x0000000001f31800 nid=0x76c4 runnable
>    [junit4]
>    [junit4] "Gang worker#4 (Parallel GC Threads)" prio=10
> tid=0x0000000001f33000 nid=0x76c5 runnable
>    [junit4]
>    [junit4] "Gang worker#5 (Parallel GC Threads)" prio=10
> tid=0x0000000001f35000 nid=0x76ca runnable
>    [junit4]
>    [junit4] "Gang worker#6 (Parallel GC Threads)" prio=10
> tid=0x0000000001f37000 nid=0x76cb runnable
>    [junit4]
>    [junit4] "Gang worker#7 (Parallel GC Threads)" prio=10
> tid=0x0000000001f38800 nid=0x76cd runnable
>    [junit4]
>    [junit4] "G1 Main Concurrent Mark GC Thread" prio=10
> tid=0x00007f1250011000 nid=0x76e0 runnable
>    [junit4]
>    [junit4] "Gang worker#0 (G1 Parallel Marking Threads)" prio=10
> tid=0x00007f125001c000 nid=0x76e2 runnable
>    [junit4]
>    [junit4] "Gang worker#1 (G1 Parallel Marking Threads)" prio=10
> tid=0x00007f125001e000 nid=0x76e4 runnable
>    [junit4]
>    [junit4] "G1 Concurrent Refinement Thread#0" prio=10
> tid=0x00007f125000c000 nid=0x76de runnable
>    [junit4]
>    [junit4] "G1 Concurrent Refinement Thread#1" prio=10
> tid=0x00007f125000a800 nid=0x76dc runnable
>    [junit4]
>    [junit4] "G1 Concurrent Refinement Thread#2" prio=10
> tid=0x00007f1250008800 nid=0x76da runnable
>    [junit4]
>    [junit4] "G1 Concurrent Refinement Thread#3" prio=10
> tid=0x00007f1250006800 nid=0x76d8 runnable
>    [junit4]
>    [junit4] "G1 Concurrent Refinement Thread#4" prio=10
> tid=0x00007f1250004800 nid=0x76d5 runnable
>    [junit4]
>    [junit4] "G1 Concurrent Refinement Thread#5" prio=10
> tid=0x00007f1250003000 nid=0x76d3 runnable
>    [junit4]
>    [junit4] "G1 Concurrent Refinement Thread#6" prio=10
> tid=0x00007f1250001000 nid=0x76d1 runnable
>    [junit4]
>    [junit4] "G1 Concurrent Refinement Thread#7" prio=10
> tid=0x0000000001f3d000 nid=0x76d0 runnable
>    [junit4]
>    [junit4] "G1 Concurrent Refinement Thread#8" prio=10
> tid=0x0000000001f3b000 nid=0x76cf runnable
>    [junit4]
>    [junit4]
>    [junit4] "VM Periodic Task Thread" prio=10 tid=0x00007f12501a6800
> nid=0x76f8 waiting on condition
>    [junit4]
>    [junit4] JNI global references: 262
>    [junit4]
>    [junit4] Heap
>    [junit4]  garbage-first heap   total 484352K, used 249723K
> [0x00000000dae00000, 0x00000000f8700000, 0x00000000fae00000)
>    [junit4]   region size 1024K, 155 young (158720K), 3 survivors (3072K)
>    [junit4]  compacting perm gen  total 21504K, used 20946K
> [0x00000000fae00000, 0x00000000fc300000, 0x0000000100000000)
>    [junit4]    the space 21504K,  97% used [0x00000000fae00000,
> 0x00000000fc2749e8, 0x00000000fc274a00, 0x00000000fc300000)
>    [junit4] No shared spaces configured.
>    [junit4] <<< JVM J1: EOF ----
>    [junit4] Execution time total: 5 hours 54 minutes 7 seconds
>    [junit4] ERROR: Forked JVM execution exception: 0, execution line:
> /mnt/ssd/jenkins/tools/java/jdk1.7.0_04/jre/bin/java -
> XX:+HeapDumpOnOutOfMemoryError -
> XX:HeapDumpPath=/var/lib/jenkins/workspace/Lucene-Solr-4.x-Linux-Java7-
> 64/heapdumps/ -XX:+UseG1GC -Dtests.prefix=tests -
> Dtests.seed=9BF985E4B18F38E1 -Xmx512M -Dtests.iters= -Dtests.verbose=false
> -Dtests.infostream=false -Dtests.lockdir=/mnt/ssd/jenkins/workspace/Lucene-
> Solr-4.x-Linux-Java7-64/checkout/lucene/build -Dtests.codec=random -
> Dtests.postingsformat=random -Dtests.locale=random -
> Dtests.timezone=random -Dtests.directory=random -
> Dtests.linedocsfile=europarl.lines.txt.gz -Dtests.luceneMatchVersion=4.0 -
> Dtests.cleanthreads=perMethod -Djava.util.logging.config.file=/dev/null -
> Dtests.nightly=false -Dtests.weekly=false -Dtests.slow=true -
> Dtests.asserts.gracious=false -Dtests.multiplier=3 -DtempDir=. -
> Dlucene.version=4.0-SNAPSHOT -Djetty.testMode=1 -Djetty.insecurerandom=1
> -Dsolr.directoryFactory=org.apache.solr.core.MockDirectoryFactory -classpath
> /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux-Java7-
> 64/checkout/lucene/build/test-
> framework/classes/java:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux-
> Java7-64/checkout/lucene/test-framework/lib/junit-
> 4.10.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux-Java7-
> 64/checkout/lucene/test-framework/lib/randomizedtesting-runner-
> 1.5.0.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux-Java7-
> 64/checkout/lucene/build/core/classes/java:/mnt/ssd/jenkins/workspace/Lucen
> e-Solr-4.x-Linux-Java7-
> 64/checkout/lucene/build/core/classes/test:/var/lib/jenkins/tools/ant/supporte
> d18/lib/ant-launcher.jar:/var/lib/jenkins/.ant/lib/ivy-
> 2.2.0.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-
> jsch.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-
> jai.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-
> testutil.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-commons-
> net.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-apache-
> log4j.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-
> jmf.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-
> antlr.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant.jar:/var/lib/jenkins/tool
> s/ant/supported18/lib/ant-
> junit.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-
> junit4.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-commons-
> logging.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-
> swing.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-apache-
> oro.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-
> netrexx.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-apache-
> resolver.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-apache-
> xalan2.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-apache-
> regexp.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-
> jdepend.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-apache-
> bcel.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-apache-
> bsf.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-
> javamail.jar:/mnt/ssd/jenkins/tools/java/jdk1.7.0_04/lib/tools.jar:/var/lib/jenki
> ns/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-
> 1.5.0.jar -ea:org.apache.lucene... -ea:org.apache.solr...
> com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile
> /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux-Java7-
> 64/checkout/lucene/build/core/test/junit4-J0-000827383edf838c6.events
>  <at> /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux-Java7-
> 64/checkout/lucene/build/core/test/junit4-J0-000827383edf838c6.suites
>    [junit4] ERROR: Forked JVM execution exception: 1, execution line:
> /mnt/ssd/jenkins/tools/java/jdk1.7.0_04/jre/bin/java -
> XX:+HeapDumpOnOutOfMemoryError -
> XX:HeapDumpPath=/var/lib/jenkins/workspace/Lucene-Solr-4.x-Linux-Java7-
> 64/heapdumps/ -XX:+UseG1GC -Dtests.prefix=tests -
> Dtests.seed=9BF985E4B18F38E1 -Xmx512M -Dtests.iters= -Dtests.verbose=false
> -Dtests.infostream=false -Dtests.lockdir=/mnt/ssd/jenkins/workspace/Lucene-
> Solr-4.x-Linux-Java7-64/checkout/lucene/build -Dtests.codec=random -
> Dtests.postingsformat=random -Dtests.locale=random -
> Dtests.timezone=random -Dtests.directory=random -
> Dtests.linedocsfile=europarl.lines.txt.gz -Dtests.luceneMatchVersion=4.0 -
> Dtests.cleanthreads=perMethod -Djava.util.logging.config.file=/dev/null -
> Dtests.nightly=false -Dtests.weekly=false -Dtests.slow=true -
> Dtests.asserts.gracious=false -Dtests.multiplier=3 -DtempDir=. -
> Dlucene.version=4.0-SNAPSHOT -Djetty.testMode=1 -Djetty.insecurerandom=1
> -Dsolr.directoryFactory=org.apache.solr.core.MockDirectoryFactory -classpath
> /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux-Java7-
> 64/checkout/lucene/build/test-
> framework/classes/java:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux-
> Java7-64/checkout/lucene/test-framework/lib/junit-
> 4.10.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux-Java7-
> 64/checkout/lucene/test-framework/lib/randomizedtesting-runner-
> 1.5.0.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux-Java7-
> 64/checkout/lucene/build/core/classes/java:/mnt/ssd/jenkins/workspace/Lucen
> e-Solr-4.x-Linux-Java7-
> 64/checkout/lucene/build/core/classes/test:/var/lib/jenkins/tools/ant/supporte
> d18/lib/ant-launcher.jar:/var/lib/jenkins/.ant/lib/ivy-
> 2.2.0.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-
> jsch.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-
> jai.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-
> testutil.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-commons-
> net.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-apache-
> log4j.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-
> jmf.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-
> antlr.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant.jar:/var/lib/jenkins/tool
> s/ant/supported18/lib/ant-
> junit.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-
> junit4.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-commons-
> logging.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-
> swing.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-apache-
> oro.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-
> netrexx.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-apache-
> resolver.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-apache-
> xalan2.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-apache-
> regexp.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-
> jdepend.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-apache-
> bcel.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-apache-
> bsf.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-
> javamail.jar:/mnt/ssd/jenkins/tools/java/jdk1.7.0_04/lib/tools.jar:/var/lib/jenki
> ns/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-
> 1.5.0.jar -ea:org.apache.lucene... -ea:org.apache.solr...
> com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile
> /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux-Java7-
> 64/checkout/lucene/build/core/test/junit4-J1-000827384742956a9.events
>  <at> /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux-Java7-
> 64/checkout/lucene/build/core/test/junit4-J1-000827384742956a9.suites
> 
> BUILD FAILED
> /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux-Java7-
> 64/checkout/build.xml:29: The following error occurred while executing this
> line:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux-Java7-
> 64/checkout/lucene/build.xml:50: The following error occurred while executing
> this line:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux-Java7-
> 64/checkout/lucene/common-build.xml:1046: The following error occurred
> while executing this line:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux-Java7-
> 64/checkout/lucene/common-build.xml:759: At least one slave process threw
> an unexpected exception, first: Quit event not received from a slave process?
> This may indicate JVM crash or runner bugs. Inspect full output file(s):
> /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux-Java7-
> 64/checkout/lucene/build/core/test/junit4-J0-000827383edf838c6.sysout, and:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux-Java7-
> 64/checkout/lucene/build/core/test/junit4-J0-000827383edf838c6.syserr
> 
> Total time: 355 minutes 23 seconds
> Build step 'Execute shell' marked build as failure Archiving artifacts Recording
> test results Email was triggered for: Failure Sending email for trigger: Failure
> 
>