ibrahim | 25 Dec 16:07 2014
Picon

[jgroups-users] Does JGroups block the current members while is generating new view?

Hi folks,

I just want to know that if there is a member is joining or leaving, in this
case, new view will be created. So does JGroups stop making progress or not
while is generating new view? 

Also I think the protocol GMS is responsible for managing the view, am I
right?

I look forward to hearing from you

Thank you indeed

Ibrahim

--
View this message in context: http://jgroups.1086181.n5.nabble.com/Does-JGroups-block-the-current-members-while-is-generating-new-view-tp10451.html
Sent from the JGroups - General mailing list archive at Nabble.com.

------------------------------------------------------------------------------
Dive into the World of Parallel Programming! The Go Parallel Website,
sponsored by Intel and developed in partnership with Slashdot Media, is your
hub for all things parallel software development, from weekly thought
leadership blogs to news, videos, case studies, tutorials and more. Take a
look and join the conversation now. http://goparallel.sourceforge.net
Ron Gonzalez | 17 Dec 19:03 2014
Picon

[jgroups-users] Meaning of extra character in view Address

We're trying to figure out what the extra " |0 " in our node address [192.168.2.5:7600|0] means?
is this an ipv6 conflict or does it arise from the fact that we're running two nodes on the same ipv4 address separated only by port?  We're using tcp and binding to address:7600 for the first node and for the second node address:7700. 

Why does our node report itself twice in the view with the same port, one of them showing a "|0" at the end of the address?

org.jgroups.protocols.pbcast.NAKACK 192.168.2.5:7600] discarded message from non-member 192.168.2.10:7600, my view is [192.168.2.5:7600|0] [192.168.2.5:7600]




------------------------------------------------------------------------------
Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
from Actuate! Instantly Supercharge Your Business Reports and Dashboards
with Interactivity, Sharing, Native Excel Exports, App Integration & more
Get technology previously reserved for billion-dollar corporations, FREE
http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk
_______________________________________________
javagroups-users mailing list
javagroups-users <at> lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/javagroups-users
Bela Ban | 17 Dec 13:21 2014
Picon

[jgroups-users] RAFT consensus protocol in JGroups

FYI,

I've been working on an implementation of the RAFT consensus algorithm 
[1] in JGroups.

In a nutshell, RAFT provides leader election via consensus (simple 
majority) and there's an associated state machine which only commits 
values (to a persistent log) with consensus.

When we for example have 5 nodes, a value can only be committed (= added 
to the state machine) if we have acks from 3 nodes. Late joiners are 
informed of the committed values when they come up.

Looking at CAP, RAFT gives up availability in favor of consistency; when 
no majority of nodes is running, changes can only be appended to the 
log, but are not committed (meaning they're not seen in the state 
machines). This is kind of the opposite of eventual consistency which 
trades consistency for availability.

Compared to transactions, consensus requires acks to 'prepares' only 
from a majority of nodes, so a slow node won't delay or prevent the 
committing of a unit of work.

Also, partition handling is simple as only the majority partition is 
allowed to make progress.

The advantages of implementing RAFT in JGroups are:
* Transports are already available: UDP, TCP
** Contains thread pools, priority delivery (OOB), batching etc
* Variety of discovery protocols
* Encryption, authentication, compression
* Fragmentation, reliability over UDP
* Multicasting for larger clusters
* Failure detection
* Sync/async cluster RPC
* Less code has to be written to implement RAFT in JGroups compared to a 
standalone implementation. For more details see [4].

I've completed an initial prototype [2] which is alpha quality at best:
* Leader election and state machines have been implemented
* Demo ReplicatedStateMachineDemo to show functionality
* Almost no unit tests yet. I intend to write tests for all use cases 
and scenarios described in the paper
* Persistent log based on LevelDB (written by Ugo Landini)

On the todo list are:
* Dynamic cluster changes (currently we have a static majority)
* Snapshotting of logs
* Lots of unit tests

To run a demo, run
* bin/demo.sh -name B -follower
* bin/demo.sh -name C -follower
* bin/demo.sh -name  A

This will create 3 logs in the temp dir: A, B and C (if run on the same 
box). The demo can then be used to add keys/values to the state machine, 
dumps the contents of the state machine and dump the logs. Again, no 
snapshotting has been implemented yet, so the logs will only grow.

For questions, please use the jgroups-raft mailing list at [3].
Cheers,

[1] https://raftconsensus.github.io/
[2] https://github.com/belaban/jgroups-raft
[3] https://groups.google.com/forum/#!forum/jgroups-raft
[4] file:///Users/bela/jgroups-raft/doc/readme.adoc

--

-- 
Bela Ban, JGroups lead (http://www.jgroups.org)

------------------------------------------------------------------------------
Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
from Actuate! Instantly Supercharge Your Business Reports and Dashboards
with Interactivity, Sharing, Native Excel Exports, App Integration & more
Get technology previously reserved for billion-dollar corporations, FREE
http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk
Gennady Shumakher | 9 Dec 19:43 2014
Picon

[jgroups-users] JDBC_PING clean up

Hello,
I'm experimenting with JDBC_PING and see that in case of node's unexpected failure its record stays in the table and slows down others during startup. Is there a way to clean up invalid records?

Thanks,
Gennady  
------------------------------------------------------------------------------
Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
from Actuate! Instantly Supercharge Your Business Reports and Dashboards
with Interactivity, Sharing, Native Excel Exports, App Integration & more
Get technology previously reserved for billion-dollar corporations, FREE
http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk
_______________________________________________
javagroups-users mailing list
javagroups-users <at> lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/javagroups-users
Cornelius, Michael | 9 Dec 15:36 2014

[jgroups-users] Early release of jgroups 3.5.2

Hi,

 

is it possible to release version 3.5.2 very soon?

We have a running production cluster with version 3.5.1 and https://issues.jboss.org/browse/JGRP-1899 is a major problem for us.

 

Regards

Frank

 

Weltbild Retail GmbH & Co. KG

Steinerne Furt 68-72, 86167 Augsburg | Amtsgericht Augsburg HRA 18273 | UST-IdNr.: DE 815 516 977

Persönlich haftende Gesellschafterin: Weltbild Retail Verwaltungs GmbH | Sitz Augsburg, Amtsgericht Augsburg HRB 28784

Geschäftsführer: Patrick Hofmann,  Sikko Böhm
------------------------------------------------------------------------------
Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
from Actuate! Instantly Supercharge Your Business Reports and Dashboards
with Interactivity, Sharing, Native Excel Exports, App Integration & more
Get technology previously reserved for billion-dollar corporations, FREE
http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk
_______________________________________________
javagroups-users mailing list
javagroups-users <at> lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/javagroups-users
POOJA VADGAMA | 3 Dec 17:58 2014
Picon

[jgroups-users] multiple coordinators found and views stuck.

Hello, I am using the jgroups version 3.3.5.Final in an application running inside web logic. 
I have a cluster of 60 nodes all using TCP as in below xml. 


<?xml version="1.0" encoding="UTF-8"?> 


<config
                xmlns="urn:org:jgroups" 
                xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"  
                xsi:schemaLocation="urn:org:jgroups http://www.jgroups.org/schema/JGroups-3.0.xsd">
          
  <TCP
  bind_port="3333" 
            loopback="true" 
                    recv_buf_size="20000000" 
                                        send_buf_size="640000" 
                                        discard_incompatible_packets="true" 
                                        max_bundle_size="64000" 
                                        max_bundle_timeout="30" 
                                        enable_bundling="true" 
                                        use_send_queues="true" 
                                        sock_conn_timeout="3000" 
                                        peer_addr_read_timeout="3000" 
                                        thread_pool.enabled="true" 
                                        thread_pool.min_threads="10" 
                                        thread_pool.max_threads="100" 
                                        thread_pool.keep_alive_time="5000" 
                                        thread_pool.queue_enabled="true" 
                                        thread_pool.queue_max_size="1000" 
                                        thread_pool.rejection_policy="Run" 
                                        oob_thread_pool.enabled="true" 
                                        oob_thread_pool.min_threads="10" 
                                        oob_thread_pool.max_threads="100" 
                                        oob_thread_pool.keep_alive_time="5000" 
                                        oob_thread_pool.queue_enabled="true" 
                                        oob_thread_pool.queue_max_size="1000" 
                                        oob_thread_pool.rejection_policy="Run" 
                                        timer.rejection_policy="Discard" 
                    timer.max_threads="500"/>
                                
        <TCPPING
                                        timeout="200000" 
                                        initial_hosts="" 
                                        port_range="0" 
                                        num_initial_members="1" 
                                        />
                        
        <MERGE2 
                                        max_interval="100000" 
                                        min_interval="20000" />
                        
        <FD_SOCK 
                                        start_port="1234"
                                        get_cache_timeout="3000" 
                                        sock_conn_timeout="3000" 
         />
        
        <FD 
                                        timeout="60000" 
                                        max_tries="10" />
                        
        <VERIFY_SUSPECT
                                        timeout="3000" />
                        
        <BARRIER />
                        
        <pbcast.NAKACK
                                        use_mcast_xmit="false" 
                                        retransmit_timeout="600,1200,2400,4800" 
                                        discard_delivered_msgs="true" />
                        
        <UNICAST 
                                        timeout="300,600,1200" />
                        
        <pbcast.STABLE
                                        stability_delay="1000" 
                                        desired_avg_gossip="50000" 
                                        max_bytes="400000" />
                                                        
        <pbcast.GMS 
                                        print_local_addr="true" 
                                        join_timeout="60000" 
                                        view_bundling="true" />

        <FRAG2
                                        frag_size="60000" />
                        
        <pbcast.STATE_TRANSFER/>
</config>




I normally don't see a problem when this cluster is coming up.
But if the cluster is in a good state and I reboot nodes randomly, I see multiple co-ordinators at the end of this exercise. 

Below is one instance, Node1 was co-ordinator and node2 was second member. 

At 23:35:29, node1 published view with id=99 and complains that it is missing ACK from node 9 within 2 seconds. But all nodes received it.Also node 9 still gets subsequent view with id 100 from node 1. So i think (missing ACK) is not really a problem 

At 23:41:16, node 1 publishes view with id=100. All nodes get it. At 23:41:45, other nodes get a view with id=101 and co-ordinator is changed to node2. while node1, gets a merge view after 5 minutes. 

"23:46:53.367-05:00 [user] MessageReceiver viewAccepted() callback called[node1|101]; Co-ordinator is node1; Members are [node1]; View type is MergeView"

Since this point, clusters got split, some got stuck in old view forever. When I reset few nodes, they sent request to node1 and joined successfully which was supposed to be the real co-ordinator. UNfortunately, I dont have jgroups logs in debug when this happened. I am trying to understand what caused node2 to take over within 30 seconds ? ( there is no FD or FD_SOCK kicking in here.. what else then?> Also, I am not able to understand under what circumstances MergeView shows only itself? FYI, I have another channel running with exact same configuration but different ports outside of Weblogic and it went through this storm of reboots too but its co-ordinator did not flip and remained steady.

After some time I  see logs in node1 saying, "sender window for node 2 not found".

------------------------------------------------------------------------------
Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
from Actuate! Instantly Supercharge Your Business Reports and Dashboards
with Interactivity, Sharing, Native Excel Exports, App Integration & more
Get technology previously reserved for billion-dollar corporations, FREE
http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk
_______________________________________________
javagroups-users mailing list
javagroups-users <at> lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/javagroups-users
Bela Ban | 2 Dec 15:23 2014
Picon

[jgroups-users] JGroups 3.6.1 released

FYI,

this is a mini release with only 4 issues [1]:

* NAKACK2: added the capability (off by default) to retransmit the last 
message missing more quickly: [https://issues.jboss.org/browse/JGRP-1904]

* https://issues.jboss.org/browse/JGRP-1903: TCPPING may now only 
contain a few addresses in initial_hosts, and the coordinator will 
disseminate them to all members

* https://issues.jboss.org/browse/JGRP-1899: discovery could sometimes 
return prematurely with a max of 16 members (only in 3.5.x and 3.6.x, 
not in previous versions)

* https://issues.jboss.org/browse/JGRP-1898: reduction of false 
suspicions with FD_HOST

[1] https://issues.jboss.org/browse/JGRP/fixforversion/12325982

--

-- 
Bela Ban, JGroups lead (http://www.jgroups.org)

------------------------------------------------------------------------------
Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
from Actuate! Instantly Supercharge Your Business Reports and Dashboards
with Interactivity, Sharing, Native Excel Exports, App Integration & more
Get technology previously reserved for billion-dollar corporations, FREE
http://pubads.g.doubleclick.net/gampad/clk?id=157005751&iu=/4140/ostg.clktrk
ibrahim | 2 Dec 11:22 2014
Picon

[jgroups-users] SEQUENCE protocol, who generates the sequence number (unique no)?


I read the SEQUENCE protocol implementation, I came to the line where the
seqno is generated.  
public Object down(Event evt) {
--
--
long next_seqno=seqno.incrementAndGet();
in_flight_sends.incrementAndGet();
try {
SequencerHeader hdr=new SequencerHeader(is_coord? SequencerHeader.BCAST :
SequencerHeader.WRAPPED_BCAST, next_seqno);
msg.putHeader(this.id, hdr);
if(log.isTraceEnabled())
log.trace("[" + local_addr + "]: forwarding " + local_addr + "::" + seqno +
" to coord " + coord);                    
forwardToCoord(next_seqno, msg);
--
--
}
It seems that any members in cluster can generate sequence number, it is not
just coordinator. If so, we may come to scenario when two members generate
same sequence number. For example, cluster [A,B,C], B and C receive two
messages at same time, B receives MB1 and MB2, C receives  MC1 and MC2. This
may lead to B generates sequence number 55 and 56 for MB1 and MB2
respectively. C generates sequence number 55 and 56 for MC1 and MC2. In this
case we come up with two members having same sequence number, which should
not be happened as SEQUENCE must achieve TOTAL ORDERING among whole members.  

Ibrahim

--
View this message in context: http://jgroups.1086181.n5.nabble.com/SEQUENCE-protocol-who-generates-the-sequence-number-unique-no-tp10430.html
Sent from the JGroups - General mailing list archive at Nabble.com.

------------------------------------------------------------------------------
Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
from Actuate! Instantly Supercharge Your Business Reports and Dashboards
with Interactivity, Sharing, Native Excel Exports, App Integration & more
Get technology previously reserved for billion-dollar corporations, FREE
http://pubads.g.doubleclick.net/gampad/clk?id=157005751&iu=/4140/ostg.clktrk
ibrahim | 1 Dec 23:18 2014
Picon

[jgroups-users] SEQUENCE protocol, who generate the sequence number (unique no)?


I read the SEQUENCE protocol implementation, I came to the line where the
seqno is generated.  

public Object down(Event evt) {
--
--
long next_seqno=seqno.incrementAndGet(); //Here ******
in_flight_sends.incrementAndGet();
try {
SequencerHeader hdr=new SequencerHeader(is_coord? SequencerHeader.BCAST :
SequencerHeader.WRAPPED_BCAST, next_seqno);
msg.putHeader(this.id, hdr);
if(log.isTraceEnabled())
log.trace("[" + local_addr + "]: forwarding " + local_addr + "::" + seqno +
" to coord " + coord);                    
forwardToCoord(next_seqno, msg);
--
--
}
It seems that any members in cluster can generate sequence number, it is not
just coordinator. If so, we may come to scenario when two members generate
same sequence number. For example, cluster [A,B,C], B and C receive two
messages at same time, B receives MB1 and MB2, C receives  MC1 and MC2. This
may lead to B generates sequence number 55 and 56 for MB1 and MB2
respectively. C generates sequence number 55 and 56 for MC1 and MC2. In this
case we come up with two members having same sequence number, which should
not be happened as SEQUENCE must achieve TOTAL ORDERING among whole members.  

Ibrahim

--
View this message in context: http://jgroups.1086181.n5.nabble.com/SEQUENCE-protocol-who-generate-the-sequence-number-unique-no-tp10424.html
Sent from the JGroups - General mailing list archive at Nabble.com.

------------------------------------------------------------------------------
Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
from Actuate! Instantly Supercharge Your Business Reports and Dashboards
with Interactivity, Sharing, Native Excel Exports, App Integration & more
Get technology previously reserved for billion-dollar corporations, FREE
http://pubads.g.doubleclick.net/gampad/clk?id=157005751&iu=/4140/ostg.clktrk
Tuomas Kiviaho | 27 Nov 12:58 2014
Picon
Picon

[jgroups-users] Unable to validate jgroups-3.6.xsd

I tried to enable async_discovery for JDBC_PING for debugging purposes, because the table tends to be
filled up with old entries when JDK is terminated and I could not find a maintenance procedure for outdated
entries. 

First thing that I encountered was. 

JGRP000004: parsing failure; the XML document is not correct: Open quote is expected for attribute "{1}"
associated with an  element type  "async_discovery".

so I though to give a try for schema version that corresponds to my library version. 

I started experiencing similar problems with validation as stated in JGRP-1692, modified the schema (and
it's references) as-per pull request (unprefixed xmlns seems to be working just as good as prefix tns) and
got the validation working. To my surprise some of the previously available attributes were missing such
as gossip_router_hosts of TUNNEL and timeout of pbcast.FLUSH. It seems that both are declared as private
members although at least hosts attribute had a properly annotated setter and the issue suggested that
these files are indeed machine generated. 

I even tried to use the properties file format but figured out from Infinispan's Parser70 and
JGroupsTransport that only XML is supported. 

I'm just asking whether or not there is some work left with the newer XMLs or am I trying to achieve something
that I'm not (yet) supposed to? 

--

-- 
Tuomas

------------------------------------------------------------------------------
Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
from Actuate! Instantly Supercharge Your Business Reports and Dashboards
with Interactivity, Sharing, Native Excel Exports, App Integration & more
Get technology previously reserved for billion-dollar corporations, FREE
http://pubads.g.doubleclick.net/gampad/clk?id=157005751&iu=/4140/ostg.clktrk
ibrahim | 26 Nov 14:56 2014
Picon

[jgroups-users] Add SEQUENCER protocol to the configuration

Hi folks,

I am trying to use SEQUENCER protocol. Where SEQUENCER should be located in
stack file? Also can you provide an example how to use it?

I have not found in the doc things about SEQUENCER, just design of it are
mentioned/

Thank.

Sanosi

--
View this message in context: http://jgroups.1086181.n5.nabble.com/Add-SEQUENCER-protocol-to-the-configuration-tp10418.html
Sent from the JGroups - General mailing list archive at Nabble.com.

------------------------------------------------------------------------------
Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
from Actuate! Instantly Supercharge Your Business Reports and Dashboards
with Interactivity, Sharing, Native Excel Exports, App Integration & more
Get technology previously reserved for billion-dollar corporations, FREE
http://pubads.g.doubleclick.net/gampad/clk?id=157005751&iu=/4140/ostg.clktrk

Gmane