Clustering

Well, here's clearly explained (or I hope it is) idea of clustering or cluster 
machining I raised on IRC.

Every single computer (whatever PC or smartphone or anything else) can join a 
cluster machine, ie. a big virtual computer (not virtual like VM but made by 
more physical computers) in which every thread of every application can be 
run on any of its processor (depending on priority configuration - ie. a task 
hard for smartphone will be automatically posted to computer if this is idle) 
and the current state should be synchronized with the other computers in the 
cluster.

Well, there were some questions about how shall it work exactly, so here we 
are:

Since Unununium uses orthogonal persistence, there'll be no problem to use 
network instead of hard drive (or any other slow permanent storage). The main 
problem can occur when two or more threads of a single application will be 
run each on another processor. Well, this is not problem, because when any 
thread want to access shared memory for writing, it shall lock it (eg. using 
semaphore) or Segmentation Fault can occur. So when a request for semaphore 
is sent, the thread is suspended till this semaphore is locked on all 
computers in cluster and after this semaphore is unlocked, then all the 
changes are flushed to all computers. Well, this requires either running VM 
or that all computers in the cluster are the same architecture.

There can be problems with uncleanly unjoined computers like on power failure 
or network disconnection (eg. moving with smartphone out of Bluetooth 
connection range). Well, this is also in fact no problem, because orthogonal 
persistence counts with this (or something very similar), so when one 
computer left, it's job will be overtaken by some other one (if required). 

Re: Clustering

Actually, a good clustering system always checks if results are
received twice... no?

Take distributed systems like Distributed.net, they check if packets
are processed twice, especially since processing can be made randomly
(when a node runs out of job because it's disconnected from network).
So I don't see how it's worse with UUU than any other possible
environment. Did I miss something?

On 3/2/06, Jan „Sten" Adámek <unununium <at> stensoft.com> wrote:
> Well, here's clearly explained (or I hope it is) idea of clustering or cluster
> machining I raised on IRC.
>
> Every single computer (whatever PC or smartphone or anything else) can join a
> cluster machine, ie. a big virtual computer (not virtual like VM but made by
> more physical computers) in which every thread of every application can be
> run on any of its processor (depending on priority configuration - ie. a task
> hard for smartphone will be automatically posted to computer if this is idle)
> and the current state should be synchronized with the other computers in the
> cluster.
>
> Well, there were some questions about how shall it work exactly, so here we
> are:
>
> Since Unununium uses orthogonal persistence, there'll be no problem to use
> network instead of hard drive (or any other slow permanent storage). The main
> problem can occur when two or more threads of a single application will be
> run each on another processor. Well, this is not problem, because when any
> thread want to access shared memory for writing, it shall lock it (eg. using
> semaphore) or Segmentation Fault can occur. So when a request for semaphore

Re: Clustering

Well, the think is this clustering to be implemented as a part of Unununium 
because I think clustering (easy-joining?) will be basic point of future 
generation of computers.

You're right that we can use something like Distributed.net to implement this. 
It won't be worse with Unununium than other environment, in fact it will be 
easier.

> Actually, a good clustering system always checks if results are
> received twice... no?
>
> Take distributed systems like Distributed.net, they check if packets
> are processed twice, especially since processing can be made randomly
> (when a node runs out of job because it's disconnected from network).
> So I don't see how it's worse with UUU than any other possible
> environment. Did I miss something?
>
>
> --
> Jacques Mony
>
> _______________________________________________
> Uuu-devel mailing list
> Uuu-devel <at> unununium.org
> http://unununium.org/cgi-bin/mailman/listinfo/uuu-devel

_______________________________________________
Uuu-devel mailing list
Uuu-devel <at> unununium.org

Re: Clustering

And more importantly.. who's going to write the code for this?

Richard Fillion
rfillion <at> cs.und.edu

On Mar 2, 2006, at 9:48 AM, Jan „Sten“ Adámek wrote:

> Well, the think is this clustering to be implemented as a part of  
> Unununium
> because I think clustering (easy-joining?) will be basic point of  
> future
> generation of computers.
>
> You're right that we can use something like Distributed.net to  
> implement this.
> It won't be worse with Unununium than other environment, in fact it  
> will be
> easier.
>
>> Actually, a good clustering system always checks if results are
>> received twice... no?
>>
>> Take distributed systems like Distributed.net, they check if packets
>> are processed twice, especially since processing can be made randomly
>> (when a node runs out of job because it's disconnected from network).
>> So I don't see how it's worse with UUU than any other possible
>> environment. Did I miss something?
>>
>>
>> --

Re: Clustering

Jan „Sten" Adámek will, this is obvious  At least, if he read the FAQ !

On 3/2/06, Richard Fillion <rfillion <at> cs.und.edu> wrote:
> And more importantly.. who's going to write the code for this?
>
> Richard Fillion
> rfillion <at> cs.und.edu
>
> On Mar 2, 2006, at 9:48 AM, Jan „Sten" Adámek wrote:
>
> > Well, the think is this clustering to be implemented as a part of
> > Unununium
> > because I think clustering (easy-joining?) will be basic point of
> > future
> > generation of computers.
> >
> > You're right that we can use something like Distributed.net to
> > implement this.
> > It won't be worse with Unununium than other environment, in fact it
> > will be
> > easier.
> >
> >> Actually, a good clustering system always checks if results are
> >> received twice... no?
> >>
> >> Take distributed systems like Distributed.net, they check if packets
> >> are processed twice, especially since processing can be made randomly
> >> (when a node runs out of job because it's disconnected from network).
> >> So I don't see how it's worse with UUU than any other possible
> >> environment. Did I miss something?

Re: Clustering

then have fun implementing checkpointing- necessary for the
orthogonal persistence approach- on top of that (or your own
implementation) for an asynchronous process along with
saving certain machine/node states, such as the context of a thread
for all nodes. you would have to cope with much
more complex synchronization methods (checkpointing etc) than
with current NUMA-like architectures, since you have a shitload
of different states in the whole "network" which are then because
of the overly complex parallelism undeterministic.

so when do you do checkpointing? how do you reroll certain processes
that have not finished because of an unreliable topology of the network?
etc. ...

that topic is already difficult to solve if you know what is being
processed,
how it is done, the states the process will have as well as their dependency
and order.

maybe the problem is solveable. but i dont think the approach to it will
be simple
and then there is this still the question whether the communication-overhead
allows for a still practicable/useable system.

2cents

Jan „Sten“ Adámek wrote:

>Well, the think is this clustering to be implemented as a part of Unununium 
>because I think clustering (easy-joining?) will be basic point of future 

Re: Clustering

On Thu, Mar 02, 2006 at 04:15:05PM +0100, Jan ???Sten??? Ad??mek wrote:
> Well, here's clearly explained (or I hope it is) idea of clustering or cluster 
> machining I raised on IRC.
> 
> Every single computer (whatever PC or smartphone or anything else) can join a 
> cluster machine, ie. a big virtual computer (not virtual like VM but made by 
> more physical computers) in which every thread of every application can be 
> run on any of its processor (depending on priority configuration - ie. a task 
> hard for smartphone will be automatically posted to computer if this is idle) 
> and the current state should be synchronized with the other computers in the 
> cluster.
> 
> Well, there were some questions about how shall it work exactly, so here we 
> are:
> 
> Since Unununium uses orthogonal persistence, there'll be no problem to use 
> network instead of hard drive (or any other slow permanent storage). The main 
> problem can occur when two or more threads of a single application will be 
> run each on another processor. Well, this is not problem, because when any 
> thread want to access shared memory for writing, it shall lock it (eg. using 
> semaphore) or Segmentation Fault can occur. So when a request for semaphore 
> is sent, the thread is suspended till this semaphore is locked on all 
> computers in cluster and after this semaphore is unlocked, then all the 
> changes are flushed to all computers. Well, this requires either running VM 
> or that all computers in the cluster are the same architecture.
> 
> There can be problems with uncleanly unjoined computers like on power failure 
> or network disconnection (eg. moving with smartphone out of Bluetooth 
> connection range). Well, this is also in fact no problem, because orthogonal 
> persistence counts with this (or something very similar), so when one 

http://www.exomeme.org)

we were excited to discover unununium today and would like to share
some ideas about a new kind of software tool called 'exomeme'.  we are
interested in some feedback, comments, suggestions, and
thought-provoking or challenging questions.  exomeme and uuu have
similar goals, and are both using python.  how can we help each other?

http://www.exomeme.org

more details about exo/meme (pdf)

here is an easy-to-read slide-show presentation that describes more
details about exomeme (www.exomeme.org) than are currently available
on the website:

  http://www.exomeme.org/exomeme.pdf

i think it will clarify how exomeme relates to unununium.  it's packed
with ideas, so you may have to read it a few times to get everything. 
feel free to borrow anything for use in unununium.

Re: more details about exo/meme (pdf)

So, you were excited about Unununium so the first thing you do is post
on the list. Did you read the FAQ?

On 3/4/06, metanull <metanull <at> gmail.com> wrote:
> here is an easy-to-read slide-show presentation that describes more
> details about exomeme (www.exomeme.org) than are currently available
> on the website:
>
>  http://www.exomeme.org/exomeme.pdf
>
> i think it will clarify how exomeme relates to unununium.  it's packed
> with ideas, so you may have to read it a few times to get everything.
> feel free to borrow anything for use in unununium.
>
> _______________________________________________
> Uuu-devel mailing list
> Uuu-devel <at> unununium.org
> http://unununium.org/cgi-bin/mailman/listinfo/uuu-devel
>

--
Jacques Mony