headers
Zhongyi Yuan | 2 Mar 2011 04:09
Picon
Favicon

set priority for R processes in spawned nodes using SNOW

Dear R users:

I am doing parallel computing by the SNOW package on Linux workstations.
According to the university's policy, one has to set a "nice" value of at
least 10 (Initiating R with the command *nice -n 10 R* does the job) for a
long-running process, to lower its priority.

The problem I have is though I can set a "nice" value for R in the mater, I
don't know how it can be done in the slaves. All R processes in the slaves
have to be niced before they start.

Can anyone help me with this? I'd appreciate it.

Best,
Zhongyi

	[[alternative HTML version deleted]]
Permalink | Reply | 2 comments |
headers
Murray Stokely | 2 Mar 2011 04:21
Favicon
Gravatar

Re: set priority for R processes in spawned nodes using SNOW

There is probably a more direct way to do this with SNOW, but one way
you can change the priority of a running R instance on Unix is with
renice.  You could use this as the first R code your spawned tasks run
:

> system(paste("renice 10 -p", Sys.getpid()))
20121: old priority 0, new priority 10

There's probably a way to hook this or similar commands into the
initialization code when spawning the worker tasks with SNOW.

            - Murray

On Tue, Mar 1, 2011 at 7:09 PM, Zhongyi Yuan <zhongyi-yuan@...> wrote:
> Dear R users:
>
> I am doing parallel computing by the SNOW package on Linux workstations.
> According to the university's policy, one has to set a "nice" value of at
> least 10 (Initiating R with the command *nice -n 10 R* does the job) for a
> long-running process, to lower its priority.
>
> The problem I have is though I can set a "nice" value for R in the mater, I
> don't know how it can be done in the slaves. All R processes in the slaves
> have to be niced before they start.
>
> Can anyone help me with this? I'd appreciate it.
>
> Best,
> Zhongyi
>
(Continue reading)

Permalink | Reply | 1 comment |
headers
Zhongyi Yuan | 2 Mar 2011 04:36
Picon
Favicon

Re: set priority for R processes in spawned nodes using SNOW

Murry,

Thank you so much for your quick reply.
That helps me out.

Best,
Zhongyi

On Tue, Mar 1, 2011 at 9:21 PM, Murray Stokely <murray@...> wrote:

> There is probably a more direct way to do this with SNOW, but one way
> you can change the priority of a running R instance on Unix is with
> renice.  You could use this as the first R code your spawned tasks run
> :
>
> > system(paste("renice 10 -p", Sys.getpid()))
> 20121: old priority 0, new priority 10
>
> There's probably a way to hook this or similar commands into the
> initialization code when spawning the worker tasks with SNOW.
>
>            - Murray
>
> On Tue, Mar 1, 2011 at 7:09 PM, Zhongyi Yuan <zhongyi-yuan@...>
> wrote:
> > Dear R users:
> >
> > I am doing parallel computing by the SNOW package on Linux workstations.
> > According to the university's policy, one has to set a "nice" value of at
> > least 10 (Initiating R with the command *nice -n 10 R* does the job) for
(Continue reading)

Permalink | Reply | 0 comments |
headers
Prashantha Hebbar | 3 Mar 2011 14:09
Picon
Favicon

An error with Rmpi

Hi friends,

I have successfully installed MPICH2 and Rmpi after working around my previous 
problem. But when I call Rmpi in R environment I get following error. Hence, Can 
any one tell me what could be the issue here to get this following error?

> library(Rmpi)
[mlscubl10-desktop:02030] [[INVALID],INVALID] ORTE_ERROR_LOG: A system-required 
executable either could not be found or was not executable by this user in file 
../../../../../../orte/mca/ess/singleton/ess_singleton_module.c at line 269
[mlscubl10-desktop:02030] [[INVALID],INVALID] ORTE_ERROR_LOG: A system-required 
executable either could not be found or was not executable by this user in file 
../../../../../../orte/mca/ess/singleton/ess_singleton_module.c at line 143
[mlscubl10-desktop:02030] [[INVALID],INVALID] ORTE_ERROR_LOG: A system-required 
executable either could not be found or was not executable by this user in file 
../../../orte/runtime/orte_init.c at line 132
--------------------------------------------------------------------------
It looks like orte_init failed for some reason; your parallel process is
likely to abort.  There are many reasons that a parallel process can
fail during orte_init; some of which are due to configuration or
environment problems.  This failure appears to be an internal failure;
here's some additional information (which may only be relevant to an
Open MPI developer):

  orte_ess_set_name failed
  --> Returned value A system-required executable either could not be found or 
was not executable by this user (-127) instead of ORTE_SUCCESS
--------------------------------------------------------------------------
--------------------------------------------------------------------------
It looks like MPI_INIT failed for some reason; your parallel process is
(Continue reading)

Permalink | Reply | 1 comment |
headers
Jeff Howbert | 3 Mar 2011 21:21
Favicon

release of Ensemble Cloud Army on SourceForge

Greetings to the R-sig-hpc,

We would like to announce the first release of the Ensemble Cloud Army
(ECA) platform.  The purpose of ECA is to support parallel computation
of machine learning ensembles written in R, using Amazon cloud
clusters.

ECA is a fully open source project, hosted on SourceForge under the
Apache license.  You can try it out by downloading the Quick Start
Guide and zipped code/data/docs bundle from
http://sourceforge.net/projects/ica/files/Releases/ECA_RMPI/.  The
Quick Start Guide outlines the steps for installing ECA on your local
computer, getting set up with Amazon Web Services, and running several
test examples.  When you’re ready to use it for your own computations, you
can consult the Users Manual for explanations of the architecture and
work flow in ECA, and how to modify the R scripts and configuration
files.

The current Quick Start Guide assumes a Windows local computer.  If
you need to install on a Linux local computer, and don’t want to deal
with the translation, let us know, and we’ll help.

The package has undergone limited beta testing outside our company, so
we hope it can very quickly be useful for real work.  We are happy to
answer any questions you have, and will be extremely grateful for any
feedback on bugs or awkward behavior.

The same code that’s in the bundle can be found as an SVN tree at
http://ica.svn.sourceforge.net/viewvc/ica/.
(Continue reading)

Permalink | Reply | 0 comments |
headers
Paul Johnson | 4 Mar 2011 00:59
Picon

Re: An error with Rmpi

On Thu, Mar 3, 2011 at 7:09 AM, Prashantha Hebbar
<prashantha.hebbar@...> wrote:
> Hi friends,
>
> I have successfully installed MPICH2 and Rmpi after working around my previous
> problem. But when I call Rmpi in R environment I get following error. Hence, Can
> any one tell me what could be the issue here to get this following error?
>
>> library(Rmpi)
> [mlscubl10-desktop:02030] [[INVALID],INVALID] ORTE_ERROR_LOG: A system-required
> executable either could not be found or was not executable by this user in file

When you say you "successfully installed", does that mean other MPI
programs do run, but Rmpi uniquely does not?

Or is it possible the installation of Rmpi is done in an environment
that is different from that in which the program is running?

On the basis of my recent experience configuring the OpenMPI on a
Rocks Linux Cluster, the usual situation finds the "successful
install" leaving a bunch of required programs in /opt/OpenMPI/bin and
some libraries installed in a location that is out of the library
search path.  I think the error message are trying to tell you
that--the executables are not where R expects them to be.

Oh, and note you DO need to re-build Rmpi after the MPI support
libraries are changed.  And when you do that, you usually need to
explicitly specify the include directory, so when you do the
"successful install" of Rmpi, how about showing us the command you run
and the output from the build. ?
(Continue reading)

Permalink | Reply | 0 comments |
headers
Cedrick Johnson | 14 Mar 2011 05:10
Gravatar

SNOW Issues: Error in writing to connection (on localhost)

Hi All-
I am having trouble on a new machine with a fresh install of R and the
requisite SNOW packages (snow, foreach, doSNOW). When I attempt to
start 2 instances on localhost and use the %dopar% utility, I get the
following message:

Error in serialize(data, node$con) : error writing to connection

Here's that systems' details:
OS: Linux Mint 10 (Ubuntu 10.10)
SessionInfo (relevant packages):

other attached packages:
 [1] doSNOW_1.0.3                 foreach_1.3.0
codetools_0.2-8
 [4] iterators_1.0.3              fogbank_2.1.0
XML_3.2-0
 [7] fExoticOptions_2110.77       fOptions_2110.78             snow_0.3-3

When I look at the created cluster object:
> cl.tmp
[[1]]
$con
                    description                           class
"<-localhost.localdomain:10187"                      "sockconn"
                           mode                            text
                          "a+b"                        "binary"
                         opened                        can read
                       "opened"                           "yes"
                      can write
(Continue reading)

Permalink | Reply | 1 comment |
headers
Cedrick Johnson | 14 Mar 2011 07:15
Gravatar

[ignore, fixed] Re: SNOW Issues: Error in writing to connection (on localhost)

Nevermind. It was a matter of PEBCAK (problem exists between chair and
keyboard).

Installed ssh on the new machine and it works. That was the only
difference between the two machines, the working one had an ssh server
running (different post altogether about why snow only runs on port
22).

Now it works as intended (was just testing out locally before turning
up the "volume" per se and having it distribute across multiple
machines (works as expected too).

My apologies.
C

On Mon, Mar 14, 2011 at 12:10 AM, Cedrick Johnson
<cedrick@...> wrote:
> Hi All-
> I am having trouble on a new machine with a fresh install of R and the
> requisite SNOW packages (snow, foreach, doSNOW). When I attempt to
> start 2 instances on localhost and use the %dopar% utility, I get the
> following message:
>
> Error in serialize(data, node$con) : error writing to connection
>
> Here's that systems' details:
> OS: Linux Mint 10 (Ubuntu 10.10)
> SessionInfo (relevant packages):
>
> other attached packages:
(Continue reading)

Permalink | Reply | 0 comments |
headers
Maas James Dr (MED | 14 Mar 2011 15:50
Picon
Picon
Favicon

something wrong with lamnodes

When installing Rmpi I get this final warning, and then subsequently when trying to use MPI it only finds
2 of the 8 nodes available.  On Ubuntu linux, any suggestions?

Thanks

Jim

** inst
** preparing package for lazy loading
** help
*** installing help indices
** building package indices ...
** testing if installed package can be loaded
Warning: running command 'lamnodes 2>/dev/null' had status 253

* DONE (Rmpi)

===============================
Dr. Jim Maas
University of East Anglia
Permalink | Reply | 1 comment |
headers
Marius Hofert | 14 Mar 2011 20:57
Picon

R hangs when connected via VPN [incl. minimal example]

Dear expeRts, 

This is a similar post as on R-SIG-MAC [I didn't receive an answer there] and
R-help. On the latter mailing list, Duncan Murdoch suggested to contact a
package maintainer and ask for help. I contacted Rich Calaway from Revolution 
Analytics. After a few trials he suggested to write to R-SIG-HPC. 

The problem is that if [and only if] I am connected via VPN, R "hangs" on executing
the minimal example below.
After contacting Rich Calaway, I realized that [after executing the minimal example
line by line] the makeCluster call is the problem. So if I am connected via
VPN, the following line is not executed properly:
cl <- makeCluster(mpi.universe.size(), type ="MPI") 

Is this a known issue?
The "red button" in the R Console is dark red [normal behavior if R is running], 
so R tries to do something, but cannot succeed. CPU goes up to 100%... This is quite
annoying when working on a server but testing code locally at the same time. 

I'm running R version 2.12.1 (Platform: x86_64-apple-darwin9.8.0/x86_64 (64-bit)) 
on a MacBook Pro under Mac OS X 10.6.6. I am connected to a server via a VPN client 
[Cisco AnyConnect; same happens with Apple's internal VPN connection].

Any help is highly appreciated.

Cheers, 

Marius 

Minimal example:
(Continue reading)

Permalink | Reply | 0 comments |

Gmane