headers
Stefan Schwarzburg | 3 Sep 08:33

demo, classes and indentation

Dear all,
I have a question about the demo mode:
Is it possible to run a demo that includes classes? Whenever I try to run such a  demo, the indentation is missing at each block (which makes it unusable for classes...).
Can someone help me here?
Cheers,
Stefan

--
Institut für Astronomie und Astrophysik
Eberhard Karls Universität Tübingen
Sand 1 - D-72076 Tübingen
schwarz <at> astro.uni-tuebingen.de
stefan.schwarzburg <at> googlemail.com

Tel.: 07071/29-78605
-----------------------------------------------------------------------
_______________________________________________
IPython-user mailing list
IPython-user <at> scipy.org
http://lists.ipython.scipy.org/mailman/listinfo/ipython-user
Permalink | Reply | 501 comments |
headers
Inigo Aldazabal Mensa | 3 Sep 10:05
Picon
Picon
Favicon
Gravatar

Embedded IPython and -pylab

Hi all,

The thing is, if I try to call IPython from a script with -pylab as the
argv, the script neither loads the pylab namespace nor it is in matplotlib
threaded mode. So that in

>>> ipshell = IPShellEmbed('-pylab')
>>> ipshell()

the -pylab seems to have no effect.

>From my search I found that I can go into pylab mode doing eg:

>>> sys.argv = [sys.argv[0], '-pylab']
>>> IPython.Shell.start().mainloop()

But then in this case I don't get the previous variable space so I can not
interact with it.

The idea of course is doing some number-crunching and at some point 
stopping and goint into IPython to make some plots, etc. in a non-blocking 
manner a la "IPython -pylab".

I found a couple of posts regarding this, but no clear answers about it. Is
it not possible to do it or I'm missing something here?

Thanks,

Inigo
Permalink | Reply | 565 comments |
headers
Ville M. Vainio | 3 Sep 10:57
Picon
Gravatar

Re: Embedded IPython and -pylab

On Wed, Sep 3, 2008 at 11:05 AM, Inigo Aldazabal Mensa
<inigo_aldazabal <at> ehu.es> wrote:

>>>> sys.argv = [sys.argv[0], '-pylab']
>>>> IPython.Shell.start().mainloop()
>
> But then in this case I don't get the previous variable space so I can not
> interact with it.

See how you can do it with tests/test_embed.py

--

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'
Permalink | Reply | 0 comments |
headers
JM Seitz | 3 Sep 18:33
Picon

sys.stderr

So (as I mentioned in my previous post) I have IPython embedded into a
C++ app. That's working fine now, however, I need to be able to access
sys.stderr but instead of the original traceback that Python itself has
thrown I can only get:

ERROR - failed to write data to
stream:<pyreadline.console.console.Console object at 0x036B6A70>

So I am getting the exception that the IPython shell was handed but I
need to get the original sys.stderr exception. The shell is an
iplib.InteractiveShell instance.

TIA
Permalink | Reply | 1 comment |
headers
Brian Granger | 3 Sep 22:20
Picon

Re: ipython 0.9 ipengine launch question

Brant,

I do want to try to understand what is going on here.

> I'm using ipython 0.9 to run fairly standard commands (.push, .pull,
> .execute, etc) on ipengine/ipcontroller clusters, and I've been trying
> to get the -f flag working properly.  My clusterfile.py script looks
> like:
>
> controller = {'host':'kakahiaka',
>                  }
>
> engines = {'kakahiaka':4,
>
>               }
>
> sshx = '/mnt/py_util/sshx'
>
>
> and sshx reads:
> #!/bin/sh
> NUMPROCS=$(cat /proc/cpuinfo | grep -e "processor[[:space:]]:" | wc -l)
> export NUMPROCS
> export PATH=$PATH:$HOME/bin:$HOME/multiz:.
> export PYTHONPATH=/mnt/py_util
> "$@"
>
> In addition, I had to change line 306 of ipcluster.py from
> cmd = "ssh %s '%s' 'ipengine --controller-ip %s --logfile %s' &" % \
> (engineHost,sshx,contHost,engLog)
> to:
> cmd = "ssh %s '%s' 'ipengine --logfile %s' &" % (engineHost,sshx,engLog)
> since the new version of ipengine uses foolscap urls instead of
> straight hostnames.

Yep, this is a bug in ipcluster.py.  I will work on fixing this today.

> Everything seems to come up fine for the -f run mode, and I can run
> for about 5-10min, before something hiccups, and I get the following
> errors:
> in the logfile:
> 2008/08/27 14:33 -0700 [-] unregistered engine with id: 0
>
> in my stderr buffer:
> [Engine Exception]ConnectionLost: Connection to the other side was
> lost in a non-clean fashion.
>
> and thrown on the local console:
> Fatal Python error: PyEval_RestoreThread: NULL tstate
> Aborted

Wow, this is really wierd, especially because we don't really use
threads.  Hmmm.  Can you give more information?

Are you using threads in any way?
Can you produce cod that triggers this?
Which stderr buffer shows that EngineException?
Python version?
OS and platform?

Let's see if we can get to the bottom of this...

Brian

> The sum of my experience with python threads has been googling for
> error messages, so my hope is that this is something fairly obvious
> that I'm just not doing.
>
> Any help, or suggestions about what to try would be greatly appreciated!
> -Brant
> _______________________________________________
> IPython-user mailing list
> IPython-user <at> scipy.org
> http://lists.ipython.scipy.org/mailman/listinfo/ipython-user
>
Permalink | Reply | 1 comment |
headers
Brian Granger | 4 Sep 01:23
Picon

Re: ipython 0.9 ipengine launch question

One correction,

To have furl files that don't change each time you run the controller
you need to specify the ports to listen on using:

ipcontroller --engine-port=10111 --client-port=10112

Otherwise, the controller uses random port numbers each time (they
change).  Because the port numbers are in the furls, the furls then
change.  But, if you set the port numbers by hand, the furls will
remain the same each time you run ipcontroller:

$ ipcontroller --engine-port=10111 --client-port=10112
(test4)pcp025382pcs:security bgranger$ cat *.furl
pb://5zdgwqsnrtpifiz6hp23by3w6jyabflk <at> 127.0.0.1:10111,129.65.215.183:10111/pdsoipnt6oxbf3frpdlw76i47he3vudn
pb://la7is75mkk4z62aqdmfynnd7cibo4ecx <at> 127.0.0.1:10112,129.65.215.183:10112/pwkwqv5f7fw4qgaw56yqwba42fob4xhd
pb://la7is75mkk4z62aqdmfynnd7cibo4ecx <at> 127.0.0.1:10112,129.65.215.183:10112/xdm5xaczv6yfgdomdufuluhu4uopzrbi

And a second time to see they don't change...

$ ipcontroller --engine-port=10111 --client-port=10112
(test4)pcp025382pcs:security bgranger$ cat *.furl
pb://5zdgwqsnrtpifiz6hp23by3w6jyabflk <at> 127.0.0.1:10111,129.65.215.183:10111/pdsoipnt6oxbf3frpdlw76i47he3vudn
pb://la7is75mkk4z62aqdmfynnd7cibo4ecx <at> 127.0.0.1:10112,129.65.215.183:10112/pwkwqv5f7fw4qgaw56yqwba42fob4xhd
pb://la7is75mkk4z62aqdmfynnd7cibo4ecx <at> 127.0.0.1:10112,129.65.215.183:10112/xdm5xaczv6yfgdomdufuluhu4uopzrbi

On Wed, Aug 27, 2008 at 3:17 PM, Brant Peterson <brantp <at> gmail.com> wrote:
> All,
>
> I'm using ipython 0.9 to run fairly standard commands (.push, .pull,
> .execute, etc) on ipengine/ipcontroller clusters, and I've been trying
> to get the -f flag working properly.  My clusterfile.py script looks
> like:
>
> controller = {'host':'kakahiaka',
>                  }
>
> engines = {'kakahiaka':4,
>
>               }
>
> sshx = '/mnt/py_util/sshx'
>
>
> and sshx reads:
> #!/bin/sh
> NUMPROCS=$(cat /proc/cpuinfo | grep -e "processor[[:space:]]:" | wc -l)
> export NUMPROCS
> export PATH=$PATH:$HOME/bin:$HOME/multiz:.
> export PYTHONPATH=/mnt/py_util
> "$@"
>
> In addition, I had to change line 306 of ipcluster.py from
> cmd = "ssh %s '%s' 'ipengine --controller-ip %s --logfile %s' &" % \
> (engineHost,sshx,contHost,engLog)
> to:
> cmd = "ssh %s '%s' 'ipengine --logfile %s' &" % (engineHost,sshx,engLog)
> since the new version of ipengine uses foolscap urls instead of
> straight hostnames.
>
> I'm just launching 4 engines on my local machine, which should be
> morally equivalent to ipcluster -n 4, with which I have no problems in
> testing
>
> Everything seems to come up fine for the -f run mode, and I can run
> for about 5-10min, before something hiccups, and I get the following
> errors:
> in the logfile:
> 2008/08/27 14:33 -0700 [-] unregistered engine with id: 0
>
> in my stderr buffer:
> [Engine Exception]ConnectionLost: Connection to the other side was
> lost in a non-clean fashion.
>
> and thrown on the local console:
> Fatal Python error: PyEval_RestoreThread: NULL tstate
> Aborted
>
> The sum of my experience with python threads has been googling for
> error messages, so my hope is that this is something fairly obvious
> that I'm just not doing.
>
> Any help, or suggestions about what to try would be greatly appreciated!
> -Brant
> _______________________________________________
> IPython-user mailing list
> IPython-user <at> scipy.org
> http://lists.ipython.scipy.org/mailman/listinfo/ipython-user
>
Permalink | Reply | 0 comments |
headers
Fernando Perez | 4 Sep 02:33
Picon
Gravatar

Re: sys.stderr

Howdy,

On Wed, Sep 3, 2008 at 9:33 AM, JM Seitz <jms <at> bughunter.ca> wrote:
> So (as I mentioned in my previous post) I have IPython embedded into a
> C++ app. That's working fine now, however, I need to be able to access
> sys.stderr but instead of the original traceback that Python itself has
> thrown I can only get:
>
> ERROR - failed to write data to
> stream:<pyreadline.console.console.Console object at 0x036B6A70>
>
> So I am getting the exception that the IPython shell was handed but I
> need to get the original sys.stderr exception. The shell is an
> iplib.InteractiveShell instance.

I'm afraid I have no embedding (in C++) experience so I won't be able
to help you out, but it would really be great if you could document
this process so we could include it in the ipython docs and examples
for the future.  This question does come up somewhat regularly...

Thanks!

f
Permalink | Reply | 0 comments |
headers
Fernando Perez | 4 Sep 06:30
Picon
Gravatar

Re: demo, classes and indentation

Howdy,

On Tue, Sep 2, 2008 at 11:33 PM, Stefan Schwarzburg
<stefan.schwarzburg <at> googlemail.com> wrote:
> Dear all,
> I have a question about the demo mode:
> Is it possible to run a demo that includes classes? Whenever I try to run
> such a  demo, the indentation is missing at each block (which makes it
> unusable for classes...).
> Can someone help me here?

Unfortunately I'm afraid the demo stuff has fallen into a bit of
disrepair.  I hadn't used it in a while and I recently noticed
problems with it.  We're trying to wrap up the 0.9 release, but I have
some teaching to do soon and I'll try to fix it as soon as I can,
adding proper tests in the process.   I'll keep this in mind as I do
so, because it's likely a problem that was  simply never addressed.

One note though: you simply can't break up the sections of a demo
anywhere  but at the top-level of indentation, so there's just no way
to have a section include only part of a class or function
declaration.  That's just a limitation of python's indentation-based
model.  So for your demos, you need to keep the split points at the
top-level, keeping classes or functions *entirely* within one  demo
block.  That limitation could technically be removed, but it's not
trivial (it requires keeping an internal running block to which you
add new sections that are indented and *re-execute* the accumulated
contents to gradually complete the code).

Cheers,

f
Permalink | Reply | 501 comments |
headers
mark starnes | 4 Sep 11:07
Picon

ipython 0.9.rc1 problem with manual example during import, using MultiEngineClient

Hi everyone,

I'm following the instructions in the Aug 29 2008 manual for parallel
computing, and have been having some problems; reading back through the
posts, I've noticed nothing to help (please, correct me if I'm wrong!).

If I start my cluster with:

$ ipcluster -n 4 &

Then, with IPython, 0.9.rc1:

In [1]: from IPython.kernel import client

In [2]: tc = client.TaskClient()

I get:
---------------------------------------------------------------------------

... traceback information....

ValueError: not a furl or a file containing a furl:
/home/m.starnes/.ipython/ipcontroller-tc.furl

If I follow the instructions printed by the shell during cluster startup:

In [1]: from ipython1.kernel import client   # Differs from the manual

In [2]: mec = client.MultiEngineClient(('127.0.0.1',10105)) # Differs.

All seems well.  The same occurs when I follow the 'Quickstart Task
Farming' section:

In [3]: tc = client.TaskClient(('127.0.0.1',10105))

is fine, but

In [3]: tc = client.TaskClient()  # as per the manual.

returns:

TypeError: TaskClient() takes exactly 1 argument (0 given)

I'm wondering, have I made a mistake with my installation?

Best regards,

Mark.
Permalink | Reply | 1 comment |
headers
mark starnes | 4 Sep 11:44
Picon

Re: ipython 0.9.rc1 problem with manual example during import, using MultiEngineClient

Hello again,

Further to the request below, more problems occur for me later in the
manual.  If I execute:

In [1]: from ipython1.kernel import client

In [2]: tc = client.TaskClient(('127.0.0.1',10105))

In [3]: for n in range(1000):
    tasklist.append(client.Task("a=%i"%n, pull="a"))

In [4]: taskids = [ tc.run(t) for t in tasklist ]

I get:

---------------------------------------------------------------------------
Fault                                     Traceback (most recent call last)

/home/m.starnes/Documents/phd/meetings/85_/<ipython console> in <module>()

/usr/local/lib64/python2.5/site-packages/ipython1-0.3pre3-py2.5.egg/ipython1/kernel/taskclient.pyc
in run(self, task)
     84
     85     def run(self, task):
---> 86         return blockingCallFromThread(self.task_controller.run,
task)
     87
     88     def get_task_result(self, taskid, block=False):

/usr/local/lib64/python2.5/site-packages/ipython1-0.3pre3-py2.5.egg/ipython1/kernel/twistedutil.pyc
in blockingCallFromThread(f, *a, **kw)
     66         @raise: any error raised during the callback chain.
     67         """
---> 68         return
twisted.internet.threads.blockingCallFromThread(reactor, f, *a, **kw)
     69
     70 else:

/usr/local/lib64/python2.5/site-packages/Twisted-8.1.0-py2.5-linux-x86_64.egg/twisted/internet/threads.pyc
in blockingCallFromThread(reactor, f, *a, **kw)
     81     result = queue.get()
     82     if isinstance(result, failure.Failure):
---> 83         result.raiseException()
     84     return result
     85

/usr/local/lib64/python2.5/site-packages/Twisted-8.1.0-py2.5-linux-x86_64.egg/twisted/web/xmlrpc.pyc
in parseResponse(self, contents)
    313             return
    314         try:
--> 315             response = xmlrpclib.loads(contents)
    316         except:
    317             deferred, self.deferred = self.deferred, None

/usr/lib64/python2.5/xmlrpclib.pyc in loads(data, use_datetime)
   1130     p.feed(data)
   1131     p.close()
-> 1132     return u.close(), u.getmethodname()
   1133
   1134

/usr/lib64/python2.5/xmlrpclib.pyc in close(self)
    785             raise ResponseError()
    786         if self._type == "fault":
--> 787             raise Fault(**self._stack[0])
    788         return tuple(self._stack)
    789

Fault: <Fault 8001: 'function run not found'>

Any help would be appreciated.

Best regards,

Mark.

mark starnes wrote:
> Hi everyone,
> 
> I'm following the instructions in the Aug 29 2008 manual for parallel
> computing, and have been having some problems; reading back through the
> posts, I've noticed nothing to help (please, correct me if I'm wrong!).
> 
> If I start my cluster with:
> 
> $ ipcluster -n 4 &
> 
> Then, with IPython, 0.9.rc1:
> 
> In [1]: from IPython.kernel import client
> 
> In [2]: tc = client.TaskClient()
> 
> I get:
> ---------------------------------------------------------------------------
> 
> ... traceback information....
> 
> ValueError: not a furl or a file containing a furl:
> /home/m.starnes/.ipython/ipcontroller-tc.furl
> 
> 
> If I follow the instructions printed by the shell during cluster startup:
> 
> In [1]: from ipython1.kernel import client   # Differs from the manual
> 
> In [2]: mec = client.MultiEngineClient(('127.0.0.1',10105)) # Differs.
> 
> All seems well.  The same occurs when I follow the 'Quickstart Task
> Farming' section:
> 
> In [3]: tc = client.TaskClient(('127.0.0.1',10105))
> 
> 
> is fine, but
> 
> In [3]: tc = client.TaskClient()  # as per the manual.
> 
> returns:
> 
> TypeError: TaskClient() takes exactly 1 argument (0 given)
> 
> 
> I'm wondering, have I made a mistake with my installation?
> 
> 
> Best regards,
> 
> Mark.
>
Permalink | Reply | 0 comments |

Gmane