headers
Ariel Rokem | 1 Jul 01:24
Picon
Favicon
Gravatar

Re: [SciPy-dev] Memory usage of scipy.io.loadmat

Yes - that does seem to free up the memory.

While running this:

In [11]: for i in range(10):
                 a = sio.loadmat('/Users/arokem/Projects/SchizoSpread/Scans/SMR033109_MC/Gray/Original/TSeries/Scan1/tSeries1.mat')
  
causes a memory error, running this:

In [14]: for i in range(10):
                a = sio.loadmat('/Users/arokem/Projects/SchizoSpread/Scans/SMR033109_MC/Gray/Original/TSeries/Scan1/tSeries1.mat')
                gc.collect()

seems like it could go on forever (looking at the memory usage on a memory monitor, it just goes up and down to the same point, without net accumulation).

Thanks a lot!

Ariel









On Tue, Jun 30, 2009 at 3:33 PM, Xavier Saint-Mleux <saintmlx <at> apstat.com> wrote:

>
> So - just invoking an error in the ipython command line has freed up
> 300 MB. Where did they come from? I tried different things - assigning
> other variables doesn't seem to free up this memory. Neither do calls
> to other functions. Except "plot()", which does seem to do the trick
> for some reason. Interestingly, when I run all this in a python
> interactive session (and not ipython), I get a similar memory usage
> initially. Calling a non-existent variable does not free up the
> memory, but other things do. For example, import matplotlib.pylab into
> the namespace did the trick. Does anyone have any idea what is going on?
>

It is probably just the garbage collector being invoked.  If you invoke
it manually, does it always free memory? e.g.:

import gc

gc.collect()



Xavier Saint-Mleux


_______________________________________________
Scipy-dev mailing list
Scipy-dev <at> scipy.org
http://mail.scipy.org/mailman/listinfo/scipy-dev



--
Ariel Rokem
Helen Wills Neuroscience Institute
University of California, Berkeley
http://argentum.ucbso.berkeley.edu/ariel
_______________________________________________
SciPy-user mailing list
SciPy-user <at> scipy.org
http://mail.scipy.org/mailman/listinfo/scipy-user
Permalink | Reply | 12 comments |
headers
Fernando Perez | 1 Jul 09:40
Picon
Gravatar

Re: Tutorial topics for SciPy'09 Conference

Hi,

On Mon, Jun 1, 2009 at 10:20 PM, Fernando Perez<fperez.net <at> gmail.com> wrote:
> The time for the Scipy'09 conference is rapidly approaching, and we
> would like to both announce the plan for tutorials and solicit
> feedback from everyone on topics of interest.

rather than rehash much here, where it's not easy to paste a table,
I've posted a note with the poll results here:

http://fdoperez.blogspot.com/2009/06/scipy-advanced-tutorials-results.html

The short and plain-text-friendly version is the final topic ranking:

1	Advanced topics in matplotlib use
2	Advanced numpy
3	Designing scientific interfaces with Traits
4	Mayavi/TVTK
5	Cython
6	Symbolic computing with sympy
7	Statistics with Scipy
8	Using GPUs with PyCUDA
9	Testing strategies for scientific codes
10	Parallel computing in Python and mpi4py
11	Sparse Linear Algebra with Scipy
12	Structured and record arrays in numpy
13	Design patterns for efficient iterator-based scientific codes
14	Sage
15	The TimeSeries scikit
16	Hermes: high order Finite Element Methods
17	Graph theory with NetworkX

We're currently contacting speakers, and we'll let you know once a
final list is made with confirmed speakers.

Cheers,

f
Permalink | Reply | 0 comments |
headers
David Goldsmith | 1 Jul 22:29
Picon
Favicon

Summer Marathon "Category of the Week"


Linear Algebra! (My favorite! :-))  Let's get it to pink or better by next Wed!

DG
Permalink | Reply | 3 comments |
headers
Stéfan van der Walt | 2 Jul 12:20
Picon
Picon
Favicon

Re: Summer Marathon "Category of the Week"

Hi marathon runners,

Well done on the progress made last week!

2009/7/1 David Goldsmith <d_l_goldsmith <at> yahoo.com>:
> Linear Algebra! (My favorite! :-))  Let's get it to pink or better by next Wed!

Excellent!  The linalg module is shown here:

http://docs.scipy.org/numpy/docs/numpy.linalg/

You can choose from:

cholesky cond det eig eigh eigvals eigvalsh
inv lstsq matrix_power norm inv qr solve svd tensorinv tensorsolve

These mostly have decent docstrings already, so I'd suggest focusing
on making the following more accessible to your typical linear algebra
student:

http://docs.scipy.org/numpy/docs/numpy.linalg.linalg.eig/ [1]
http://docs.scipy.org/numpy/docs/numpy.linalg.linalg.qr/ [2]
http://docs.scipy.org/numpy/docs/numpy.linalg.linalg.norm/ [3]
http://docs.scipy.org/numpy/docs/numpy.linalg.linalg.svd/ [4]
http://docs.scipy.org/numpy/docs/numpy.linalg.linalg.tensorinv/ [5]

Have fun!
Stéfan

[1] In the notes section, describe right vs left eigenvectors.
Personally, I think we can drop the reference to the characteristic
polynomial (we
don't solve it that way in any case).

[2] Needs formatting and a link to an explanation of QR.

[3] Describe what Frobenius means, or give a link to an appropriate reference.

[4] Note Pauli's comment: indicate clearly that the decomposition A =
U S V.T is returned (V.H for complex matrices).

[5] Badly in need of some examples.
Permalink | Reply | 2 comments |
headers
Nils Wagner | 2 Jul 13:11
Picon
Favicon

Re: Summer Marathon "Category of the Week"

On Thu, 2 Jul 2009 12:20:12 +0200
  Stéfan van der Walt <stefan <at> sun.ac.za> wrote:
> Hi marathon runners,
> 
> Well done on the progress made last week!
> 
> 2009/7/1 David Goldsmith <d_l_goldsmith <at> yahoo.com>:
>> Linear Algebra! (My favorite! :-))  Let's get it to pink 
>>or better by next Wed!
> 
> Excellent!  The linalg module is shown here:
> 
> http://docs.scipy.org/numpy/docs/numpy.linalg/
> 
> You can choose from:
> 
> cholesky cond det eig eigh eigvals eigvalsh
> inv lstsq matrix_power norm inv qr solve svd tensorinv 
>tensorsolve
> 
> These mostly have decent docstrings already, so I'd 
>suggest focusing
> on making the following more accessible to your typical 
>linear algebra
> student:
> 
> http://docs.scipy.org/numpy/docs/numpy.linalg.linalg.eig/ 
>[1]
> http://docs.scipy.org/numpy/docs/numpy.linalg.linalg.qr/ 
>[2]
> http://docs.scipy.org/numpy/docs/numpy.linalg.linalg.norm/ 
>[3]
> http://docs.scipy.org/numpy/docs/numpy.linalg.linalg.svd/ 
>[4]
> http://docs.scipy.org/numpy/docs/numpy.linalg.linalg.tensorinv/ 
>[5]
> 
> Have fun!
> Stéfan
> 
> [1] In the notes section, describe right vs left 
>eigenvectors.
> Personally, I think we can drop the reference to the 
>characteristic
> polynomial (we
> don't solve it that way in any case).
> 
> [2] Needs formatting and a link to an explanation of QR.
> 
> [3] Describe what Frobenius means, or give a link to an 
>appropriate reference.
> 
> [4] Note Pauli's comment: indicate clearly that the 
>decomposition A =
> U S V.T is returned (V.H for complex matrices).
> 
> [5] Badly in need of some examples.
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev <at> scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev

Hi Stéfan,

Is there a chance to fix the following ticket 
http://projects.scipy.org/numpy/ticket/937
in this context ?

Nils
Permalink | Reply | 1 comment |
headers
Stéfan van der Walt | 2 Jul 13:16
Picon
Picon
Favicon

Re: Summer Marathon "Category of the Week"

2009/7/2 Nils Wagner <nwagner <at> iam.uni-stuttgart.de>:
> Is there a chance to fix the following ticket
> http://projects.scipy.org/numpy/ticket/937
> in this context ?

Sure, I'd gladly review any patches (I'm a bit swamped myself at the moment).

Regards
Stéfan
Permalink | Reply | 0 comments |
headers
David Goldsmith | 2 Jul 19:02
Picon
Favicon

Re: Summer Marathon "Category of the Week"


OK, the gauntlet has been thrown: anyone have the time and expertise to commit to fixing this code (I can
commit to fixing the docstring) this week?

DG
--- On Thu, 7/2/09, Stéfan van der Walt <stefan <at> sun.ac.za> wrote:

> From: Stéfan van der Walt <stefan <at> sun.ac.za>
> Subject: Re: [SciPy-dev] Summer Marathon "Category of the Week"
> To: "SciPy Developers List" <scipy-dev <at> scipy.org>
> Date: Thursday, July 2, 2009, 4:16 AM
> 2009/7/2 Nils Wagner <nwagner <at> iam.uni-stuttgart.de>:
> > Is there a chance to fix the following ticket
> > http://projects.scipy.org/numpy/ticket/937
> > in this context ?
> 
> Sure, I'd gladly review any patches (I'm a bit swamped
> myself at the moment).
> 
> Regards
> Stéfan
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev <at> scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
>
Permalink | Reply | 0 comments |
headers
David Goldsmith | 3 Jul 07:08
Picon
Favicon

Is this a (formatting) bug?


Actual output:

>>> from numpy import linalg as LA
>>> LA.eig(np.diag((1,2,3)))
(array([ 1.,  2.,  3.]), array([[ 1.,  0.,  0.],
       [ 0.,  1.,  0.],
       [ 0.,  0.,  1.]]))

i.e., a line feed wasn't inserted between w and v.

Bug?

DG
Permalink | Reply | 0 comments |
headers
David Goldsmith | 3 Jul 07:22
Picon
Favicon

Re: Is this a (formatting) bug?


I figured out a "work-around":

>>> w,v=LA.eig(np.diag((1,2,3)))
>>> w;v
array([ 1.,  2.,  3.])
array([[ 1.,  0.,  0.],
       [ 0.,  1.,  0.],
       [ 0.,  0.,  1.]])

But I'm still "concerned" about how >>> w,v (effectively) is printed (though I now see more clearly that
it's an artifact of how a tuple is printed, not an artifact of how LA.eig is printed, and thus probably
"unfixable" solely within numpy).

DG
--- On Thu, 7/2/09, David Goldsmith <d_l_goldsmith <at> yahoo.com> wrote:

> From: David Goldsmith <d_l_goldsmith <at> yahoo.com>
> Subject: [SciPy-dev] Is this a (formatting) bug?
> To: scipy-dev <at> scipy.org
> Date: Thursday, July 2, 2009, 10:08 PM
> 
> Actual output:
> 
> >>> from numpy import linalg as LA
> >>> LA.eig(np.diag((1,2,3)))
> (array([ 1.,  2.,  3.]), array([[ 1., 
> 0.,  0.],
>        [ 0.,  1.,  0.],
>        [ 0.,  0., 
> 1.]]))
> 
> i.e., a line feed wasn't inserted between w and v.
> 
> Bug?
> 
> DG
> 
> 
>       
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev <at> scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
>
Permalink | Reply | 0 comments |
headers
David Goldsmith | 3 Jul 07:50
Picon
Favicon

Example of eig not converging


What has to be true of `a` so that eig(a) fails to converge?

DG
Permalink | Reply | 1 comment |

Gmane