Plotting large images

I am trying to use  imshow to plot some semi-large fits images.
Here is the code:

from math import *
import numpy as np
from matplotlib import pyplot as plt
from matplotlib import cm as cm
import pyfits

cat = "/Volumes/Sweden/PS1SC/Data/PS20090603-3/MD09/skycell.092/"
fname = "o4985g0263o.warp.MD09.skycell.092"

hdu = pyfits.open(cat+fname+".fits")
print hdu.info()
img = hdu[1].data.astype("int")

plt.figure(figsize=[12,12])
plt.imshow(img,cmap=cm.cool)
plt.savefig("test.png")

Which gives the result:

Filename: /Volumes/Sweden/PS1SC/Data/PS20090603-3/MD09/skycell.092/ 
o4985g0263o.warp.MD09.skycell.092.fits
No.    Name         Type      Cards   Dimensions   Format
0    PRIMARY     PrimaryHDU       6  ()            int16
1                CompImageHDU   101  (6000, 6000)  float32
None
Python(23117,0xa04f2720) malloc: *** mmap(size=1152000000) failed  
(error code=12)

Re: Plotting large images

Because the image is so large, and matplotlib carries out various  
operations on the image (scaling, resampling, etc), it uses a lot of  
memory. This is one area where a specialized display program will be  
more efficient. If you need to use matplotlib, decide whether you  
really only want to display a subsection, or only need a lower  
resolution version (e.g., boxcar smooth and subsample) before  
displaying. I've found that image sizes well over 1kx1k can take some  
time to display, and those that are much larger can cause you to run  
out of memory. At least, that's what I think is happening.

Perry

On Jun 30, 2009, at 7:20 PM, Tommy Grav wrote:

> I am trying to use  imshow to plot some semi-large fits images.
> Here is the code:
>
> from math import *
> import numpy as np
> from matplotlib import pyplot as plt
> from matplotlib import cm as cm
> import pyfits
>
> cat = "/Volumes/Sweden/PS1SC/Data/PS20090603-3/MD09/skycell.092/"
> fname = "o4985g0263o.warp.MD09.skycell.092"
>
> hdu = pyfits.open(cat+fname+".fits")
> print hdu.info()
> img = hdu[1].data.astype("int")
>

Re: Plotting large images

That is what I was assuming, but it still seems a little odd that  
matplotlib generates
that large of a memory footprint. Loading the fits file into the  
program using pyfits,
with the code only uses 19MB of real memory and 600MB of virtual  
memory (strangly
adding the line img = hdu[1].data, increases this to 208MB/800MB).

Displaying images of various sizes I get these
numbers from Activity Monitor

Size                 Real Mem      Virtual
3k x 3k           0.68GB            1.57GB
4k x 4k		0.92GB            1.80GB
5k x 5k		1.20GB	          2.10GB
5.5k x 5.5k     1.38GB            2.28GB

And the limit seems to be somewhere just above 5.5k by 5.5k (darn :( )

Cheers
   Tommy

On Jun 30, 2009, at 7:27 PM, Perry Greenfield wrote:

> Because the image is so large, and matplotlib carries out various  
> operations on the image (scaling, resampling, etc), it uses a lot of  
> memory. This is one area where a specialized display program will be  
> more efficient. If you need to use matplotlib, decide whether you  
> really only want to display a subsection, or only need a lower  
> resolution version (e.g., boxcar smooth and subsample) before  

Re: Plotting large images

On Jun 30, 2009, at 7:54 PM, Tommy Grav wrote:

> That is what I was assuming, but it still seems a little odd that
> matplotlib generates
> that large of a memory footprint. Loading the fits file into the
> program using pyfits,
> with the code only uses 19MB of real memory and 600MB of virtual
> memory (strangly
> adding the line img = hdu[1].data, increases this to 208MB/800MB).
>
The reason for this is that pyfits doesn't actually load the data  
until you 'touch' the data attribute (to minimize memory, particularly  
if you just are interested in the header information).

As for the memory footprint of matplotlib, in order to be able to  
resize and handle interactive updates, it has to retain references to  
the original image, perhaps as well to intermediate products (and  
these references won't be memory collected until you clear the figure  
(e.g., clf()). It's one of the prices for flexibility and generality.  
It probably would take a lot of complexity to optimize it for large  
images (but John is better suited to answer this conclusively).

Perry

> Displaying images of various sizes I get these
> numbers from Activity Monitor
>
> Size                 Real Mem      Virtual
> 3k x 3k           0.68GB            1.57GB

Re: Tutorial topics for SciPy'09 Conference

Hi,

On Mon, Jun 1, 2009 at 10:20 PM, Fernando Perez<fperez.net@...> wrote:
> The time for the Scipy'09 conference is rapidly approaching, and we
> would like to both announce the plan for tutorials and solicit
> feedback from everyone on topics of interest.

rather than rehash much here, where it's not easy to paste a table,
I've posted a note with the poll results here:

http://fdoperez.blogspot.com/2009/06/scipy-advanced-tutorials-results.html

The short and plain-text-friendly version is the final topic ranking:

1	Advanced topics in matplotlib use
2	Advanced numpy
3	Designing scientific interfaces with Traits
4	Mayavi/TVTK
5	Cython
6	Symbolic computing with sympy
7	Statistics with Scipy
8	Using GPUs with PyCUDA
9	Testing strategies for scientific codes
10	Parallel computing in Python and mpi4py
11	Sparse Linear Algebra with Scipy
12	Structured and record arrays in numpy
13	Design patterns for efficient iterator-based scientific codes
14	Sage
15	The TimeSeries scikit
16	Hermes: high order Finite Element Methods

Re: Zooming plot and transforms...

On Tue, Jun 30, 2009 at 14:12, Fabrice Silva<silva@...> wrote:
> Le lundi 29 juin 2009 à 16:11 -0400, Jae-Joon Lee a écrit :
>> In the svn version of matplotlib, there are some helper classes to
>> ease this job a bit.
> Thanks for your pointer. Sadly the mpl.toolkits.axes_grid is not shipped
> by debian package, and downloading it requires other stuff. So I adapted

I'm the debian maintainer for matplotlib: if you need something
missing in Debian, get in touch with us, for example reporting a bug
against matplotlib requesting this toolkit.

I didn't check further, but probably it was not release because of
this phrase: "In the svn version of matplotlib".

Regards,
--

-- 
Sandro Tosi (aka morph, morpheus, matrixhasu)
My website: http://matrixhasu.altervista.org/
Me at Debian: http://wiki.debian.org/SandroTosi

------------------------------------------------------------------------------

log scale the Y axis of a stem plot

------------------------------------------------------------------------------

_______________________________________________
Matplotlib-users mailing list
Matplotlib-users@...
https://lists.sourceforge.net/lists/listinfo/matplotlib-users

Remove Patch Rectangle Around Plot

Hopefully a really simple question.  How do I remove the "box" (the
black rectangle) around the plot.

I tried

ax = fig.add_subplot(111)
ax.patch.set_visible(False)

but this makes the entire patch invisible.

Thanks,

Mark

------------------------------------------------------------------------------

ImportError: No module named mplot3d

------------------------------------------------------------------------------

_______________________________________________
Matplotlib-users mailing list
Matplotlib-users@...
https://lists.sourceforge.net/lists/listinfo/matplotlib-users

Re: Remove Patch Rectangle Around Plot

Mark Larsen wrote:
> Hopefully a really simple question.  How do I remove the "box" (the
> black rectangle) around the plot.
> 
> I tried
> 
> ax = fig.add_subplot(111)
> ax.patch.set_visible(False)
> 
> but this makes the entire patch invisible.
> 
> Thanks,
> 
> Mark
> 
> ------------------------------------------------------------------------------

I think what you want is

ax = fig.add_subplot(111,frameon=False)

JLS

------------------------------------------------------------------------------