1 Dec 05:52
Re: Memory leaks in Basemap
Thanks for reply,
This was just an example for mem leak in my application. Actually i want maps with different data to be loaded each time and the map objects taken once changees after ploting so i have to take different object each time. After around 20 maps generation it gives memory error to me.
thanks,
-Aalok
Jeff Whitaker <jswhit-97jfqw80gc6171pxa8y+qA@public.gmane.org> wrote:
Aalok kapoor wrote:
> Hi all,
>
> I am using matplotlib-0.87.7, Basemap-0.9.3, Numpy-1.1 and agg backend.
> I am facing memory leak problems after running following script -
>
>
> import os, sys, time
> import cStringIO
> import Image
> import matplotlib
> matplotlib.use('Agg')
>
> import matplotlib.pylab as p
> import matplotlib.numerix as nx
> from matplotlib.toolkits.basemap import Basemap
>
> def get_image_bytes(width_fig, height_fig, str_img):
> """Returns the bytes representing the image generated
>
> The createImage method should be called before calling this method.
>
> :Returns:
> Byte data.
> """
> # Get the actual image data
> f = cStringIO.StringIO()
> dims = (width_fig, height_fig)
> im = Image.fromstring("RGB", dims, str_img)
> im.save(f, "JPEG")
> return f.getvalue()
>
> def report_memory(i):
> pid = os.getpid()
> a2 = os.popen('ps -p %d -o rss,vsz,%%mem' % pid).readlines()
> print i, ' ', a2[1],
> return int(a2[1].split()[1])
>
>
>
> # take a memory snapshot on indStart and compare it with indEnd
> indStart, indEnd = 30, 150
> print ' rss vsz %mem'
> for i in range(indEnd):
> p.figure(figsize=(float(640)/80.0,float(480)/80.0),
> facecolor='w', edgecolor='w')
> p.axes([0.0, 0.0, 1.0, 1.0])
> map1 = Basemap(projection='cyl', lat_0=50, lon_0=-100,
> area_thresh=1000.)
> # draw coastlines, country boundaries, fill continents.
> map1.drawcoastlines(linewidth=.5)
> map1.drawcountries()
> map1.fillcontinents(color="#CEFFCE")
> # draw the edge of the map projection region (the projection limb)
> map1.drawmapboundary()
> # compute the native map projection coordinates for cities.
> lats, lons = ([18.728], [20.890])
> x,y = map1(lons,lats)
> # plot filled circles at the locations of the cities.
> map1.plot(x,y,'bo')
> canvas = p.get_current_fig_manager().canvas
> agg = canvas.switch_backends \
> (matplotlib.backends.backend_agg.FigureCanvasAgg)
> agg.draw() # Takes ~.9 seconds
> # get the image data
> str_img = agg.tostring_rgb()
> width_fig, height_fig = map(int, agg.figure.bbox.get_bounds()[2:])
> p.cla()
> p.close()
> bytes = get_image_bytes(width_fig, height_fig, str_img)
>
>
> val = report_memory(i)
> # wait a few cycles for memory usage to stabilize
> if i==indStart: start = val
>
> end = val
> print 'Average memory consumed per loop: %1.4fk bytes' % \
> ((end-start)/float(indEnd-indStart))
>
>
>
>
> Here is the O/P
>
> $python memory_leak_map.py
> rss vsz %mem
> 0 47272 50632 9.7
> 1 74412 77700 15.3
> 2 93960 97380 19.3
> 3 113308 116776 23.3
> 4 132824 136416 27.3
> 5 152352 155828 31.3
> 6 171860 175216 35.3
> 7 191372 194868 39.3
> 8 210872 214248 43.3
> 9 230336 233916 47.3
> 10 249732 253284 51.3
> 11 269252 272692 55.3
> 12 288680 292336 59.3
> 13 308108 311724 63.2
> 14 305160 331112 62.6
> 15 301096 350764 61.8
> 16 304884 370160 62.6
> 17 298276 389804 61.2
> 18 305876 409184 62.8
> 19 298316 428596 61.2
> 20 307856 448224 63.2
> 21 308004 467640 63.2
> 22 308844 487016 63.4
> 23 306260 506656 62.9
> 24 300612 526052 61.7
> Traceback (most recent call last):
> File "memory_leak_map.py", line 41, in ?
> area_thresh=1000.)
> File
> "/usr/local/lib/python2.4/site-packages/matplotlib/toolkits/basemap/basemap.py",
> line 815, in __init__
> self.riversegs = segments+segments2+segments3
> MemoryError
> Killed: 9
>
>
> Can someone please help me out solving this problem?
>
>
> Thanks
> -Aalok
>
Aaolk: All your plots use the same map projection, yet you are
initializing a new Basemap instance every time through the loop. Try
moving the line
map1 = Basemap(projection='cyl', lat_0=50, lon_0=-100,
area_thresh=1000.)
outside the loop.
-Jeff
--
Jeffrey S. Whitaker Phone : (303)497-6313
Meteorologist FAX : (303)497-6449
NOAA/OAR/PSD R/PSD1 Email : Jeffrey.S.Whitaker-32lpuo7BZBA@public.gmane.org
325 Broadway Office : Skaggs Research Cntr 1D-124
Boulder, CO, USA 80303-3328 Web : http://tinyurl.com/5telg
Find out what India is talking about on - Yahoo! Answers India
Send FREE SMS to your friend's mobile from Yahoo! Messenger Version 8. Get it NOW
------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________ Matplotlib-devel mailing list Matplotlib-devel@... https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
Its not the most intuitive way to do it,
> but it works and I can't see any major drawbacks.
>
>> I am not sure why only the top is colored in your example--it might be a
>> bug or it might be an indication that additional kwargs are needed. I
>> am reasonably sure there is a simple solution, but I can't look at it
>> any more right now--maybe I can get back to it this evening.
>
> I'm pretty sure the reason only the top was coloured is to do in the
> inconsistency I described above. Once I fixed that the colorbar is
> fine except that it is not on a log scale. But of course it can't know
> that it is suppost to be on a log scale! I tried to do:
>
> gca().axes[1].set_ylim((1e-5,1))
> gca().axes[1].set_yscale('log')
>
> but that doesn't work. I get a load of errors :
>
> exceptions.ValueError Traceback (most
> recent call last)
>
> /usr/lib/python2.4/site-packages/matplotlib/backends/backend_gtk.py in
> expose_event(self, widget, event)
> 282 x, y, w, h = self.allocation
> 283 self._pixmap_prepare (w, h)
> --> 284 self._render_figure(self._pixmap, w, h)
> 285 self._need_redraw = False
> 286
>
> /usr/lib/python2.4/site-packages/matplotlib/backends/backend_gtkagg.py
> in _render_figure(self, pixmap, width, height)
> 71 def _render_figure(self, pixmap, width, height):
> 72 if DEBUG: print 'FigureCanvasGTKAgg.render_figure'
> ---> 73 FigureCanvasAgg.draw(self)
> 74 if DEBUG: print 'FigureCanvasGTKAgg.render_figure
> pixmap', pixmap
> 75 #agg_to_gtk_drawable(pixmap, self.renderer._renderer, None)
>
> /usr/lib/python2.4/site-packages/matplotlib/backends/backend_agg.py in
> draw(self)
> 390
> 391 renderer = self.get_renderer()
> --> 392 self.figure.draw(renderer)
> 393
> 394 def get_renderer(self):
>
> /usr/lib/python2.4/site-packages/matplotlib/figure.py in draw(self,
> renderer)
> 542
> 543 # render the axes
> --> 544 for a in self.axes: a.draw(renderer)
> 545
> 546 # render the figure text
>
> /usr/lib/python2.4/site-packages/matplotlib/axes.py in draw(self,
> renderer, inframe)
> 1061
> 1062 for zorder, i, a in dsu:
> -> 1063 a.draw(renderer)
> 1064
> 1065 self.transData.thaw() # release the lazy objects
>
> /usr/lib/python2.4/site-packages/matplotlib/patches.py in draw(self,
> renderer)
> 163
> 164 verts = self.get_verts()
> --> 165 tverts = self._transform.seq_xy_tups(verts)
> 166
> 167 renderer.draw_polygon(gc, rgbFace, tverts)
>
> ValueError: Domain error on nonlinear Transformation::seq_xy_tups
> operator()(thisx, thisy)
>
> As for how to solve the problem properly. Matlab allows one to set to
> caxis scale to log. Maybe colorbar could detect that the norm instance
> was an instance of LogNorm and scale the yaxis logarithmicly. Or would
> it be better to put a scale={'log','linear'} kwarg into colorbar()?
>
> Thanks again for your help.
>
> cheers
>
> JIM
> ---
-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
RSS Feed