John Goodfellow | 4 Jun 2008 20:44
Picon

Possible memory leak?

Hi Y'all,

I have a long running text mode application (an Asterisk interface) which uses Urwid and Twisted. The library itself performs the functions I require, but I notice over time my little application consumes more and more memory.

For my main call list window, I started out using a SimpleListWalker and switched to the default of the ListBox - the PollingListWalker which solved one leak, but then I observed that whenever I opened an additional window, objects accumulated which were not released. The Python version I have tried this on is 2.5.2 and the 2.5 generation from Debian Etch.

I attach a sample which demonstrates the problem. Iterating using Test() produces the memory leak as more and more objects are tracked by the garbage collector but don't seem to be returned or collected and are yet not uncollectable. I created an alternative (Test2()) which simply allocates and then pops from a list. At the end of each procedure, the procedure's non externally referenced variables should be disposed, but the objects from Test() persist and grow.

I am sure there must be a very simple explanation why this happens. Perhaps someone could explain it.

Many thanks.

Attachment (test3.py): text/x-python, 1341 bytes
_______________________________________________
Urwid mailing list
Urwid <at> lists.excess.org
http://lists.excess.org/mailman/listinfo/urwid
John Goodfellow | 5 Jun 2008 20:49
Picon

Re: Urwid Digest, Vol 34, Issue 1

Hi,

Further to this, I designed a shorter test...

def Test3(idx):
  logging.debug("GC TRACKING %d OBJECTS" % len(gc.get_objects()))
  area = ui.get_cols_rows()
  content = Filler(Text("THIS IS A TEST STRING %10d" % idx, align="center"))
  ui.draw_screen(area, content.render(area))
  sleep(0.50)

Which still leaks by default. But, setting no cache rendering on stops
the leak, so I suspect that the caching code is hanging on to widgets.
Now to find out how to flush the cache...

On Thu, Jun 5, 2008 at 6:00 PM, <urwid-request <at> lists.excess.org> wrote:
>
> Send Urwid mailing list submissions to
>        urwid <at> lists.excess.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
>        http://lists.excess.org/mailman/listinfo/urwid
> or, via email, send a message with subject or body 'help' to
>        urwid-request <at> lists.excess.org
>
> You can reach the person managing the list at
>        urwid-owner <at> lists.excess.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of Urwid digest..."
>
>
> Today's Topics:
>
>   1. Possible memory leak? (John Goodfellow)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Wed, 4 Jun 2008 20:44:06 +0200
> From: "John Goodfellow" <4frqyivhwc <at> gmail.com>
> Subject: [Urwid] Possible memory leak?
> To: urwid <at> lists.excess.org
> Message-ID:
>        <7e55e2f00806041144i5e739c23ka7d48cb5ff749995 <at> mail.gmail.com>
> Content-Type: text/plain; charset="iso-8859-1"
>
> Hi Y'all,
>
> I have a long running text mode application (an Asterisk interface) which
> uses Urwid and Twisted. The library itself performs the functions I require,
> but I notice over time my little application consumes more and more memory.
>
> For my main call list window, I started out using a SimpleListWalker and
> switched to the default of the ListBox - the PollingListWalker which solved
> one leak, but then I observed that whenever I opened an additional window,
> objects accumulated which were not released. The Python version I have tried
> this on is 2.5.2 and the 2.5 generation from Debian Etch.
>
> I attach a sample which demonstrates the problem. Iterating using Test()
> produces the memory leak as more and more objects are tracked by the garbage
> collector but don't seem to be returned or collected and are yet not
> uncollectable. I created an alternative (Test2()) which simply allocates and
> then pops from a list. At the end of each procedure, the procedure's non
> externally referenced variables should be disposed, but the objects from
> Test() persist and grow.
>
> I am sure there must be a very simple explanation why this happens. Perhaps
> someone could explain it.
>
> Many thanks.
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: http://lists.excess.org/pipermail/urwid/attachments/20080604/7941414a/attachment-0001.html
> -------------- next part --------------
> A non-text attachment was scrubbed...
> Name: test3.py
> Type: text/x-python
> Size: 1340 bytes
> Desc: not available
> Url : http://lists.excess.org/pipermail/urwid/attachments/20080604/7941414a/attachment-0001.py
>
> ------------------------------
>
> _______________________________________________
> Urwid mailing list
> Urwid <at> lists.excess.org
> http://lists.excess.org/mailman/listinfo/urwid
>
>
> End of Urwid Digest, Vol 34, Issue 1
> ************************************
>
John Goodfellow | 5 Jun 2008 21:23
Picon

Re: Possible memory leak?

Alright, the following code solves the leak:

def Test3(idx):
  logging.debug("GC TRACKING %d OBJECTS" % len(gc.get_objects()))
  area = ui.get_cols_rows()
  txt = Text("THIS IS A TEST STRING %10d" % idx, align="center")
  fill = Filler(txt)
  canvas = fill.render(area)
  ui.draw_screen(area, canvas)
  txt._invalidate()
  sleep(0.50)

The _invalidate() method is called to remove the object from the
CanvasCache. Once that is done, no leak. But, this is quite tedious if
there are multi objects which can leak. There must be a better way -
maybe in a destructor for the widget base?

On Wed, Jun 4, 2008 at 8:44 PM, John Goodfellow <4frqyivhwc <at> gmail.com> wrote:
> Hi Y'all,
>
> I have a long running text mode application (an Asterisk interface) which
> uses Urwid and Twisted. The library itself performs the functions I require,
> but I notice over time my little application consumes more and more memory.
>
> For my main call list window, I started out using a SimpleListWalker and
> switched to the default of the ListBox - the PollingListWalker which solved
> one leak, but then I observed that whenever I opened an additional window,
> objects accumulated which were not released. The Python version I have tried
> this on is 2.5.2 and the 2.5 generation from Debian Etch.
>
> I attach a sample which demonstrates the problem. Iterating using Test()
> produces the memory leak as more and more objects are tracked by the garbage
> collector but don't seem to be returned or collected and are yet not
> uncollectable. I created an alternative (Test2()) which simply allocates and
> then pops from a list. At the end of each procedure, the procedure's non
> externally referenced variables should be disposed, but the objects from
> Test() persist and grow.
>
> I am sure there must be a very simple explanation why this happens. Perhaps
> someone could explain it.
>
> Many thanks.
>
>
Ian Ward | 5 Jun 2008 21:36
Favicon
Gravatar

Re: Possible memory leak?

That really shouldn't be necessary, there must be a problem with the way 
I'm using the WeakKeyDictionary in the canvas cache module.

I'll try to look look at it tonight.

Ian

John Goodfellow wrote:
> Alright, the following code solves the leak:
> 
> def Test3(idx):
>   logging.debug("GC TRACKING %d OBJECTS" % len(gc.get_objects()))
>   area = ui.get_cols_rows()
>   txt = Text("THIS IS A TEST STRING %10d" % idx, align="center")
>   fill = Filler(txt)
>   canvas = fill.render(area)
>   ui.draw_screen(area, canvas)
>   txt._invalidate()
>   sleep(0.50)
> 
> The _invalidate() method is called to remove the object from the
> CanvasCache. Once that is done, no leak. But, this is quite tedious if
> there are multi objects which can leak. There must be a better way -
> maybe in a destructor for the widget base?
> 
> On Wed, Jun 4, 2008 at 8:44 PM, John Goodfellow <4frqyivhwc <at> gmail.com> wrote:
>> Hi Y'all,
>>
>> I have a long running text mode application (an Asterisk interface) which
>> uses Urwid and Twisted. The library itself performs the functions I require,
>> but I notice over time my little application consumes more and more memory.
>>
>> For my main call list window, I started out using a SimpleListWalker and
>> switched to the default of the ListBox - the PollingListWalker which solved
>> one leak, but then I observed that whenever I opened an additional window,
>> objects accumulated which were not released. The Python version I have tried
>> this on is 2.5.2 and the 2.5 generation from Debian Etch.
>>
>> I attach a sample which demonstrates the problem. Iterating using Test()
>> produces the memory leak as more and more objects are tracked by the garbage
>> collector but don't seem to be returned or collected and are yet not
>> uncollectable. I created an alternative (Test2()) which simply allocates and
>> then pops from a list. At the end of each procedure, the procedure's non
>> externally referenced variables should be disposed, but the objects from
>> Test() persist and grow.
>>
>> I am sure there must be a very simple explanation why this happens. Perhaps
>> someone could explain it.
>>
>> Many thanks.
>>
>>
> 
> 
> _______________________________________________
> Urwid mailing list
> Urwid <at> lists.excess.org
> http://lists.excess.org/mailman/listinfo/urwid
> 
Ian Ward | 6 Jun 2008 18:54
Favicon
Gravatar

Re: Possible memory leak?

Ian Ward wrote:
> That really shouldn't be necessary, there must be a problem with the way 
> I'm using the WeakKeyDictionary in the canvas cache module.
> 
> I'll try to look look at it tonight.
> 

John, please try the attached patch and verify that it works for you.

Ian
Attachment (leakfix.1.diff): text/x-patch, 511 bytes
_______________________________________________
Urwid mailing list
Urwid <at> lists.excess.org
http://lists.excess.org/mailman/listinfo/urwid
John Goodfellow | 6 Jun 2008 21:24
Picon

Re: Possible memory leak?

Hi Ian,

Works like magic. Thanks a mil for the elegant solution. My long
running application will now actually run for a long time :)

Regards
J

Ian Ward wrote:
> John, please try the attached patch and verify that it works for you.
Rybarczyk Tomasz | 22 Jun 2008 18:37
Picon
Gravatar

nested frames -> footer contents change problem

Hi,

I'm trying to write some interface (similar to vifm) in which I need
to have some contents (for example listbox) and two footers which can
easily change contents. I thing that the simplest way is to just
create two nested urwid.Frames. Everything seems ok but when I'm
trying to change contents of footer of inner Frame it prints the
contents on footer of outer frame....
Example code should explain what I mean. What am I doing wrong?

import urwid.raw_display
import urwid
import sys
import os

ui = urwid.raw_display.Screen()

def run():
    some_text_list = map(urwid.Text, ("this is some content of list".split()))
    listbox = urwid.ListBox(some_text_list)

    inner_foot = urwid.Text("inner_foot")
    inner_head = urwid.Text("inner_head")

    inner_frame = urwid.Frame(body=listbox, footer=inner_foot,
header=inner_head)

    outer_foot = urwid.Text("outer_foot")
    outer_head = urwid.Text("outer_head")
    outer_frame = urwid.Frame(body=inner_frame, footer=outer_foot,
header=outer_head)

    size = ui.get_cols_rows()
    canvas = outer_frame.render(size, focus=1)
    ui.draw_screen(size, canvas)

    while True:
        key = ui.get_input()
        if key:
            outer_foot.set_text(outer_foot.text + outer_foot.text)
            #when you change only inner_foot everything is ok -
uncomment line below and comment out line above for check.
#            inner_foot.set_text(inner_foot.text + inner_foot.text)

            size = ui.get_cols_rows()

            canvas = outer_frame.render(size, focus=1)
            ui.draw_screen(size, canvas)

ui.run_wrapper(run)

Best regards
paluh
Attachment (nested_frames.py): text/x-python, 1029 bytes
_______________________________________________
Urwid mailing list
Urwid <at> lists.excess.org
http://lists.excess.org/mailman/listinfo/urwid
Rybarczyk Tomasz | 22 Jun 2008 18:42
Picon
Gravatar

Re: nested frames -> footer contents change problem

Sorry I've commented wrong line in this code sample. should be
commented out in opposite way ;-) to achive described problem. I mean:

    while True:
        key = ui.get_input()
        if key:
#            outer_foot.set_text(outer_foot.text + outer_foot.text)
            inner_foot.set_text(inner_foot.text + inner_foot.text)

On Sun, Jun 22, 2008 at 6:37 PM, Rybarczyk Tomasz <paluho <at> gmail.com> wrote:
> Hi,
>
> I'm trying to write some interface (similar to vifm) in which I need
> to have some contents (for example listbox) and two footers which can
> easily change contents. I thing that the simplest way is to just
> create two nested urwid.Frames. Everything seems ok but when I'm
> trying to change contents of footer of inner Frame it prints the
> contents on footer of outer frame....
> Example code should explain what I mean. What am I doing wrong?
>
>
> import urwid.raw_display
> import urwid
> import sys
> import os
>
> ui = urwid.raw_display.Screen()
>
> def run():
>    some_text_list = map(urwid.Text, ("this is some content of list".split()))
>    listbox = urwid.ListBox(some_text_list)
>
>    inner_foot = urwid.Text("inner_foot")
>    inner_head = urwid.Text("inner_head")
>
>    inner_frame = urwid.Frame(body=listbox, footer=inner_foot,
> header=inner_head)
>
>    outer_foot = urwid.Text("outer_foot")
>    outer_head = urwid.Text("outer_head")
>    outer_frame = urwid.Frame(body=inner_frame, footer=outer_foot,
> header=outer_head)
>
>    size = ui.get_cols_rows()
>    canvas = outer_frame.render(size, focus=1)
>    ui.draw_screen(size, canvas)
>
>    while True:
>        key = ui.get_input()
>        if key:
>            outer_foot.set_text(outer_foot.text + outer_foot.text)
>            #when you change only inner_foot everything is ok -
> uncomment line below and comment out line above for check.
> #            inner_foot.set_text(inner_foot.text + inner_foot.text)
>
>            size = ui.get_cols_rows()
>
>            canvas = outer_frame.render(size, focus=1)
>            ui.draw_screen(size, canvas)
>
> ui.run_wrapper(run)
>
> Best regards
> paluh
>
Attachment (nested_frames.py): text/x-python, 1029 bytes
_______________________________________________
Urwid mailing list
Urwid <at> lists.excess.org
http://lists.excess.org/mailman/listinfo/urwid
Ian Ward | 22 Jun 2008 20:10
Favicon
Gravatar

Re: nested frames -> footer contents change problem

Rybarczyk Tomasz wrote:
> Sorry I've commented wrong line in this code sample. should be
> commented out in opposite way ;-) to achive described problem. I mean:
> 
> On Sun, Jun 22, 2008 at 6:37 PM, Rybarczyk Tomasz <paluho <at> gmail.com> wrote:
>> Hi,
>>
>> I'm trying to write some interface (similar to vifm) in which I need
>> to have some contents (for example listbox) and two footers which can
>> easily change contents. I thing that the simplest way is to just
>> create two nested urwid.Frames. Everything seems ok but when I'm
>> trying to change contents of footer of inner Frame it prints the
>> contents on footer of outer frame....
>> Example code should explain what I mean. What am I doing wrong?

I can't seem to reproduce your problem with Urwid 0.9.8.2 or 0.9.7.2,
what version are you running?

Ian
Ian Ward | 22 Jun 2008 23:10
Favicon
Gravatar

Re: Colored border class question

Ian Ward wrote:
> Joern Koerner wrote:
>> Hi everyone!
>>
>> Based on some posts to this list I've created a Border class, that draws ba border with title around
widgets. So far so good. 
>>
>> My problem is, when I tray to change the borders color,  only the top line and the bottom line are being
ignored. They simply does not change the color and I can 
>> not figure out where the bug is....
>> Perhaps someone could give me a hint what is wrong?!
>>
> 
> Interesting, it seems to be a Columns caching bug.  The colour changes
> when you change the width of the window and it is forced to be redrawn.
> 
> I'll have a look and let you know.

Please try the attached patch and verify that it fixes the problem you
found.

Ian
Attachment (fill_attr_bug.patch): text/x-diff, 713 bytes
_______________________________________________
Urwid mailing list
Urwid <at> lists.excess.org
http://lists.excess.org/mailman/listinfo/urwid

Gmane