headers
Tomas Stanek | 4 Sep 2008 12:05
Picon
Favicon

garbage collector issues

Hello,

I've encountered quite large memory consumption of gauche under some
circumstances. My program reads an xml document, converts it into a tree
with lots of flonums (sometimes about 20000), do some work with it and 
continues to another document. The program also allocates few large 
(up to a megabyte) blocks and lot of pairs/flonums/etc during each 
iteration.

Problem is, that it's memory consuption grows up to 200 - 300MB and 
never goes down.

I've noticed two things...

First,
Scm_MakeFlonum uses SCM_NEW (not SCM_NEW_ATOMIC) for allocation of the 
ScmFlonum, which is pointer free. I expectd this to be a "typo".

Changing SCM_NEW to SCM_NEW_ATOMIC improves the situation a bit, but 
not as much.

Second,
Boehm Weiser GC is by default configured not to use -DUSE_MMAP
and -DUSE_MUNMAP flags. This way, GC doesn't ever return allocated
memory to the operating system.

By using -DUSE_MMAP and -DUSE_MUNMAP, memory consumption keeps around
20 - 30MB. I'm not sure why -DUSE_M[UN]MAP makes a such difference. My
speculation is that GC's freelists are poluted with large blocks which
are not very usable for future allocations and GC tends to grow heap.
(Continue reading)

Permalink | Reply | 2 comments |
headers
Shiro Kawai | 4 Sep 2008 12:58
Favicon

Re: garbage collector issues

Thanks, Tomas.

From: Tomas Stanek <sad0ur <at> psi.cz>
Subject: [Gauche-devel] garbage collector issues
Date: Thu, 4 Sep 2008 12:05:44 +0200

> Scm_MakeFlonum uses SCM_NEW (not SCM_NEW_ATOMIC) for allocation of the 
> ScmFlonum, which is pointer free. I expectd this to be a "typo".

I think you're correct.  It doesn't need to be SCM_NEW
(technically ScmFlonum has a pointer to the <flonum> class, but
it is allocated staticallly and will never be GCed, so we don't
need to consider it.)

In a branch of the source tree I implemented a different treatment
of flonums, in which ScmFlonum is only 8 bytes (it's just double).
It also uses a sort of specialized generational GC to reduce overhead
of ephemeral flonums.  (The detail is in my DLS2008 paper which will
be available on ACM site sometime in future).

> Second,
> Boehm Weiser GC is by default configured not to use -DUSE_MMAP
> and -DUSE_MUNMAP flags. This way, GC doesn't ever return allocated
> memory to the operating system.
> 
> By using -DUSE_MMAP and -DUSE_MUNMAP, memory consumption keeps around
> 20 - 30MB. I'm not sure why -DUSE_M[UN]MAP makes a such difference. My
> speculation is that GC's freelists are poluted with large blocks which
> are not very usable for future allocations and GC tends to grow heap.
>
(Continue reading)

Permalink | Reply | 1 comment |
headers
Tomas Stanek | 4 Sep 2008 13:54
Picon
Favicon

Re: garbage collector issues

On Thu 04.09 (00:58), Shiro Kawai wrote:
> Thanks, Tomas.
> 
> From: Tomas Stanek <sad0ur <at> psi.cz>
> Subject: [Gauche-devel] garbage collector issues
> Date: Thu, 4 Sep 2008 12:05:44 +0200
> 
> > Scm_MakeFlonum uses SCM_NEW (not SCM_NEW_ATOMIC) for allocation of the 
> > ScmFlonum, which is pointer free. I expectd this to be a "typo".
> 
> I think you're correct.  It doesn't need to be SCM_NEW
> (technically ScmFlonum has a pointer to the <flonum> class, but
> it is allocated staticallly and will never be GCed, so we don't
> need to consider it.)

> In a branch of the source tree I implemented a different treatment
> of flonums, in which ScmFlonum is only 8 bytes (it's just double).
> It also uses a sort of specialized generational GC to reduce overhead
> of ephemeral flonums.  (The detail is in my DLS2008 paper which will
> be available on ACM site sometime in future).
Is this branch already available, or will this technique be used in
future releases of gauche? I would especially be interested if there
would be some relatively cheap technique (similar to integer shifting)
which would distinguish those doubles from potential pointers.

> > Second,
> > Boehm Weiser GC is by default configured not to use -DUSE_MMAP
> > and -DUSE_MUNMAP flags. This way, GC doesn't ever return allocated
> > memory to the operating system.
> >
(Continue reading)

Permalink | Reply | 0 comments |

Gmane