Leaking Boehm GC on CE 6

Hello!

Many years ago (back in 2003 to be exact) Hans Boehm and others helped
us get Boehm-GC working correctly on gcj 3.3 on arm-wince-pe (Windows
CE 3.0, at that time).  Everything - particularly that modified-for-CE
build of Boehm GC - has been working extremely well until recently.
We are extremely grateful for both the help we received then and the
reliable and performant operation that Boehm GC has delivered over the
years.

Then along came Windows CE 6.  gcj/boehm still work without crashing,
but on CE 6 we are experiencing severe leaking.  Specifically: a gcj
binary that works perfectly (without leaking) on Windows CE 4/5
exhibits fairly large memory leaks when run on CE 6.

This is baffling to say the very least.  As far as I can tell, the
memory map on Windows CE 6 is the same and I'm not aware of any reason
WHY the Boehm GC would leak on CE 6 when it does not on other earlier
versions.

I am gearing up for what I expect will be a long and difficult project
to find and fix this problem.  I need some ideas about where I could
look for the problem, and some idea of whether there might be fixes
for the GC in versions after 6.2 that might be relevant.  I have
ported the "test.c" that's used to build gctest to arm-wince-pe
(GNUWINCE), and here is the output from gctest.exe on Windows Mobile 5
(no leaking):

Completed 1 tests
Allocated 560621 collectable objects

RE: Leaking Boehm GC on CE 6

I thought WinCE 6 totally changed the memory map, going from the awful 32 by 32 MB slot with the rest shared
architecture of previous WinCE to a proper 2/2 GB split protected memory system.

> -----Original Message-----
> From: gc-bounces@... [mailto:gc-
> bounces@...] On Behalf Of Craig Vanderborgh
> Sent: 12 January 2009 05:12
> To: gc@...
> Subject: [Gc] Leaking Boehm GC on CE 6
> 
> Hello!
> 
> Many years ago (back in 2003 to be exact) Hans Boehm and others helped
> us get Boehm-GC working correctly on gcj 3.3 on arm-wince-pe (Windows
> CE 3.0, at that time).  Everything - particularly that modified-for-CE
> build of Boehm GC - has been working extremely well until recently.
> We are extremely grateful for both the help we received then and the
> reliable and performant operation that Boehm GC has delivered over the
> years.
> 
> Then along came Windows CE 6.  gcj/boehm still work without crashing,
> but on CE 6 we are experiencing severe leaking.  Specifically: a gcj
> binary that works perfectly (without leaking) on Windows CE 4/5
> exhibits fairly large memory leaks when run on CE 6.
> 
> This is baffling to say the very least.  As far as I can tell, the
> memory map on Windows CE 6 is the same and I'm not aware of any reason
> WHY the Boehm GC would leak on CE 6 when it does not on other earlier
> versions.
> 

RE: Leaking Boehm GC on CE 6

I'm pretty sure that it is shared.  MS says that it is shared.  On WinCE 5 (WinMob 5, 6) there is a total of 4 GiB
address space. The top 2 GiB is off-limits and belongs to the kernel. The bottom 32 MiB is slot 0.  The next 1
GiB is the 32 application slots (so that finishes at 1 GiB + 32 MiB).  The remainder (from 1 GiB + 32 MiB up to 2
GiB) is shared.  Each application gets its own 32 MiB private chunk, and can do further allocations from the
shared space (e.g. VirtualAlloc requesting >2 MiB).  The entire address range is shared (read and write),
which makes IPC much easier, but also results in inadequate memory protection. 

In WinCE 6 (which won't be used 'til WinMob 7, sadly), the top 2 GiB is off-limits kernel memory, and the
bottom 2 GiB is totally private to each application.

References:
http://msdn.microsoft.com/en-us/library/aa450572.aspx
http://msdn.microsoft.com/en-us/library/aa914933.aspx
http://msdn.microsoft.com/en-us/library/aa450975.aspx

etc.

Now, whether or not the change from bottom 2 GiB shared to bottom 2 GiB private has repercussions for the GC, I
don't know.  But it wouldn't surprise me.

> -----Original Message-----
> From: Craig Vanderborgh [mailto:craigvanderborgh@...]
> Sent: 12 January 2009 16:16
> To: DrPizza
> Subject: Re: [Gc] Leaking Boehm GC on CE 6
> 
> I don't believe the "rest shared" is the case on CE/WM 5.  There is
> indeed a 32MB limit for the "low" part of the address space.  But the
> upper address space - the LMA - is not shared.  We tweaked Boehm GC so
> that it would allocate all of its blocks from the so-called "Large

GC_merge_unmapped: assertion violation

Hi!

I sometimes get GC_ASSERT(!IS_MAPPED(nexthdr)) violation in GC_merge_unmapped().

Could somebody tell me, please, whether this is wrong assertion (i.e. there may be two adjacent mapped free
blocks at this point) or the problem's somewhere else?

My config is GC_WIN32_THREADS+USE_MUNMAP+GC_ASSERTIONS and GC_enable_incremental() is called at start-up.

Bye.

Bug fix for GC_allochblk

Hi!

The current algorithm of avoiding blocks splitting in case of USE_MUNMAP in GC_allochblk() may lead
(under some rare circumstances) the situation when GC_alloc_large() (which calls GC_allochblk() and
GC_collect_or_expand() in a loop) grows the heap to the limit and returns NULL (because GC_allochblk()
tries only an exact match). This situation may arise only (but I'm not exactly sure for this) in the
generational (not "true" incremental) mode.

So, I suggest a small patch for GC_allochblk(). (May be, someone else suggests a better strategy of
choosing "split_limit" value, or suggests some other solution).

Bye.

Bug fix for GC_allochblk - the patch

Sorry, I forgot to attach the patch itself.
Here is it.

Bye.

diff -ru bdwgc/allchblk.c updated/bdwgc/allchblk.c
--- bdwgc/allchblk.c	2008-11-23 19:08:54.000000000 +0300
+++ updated/bdwgc/allchblk.c	2009-01-18 18:17:00.016563000 +0300
 <at>  <at>  -596,6 +596,11  <at>  <at> 
 #     ifdef USE_MUNMAP
 	/* avoid splitting, since that might require remapping */
 	split_limit = 0;
+
+	/* Don't avoid splitting in case of generational mode since	*/
+	/* there may be no exact match even after heap expansion.	*/
+	if (GC_incremental && GC_finalizer_bytes_freed <= (GC_heapsize >> 4))
+	  split_limit = N_HBLK_FLS;
 #     else
 	if (GC_finalizer_bytes_freed > (GC_heapsize >> 4))  {
 	  /* If we are deallocating lots of memory from		*/

_______________________________________________
Gc mailing list
Gc@...
http://www.hpl.hp.com/hosted/linux/mail-archives/gc/

Re: [PATCH] FIXUP_POINTER compilation

Hi!

Adam Warner <lists@...> wrote:
> Hi Hans,
> 
> Line 1341 of mark.c contains this macro:
> GC_PUSH_ONE_STACK((ptr_t)p, MARKED_FROM_REGISTER);
> 
> When NEED_FIXUP_POINTER is defined the GC_PUSH_ONE_STACK includes the 
> macro expansion of FIXUP_POINTER(p)
> 
> Substituting (ptr_t)p into FIXUP_POINTER(p) results in:
> ((ptr_t)p) = ...
> The cast should be in the right hand expression, not the lvalue.
> 
> Depending upon the context a right hand cast should either be to a word 
> pointer or a char pointer (i.e. to match the type of the variable being 
> reassigned). I have fixed this using typeof(). If you want to avoid the 
> use of typeof() then my prior testing indicates that twin versions of any 
> macro that contain FIXUP_POINTER() will be required.
> 
> I also added casts in gc_pmark.h that were missing from the 
> NEED_FIXUP_POINTER definition when compared with the !NEED_FIXUP_POINTER 
> definition.
> 
> Compilation primarily tested with:
> make clean &&
> CFLAGS="-Werror -DPOINTER_SHIFT=0" ./configure --enable-threads=posix --
> enable-parallel-mark --enable-large-config &&
> make &&

gc v7.2a1 tarball

Hi!

The BDWGC CVS hasn't been changed for about 2 months. The recent tar-ball (v7.1) is dated of May 2008.

So, for convenience, I've created a tar-ball containing the current CVS snapshot (dated Dec 2008) plus a
number of pending patches (dated Oct-Jan 2008). There are, however, a number of hot FIXME present (like in
GC_merge_unmapped()), but they haven't led to a crash in my practice. My mostly tested config is
LARGE_CONFIG + GC_THREADS + THREAD_LOCAL_ALLOC + PARALLEL_MARK + USE_MUNMAP on Win32 (both VC++ 8 and
MinGW gcc v4.2.1).

For the interested parties, this tar-ball is at:
http://www.ivmaisoft.com/sources/jcgo/bdwgc72a1-20090119-ivmai.tar.bz2

Bye.

RE: gc v7.2a1 tarball

Thanks.  I expect to have a bit more time to catch up with this shortly. 

Hans

> -----Original Message-----
> From: gc-bounces@... 
> [mailto:gc-bounces@...] On Behalf Of Ivan Maidanski
> Sent: Friday, January 30, 2009 2:21 PM
> To: gc@...
> Subject: [Gc] gc v7.2a1 tarball
> 
> Hi!
> 
> The BDWGC CVS hasn't been changed for about 2 months. The 
> recent tar-ball (v7.1) is dated of May 2008.
> 
> So, for convenience, I've created a tar-ball containing the 
> current CVS snapshot (dated Dec 2008) plus a number of 
> pending patches (dated Oct-Jan 2008). There are, however, a 
> number of hot FIXME present (like in GC_merge_unmapped()), 
> but they haven't led to a crash in my practice. My mostly 
> tested config is LARGE_CONFIG + GC_THREADS + 
> THREAD_LOCAL_ALLOC + PARALLEL_MARK + USE_MUNMAP on Win32 
> (both VC++ 8 and MinGW gcc v4.2.1).
> 
> For the interested parties, this tar-ball is at:
> http://www.ivmaisoft.com/sources/jcgo/bdwgc72a1-20090119-ivmai.tar.bz2
> 
> Bye.
> _______________________________________________

[PATCH] Allow registration of disappearing links on non-heap objects

Hello,

In 7.1, registering disappearing links on non-heap objects leads to a
segmentation fault as illustrated with this program:

  #include <gc/gc.h>

  static const int foo;

  int
  main (int argc, char *argv[])
  {
    void *link;

    GC_INIT ();

    link = (void *) &foo;
    GC_GENERAL_REGISTER_DISAPPEARING_LINK (&link, (void *) &foo);
    GC_gcollect ();  /* <-- triggers the segfault */

    return 0;
  }

The backtrace is as follows:

  #0  GC_is_marked (p=0x109 <Address 0x109 out of bounds>) at mark.c:232
  #1  0xb7fa2c02 in GC_finalize () at finalize.c:507
  #2  0xb7f9f7b2 in GC_finish_collection () at alloc.c:656
  #3  0xb7f9fe09 in GC_try_to_collect_inner (stop_func=0xb7f9ef20 <GC_never_stop_func>) at alloc.c:373
  #4  0xb7fa00b4 in GC_try_to_collect (stop_func=0xb7f9ef20 <GC_never_stop_func>) at alloc.c:762