David Given | 19 Aug 00:34 2014

[Gc] Storing pointers to GC objects in non-GC memory

I'm trying to make libgc work with an Ada program.

Persuading Ada to allocate objects with libgc is pretty easy. Persuading
it to allocate *all* objects with libgc is rather hard --- in a modern
environment the standard libraries and runtimes are all precompiled
shared objects, so replacing their references to malloc() isn't really
an option.

According to the libgc docs, libgc doesn't scan blocks allocated with
system malloc(). This means that if I allocate a block from libgc, then
store the pointer in a block allocated with system malloc() --- such as
an Ada standard container data structure --- then libgc won't see the
pointer, will think the object is unreferenced, and free it, right?

Is there anything I can do here or am I fundamentally doomed? The exact
wording of the libgc docs are that libgc doesn't 'usually' scan blocks
from system malloc() --- what does 'usually' mean here?

--

-- 
┌─── dg@cowlark.com ─────
http://www.cowlark.com ─────
│ "Blue is beautiful... blue is best...
│ I'm blue! I'm beautiful! I'm best!"
│ --- _Dougal and the Blue Cat_

_______________________________________________
bdwgc mailing list
bdwgc@...
(Continue reading)

Juan Wajnerman | 10 Aug 01:36 2014
Picon

[Gc] Segfault during parallel collection

I'm having trouble combining parallel gc with coroutines made with libpcl (http://xmailserver.org/libpcl.html).
To allow the gc to deal with the coroutine stacks I register a callback with GC_set_push_other_roots. Whenever a coroutine starts or stops the GC_stackbottom is replaced. Only the portions of the coroutine stacks that are actually used are lately pushed when the "push_other_roots" callback is invoked.

This seems to work fairly well most of the time, but sometimes seems like something is not properly initialized and it crashes with the following backtrace:

0   libgc.1.dylib                 0x0000000104392d9b GC_do_parallel_mark + 28
1   libgc.1.dylib                 0x0000000104392249 GC_mark_some + 422
2   libgc.1.dylib                 0x000000010438b58d GC_stopped_mark + 119
3   libgc.1.dylib                 0x000000010438b4b1 GC_try_to_collect_inner + 260
4   libgc.1.dylib                 0x000000010438c292 GC_collect_or_expand + 188
5   libgc.1.dylib                 0x000000010438c48a GC_allocobj + 167
6   libgc.1.dylib                 0x0000000104390ba9 GC_generic_malloc_inner + 350
7   libgc.1.dylib                 0x0000000104391bb7 GC_generic_malloc_many + 977
8   libgc.1.dylib                 0x0000000104398aa3 GC_malloc_atomic + 135
...


The reason I suspect the problem is with the initialization is because during a stress test the crash occurs very early in the first grabage collection or it doesn't happen at all even after tens of GBs are allocated and freed.
The stress test only runs on a single thread, entering and leaving many coroutines that allocate memory. This problem doesn't occur if I run the test using GC_NPROCS=1

It's complicated to give an example because it's not even C/C++ code. I'm working on a new programming language. However you can get an idea of what I'm trying to do reading this code: https://github.com/manastech/crystal/blob/master/src/fiber/pcl.cr#L102

Any ideas? Should I place some lock around the "push_other_roots" initialization?

Thanks!
- Juan Wajnerman

_______________________________________________
bdwgc mailing list
bdwgc@...
https://lists.opendylan.org/mailman/listinfo/bdwgc
Christian Schafmeister | 8 Aug 07:30 2014
Picon
Picon

Re: [Gc] Trouble with disappearing links

Thank you,  your suggestions gave me what I needed to get it working reliably.

I’m not sure how asynchronously the Boehm collector updates disappearing links.
Do I need to temporarily suspend the Boehm collector by suspending interrupts or signals while I’m updating my weak-key-hash-table?

Best,

.Chris.


On Aug 6, 2014, at 5:08 PM, Bruce Hoult <bruce-zQEARAI6hqHYtjvyW6yDsg@public.gmane.org> wrote:

>I don’t see any mention of this in the documentation unless the statement “where p
> is a pointer that is not followed by finalization code” is supposed to indicate that
> you put the pointer in memory that is not scanned for pointers

No, it means what it says. GC_register_disappearing_link() is used for fields of normal scannable objects. If your finalizer tries to dereference a pointer field that just got zeroed then your program is going to crash. So if the finalizer uses a particular pointer, don't ask for it to disappear.

The ECL code you quoted uses GC_general_register_disappearing_link() not GC_register_disappearing_link(). They do very different things. Possibly they should have names that are more different.



On Thu, Aug 7, 2014 at 7:20 AM, Christian Schafmeister <chris.schaf-H+0wwilmMs3R7s880joybQ@public.gmane.org> wrote:

Following up on my own question I looked into ECL which uses the Boehm garbage collector and supports weak pointers.

It uses this code to allocate a weak pointer with GC_MALLOC_ATOMIC which contains objects that are not scanned for pointers.
I don’t see any mention of this in the documentation unless the statement “where p is a pointer that is not followed by finalization code” is supposed to indicate that you put the pointer in memory that is not scanned for pointers.: 

/* The following routine may be used to break cycles between */ /* finalizable objects, thus causing cyclic finalizable */ /* objects to be finalized in the correct order. Standard */ /* use involves calling GC_register_disappearing_link(&p), */ /* where p is a pointer that is not followed by finalization */ /* code, and should not be considered in determining */ /* finalization order. */

——— ECL code below ———— 

static cl_object
ecl_alloc_weak_pointer(cl_object o)
{
const cl_env_ptr the_env = ecl_process_env();
struct ecl_weak_pointer *obj;
ecl_disable_interrupts_env(the_env);
obj = GC_MALLOC_ATOMIC(sizeof(struct ecl_weak_pointer));
ecl_enable_interrupts_env(the_env);
obj->t = t_weak_pointer;
obj->value = o;
        if (!ECL_FIXNUMP(o) && !ECL_CHARACTERP(o) && !Null(o)) {
                GC_general_register_disappearing_link((void**)&(obj->value), (void*)o);
                si_set_finalizer((cl_object)obj, ECL_T);
        }
return (cl_object)obj;
}




On Aug 6, 2014, at 2:03 PM, Christian Schafmeister <chris.schaf-H+0wwilmMs3R7s880joybQ@public.gmane.org> wrote:


I’m using disappearing links for the first time (I may not know what I’m doing) and they seem to be keeping the object that they are pointing to alive.
I don’t know why the disappearing link is not disappearing and appears to keep the object it points to alive.
Should I not be allocating the memory that contains the disappearing link within the Boehm managed memory?

Pseudo code:
1) I’m implementing a weak key hash table
2) I allocate two parallel arrays (key array/value array) using GC_MALLOC.
3) I write a key and value into each corresponding array.
4) I call GC_register_disappearing_link(&key[0]).  This should make the key entry at index 0 a disappearing link.
5) Then I destroy all references to the key other than what are in the array.
6) The key object stays alive until I reset the key[0] entry - then it gets finalized.


Here is a transcript of the Common Lisp session:

> (setq ht (make-weak-key-hash-table))
(setq ht (make-weak-key-hash-table))

#<CORE:WEAK-KEY-HASH-TABLE <at> 0x11464c1d8) > 
> (low-level-describe ht)
(low-level-describe ht)
WeakKeyHashTable   size: 16
   keys memory range:  0x1142e0c00  - 0x1142e0d00 
   0  key.px <at> 0x1142e0c00  unbound
   1  key.px <at> 0x1142e0c10  unbound
   2  key.px <at> 0x1142e0c20  unbound
   3  key.px <at> 0x1142e0c30  unbound
   4  key.px <at> 0x1142e0c40  unbound
   5  key.px <at> 0x1142e0c50  unbound
   6  key.px <at> 0x1142e0c60  unbound
   7  key.px <at> 0x1142e0c70  unbound
   8  key.px <at> 0x1142e0c80  unbound
   9  key.px <at> 0x1142e0c90  unbound
   10  key.px <at> 0x1142e0ca0  unbound
   11  key.px <at> 0x1142e0cb0  unbound
   12  key.px <at> 0x1142e0cc0  unbound
   13  key.px <at> 0x1142e0cd0  unbound
   14  key.px <at> 0x1142e0ce0  unbound
   15  key.px <at> 0x1142e0cf0  unbound

> (setq key (cons 1 2))
(setq key (cons 1 2))

(1 . 2)
> (weak-hash-table-put ht 0 key 9999)
(weak-hash-table-put ht 0 key 9999)
../../src/gctools/gcalloc.h:969  Registered disappearing link 0x1142e0c00  obj 0x1120c5700  return val = 0

> (low-level-describe ht)
(low-level-describe ht)
WeakKeyHashTable   size: 16
   keys memory range:  0x1142e0c00  - 0x1142e0d00 
   0  key.px <at> 0x1142e0c00  (1 . 2) <at> 0x1120c5700   -->   9999
   1  key.px <at> 0x1142e0c10  unbound
   2  key.px <at> 0x1142e0c20  unbound
   3  key.px <at> 0x1142e0c30  unbound
   4  key.px <at> 0x1142e0c40  unbound
   5  key.px <at> 0x1142e0c50  unbound
   6  key.px <at> 0x1142e0c60  unbound
   7  key.px <at> 0x1142e0c70  unbound
   8  key.px <at> 0x1142e0c80  unbound
   9  key.px <at> 0x1142e0c90  unbound
   10  key.px <at> 0x1142e0ca0  unbound
   11  key.px <at> 0x1142e0cb0  unbound
   12  key.px <at> 0x1142e0cc0  unbound
   13  key.px <at> 0x1142e0cd0  unbound
   14  key.px <at> 0x1142e0ce0  unbound
   15  key.px <at> 0x1142e0cf0  unbound

> (setq key nil)    ;;;   Here I wipe out the only non-disappearing reference to the key
(setq key nil)    ;;;   Here I wipe out the only non-disappearing reference to the key

NIL
> (gctools:garbage-collect)  ;; I force garbage collections - the key object is not collected although I think it should be
(gctools:garbage-collect)

> (gctools:garbage-collect)
(gctools:garbage-collect)

> (gctools:garbage-collect)
(gctools:garbage-collect)

> (gctools:garbage-collect)
(gctools:garbage-collect)

> ;; No finalization messages have been printed
;; No finalization message

> (low-level-describe ht)  ;; Here we see the key object is still alive
(low-level-describe ht)
WeakKeyHashTable   size: 16
   keys memory range:  0x1142e0c00  - 0x1142e0d00 
   0  key.px <at> 0x1142e0c00  (1 . 2) <at> 0x1120c5700   -->   9999
   1  key.px <at> 0x1142e0c10  unbound
   2  key.px <at> 0x1142e0c20  unbound
   3  key.px <at> 0x1142e0c30  unbound
   4  key.px <at> 0x1142e0c40  unbound
   5  key.px <at> 0x1142e0c50  unbound
   6  key.px <at> 0x1142e0c60  unbound
   7  key.px <at> 0x1142e0c70  unbound
   8  key.px <at> 0x1142e0c80  unbound
   9  key.px <at> 0x1142e0c90  unbound
   10  key.px <at> 0x1142e0ca0  unbound
   11  key.px <at> 0x1142e0cb0  unbound
   12  key.px <at> 0x1142e0cc0  unbound
   13  key.px <at> 0x1142e0cd0  unbound
   14  key.px <at> 0x1142e0ce0  unbound
   15  key.px <at> 0x1142e0cf0  unbound

> ;; Key is still there
;; Key is still there


> (weak-hash-table-put ht 0 nil 9999)
(weak-hash-table-put ht 0 nil 9999)
../../src/gctools/gcalloc.h:969  Registered disappearing link 0x1142e0c00  obj 0x9  return val = 1

> (gctools:garbage-collect)   ;; Now the object will be collected and a finalization function called
(gctools:garbage-collect)
../../src/gctools/gcalloc.h:894 Boehm finalized weak linked address 0x1120c5700 at 0x1142e0c00

> (low-level-describe ht)
(low-level-describe ht)
WeakKeyHashTable   size: 16
   keys memory range:  0x1142e0c00  - 0x1142e0d00 
   0  key.px <at> 0x1142e0c00  nil
   1  key.px <at> 0x1142e0c10  unbound
   2  key.px <at> 0x1142e0c20  unbound
   3  key.px <at> 0x1142e0c30  unbound
   4  key.px <at> 0x1142e0c40  unbound
   5  key.px <at> 0x1142e0c50  unbound
   6  key.px <at> 0x1142e0c60  unbound
   7  key.px <at> 0x1142e0c70  unbound
   8  key.px <at> 0x1142e0c80  unbound
   9  key.px <at> 0x1142e0c90  unbound
   10  key.px <at> 0x1142e0ca0  unbound
   11  key.px <at> 0x1142e0cb0  unbound
   12  key.px <at> 0x1142e0cc0  unbound
   13  key.px <at> 0x1142e0cd0  unbound
   14  key.px <at> 0x1142e0ce0  unbound
   15  key.px <at> 0x1142e0cf0  unbound



--
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.

_______________________________________________
bdwgc mailing list
bdwgc-ZwoEplunGu1I4Lznb4ZCK0B+6BGkLq7r@public.gmane.org
https://lists.opendylan.org/mailman/listinfo/bdwgc


_______________________________________________
bdwgc mailing list
bdwgc@...
https://lists.opendylan.org/mailman/listinfo/bdwgc
Christian Schafmeister | 6 Aug 20:03 2014
Picon
Picon

[Gc] Trouble with disappearing links


I’m using disappearing links for the first time (I may not know what I’m doing) and they seem to be keeping the object that they are pointing to alive.
I don’t know why the disappearing link is not disappearing and appears to keep the object it points to alive.
Should I not be allocating the memory that contains the disappearing link within the Boehm managed memory?

Pseudo code:
1) I’m implementing a weak key hash table
2) I allocate two parallel arrays (key array/value array) using GC_MALLOC.
3) I write a key and value into each corresponding array.
4) I call GC_register_disappearing_link(&key[0]).  This should make the key entry at index 0 a disappearing link.
5) Then I destroy all references to the key other than what are in the array.
6) The key object stays alive until I reset the key[0] entry - then it gets finalized.


Here is a transcript of the Common Lisp session:

> (setq ht (make-weak-key-hash-table))
(setq ht (make-weak-key-hash-table))

#<CORE:WEAK-KEY-HASH-TABLE <at> 0x11464c1d8) > 
> (low-level-describe ht)
(low-level-describe ht)
WeakKeyHashTable   size: 16
   keys memory range:  0x1142e0c00  - 0x1142e0d00 
   0  key.px <at> 0x1142e0c00  unbound
   1  key.px <at> 0x1142e0c10  unbound
   2  key.px <at> 0x1142e0c20  unbound
   3  key.px <at> 0x1142e0c30  unbound
   4  key.px <at> 0x1142e0c40  unbound
   5  key.px <at> 0x1142e0c50  unbound
   6  key.px <at> 0x1142e0c60  unbound
   7  key.px <at> 0x1142e0c70  unbound
   8  key.px <at> 0x1142e0c80  unbound
   9  key.px <at> 0x1142e0c90  unbound
   10  key.px <at> 0x1142e0ca0  unbound
   11  key.px <at> 0x1142e0cb0  unbound
   12  key.px <at> 0x1142e0cc0  unbound
   13  key.px <at> 0x1142e0cd0  unbound
   14  key.px <at> 0x1142e0ce0  unbound
   15  key.px <at> 0x1142e0cf0  unbound

> (setq key (cons 1 2))
(setq key (cons 1 2))

(1 . 2)
> (weak-hash-table-put ht 0 key 9999)
(weak-hash-table-put ht 0 key 9999)
../../src/gctools/gcalloc.h:969  Registered disappearing link 0x1142e0c00  obj 0x1120c5700  return val = 0

> (low-level-describe ht)
(low-level-describe ht)
WeakKeyHashTable   size: 16
   keys memory range:  0x1142e0c00  - 0x1142e0d00 
   0  key.px <at> 0x1142e0c00  (1 . 2) <at> 0x1120c5700   -->   9999
   1  key.px <at> 0x1142e0c10  unbound
   2  key.px <at> 0x1142e0c20  unbound
   3  key.px <at> 0x1142e0c30  unbound
   4  key.px <at> 0x1142e0c40  unbound
   5  key.px <at> 0x1142e0c50  unbound
   6  key.px <at> 0x1142e0c60  unbound
   7  key.px <at> 0x1142e0c70  unbound
   8  key.px <at> 0x1142e0c80  unbound
   9  key.px <at> 0x1142e0c90  unbound
   10  key.px <at> 0x1142e0ca0  unbound
   11  key.px <at> 0x1142e0cb0  unbound
   12  key.px <at> 0x1142e0cc0  unbound
   13  key.px <at> 0x1142e0cd0  unbound
   14  key.px <at> 0x1142e0ce0  unbound
   15  key.px <at> 0x1142e0cf0  unbound

> (setq key nil)    ;;;   Here I wipe out the only non-disappearing reference to the key
(setq key nil)    ;;;   Here I wipe out the only non-disappearing reference to the key

NIL
> (gctools:garbage-collect)  ;; I force garbage collections - the key object is not collected although I think it should be
(gctools:garbage-collect)

> (gctools:garbage-collect)
(gctools:garbage-collect)

> (gctools:garbage-collect)
(gctools:garbage-collect)

> (gctools:garbage-collect)
(gctools:garbage-collect)

> ;; No finalization messages have been printed
;; No finalization message

> (low-level-describe ht)  ;; Here we see the key object is still alive
(low-level-describe ht)
WeakKeyHashTable   size: 16
   keys memory range:  0x1142e0c00  - 0x1142e0d00 
   0  key.px <at> 0x1142e0c00  (1 . 2) <at> 0x1120c5700   -->   9999
   1  key.px <at> 0x1142e0c10  unbound
   2  key.px <at> 0x1142e0c20  unbound
   3  key.px <at> 0x1142e0c30  unbound
   4  key.px <at> 0x1142e0c40  unbound
   5  key.px <at> 0x1142e0c50  unbound
   6  key.px <at> 0x1142e0c60  unbound
   7  key.px <at> 0x1142e0c70  unbound
   8  key.px <at> 0x1142e0c80  unbound
   9  key.px <at> 0x1142e0c90  unbound
   10  key.px <at> 0x1142e0ca0  unbound
   11  key.px <at> 0x1142e0cb0  unbound
   12  key.px <at> 0x1142e0cc0  unbound
   13  key.px <at> 0x1142e0cd0  unbound
   14  key.px <at> 0x1142e0ce0  unbound
   15  key.px <at> 0x1142e0cf0  unbound

> ;; Key is still there
;; Key is still there


> (weak-hash-table-put ht 0 nil 9999)
(weak-hash-table-put ht 0 nil 9999)
../../src/gctools/gcalloc.h:969  Registered disappearing link 0x1142e0c00  obj 0x9  return val = 1

> (gctools:garbage-collect)   ;; Now the object will be collected and a finalization function called
(gctools:garbage-collect)
../../src/gctools/gcalloc.h:894 Boehm finalized weak linked address 0x1120c5700 at 0x1142e0c00

> (low-level-describe ht)
(low-level-describe ht)
WeakKeyHashTable   size: 16
   keys memory range:  0x1142e0c00  - 0x1142e0d00 
   0  key.px <at> 0x1142e0c00  nil
   1  key.px <at> 0x1142e0c10  unbound
   2  key.px <at> 0x1142e0c20  unbound
   3  key.px <at> 0x1142e0c30  unbound
   4  key.px <at> 0x1142e0c40  unbound
   5  key.px <at> 0x1142e0c50  unbound
   6  key.px <at> 0x1142e0c60  unbound
   7  key.px <at> 0x1142e0c70  unbound
   8  key.px <at> 0x1142e0c80  unbound
   9  key.px <at> 0x1142e0c90  unbound
   10  key.px <at> 0x1142e0ca0  unbound
   11  key.px <at> 0x1142e0cb0  unbound
   12  key.px <at> 0x1142e0cc0  unbound
   13  key.px <at> 0x1142e0cd0  unbound
   14  key.px <at> 0x1142e0ce0  unbound
   15  key.px <at> 0x1142e0cf0  unbound

_______________________________________________
bdwgc mailing list
bdwgc@...
https://lists.opendylan.org/mailman/listinfo/bdwgc
Peter Wang | 5 Aug 07:54 2014
Picon

[Gc] Support winpthreads

Hi,

Here is a preliminary patch to support building with winpthreads.
It overloads GC_WIN32_PTHREADS to mean either pthreads-win32 or
winpthreads.  An alternative might be to add a new symbol
GC_WINPTHREADS?

It seems to be fine but I have not tested it extensively.
Perhaps someone else has come across it already?

Peter
_______________________________________________
bdwgc mailing list
bdwgc@...
https://lists.opendylan.org/mailman/listinfo/bdwgc
Christian Weisgerber | 25 Jul 23:02 2014
Picon

Re: [Gc] sparc_mach_dep.lo in wrong dir

Ivan Maidanski:

> Wed, 23 Apr 2014 15:01:23 -0700 from Mojca Miklavec:
> >Building gc 7.4 on Sparc Solaris fails with
> >libtool: link: `src/sparc_mach_dep.lo' is not a valid libtool object
> >The problem is that the file is actually in top level
> >(src/sparc_mach_dep.lo), not under src. If I manually edit the Makefile
> >and remove src/ from all occurencies of src/sparc_mach_dep.lo, the
> >compilation succeeds.
> 
> Should fixed in https://github.com/ivmai/bdwgc/commit/e2bf29b7801357c1ad28c2948a3f88c655dd722a

Building gc 7.4.2 on OpenBSD/sparc64 still fails, but differently:

make: don't know how to make sparc_mach_dep.lo (prerequisite of: libgc.la)

Now there is no Makefile rule to derive sparc_mach_dep.lo from
src/sparc_mach_dep.S.

--

-- 
Christian "naddy" Weisgerber                          naddy@...
Richard Brooksby | 10 Jul 12:28 2014

[Gc] C optimizer hazards in practice

https://github.com/ivmai/bdwgc/blob/3ec2783588c8ba53a0b6a347d61d4e5c1b5dc80e/README.md says:

> Some C optimizers may lose the last undisguised pointer to a memory object as a consequence of clever
optimizations. This has almost never been observed in practice.

Does anyone have any stories to tell about when this *has* been observed, or any links to accounts of this happening?

Background: The Memory Pool System <http://www.ravenbrook.com/project/mps/> team is currently
investigating a case of the Microsoft C compiler optimising away the last reference to an object, causing
it to be prematurely recycled and corrupt the heap.  See
<http://mailman.ravenbrook.com/pipermail/mps-discussion/2014-July/000144.html> for details,
including repro, disassembly, analysis, etc.
Lucas Meijer | 30 Jun 10:24 2014

[Gc] adding profiling callbacks

Hi,

Unity is a game development tool that for some platforms uses Boehm to collect garbage generated by our users .NET code.

Our builtin profiler can profile all the different game subsystems we have, and I've added support to it for also profiling the boehm collector.

I've added profiling callbacks to our version of boehm, and would like to ask this list if there's any interest in taking these patches. (the friendly folks at the mono project did pretty much the same for their fork of boehm).

Here's the patches we have now. I'd be more than happy to adapt/fix any concerns they might have.

Bye, Lucas


diff --git a/External/bdwgc/alloc.c b/External/bdwgc/alloc.c
--- a/External/bdwgc/alloc.c
+++ b/External/bdwgc/alloc.c
 <at>  <at>  -347,6 +347,18  <at>  <at> 
 STATIC GC_bool GC_stopped_mark(GC_stop_func stop_func);
 STATIC void GC_finish_collection(void);

+static GC_event_callback_func GC_event_callback = NULL;
+
+void GC_set_event_callback(GC_event_callback_func func)
+{
+  GC_event_callback = func;
+}
+
+GC_event_callback_func GC_get_event_callback()
+{
+  return GC_event_callback;
+}
+
 /*
  * Initiate a garbage collection if appropriate.
  * Choose judiciously
 <at>  <at>  -420,6 +432,10  <at>  <at> 
 #   endif
     ASSERT_CANCEL_DISABLED();
     if (GC_dont_gc || (*stop_func)()) return FALSE;
+
+    if (GC_event_callback)
+      GC_event_callback (GC_EVENT_START, NULL);
+
     if (GC_incremental && GC_collection_in_progress()) {
       GC_COND_LOG_PRINTF(
             "GC_try_to_collect_inner: finishing collection in progress\n");
 <at>  <at>  -476,6 +492,10  <at>  <at> 
                       MS_TIME_DIFF(current_time,start_time));
       }
 #   endif
+    
+    if (GC_event_callback)
+      GC_event_callback (GC_EVENT_END, NULL);
+
     return(TRUE);
 }

 <at>  <at>  -607,10 +627,20  <at>  <at> 
         GET_TIME(start_time);
 #   endif

+    if (GC_event_callback)
+      GC_event_callback (GC_EVENT_PRE_STOP_WORLD, NULL);
+
     STOP_WORLD();
 #   ifdef THREAD_LOCAL_ALLOC
       GC_world_stopped = TRUE;
 #   endif
+
+    if (GC_event_callback)
+    {
+      GC_event_callback (GC_EVENT_POST_STOP_WORLD, NULL);
+      GC_event_callback (GC_EVENT_MARK_START, NULL);
+    }
+
         /* Output blank line for convenience here */
     GC_COND_LOG_PRINTF(
               "\n--> Marking for collection #%lu after %lu allocated bytes\n",
 <at>  <at>  -632,10 +662,19  <at>  <at> 
             GC_COND_LOG_PRINTF("Abandoned stopped marking after"
                                " %u iterations\n", i);
             GC_deficit = i;     /* Give the mutator a chance.   */
+
+            if (GC_event_callback)
+            {
+              GC_event_callback (GC_EVENT_MARK_END, NULL);
+              GC_event_callback (GC_EVENT_PRE_START_WORLD, NULL);
+            }
 #           ifdef THREAD_LOCAL_ALLOC
               GC_world_stopped = FALSE;
 #           endif
             START_WORLD();
+            if (GC_event_callback)
+              GC_event_callback (GC_EVENT_POST_START_WORLD, NULL);
+
             return(FALSE);
           }
           if (GC_mark_some(GC_approx_sp())) break;
 <at>  <at>  -656,7 +695,18  <at>  <at> 
 #   ifdef THREAD_LOCAL_ALLOC
       GC_world_stopped = FALSE;
 #   endif
+
+    if (GC_event_callback)
+    {
+      GC_event_callback (GC_EVENT_MARK_END, NULL);
+      GC_event_callback (GC_EVENT_PRE_START_WORLD, NULL);
+    }
+
     START_WORLD();
+
+    if (GC_event_callback)
+      GC_event_callback (GC_EVENT_POST_START_WORLD, NULL);
+
 #   ifndef SMALL_CONFIG
       if (GC_PRINT_STATS_FLAG) {
         unsigned long time_diff;
diff --git a/External/bdwgc/include/gc.h b/External/bdwgc/include/gc.h
--- a/External/bdwgc/include/gc.h
+++ b/External/bdwgc/include/gc.h
 <at>  <at>  -105,6 +105,25  <at>  <at> 
 /* Public R/W variables */
 /* The supplied setter and getter functions are preferred for new code. */

+typedef enum {
+    GC_EVENT_START,
+    GC_EVENT_MARK_START,
+    GC_EVENT_MARK_END,
+    GC_EVENT_RECLAIM_START,
+    GC_EVENT_RECLAIM_END,
+    GC_EVENT_END,
+    GC_EVENT_PRE_STOP_WORLD,
+    GC_EVENT_POST_STOP_WORLD,
+    GC_EVENT_PRE_START_WORLD,
+    GC_EVENT_POST_START_WORLD,
+    GC_EVENT_SUSPENDED_THREAD,
+    GC_EVENT_UNSUSPENDED_THREAD,
+} GCEventType;
+
+typedef void * (GC_CALLBACK * GC_event_callback_func)(GCEventType eventType, void* data);
+GC_API void GC_CALL GC_set_event_callback(GC_event_callback_func);
+GC_API GC_event_callback_func GC_CALL GC_get_event_callback(void);
+
 typedef void * (GC_CALLBACK * GC_oom_func)(size_t /* bytes_requested */);
 GC_API GC_ATTR_DEPRECATED GC_oom_func GC_oom_fn;
                         /* When there is insufficient memory to satisfy */
diff --git a/External/bdwgc/pthread_stop_world.c b/External/bdwgc/pthread_stop_world.c
--- a/External/bdwgc/pthread_stop_world.c
+++ b/External/bdwgc/pthread_stop_world.c
 <at>  <at>  -505,6 +505,9  <at>  <at> 
                     n_live_threads--;
                     break;
                 case 0:
+                    GC_get_event_callback_func cb = GC_get_event_callback();
+                    if (cb)
+                      cb(GC_EVENT_SUSPENDED_THREAD, p->id);
                     break;
                 default:
                     ABORT_ARG1("pthread_kill failed at suspend",
 <at>  <at>  -829,6 +832,9  <at>  <at> 
                     n_live_threads--;
                     break;
                 case 0:
+                    GC_get_event_callback_func cb = GC_get_event_callback();
+                    if (cb)
+                      cb(GC_EVENT_UNSUSPENDED_THREAD, p->id);
                     break;
                 default:
                     ABORT_ARG1("pthread_kill failed at resume",
_______________________________________________
bdwgc mailing list
bdwgc@...
https://lists.opendylan.org/mailman/listinfo/bdwgc
Stefan Kral | 25 Jun 13:26 2014
Picon

[Gc] Supplying hugepages to the GC

Hello everyone.

I have had good results speeding up some big lookup-table accesses by backing them with hugepages (and initializing them upon application startup.)

Is there a straightforward way I could configure / instruct the gc to alloc the first few heap chunks using hugepages when available? Possibly by letting it alloc memory with (hugepage backed) malloc instead of mmap or sbrk?

Regards, Stefan Kral.

_______________________________________________
bdwgc mailing list
bdwgc@...
https://lists.opendylan.org/mailman/listinfo/bdwgc
Paul Bone | 25 Jun 03:34 2014
Picon

[Gc] glibc 2.19 lock elision bug

Hi.

I wrote earlier about a bug I was trying to find regarding lock elision in
glibc 2.19 that affects Boehm GC.  I beleive that this affects all
applications that use Boehm GC, not just Mercury, as mono applications have
also been crashing since I upgraded glibc.  I would like some feedback and
help with the following change:

The git branch containing this change can be found here:
https://github.com/PaulBone/bdwgc/tree/fix_tsx_bug

Thank you.

From eb31ad476a0a3b4125202bd7628a9ab3cfb634d6 Mon Sep 17 00:00:00 2001
From: Paul Bone <paul@...>
Date: Wed, 25 Jun 2014 11:17:50 +1000
Subject: [PATCH] Workaround Linux NTPL lock elision bug.

glibc 2.19 on Linux x86-64 platforms includes support for lock elision,
by using Intel's TSX support when it is available.  Without modifying an
application this converts suitable critical sections that use mutex into
transactional memory critical sections.  See http://lwn.net/Articles/534758/
If a problem occurs that means that transactional memory can't be used, such
as a system call or buffer overflow, the pthreads implementation will catch
this error and retry the critical section using a normal mutex.

I noticed that since upgrading glibc that programs using Boehm GC crash, one
of these crashes was an assertion that the owner field of a mutex was
invalid.  The assertion was generated by the pthreads implementation.
I believe that there is a bug in glibc that when a mutex cannot be used
safely for transactions that some series of events causes it's owner field
to be set incorrectly (or cleared when it shouldn't be).

I've found that I can work around this problem by having Boehm GC use an
error checking mutex, which I believe doesn't use lock elision and in my
testing doesn't crash.

XXX: This work-around mostly works except for linking the feature detection
in configure.ac to the conditional compilation in pthread_support.c as there
isn't an obvious way to make it work for automake and Makefile.direct.
Could I have some help updating the build system please?

include/private/pthread_support.h:
pthread_support.c:
    Define GC_setup_mark_lock()  This procedure creates the lock specifying a
    pthread_mutexattr_t structure.  This is used to disable lock elision on
    Linux with glibc 2.19.

configure.ac:
    If we're using Linux then check for the gnu extensions required to
    identify the version of glibc at runtime.

misc.c:
    Call GC_setup_mark_lock() when initialising the collector.
---
 configure.ac                      | 13 +++++++++++++
 include/private/pthread_support.h |  2 ++
 misc.c                            |  3 +++
 pthread_support.c                 | 39 +++++++++++++++++++++++++++++++++++++++
 4 files changed, 57 insertions(+)

diff --git a/configure.ac b/configure.ac
index 7667949..6853e97 100644
--- a/configure.ac
+++ b/configure.ac
 <at>  <at>  -646,6 +646,19  <at>  <at>  case "$host" in
  *) AC_MSG_RESULT(no) ;;
 esac

+dnl Check for specific glibc functions and definitions that we need to for
+dnl the glibc 2.19 workaround.
+HAVE_LIBC_VERSION_H=no
+HAVE_GNU_GET_LIBC_VERSION=no
+case "${host}" in
+  *-linux*)
+    AC_CHECK_HEADER([gnu/libc-version.h], HAVE_LIBC_VERSION_H=yes)
+    AC_CHECK_FUNC([gnu_get_libc_version], HAVE_GNU_GET_LIBC_VERSION=yes)
+    ;;
+esac
+AC_SUBST(HAVE_LIBC_VERSION_H)
+AC_SUBST(HAVE_GNU_GET_LIBC_VERSION)
+
 dnl Include defines that have become de facto standard.
 dnl ALL_INTERIOR_POINTERS and NO_EXECUTE_PERMISSION can be overridden
 dnl in the startup code.
diff --git a/include/private/pthread_support.h b/include/private/pthread_support.h
index 525a9aa..017f194 100644
--- a/include/private/pthread_support.h
+++ b/include/private/pthread_support.h
 <at>  <at>  -148,6 +148,8  <at>  <at>  GC_INNER_PTHRSTART GC_thread GC_start_rtn_prepare_thread(
                                         struct GC_stack_base *sb, void *arg);
 GC_INNER_PTHRSTART void GC_thread_exit_proc(void *);

+GC_INNER void GC_setup_mark_lock(void);
+
 #endif /* GC_PTHREADS && !GC_WIN32_THREADS */

 #endif /* GC_PTHREAD_SUPPORT_H */
diff --git a/misc.c b/misc.c
index df434a1..dccf5f3 100644
--- a/misc.c
+++ b/misc.c
 <at>  <at>  -875,6 +875,9  <at>  <at>  GC_API void GC_CALL GC_init(void)
         /* else */ InitializeCriticalSection (&GC_allocate_ml);
      }
 #   endif /* GC_WIN32_THREADS */
+#   if (defined(GC_PTHREADS) && !defined(GC_WIN32_THREADS))
+     GC_setup_mark_lock();
+#   endif /* GC_PTHREADS */
 #   if (defined(MSWIN32) || defined(MSWINCE)) && defined(THREADS)
       InitializeCriticalSection(&GC_write_cs);
 #   endif
diff --git a/pthread_support.c b/pthread_support.c
index c00b93d..49b33d0 100644
--- a/pthread_support.c
+++ b/pthread_support.c
 <at>  <at>  -95,6 +95,10  <at>  <at> 
   typedef unsigned int sem_t;
 #endif /* GC_DGUX386_THREADS */

+#ifdef HAVE_LIBC_VERSION_H
+# include <gnu/libc-version.h>
+#endif
+
 /* Undefine macros used to redirect pthread primitives. */
 # undef pthread_create
 # ifndef GC_NO_PTHREAD_SIGMASK
 <at>  <at>  -1973,12 +1977,47  <at>  <at>  GC_INNER void GC_lock(void)
   /* defined.                                                           */
   static pthread_mutex_t mark_mutex =
         {0, 0, 0, PTHREAD_MUTEX_ERRORCHECK_NP, {0, 0}};
+#elif defined(HAVE_GNU_GET_LIBC_VERSION) && defined(HAVE_LIBC_VERSION_H)
+  static pthread_mutex_t mark_mutex;
 #else
   static pthread_mutex_t mark_mutex = PTHREAD_MUTEX_INITIALIZER;
 #endif

 static pthread_cond_t builder_cv = PTHREAD_COND_INITIALIZER;

+GC_INNER void GC_setup_mark_lock(void)
+{
+#if defined(HAVE_GNU_GET_LIBC_VERSION) && defined(HAVE_LIBC_VERSION_H)
+    char *version_str;
+    pthread_mutexattr_t attr;
+
+    if (0 != pthread_mutexattr_init(&attr)) {
+        goto error;
+    }
+
+    version_str = gnu_get_libc_version;
+    if (0 == strcmp("2.19", version_str))
+    {
+        /*
+         * Disable lock elision on this version of glibc.
+         */
+        if (0 != pthread_mutexattr_settype(&attr, PTHREAD_MUTEX_ERRORCHECK)) {
+            goto error;
+        }
+    }
+
+    if (0 != pthread_mutex_init(&mark_mutex, &attr)) {
+        goto error;
+    }
+    pthread_mutexattr_destroy(&attr);
+    return;
+
+error:
+    perror("Error setting up marker mutex");
+    exit(1);
+#endif /* HAVE_GNU_GET_LIBC_VERSION && HAVE_LIBC_VERSION_H */
+}
+
 GC_INNER void GC_acquire_mark_lock(void)
 {
     GC_ASSERT(GC_mark_lock_holder != NUMERIC_THREAD_ID(pthread_self()));
-- 
2.0.0

--

-- 
Paul Bone
Daniel R. Grayson | 25 Jun 01:46 2014
Picon

[Gc] parallel speedup

In our application that uses libgc (see http://macaulay2.com/) I observe no
speedup when running tasks in parallel, if the tasks allocate memory using
libgc.  Perhaps I'm doing something wrong.  Are there any commonly observed
situations where no speedup occurs?

A glance at the source code shows that mutex locks lock down the world on
almost every occasion, so it's hard to see why there would ever be any speedup
when using threads.


_______________________________________________
bdwgc mailing list
bdwgc@...
https://lists.opendylan.org/mailman/listinfo/bdwgc

Gmane