Ivan Maidanski | 11 Sep 19:11 2014
Picon

Re: [Gc] [bdwgc] Assertions when glibc uses lock elision (#51)

Hi Jan,

I have merged fix-tsx-bug commit to master (as well as release-7_4 branch).
Note that this fix (workaround) enabled only for linux/x64 at this moment. To enable it for x86, use -DFIX_GLIBC_TSX_BUG . Please test it whether it is OK for x86. If it solves the issue I'll change the code to define the macro implicitly for linux/x86.

OTOH, this workaround has performance drawback (in locking) for linux/intel platforms which do not have TSX. Could we exclude such targets by checking TSX presence at runtime?

Regards,
Ivan

-----

Monday, 08 Sept 2014, 12:22 +04:00 from Jan Alexander Steffens <notifications <at> github.com>:

guile: ../nptl/pthread_mutex_lock.c:80: __pthread_mutex_cond_lock: Assertion mutex->__data.__owner == 0' failed.
gc's testsuite also randomly asserts. Strangely, it seems to happen soon after start or not at all.
The fix_tsx_bug branch in its current state (commit  757af8a ) works on x86_64 but not i686, which continues to assert.
Meanwhile, the glibc maintainers assert that this is not their bug.

Reply to this email directly or  view it on GitHub .

_______________________________________________
bdwgc mailing list
bdwgc@...
https://lists.opendylan.org/mailman/listinfo/bdwgc
Jonathan Chambers | 5 Sep 15:14 2014
Picon

[Gc] Pushing Registers for Collector Thread

Hello,

Posting to the mailing list an issue one of our customers raised with us and on github. We (Unity) use bdwgc within the Mono VM. We are running an older version, but the issue highlighted still seems to persist in master. 


I believe this may apply to more architectures/platforms, but at the moment I'll reference Win64. In GC_push_stack_for, all threads push their stack sections. However, when pushing register values the current thread is skipped:


This leads to a case where memory may be collected while still in use. Suppose an function allocates memory from the GC and it gets stored in a volatile register (RCX for example on Win64). No pointer on the stack or managed heap reference this allocation. Now a collection is triggered. 

If any subsequent functions in the call chain between the allocating function and GC_push_stack_for does use RCX, the original value should be pushed onto the stack and restored later as it's a volatile register. This stack reference will be found and marked since we push the current thread stack.

However, if any subsequent functions in the call chain between the allocating function and GC_push_stack_for does *not* use RCX the reference will be not be marked and will be collected.

On Windows/OSX this seems to always be the case. In pthread_stop_world.c I see calls to GC_save_regs_in_stack for SPARC and Itanium. Is there a reason this is not done/needed for all platforms?

If this is indeed a bug, we can provide patches for all the platforms we support.

Thanks,
Jonathan
_______________________________________________
bdwgc mailing list
bdwgc@...
https://lists.opendylan.org/mailman/listinfo/bdwgc
David Given | 19 Aug 00:34 2014

[Gc] Storing pointers to GC objects in non-GC memory

I'm trying to make libgc work with an Ada program.

Persuading Ada to allocate objects with libgc is pretty easy. Persuading
it to allocate *all* objects with libgc is rather hard --- in a modern
environment the standard libraries and runtimes are all precompiled
shared objects, so replacing their references to malloc() isn't really
an option.

According to the libgc docs, libgc doesn't scan blocks allocated with
system malloc(). This means that if I allocate a block from libgc, then
store the pointer in a block allocated with system malloc() --- such as
an Ada standard container data structure --- then libgc won't see the
pointer, will think the object is unreferenced, and free it, right?

Is there anything I can do here or am I fundamentally doomed? The exact
wording of the libgc docs are that libgc doesn't 'usually' scan blocks
from system malloc() --- what does 'usually' mean here?

--

-- 
┌─── dg@cowlark.com ─────
http://www.cowlark.com ─────
│ "Blue is beautiful... blue is best...
│ I'm blue! I'm beautiful! I'm best!"
│ --- _Dougal and the Blue Cat_

_______________________________________________
bdwgc mailing list
bdwgc@...
https://lists.opendylan.org/mailman/listinfo/bdwgc
Juan Wajnerman | 10 Aug 01:36 2014
Picon

[Gc] Segfault during parallel collection

I'm having trouble combining parallel gc with coroutines made with libpcl (http://xmailserver.org/libpcl.html).
To allow the gc to deal with the coroutine stacks I register a callback with GC_set_push_other_roots. Whenever a coroutine starts or stops the GC_stackbottom is replaced. Only the portions of the coroutine stacks that are actually used are lately pushed when the "push_other_roots" callback is invoked.

This seems to work fairly well most of the time, but sometimes seems like something is not properly initialized and it crashes with the following backtrace:

0   libgc.1.dylib                 0x0000000104392d9b GC_do_parallel_mark + 28
1   libgc.1.dylib                 0x0000000104392249 GC_mark_some + 422
2   libgc.1.dylib                 0x000000010438b58d GC_stopped_mark + 119
3   libgc.1.dylib                 0x000000010438b4b1 GC_try_to_collect_inner + 260
4   libgc.1.dylib                 0x000000010438c292 GC_collect_or_expand + 188
5   libgc.1.dylib                 0x000000010438c48a GC_allocobj + 167
6   libgc.1.dylib                 0x0000000104390ba9 GC_generic_malloc_inner + 350
7   libgc.1.dylib                 0x0000000104391bb7 GC_generic_malloc_many + 977
8   libgc.1.dylib                 0x0000000104398aa3 GC_malloc_atomic + 135
...


The reason I suspect the problem is with the initialization is because during a stress test the crash occurs very early in the first grabage collection or it doesn't happen at all even after tens of GBs are allocated and freed.
The stress test only runs on a single thread, entering and leaving many coroutines that allocate memory. This problem doesn't occur if I run the test using GC_NPROCS=1

It's complicated to give an example because it's not even C/C++ code. I'm working on a new programming language. However you can get an idea of what I'm trying to do reading this code: https://github.com/manastech/crystal/blob/master/src/fiber/pcl.cr#L102

Any ideas? Should I place some lock around the "push_other_roots" initialization?

Thanks!
- Juan Wajnerman

_______________________________________________
bdwgc mailing list
bdwgc@...
https://lists.opendylan.org/mailman/listinfo/bdwgc
Christian Schafmeister | 8 Aug 07:30 2014
Picon
Picon

Re: [Gc] Trouble with disappearing links

Thank you,  your suggestions gave me what I needed to get it working reliably.

I’m not sure how asynchronously the Boehm collector updates disappearing links.
Do I need to temporarily suspend the Boehm collector by suspending interrupts or signals while I’m updating my weak-key-hash-table?

Best,

.Chris.


On Aug 6, 2014, at 5:08 PM, Bruce Hoult <bruce-zQEARAI6hqHYtjvyW6yDsg@public.gmane.org> wrote:

>I don’t see any mention of this in the documentation unless the statement “where p
> is a pointer that is not followed by finalization code” is supposed to indicate that
> you put the pointer in memory that is not scanned for pointers

No, it means what it says. GC_register_disappearing_link() is used for fields of normal scannable objects. If your finalizer tries to dereference a pointer field that just got zeroed then your program is going to crash. So if the finalizer uses a particular pointer, don't ask for it to disappear.

The ECL code you quoted uses GC_general_register_disappearing_link() not GC_register_disappearing_link(). They do very different things. Possibly they should have names that are more different.



On Thu, Aug 7, 2014 at 7:20 AM, Christian Schafmeister <chris.schaf-H+0wwilmMs3R7s880joybQ@public.gmane.org> wrote:

Following up on my own question I looked into ECL which uses the Boehm garbage collector and supports weak pointers.

It uses this code to allocate a weak pointer with GC_MALLOC_ATOMIC which contains objects that are not scanned for pointers.
I don’t see any mention of this in the documentation unless the statement “where p is a pointer that is not followed by finalization code” is supposed to indicate that you put the pointer in memory that is not scanned for pointers.: 

/* The following routine may be used to break cycles between */ /* finalizable objects, thus causing cyclic finalizable */ /* objects to be finalized in the correct order. Standard */ /* use involves calling GC_register_disappearing_link(&p), */ /* where p is a pointer that is not followed by finalization */ /* code, and should not be considered in determining */ /* finalization order. */

——— ECL code below ———— 

static cl_object
ecl_alloc_weak_pointer(cl_object o)
{
const cl_env_ptr the_env = ecl_process_env();
struct ecl_weak_pointer *obj;
ecl_disable_interrupts_env(the_env);
obj = GC_MALLOC_ATOMIC(sizeof(struct ecl_weak_pointer));
ecl_enable_interrupts_env(the_env);
obj->t = t_weak_pointer;
obj->value = o;
        if (!ECL_FIXNUMP(o) && !ECL_CHARACTERP(o) && !Null(o)) {
                GC_general_register_disappearing_link((void**)&(obj->value), (void*)o);
                si_set_finalizer((cl_object)obj, ECL_T);
        }
return (cl_object)obj;
}




On Aug 6, 2014, at 2:03 PM, Christian Schafmeister <chris.schaf-H+0wwilmMs3R7s880joybQ@public.gmane.org> wrote:


I’m using disappearing links for the first time (I may not know what I’m doing) and they seem to be keeping the object that they are pointing to alive.
I don’t know why the disappearing link is not disappearing and appears to keep the object it points to alive.
Should I not be allocating the memory that contains the disappearing link within the Boehm managed memory?

Pseudo code:
1) I’m implementing a weak key hash table
2) I allocate two parallel arrays (key array/value array) using GC_MALLOC.
3) I write a key and value into each corresponding array.
4) I call GC_register_disappearing_link(&key[0]).  This should make the key entry at index 0 a disappearing link.
5) Then I destroy all references to the key other than what are in the array.
6) The key object stays alive until I reset the key[0] entry - then it gets finalized.


Here is a transcript of the Common Lisp session:

> (setq ht (make-weak-key-hash-table))
(setq ht (make-weak-key-hash-table))

#<CORE:WEAK-KEY-HASH-TABLE <at> 0x11464c1d8) > 
> (low-level-describe ht)
(low-level-describe ht)
WeakKeyHashTable   size: 16
   keys memory range:  0x1142e0c00  - 0x1142e0d00 
   0  key.px <at> 0x1142e0c00  unbound
   1  key.px <at> 0x1142e0c10  unbound
   2  key.px <at> 0x1142e0c20  unbound
   3  key.px <at> 0x1142e0c30  unbound
   4  key.px <at> 0x1142e0c40  unbound
   5  key.px <at> 0x1142e0c50  unbound
   6  key.px <at> 0x1142e0c60  unbound
   7  key.px <at> 0x1142e0c70  unbound
   8  key.px <at> 0x1142e0c80  unbound
   9  key.px <at> 0x1142e0c90  unbound
   10  key.px <at> 0x1142e0ca0  unbound
   11  key.px <at> 0x1142e0cb0  unbound
   12  key.px <at> 0x1142e0cc0  unbound
   13  key.px <at> 0x1142e0cd0  unbound
   14  key.px <at> 0x1142e0ce0  unbound
   15  key.px <at> 0x1142e0cf0  unbound

> (setq key (cons 1 2))
(setq key (cons 1 2))

(1 . 2)
> (weak-hash-table-put ht 0 key 9999)
(weak-hash-table-put ht 0 key 9999)
../../src/gctools/gcalloc.h:969  Registered disappearing link 0x1142e0c00  obj 0x1120c5700  return val = 0

> (low-level-describe ht)
(low-level-describe ht)
WeakKeyHashTable   size: 16
   keys memory range:  0x1142e0c00  - 0x1142e0d00 
   0  key.px <at> 0x1142e0c00  (1 . 2) <at> 0x1120c5700   -->   9999
   1  key.px <at> 0x1142e0c10  unbound
   2  key.px <at> 0x1142e0c20  unbound
   3  key.px <at> 0x1142e0c30  unbound
   4  key.px <at> 0x1142e0c40  unbound
   5  key.px <at> 0x1142e0c50  unbound
   6  key.px <at> 0x1142e0c60  unbound
   7  key.px <at> 0x1142e0c70  unbound
   8  key.px <at> 0x1142e0c80  unbound
   9  key.px <at> 0x1142e0c90  unbound
   10  key.px <at> 0x1142e0ca0  unbound
   11  key.px <at> 0x1142e0cb0  unbound
   12  key.px <at> 0x1142e0cc0  unbound
   13  key.px <at> 0x1142e0cd0  unbound
   14  key.px <at> 0x1142e0ce0  unbound
   15  key.px <at> 0x1142e0cf0  unbound

> (setq key nil)    ;;;   Here I wipe out the only non-disappearing reference to the key
(setq key nil)    ;;;   Here I wipe out the only non-disappearing reference to the key

NIL
> (gctools:garbage-collect)  ;; I force garbage collections - the key object is not collected although I think it should be
(gctools:garbage-collect)

> (gctools:garbage-collect)
(gctools:garbage-collect)

> (gctools:garbage-collect)
(gctools:garbage-collect)

> (gctools:garbage-collect)
(gctools:garbage-collect)

> ;; No finalization messages have been printed
;; No finalization message

> (low-level-describe ht)  ;; Here we see the key object is still alive
(low-level-describe ht)
WeakKeyHashTable   size: 16
   keys memory range:  0x1142e0c00  - 0x1142e0d00 
   0  key.px <at> 0x1142e0c00  (1 . 2) <at> 0x1120c5700   -->   9999
   1  key.px <at> 0x1142e0c10  unbound
   2  key.px <at> 0x1142e0c20  unbound
   3  key.px <at> 0x1142e0c30  unbound
   4  key.px <at> 0x1142e0c40  unbound
   5  key.px <at> 0x1142e0c50  unbound
   6  key.px <at> 0x1142e0c60  unbound
   7  key.px <at> 0x1142e0c70  unbound
   8  key.px <at> 0x1142e0c80  unbound
   9  key.px <at> 0x1142e0c90  unbound
   10  key.px <at> 0x1142e0ca0  unbound
   11  key.px <at> 0x1142e0cb0  unbound
   12  key.px <at> 0x1142e0cc0  unbound
   13  key.px <at> 0x1142e0cd0  unbound
   14  key.px <at> 0x1142e0ce0  unbound
   15  key.px <at> 0x1142e0cf0  unbound

> ;; Key is still there
;; Key is still there


> (weak-hash-table-put ht 0 nil 9999)
(weak-hash-table-put ht 0 nil 9999)
../../src/gctools/gcalloc.h:969  Registered disappearing link 0x1142e0c00  obj 0x9  return val = 1

> (gctools:garbage-collect)   ;; Now the object will be collected and a finalization function called
(gctools:garbage-collect)
../../src/gctools/gcalloc.h:894 Boehm finalized weak linked address 0x1120c5700 at 0x1142e0c00

> (low-level-describe ht)
(low-level-describe ht)
WeakKeyHashTable   size: 16
   keys memory range:  0x1142e0c00  - 0x1142e0d00 
   0  key.px <at> 0x1142e0c00  nil
   1  key.px <at> 0x1142e0c10  unbound
   2  key.px <at> 0x1142e0c20  unbound
   3  key.px <at> 0x1142e0c30  unbound
   4  key.px <at> 0x1142e0c40  unbound
   5  key.px <at> 0x1142e0c50  unbound
   6  key.px <at> 0x1142e0c60  unbound
   7  key.px <at> 0x1142e0c70  unbound
   8  key.px <at> 0x1142e0c80  unbound
   9  key.px <at> 0x1142e0c90  unbound
   10  key.px <at> 0x1142e0ca0  unbound
   11  key.px <at> 0x1142e0cb0  unbound
   12  key.px <at> 0x1142e0cc0  unbound
   13  key.px <at> 0x1142e0cd0  unbound
   14  key.px <at> 0x1142e0ce0  unbound
   15  key.px <at> 0x1142e0cf0  unbound



--
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.

_______________________________________________
bdwgc mailing list
bdwgc-ZwoEplunGu1I4Lznb4ZCK0B+6BGkLq7r@public.gmane.org
https://lists.opendylan.org/mailman/listinfo/bdwgc


_______________________________________________
bdwgc mailing list
bdwgc@...
https://lists.opendylan.org/mailman/listinfo/bdwgc
Christian Schafmeister | 6 Aug 20:03 2014
Picon
Picon

[Gc] Trouble with disappearing links


I’m using disappearing links for the first time (I may not know what I’m doing) and they seem to be keeping the object that they are pointing to alive.
I don’t know why the disappearing link is not disappearing and appears to keep the object it points to alive.
Should I not be allocating the memory that contains the disappearing link within the Boehm managed memory?

Pseudo code:
1) I’m implementing a weak key hash table
2) I allocate two parallel arrays (key array/value array) using GC_MALLOC.
3) I write a key and value into each corresponding array.
4) I call GC_register_disappearing_link(&key[0]).  This should make the key entry at index 0 a disappearing link.
5) Then I destroy all references to the key other than what are in the array.
6) The key object stays alive until I reset the key[0] entry - then it gets finalized.


Here is a transcript of the Common Lisp session:

> (setq ht (make-weak-key-hash-table))
(setq ht (make-weak-key-hash-table))

#<CORE:WEAK-KEY-HASH-TABLE <at> 0x11464c1d8) > 
> (low-level-describe ht)
(low-level-describe ht)
WeakKeyHashTable   size: 16
   keys memory range:  0x1142e0c00  - 0x1142e0d00 
   0  key.px <at> 0x1142e0c00  unbound
   1  key.px <at> 0x1142e0c10  unbound
   2  key.px <at> 0x1142e0c20  unbound
   3  key.px <at> 0x1142e0c30  unbound
   4  key.px <at> 0x1142e0c40  unbound
   5  key.px <at> 0x1142e0c50  unbound
   6  key.px <at> 0x1142e0c60  unbound
   7  key.px <at> 0x1142e0c70  unbound
   8  key.px <at> 0x1142e0c80  unbound
   9  key.px <at> 0x1142e0c90  unbound
   10  key.px <at> 0x1142e0ca0  unbound
   11  key.px <at> 0x1142e0cb0  unbound
   12  key.px <at> 0x1142e0cc0  unbound
   13  key.px <at> 0x1142e0cd0  unbound
   14  key.px <at> 0x1142e0ce0  unbound
   15  key.px <at> 0x1142e0cf0  unbound

> (setq key (cons 1 2))
(setq key (cons 1 2))

(1 . 2)
> (weak-hash-table-put ht 0 key 9999)
(weak-hash-table-put ht 0 key 9999)
../../src/gctools/gcalloc.h:969  Registered disappearing link 0x1142e0c00  obj 0x1120c5700  return val = 0

> (low-level-describe ht)
(low-level-describe ht)
WeakKeyHashTable   size: 16
   keys memory range:  0x1142e0c00  - 0x1142e0d00 
   0  key.px <at> 0x1142e0c00  (1 . 2) <at> 0x1120c5700   -->   9999
   1  key.px <at> 0x1142e0c10  unbound
   2  key.px <at> 0x1142e0c20  unbound
   3  key.px <at> 0x1142e0c30  unbound
   4  key.px <at> 0x1142e0c40  unbound
   5  key.px <at> 0x1142e0c50  unbound
   6  key.px <at> 0x1142e0c60  unbound
   7  key.px <at> 0x1142e0c70  unbound
   8  key.px <at> 0x1142e0c80  unbound
   9  key.px <at> 0x1142e0c90  unbound
   10  key.px <at> 0x1142e0ca0  unbound
   11  key.px <at> 0x1142e0cb0  unbound
   12  key.px <at> 0x1142e0cc0  unbound
   13  key.px <at> 0x1142e0cd0  unbound
   14  key.px <at> 0x1142e0ce0  unbound
   15  key.px <at> 0x1142e0cf0  unbound

> (setq key nil)    ;;;   Here I wipe out the only non-disappearing reference to the key
(setq key nil)    ;;;   Here I wipe out the only non-disappearing reference to the key

NIL
> (gctools:garbage-collect)  ;; I force garbage collections - the key object is not collected although I think it should be
(gctools:garbage-collect)

> (gctools:garbage-collect)
(gctools:garbage-collect)

> (gctools:garbage-collect)
(gctools:garbage-collect)

> (gctools:garbage-collect)
(gctools:garbage-collect)

> ;; No finalization messages have been printed
;; No finalization message

> (low-level-describe ht)  ;; Here we see the key object is still alive
(low-level-describe ht)
WeakKeyHashTable   size: 16
   keys memory range:  0x1142e0c00  - 0x1142e0d00 
   0  key.px <at> 0x1142e0c00  (1 . 2) <at> 0x1120c5700   -->   9999
   1  key.px <at> 0x1142e0c10  unbound
   2  key.px <at> 0x1142e0c20  unbound
   3  key.px <at> 0x1142e0c30  unbound
   4  key.px <at> 0x1142e0c40  unbound
   5  key.px <at> 0x1142e0c50  unbound
   6  key.px <at> 0x1142e0c60  unbound
   7  key.px <at> 0x1142e0c70  unbound
   8  key.px <at> 0x1142e0c80  unbound
   9  key.px <at> 0x1142e0c90  unbound
   10  key.px <at> 0x1142e0ca0  unbound
   11  key.px <at> 0x1142e0cb0  unbound
   12  key.px <at> 0x1142e0cc0  unbound
   13  key.px <at> 0x1142e0cd0  unbound
   14  key.px <at> 0x1142e0ce0  unbound
   15  key.px <at> 0x1142e0cf0  unbound

> ;; Key is still there
;; Key is still there


> (weak-hash-table-put ht 0 nil 9999)
(weak-hash-table-put ht 0 nil 9999)
../../src/gctools/gcalloc.h:969  Registered disappearing link 0x1142e0c00  obj 0x9  return val = 1

> (gctools:garbage-collect)   ;; Now the object will be collected and a finalization function called
(gctools:garbage-collect)
../../src/gctools/gcalloc.h:894 Boehm finalized weak linked address 0x1120c5700 at 0x1142e0c00

> (low-level-describe ht)
(low-level-describe ht)
WeakKeyHashTable   size: 16
   keys memory range:  0x1142e0c00  - 0x1142e0d00 
   0  key.px <at> 0x1142e0c00  nil
   1  key.px <at> 0x1142e0c10  unbound
   2  key.px <at> 0x1142e0c20  unbound
   3  key.px <at> 0x1142e0c30  unbound
   4  key.px <at> 0x1142e0c40  unbound
   5  key.px <at> 0x1142e0c50  unbound
   6  key.px <at> 0x1142e0c60  unbound
   7  key.px <at> 0x1142e0c70  unbound
   8  key.px <at> 0x1142e0c80  unbound
   9  key.px <at> 0x1142e0c90  unbound
   10  key.px <at> 0x1142e0ca0  unbound
   11  key.px <at> 0x1142e0cb0  unbound
   12  key.px <at> 0x1142e0cc0  unbound
   13  key.px <at> 0x1142e0cd0  unbound
   14  key.px <at> 0x1142e0ce0  unbound
   15  key.px <at> 0x1142e0cf0  unbound

_______________________________________________
bdwgc mailing list
bdwgc@...
https://lists.opendylan.org/mailman/listinfo/bdwgc
Peter Wang | 5 Aug 07:54 2014
Picon

[Gc] Support winpthreads

Hi,

Here is a preliminary patch to support building with winpthreads.
It overloads GC_WIN32_PTHREADS to mean either pthreads-win32 or
winpthreads.  An alternative might be to add a new symbol
GC_WINPTHREADS?

It seems to be fine but I have not tested it extensively.
Perhaps someone else has come across it already?

Peter
_______________________________________________
bdwgc mailing list
bdwgc@...
https://lists.opendylan.org/mailman/listinfo/bdwgc
Christian Weisgerber | 25 Jul 23:02 2014
Picon

Re: [Gc] sparc_mach_dep.lo in wrong dir

Ivan Maidanski:

> Wed, 23 Apr 2014 15:01:23 -0700 from Mojca Miklavec:
> >Building gc 7.4 on Sparc Solaris fails with
> >libtool: link: `src/sparc_mach_dep.lo' is not a valid libtool object
> >The problem is that the file is actually in top level
> >(src/sparc_mach_dep.lo), not under src. If I manually edit the Makefile
> >and remove src/ from all occurencies of src/sparc_mach_dep.lo, the
> >compilation succeeds.
> 
> Should fixed in https://github.com/ivmai/bdwgc/commit/e2bf29b7801357c1ad28c2948a3f88c655dd722a

Building gc 7.4.2 on OpenBSD/sparc64 still fails, but differently:

make: don't know how to make sparc_mach_dep.lo (prerequisite of: libgc.la)

Now there is no Makefile rule to derive sparc_mach_dep.lo from
src/sparc_mach_dep.S.

--

-- 
Christian "naddy" Weisgerber                          naddy@...
Richard Brooksby | 10 Jul 12:28 2014

[Gc] C optimizer hazards in practice

https://github.com/ivmai/bdwgc/blob/3ec2783588c8ba53a0b6a347d61d4e5c1b5dc80e/README.md says:

> Some C optimizers may lose the last undisguised pointer to a memory object as a consequence of clever
optimizations. This has almost never been observed in practice.

Does anyone have any stories to tell about when this *has* been observed, or any links to accounts of this happening?

Background: The Memory Pool System <http://www.ravenbrook.com/project/mps/> team is currently
investigating a case of the Microsoft C compiler optimising away the last reference to an object, causing
it to be prematurely recycled and corrupt the heap.  See
<http://mailman.ravenbrook.com/pipermail/mps-discussion/2014-July/000144.html> for details,
including repro, disassembly, analysis, etc.
Lucas Meijer | 30 Jun 10:24 2014

[Gc] adding profiling callbacks

Hi,

Unity is a game development tool that for some platforms uses Boehm to collect garbage generated by our users .NET code.

Our builtin profiler can profile all the different game subsystems we have, and I've added support to it for also profiling the boehm collector.

I've added profiling callbacks to our version of boehm, and would like to ask this list if there's any interest in taking these patches. (the friendly folks at the mono project did pretty much the same for their fork of boehm).

Here's the patches we have now. I'd be more than happy to adapt/fix any concerns they might have.

Bye, Lucas


diff --git a/External/bdwgc/alloc.c b/External/bdwgc/alloc.c
--- a/External/bdwgc/alloc.c
+++ b/External/bdwgc/alloc.c
 <at>  <at>  -347,6 +347,18  <at>  <at> 
 STATIC GC_bool GC_stopped_mark(GC_stop_func stop_func);
 STATIC void GC_finish_collection(void);

+static GC_event_callback_func GC_event_callback = NULL;
+
+void GC_set_event_callback(GC_event_callback_func func)
+{
+  GC_event_callback = func;
+}
+
+GC_event_callback_func GC_get_event_callback()
+{
+  return GC_event_callback;
+}
+
 /*
  * Initiate a garbage collection if appropriate.
  * Choose judiciously
 <at>  <at>  -420,6 +432,10  <at>  <at> 
 #   endif
     ASSERT_CANCEL_DISABLED();
     if (GC_dont_gc || (*stop_func)()) return FALSE;
+
+    if (GC_event_callback)
+      GC_event_callback (GC_EVENT_START, NULL);
+
     if (GC_incremental && GC_collection_in_progress()) {
       GC_COND_LOG_PRINTF(
             "GC_try_to_collect_inner: finishing collection in progress\n");
 <at>  <at>  -476,6 +492,10  <at>  <at> 
                       MS_TIME_DIFF(current_time,start_time));
       }
 #   endif
+    
+    if (GC_event_callback)
+      GC_event_callback (GC_EVENT_END, NULL);
+
     return(TRUE);
 }

 <at>  <at>  -607,10 +627,20  <at>  <at> 
         GET_TIME(start_time);
 #   endif

+    if (GC_event_callback)
+      GC_event_callback (GC_EVENT_PRE_STOP_WORLD, NULL);
+
     STOP_WORLD();
 #   ifdef THREAD_LOCAL_ALLOC
       GC_world_stopped = TRUE;
 #   endif
+
+    if (GC_event_callback)
+    {
+      GC_event_callback (GC_EVENT_POST_STOP_WORLD, NULL);
+      GC_event_callback (GC_EVENT_MARK_START, NULL);
+    }
+
         /* Output blank line for convenience here */
     GC_COND_LOG_PRINTF(
               "\n--> Marking for collection #%lu after %lu allocated bytes\n",
 <at>  <at>  -632,10 +662,19  <at>  <at> 
             GC_COND_LOG_PRINTF("Abandoned stopped marking after"
                                " %u iterations\n", i);
             GC_deficit = i;     /* Give the mutator a chance.   */
+
+            if (GC_event_callback)
+            {
+              GC_event_callback (GC_EVENT_MARK_END, NULL);
+              GC_event_callback (GC_EVENT_PRE_START_WORLD, NULL);
+            }
 #           ifdef THREAD_LOCAL_ALLOC
               GC_world_stopped = FALSE;
 #           endif
             START_WORLD();
+            if (GC_event_callback)
+              GC_event_callback (GC_EVENT_POST_START_WORLD, NULL);
+
             return(FALSE);
           }
           if (GC_mark_some(GC_approx_sp())) break;
 <at>  <at>  -656,7 +695,18  <at>  <at> 
 #   ifdef THREAD_LOCAL_ALLOC
       GC_world_stopped = FALSE;
 #   endif
+
+    if (GC_event_callback)
+    {
+      GC_event_callback (GC_EVENT_MARK_END, NULL);
+      GC_event_callback (GC_EVENT_PRE_START_WORLD, NULL);
+    }
+
     START_WORLD();
+
+    if (GC_event_callback)
+      GC_event_callback (GC_EVENT_POST_START_WORLD, NULL);
+
 #   ifndef SMALL_CONFIG
       if (GC_PRINT_STATS_FLAG) {
         unsigned long time_diff;
diff --git a/External/bdwgc/include/gc.h b/External/bdwgc/include/gc.h
--- a/External/bdwgc/include/gc.h
+++ b/External/bdwgc/include/gc.h
 <at>  <at>  -105,6 +105,25  <at>  <at> 
 /* Public R/W variables */
 /* The supplied setter and getter functions are preferred for new code. */

+typedef enum {
+    GC_EVENT_START,
+    GC_EVENT_MARK_START,
+    GC_EVENT_MARK_END,
+    GC_EVENT_RECLAIM_START,
+    GC_EVENT_RECLAIM_END,
+    GC_EVENT_END,
+    GC_EVENT_PRE_STOP_WORLD,
+    GC_EVENT_POST_STOP_WORLD,
+    GC_EVENT_PRE_START_WORLD,
+    GC_EVENT_POST_START_WORLD,
+    GC_EVENT_SUSPENDED_THREAD,
+    GC_EVENT_UNSUSPENDED_THREAD,
+} GCEventType;
+
+typedef void * (GC_CALLBACK * GC_event_callback_func)(GCEventType eventType, void* data);
+GC_API void GC_CALL GC_set_event_callback(GC_event_callback_func);
+GC_API GC_event_callback_func GC_CALL GC_get_event_callback(void);
+
 typedef void * (GC_CALLBACK * GC_oom_func)(size_t /* bytes_requested */);
 GC_API GC_ATTR_DEPRECATED GC_oom_func GC_oom_fn;
                         /* When there is insufficient memory to satisfy */
diff --git a/External/bdwgc/pthread_stop_world.c b/External/bdwgc/pthread_stop_world.c
--- a/External/bdwgc/pthread_stop_world.c
+++ b/External/bdwgc/pthread_stop_world.c
 <at>  <at>  -505,6 +505,9  <at>  <at> 
                     n_live_threads--;
                     break;
                 case 0:
+                    GC_get_event_callback_func cb = GC_get_event_callback();
+                    if (cb)
+                      cb(GC_EVENT_SUSPENDED_THREAD, p->id);
                     break;
                 default:
                     ABORT_ARG1("pthread_kill failed at suspend",
 <at>  <at>  -829,6 +832,9  <at>  <at> 
                     n_live_threads--;
                     break;
                 case 0:
+                    GC_get_event_callback_func cb = GC_get_event_callback();
+                    if (cb)
+                      cb(GC_EVENT_UNSUSPENDED_THREAD, p->id);
                     break;
                 default:
                     ABORT_ARG1("pthread_kill failed at resume",
_______________________________________________
bdwgc mailing list
bdwgc@...
https://lists.opendylan.org/mailman/listinfo/bdwgc
Stefan Kral | 25 Jun 13:26 2014
Picon

[Gc] Supplying hugepages to the GC

Hello everyone.

I have had good results speeding up some big lookup-table accesses by backing them with hugepages (and initializing them upon application startup.)

Is there a straightforward way I could configure / instruct the gc to alloc the first few heap chunks using hugepages when available? Possibly by letting it alloc memory with (hugepage backed) malloc instead of mmap or sbrk?

Regards, Stefan Kral.

_______________________________________________
bdwgc mailing list
bdwgc@...
https://lists.opendylan.org/mailman/listinfo/bdwgc

Gmane