23 May 2013 04:22
23 May 2013 00:26
Trying to fix test_stack on powerpc.
Does anyone have a hint as to where to look for the test_stack failure on powerpc? It seems to fail in one of two ways. Either it fails when one thread fails to do a pop of the stack (it gets a 0). Or it fails when checking the stack at the end and the stack has a circular list in it, due to the next pointers having gotten messed up. For example for the circular list problem it can end up like this (note I did add some extra debug printing to things): Initial list (nthreads = 3): 6 0x1059b0e8 next=1059b0d8 5 0x1059b0d8 next=1059b0c8 4 0x1059b0c8 next=1059b028 3 0x1059b028 next=1059b018 2 0x1059b018 next=1059b008 1 0x1059b008 next=00000000 starting thread 1 starting thread 0 starting thread 2 finished thread 2: 360828 total ops finished thread 0: 289581 total ops finished thread 1: 349590 total ops 3 367 final list (should be reordered initial list): (dump of original list locations)(Continue reading)
22 May 2013 23:56
Re[2]: [bdwgc] Convert readme to markdown. (#24)
Hi David,
Thank you for the explanation (for the work of rewriting README in .md style) - I've committed it (plus minor changes and fix of a typo in the title):
* https://github.com/ivmai/bdwgc/commit/a36f601f2219402f1dfbae82f089002e1d4f7869
* https://github.com/ivmai/bdwgc/commit/b6664ad715fb835990da5c3b8e97cf036997958e
Regards,
Ivan
Sun, 12 May 2013, 9:36 -07:00 from David Terei <notifications <at> github.com>:
Thank you for the explanation (for the work of rewriting README in .md style) - I've committed it (plus minor changes and fix of a typo in the title):
* https://github.com/ivmai/bdwgc/commit/a36f601f2219402f1dfbae82f089002e1d4f7869
* https://github.com/ivmai/bdwgc/commit/b6664ad715fb835990da5c3b8e97cf036997958e
Regards,
Ivan
Sun, 12 May 2013, 9:36 -07:00 from David Terei <notifications <at> github.com>:
Why plain? The point of markdown is to be human readable so that it looks nice without any parsing. HTML maybe, but I also converted it to markdown so that when you go to the github page the readme looks nice since Github will convert it to HTML automatically.
The Github pages feature also accepts markdown that it will convert to HTML automatically...
—
Reply to this email directly or view it on GitHub.
_______________________________________________ Gc mailing list Gc@... http://www.hpl.hp.com/hosted/linux/mail-archives/gc/
12 May 2013 14:17
Re: [bdwgc] Convert readme to markdown. (#24)
Hi David,
I think it is good to have ability to use some mark-down features for documentation (not only main README file). But probably a make file rule should be added to generate plain (and/or html) file from the .md file.
Any suggestions, pros/cons from the community about it?
Thank you
Regards,
Ivan
Fri, 10 May 2013, 13:22 -07:00 from David Terei <notifications <at> github.com>:
I think it is good to have ability to use some mark-down features for documentation (not only main README file). But probably a make file rule should be added to generate plain (and/or html) file from the .md file.
Any suggestions, pros/cons from the community about it?
Thank you
Regards,
Ivan
Fri, 10 May 2013, 13:22 -07:00 from David Terei <notifications <at> github.com>:
You can merge this Pull Request by running
git pull https://github.com/dterei/bdwgc mdOr view, comment on, or merge it at:
https://github.com/ivmai/bdwgc/pull/24
Commit Summary
- Convert readme to markdown.
File Changes
- M Makefile.direct (2)
- M Makefile.dj (2)
- R README.md (536)
- M doc/doc.am (2)
Patch Links:
_______________________________________________ Gc mailing list Gc@... http://www.hpl.hp.com/hosted/linux/mail-archives/gc/
14 Apr 2013 08:55
Re: Silently fails to allocate memory?
On Sat, Apr 13, 2013 at 4:04 PM, Colin LeMahieu <clemahieu <at> gmail.com> wrote:
You certainly haven't explained what you mean by "give up". Someone already asked, null pointer, freeze? What is a "give up".
I have said that I am working on getting a more definitive definition. At current I am not sure what it does exactly except that I have a large loop, that creates objects, and compares or modifies them in some fashion and at some seemingly arbitrary point all of the compares/modifications will simply fail. The code will continue to loop and "complete" so long as the loop ending condition does not depend on the values of any of the objects created. If for example, I created a "number" object that held a value, and each iteration created another number object that holds the next value, and do this ten thousand times, it will complete the ten thousand iterations but the "last value" that it would show (assuming I printed each value to the console as it was created) would be about 3500. If instead the loop is set to loop until I get an object that has a value of say, 5000, it would never reach that point no matter how many objects I created with iterative values, as if the objects are just "not created" after a certain point. Since my code has many checks to deal with NULL values, which are allowed in every situation (and handled differently depending on the situation), if its a case of "new" returning NULL every time after that point, it will be difficult to trace but I am working on a solution to help me narrow it down. (Of course, if I just dont free the memory at all, or use libgc at all, all of the objects are created, and the amount printed to the console, and the final number of the last object are exactly as expected, but I need libgc).
On Sat, Apr 13, 2013 at 8:11 PM, Brian Beuning <bbeuning-gQiW9Mwg1h1Wk0Htik3J/w@public.gmane.org> wrote:
I recently used GC to allocate 2 GB of objects and it worked fine.
I do not know if its the size, or the count. My loop test loops about 10 million times, every time creating about 7 trivial objects. At about 3500 it fails (The 10 million loop is overkill, but it never even makes it to 4000). If I create a few less objects each iteration, it will fail at a later time, relative to the number of objects being created. If I just let the memory be unfreed and accumulate (i.e. temporarily disable libgc) then it completes all 10 million iterations without issue. This is on a 32bit Ubuntu Linux install.
I think we get what you are doing and the issue you are seeing.
There are too many things your code might be doing wrong for anyone to guess at.
If you can reproduce the issue with some small test case (under 100 lines)
then someone will probably volunteer to look at it.
I am combing through my code as best I can, making sure every vector, every string, everything is using libgc (even things not remotely involved in the test case, just to be certain) and will retest when I am sure that I am not allocating memory through any other means. I posed the question only because I thought that worst case scenario if I didnt use libgc everywhere would be leaked/unfreed memory, not this so I assumed there was some key I was missing. A compile flag to use (or not use), or some additional step like the GC_INIT command, etc...If no one has any idea why this would happen, I will dig deeper and email again when I have more information (or possibly a test case) that can help solve the issue.
_______________________________________________ Gc mailing list Gc@... http://www.hpl.hp.com/hosted/linux/mail-archives/gc/
13 Apr 2013 07:55
Re: Silently fails to allocate memory?
On Sat, Apr 13, 2013 at 2:43 PM, The Devils Jester <thedevilsjester <at> gmail.com> wrote:
If, for example, I am not deleting something because it was in the destructor, would not the worse case scenario be unfreed memory? What could cause libgc to "give up" on Linux (but not apparently on OSX)? This is not actually the case though (I dont do anything in destructors).
It shouldn't "give up". I've never seen such a thing. And you don't describe what you mean by "give up" or "silently fail". Does it return a null pointer?
If all you are doing is allocating memory then unfreed memory is the worst that can happen. If you were relying on C++ destructors to close files then you could run out of file handles.Or similar for other resources.
If my object has a vector of pointers, would not those be automatically collected as well?
Of course, if those pointers were to memory allocated by GC and there are no other live pointers to those objects. And if the memory for the vector itself has been allocated by GC.
The question is not about it working or not (I believe it is) its about any call to new silently failing after a certain number of objects created. (Which as I said works fine if I manually call delete or I disable libgc.)
What could I be doing wrong that would cause something like this? All I am doing is using gc inherited objects, with the new command. Is C++11 or certain compiler flags an issue? (Just O2 iirc).
With your talk about "vectors" I wonder whether you are in fact doing something like storing the pointers to your GC-allocated objects (only) in an STL vector which is using ordinary malloc() allocated memory internally.. In which case as far as the GC is concerned there is no pointer to your objects and the memory will be reused for new objects.
In that case when you allocate a new object the address will be the same an for an object that you allocated earlier, and still think you are using.
On Apr 12, 2013 9:03 PM, "Bruce Hoult" <bruce-zQEARAI6hqHYtjvyW6yDsg@public.gmane.org> wrote:The GC does actually work, and thousands of programs (and indeed products) use it, so the most likely explanation is that you're doing something wrong. Unfortunately we have no way of knowing without seeing your code.
You're aware that C++ destructors won't be called by the GC? They're not needed if all they'd do is free other memory that is also GC allocated, but they will be if you need to close files or free other memory, such as that allocated using malloc by library code (including the STL). In that case you need to register finalizers for the affected objects.On Sat, Apr 13, 2013 at 9:51 AM, The Devils Jester <thedevilsjester <at> gmail.com> wrote:It is difficult to describe the issue I am having with libgc.
I have a for loop that allocates thousands of (small) objects. Each iteration of the loop only allocates a few new objects, but when the loop is sufficiently large (usually around 3500 iterations), then for some reason it just...well gives up. It does not hang or segfault, or give any indication that something is wrong, but it (and this is difficult to determine) seems to silently stop allowing the objects to be created.
If I disable libgc (or manually call delete, which is not a working long term solution), the loop executes correctly. This is on Ubuntu Linux using GCC 4.7 .
On Mac OS X, I cannot tell if its working (which it appears to be) or if the limit to "give up" is might higher.
Is there some trick to this? Am I using it wrong? How I am using libgc is by simply including "gc_cpp.h" calling "GC_INIT" and having my class(es) inherit from gc.
--
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.
_______________________________________________
Gc mailing list
Gc-V9/bV5choks0duEV06nMPQ@public.gmane.orgm
http://www.hpl.hp.com/hosted/linux/mail-archives/gc/
--
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.
_______________________________________________ Gc mailing list Gc@... http://www.hpl.hp.com/hosted/linux/mail-archives/gc/
13 Apr 2013 07:12
Using GC in C++ via multiple inheritance
Hello. I've been refactoring parts of Asymptote (asymptote.sf.net)
code for inclusion into my own project and was hence introduced to GC.
I understand that by subclassing a new class I create say MyClass from
class gc, instances of MyClass will be garbage collected since
operator new for it is now overloaded to use GC_MALLOC. However, I'd
like to clarify one thing:
If I have to create MyClass as a subclass of a LibraryClass which
doesn't use gc, is it sufficient for me to multiply inherit from gc
and do:
class MyClass : public LibraryClass, public gc {
...
} ;
so that new-ed instances of MyClass will be garbage collected
including the LibraryClass sub-objects within them? I hope so, but I
didn't want to assume, so am asking for this clarification here.
Note that I am not an expert programmer, only an intermediate one, so
I may require more explanations than others. Thank you for your time
and patience.
--
Shriramana Sharma ஶ்ரீரமணஶர்மா श्रीरमणशर्मा
12 Apr 2013 23:51
Silently fails to allocate memory?
It is difficult to describe the issue I am having with libgc.
I have a for loop that allocates thousands of (small) objects. Each iteration of the loop only allocates a few new objects, but when the loop is sufficiently large (usually around 3500 iterations), then for some reason it just...well gives up. It does not hang or segfault, or give any indication that something is wrong, but it (and this is difficult to determine) seems to silently stop allowing the objects to be created.
If I disable libgc (or manually call delete, which is not a working long term solution), the loop executes correctly. This is on Ubuntu Linux using GCC 4.7 .
On Mac OS X, I cannot tell if its working (which it appears to be) or if the limit to "give up" is might higher.
Is there some trick to this? Am I using it wrong? How I am using libgc is by simply including "gc_cpp.h" calling "GC_INIT" and having my class(es) inherit from gc.
_______________________________________________ Gc mailing list Gc@... http://www.hpl.hp.com/hosted/linux/mail-archives/gc/
11 Apr 2013 02:58
[win32] Isn't this a race condition?
Hi list,
In the (admittedly somewhat old) version of boehm-gc in the GCC source tree,
I see the following failure from thread_leak_test about 50% of the time (out
of 200 testruns):
> $ ./.libs/thread_leak_test.exe
> SuspendThread failed
> Aborted (core dumped)
It's coming from GC_suspend in win32_threads.c, relevant snippets shown below:
/* Suspend the given thread, if it's still active. */
STATIC void GC_suspend(GC_thread t)
{
[ ... snip ... ]
DWORD exitCode;
[ ... snip ... ]
if (GetExitCodeThread(t -> handle, &exitCode) &&
exitCode != STILL_ACTIVE) {
# ifdef GC_PTHREADS
t -> stack_base = 0; /* prevent stack from being pushed */
# else
/* this breaks pthread_join on Cygwin, which is guaranteed to */
/* only see user pthreads */
GC_ASSERT(GC_win32_dll_threads);
GC_delete_gc_thread_no_free(t);
# endif
return;
}
[ ... snip ... ]
# else /* !MSWINCE */
if (SuspendThread(t -> handle) == (DWORD)-1)
ABORT("SuspendThread failed");
# endif /* !MSWINCE */
t -> suspended = (unsigned char)TRUE;
[ ... snip ... ]
}
Judging by reading the code, it seemed likely to me that there was a race
condition where a thread could still be active at the time GetExitCodeThread
was called, but then have exited by the time SuspendThread was called,
resulting in that call failing. I modified that clause to read:
if (SuspendThread(thread_table[i].handle) == (DWORD)-1) {
if (GetExitCodeThread(thread_table[i].handle,&exitCode) &&
exitCode != STILL_ACTIVE) {
ABORT("Race condition");
}
ABORT("SuspendThread failed");
}
i.e., it now checks again if the thread has exited when SuspendThread fails,
and now I occasionally see the "Race condition" message - although not often
(about 5% of tests); most often it still just shows the "SuspendThread failed"
message. I suspect (but cannot prove) that during the thread shutdown code in
the windows kernel, there is a stage at which it stops being suspendable
before the final exit code is stored.
In the HEAD version of bdwgc, analogous code still exists in
win32_threads.c/GC_suspend(), but I don't see it fail (in 200 tests).
Does anyone know why it's no longer a problem? I note that an awful lot has
changed in the rest of the test infrastructure, perhaps there's some more
locking somewhere (parallel marking?) that prevents the problem from arising.
cheers,
DaveK
3 Apr 2013 11:16
Re: GC problem on Android
Hi Manuel,
Tue, 2 Apr 2013, 14:28 +02:00 from Manuel.Serrano <at> inria.fr:
Which test case to fail? What are the gcc options specified?
I have Galaxy S3 (4.1), Nexus i9250 (4.1), Nexus 10 Tab (4.2), and Nexus 7 (4.2.2) available for testing.
I've just run gctest on Nexus 7 (from master branch):
C:\android-ndk\toolchains\arm-linux-androideabi-4.7\prebuilt\windows\bin\arm-linux-androideabi-gcc.exe -fno-short-enums -I c:/android-ndk/platforms/android-9/arch-arm/usr/include -nostdlib -lc -Wl,-rpath-link=c:/android-ndk/platforms/android-9/arch-arm/usr/lib,-dynamic-linker=/system/bin/linker,--gc-sections,-z,nocopyreloc -L c:/android-ndk/platforms/android-9/arch-arm/usr/lib c:/android-ndk/platforms/android-9/arch-arm/usr/lib/crtbegin_static.o c:/android-ndk/platforms/android-9/arch-arm/usr/lib/crtend_android.o -O2 -fno-strict-aliasing -I include -I libatomic_ops/src -DGC_THREADS -DNO_EXECUTE_PERMISSION -DTHREAD_LOCAL_ALLOC -DPARALLEL_MARK -DGC_GCJ_SUPPORT -DNO_DEBUGGING -DUSE_MMAP -DUSE_MUNMAP -DALL_INTERIOR_POINTERS -DJAVA_FINALIZATION -DATOMIC_UNCOLLECTABLE -DIGNORE_DYNAMIC_LOADING -mcpu=cortex-a9 -DDONT_USE_ATEXIT -DGC_NOT_DLL -s tests/test.c extra/gc.c -lgcc -ldl
Completed 6 tests
Allocated 10981406 collectable objects
Allocated 1218 uncollectable objects
Allocated 7085658 atomic objects
Allocated 136648 stubborn objects
Finalized 13519/13519 objects - finalization is probably ok
Total number of bytes allocated is 590082877
Final heap size is 671744 bytes
Completed 252 collections (using 4 marker threads)
Collector appears to work
Regards,
Ivan
Tue, 2 Apr 2013, 14:28 +02:00 from Manuel.Serrano <at> inria.fr:
Dear Ivan,
It took me couple of days before sending the patch of my previous mail
because I was actually busy struggling with another problem I had on
Android. I came to the conclusion that there is an error somewhere in
the collector implementation that makes applications break on Android.
The problem is not easy to reproduce. Actually I only see it on an
actual Nexus4 device running Android 4.2.2. All the other platforms I
have tested (Google Emulator/Android 4.2.2, Nexus 7/Android 4.2.2, Asus
transformer/Android 4.2.1, Samsung GS3/Android 4.1.2, HTC desired
HD/Android 2.3.5, Samsung GalaxyTab'7/2.3.4) work like a charm. The most
confusing thing is that the Nexus 7 works but not the Nexus 4. Apart
the Linux kernel (3.4.x on the Nexus 4, 3.0.x on the Nexus 7), the two
platforms are almost identical (quad-core ARM9, same Android version).
Which test case to fail? What are the gcc options specified?
I have Galaxy S3 (4.1), Nexus i9250 (4.1), Nexus 10 Tab (4.2), and Nexus 7 (4.2.2) available for testing.
I've just run gctest on Nexus 7 (from master branch):
C:\android-ndk\toolchains\arm-linux-androideabi-4.7\prebuilt\windows\bin\arm-linux-androideabi-gcc.exe -fno-short-enums -I c:/android-ndk/platforms/android-9/arch-arm/usr/include -nostdlib -lc -Wl,-rpath-link=c:/android-ndk/platforms/android-9/arch-arm/usr/lib,-dynamic-linker=/system/bin/linker,--gc-sections,-z,nocopyreloc -L c:/android-ndk/platforms/android-9/arch-arm/usr/lib c:/android-ndk/platforms/android-9/arch-arm/usr/lib/crtbegin_static.o c:/android-ndk/platforms/android-9/arch-arm/usr/lib/crtend_android.o -O2 -fno-strict-aliasing -I include -I libatomic_ops/src -DGC_THREADS -DNO_EXECUTE_PERMISSION -DTHREAD_LOCAL_ALLOC -DPARALLEL_MARK -DGC_GCJ_SUPPORT -DNO_DEBUGGING -DUSE_MMAP -DUSE_MUNMAP -DALL_INTERIOR_POINTERS -DJAVA_FINALIZATION -DATOMIC_UNCOLLECTABLE -DIGNORE_DYNAMIC_LOADING -mcpu=cortex-a9 -DDONT_USE_ATEXIT -DGC_NOT_DLL -s tests/test.c extra/gc.c -lgcc -ldl
Completed 6 tests
Allocated 10981406 collectable objects
Allocated 1218 uncollectable objects
Allocated 7085658 atomic objects
Allocated 136648 stubborn objects
Finalized 13519/13519 objects - finalization is probably ok
Total number of bytes allocated is 590082877
Final heap size is 671744 bytes
Completed 252 collections (using 4 marker threads)
Collector appears to work
Why?The problem is not easy to debug mostly because I'm not able to use gdb
on Android.
Regards,
Ivan
The symptom is as follows: when I compile my application enabling
multithreading _and_ when I don't compile with -DTHREAD_LOCAL_ALLOC,
then the application crashes, even before the first collection takes
place. If I disable multithreading support or if I compile with
-DTHREAD_LOCAL_ALLOC, then it runs smoothly and all the tests succeed.
I have tested three versions of collector (7.2d, 7.3alpha1, and
7.3alpha3). They all behave similarly.
Since I have found a configuration where everything is fine, this
problem is not crucial for me. However, I'm worried by the symptom and I
will sleep better if we could find and fix that bug. I would be pleased
to help. Just tell me what to do.
Cheers,
--
Manuel
_______________________________________________ Gc mailing list Gc@... http://www.hpl.hp.com/hosted/linux/mail-archives/gc/
2 Apr 2013 14:46
opposite of GC_exclude_static_roots
Hello All My understanding is that when mmap-ing (on GNU/Linux/Debian/Sid/x86-64) some textual (e.g. JSON or XML) file segment, it is better to inform the GC using GC_exclude_static_roots to avoid scanning it. However, I don't understand what should I do when I am munmap-ing that file segment because I have finish dealing with that file and that segment. Or maybe it does not matter (e.g. because the GC is internally using /proc/self/maps)? Or does GC_exclude_static_roots should only be used for some huge global or static pointer-less array? Regards. -- -- Basile STARYNKEVITCH http://starynkevitch.net/Basile/ email: basile<at>starynkevitch<dot>net mobile: +33 6 8501 2359 8, rue de la Faiencerie, 92340 Bourg La Reine, France *** opinions {are only mines, sont seulement les miennes} ***
RSS Feed