headers
Pascal Bourguignon | 5 Jun 2006 07:49
X-Face
Favicon

The weak-pointer weakness of sbcl garbage collector


Here's a simple test case with weak pointers that crashes the garbage
collector:

It crashes as well on (sb-ext:GC :full t) than on with (sb-ext:GC).

[pjb <at> thalassa clext]$ sbcl --userinit sbcl-crash.lisp 
[... a long time ...]
Argh! gc_find_free_space failed (first_page), nbytes=8.
 Gen StaPg UbSta LaSta LUbSt Boxed Unboxed LB   LUB  !move  Alloc  Waste   Trig    WP  GCs Mem-age
   0:     0     0     0     0     0     0     0     0        0     0        0 2000000   0  0.0000
   1:     0     0     0     0     0     0     0     0        0     0        0 2000000   0  0.0000
   2: 116387     0     0     0 86123     0     0     0        0 352757576     2232 2000000   0 -842625067682656816483086280949968522232121039800283591607923859376593011885590357036292519050064299540222668472178907187362251077093634053623508903500246905364540895066764456197487074077671512285511409842326964610696830697804014720028978861320622542885529817624442240568513359575358456516459888640.0000
   3:     0     0     0     0     0     0     0     0        0     0        0 2000000   0  0.0000
   4:     0     0     0     0     0     0     0     0        0     0        0 2000000   0  0.0000
   5:     0     0     0     0    85     6     0     0       19 362968     9768 2362968  66  0.0000
   6:     0     0     0     0  5565     0     0     0        0 22794240        0 2000000 5522  0.0000
   Total bytes allocated=536858112
fatal error encountered in SBCL pid 2316:

The system is too badly corrupted or confused to continue at the Lisp
level. If the system had been compiled with the SB-LDB feature, we'd drop
into the LDB low-level debugger now. But there's no LDB in this build, so
we can't really do anything but just exit, sorry.

[pjb <at> thalassa clext]$ sbcl --version
SBCL 0.9.12

[pjb <at> thalassa clext]$ cat sbcl-crash.lisp
(Continue reading)

Permalink | Reply | 2 comments |
headers
Juho Snellman | 6 Jun 2006 14:29
Picon
Picon
Favicon

Re: The weak-pointer weakness of sbcl garbage collector


Pascal Bourguignon <pjb <at> informatimago.com> writes:
> Here's a simple test case with weak pointers that crashes the garbage
> collector:

It's not actually crashing the garbage collector. It's running out of
memory, since your WEAK-LIST-LIST is buggy (if the first weak pointer
in the list hasn't been broken, it will loop infinitely and cons
madly; if it has been broken, it'll just loop infinitely).

Also, please rememeber that the SBCL garbage collector is conservative
on some platforms. There are no guarantees of finalizers getting run
or weak pointers getting broken immediately after some memory becomes
logically garbage.

--

-- 
Juho Snellman
Permalink | Reply | 1 comment |
headers
Pascal Bourguignon | 7 Jun 2006 10:45
X-Face
Favicon

Re: The weak-pointer weakness of sbcl garbage collector

Juho Snellman writes:
> 
> Pascal Bourguignon <pjb <at> informatimago.com> writes:
> > Here's a simple test case with weak pointers that crashes the garbage
> > collector:
> 
> It's not actually crashing the garbage collector. It's running out of
> memory, since your WEAK-LIST-LIST is buggy (if the first weak pointer
> in the list hasn't been broken, it will loop infinitely and cons
> madly; if it has been broken, it'll just loop infinitely).
> 
> Also, please rememeber that the SBCL garbage collector is conservative
> on some platforms. There are no guarantees of finalizers getting run
> or weak pointers getting broken immediately after some memory becomes
> logically garbage.

Thank you very much, and my excuses for the wrong bug report.

I believed it gave it even when I didn't call weak-list-list, I only
added it to the "bug" report when the compiler complained for not
having a definition for it...

(And must say I'm spoiled on clisp which gives me nice 
"*** - Lisp stack overflow. RESET" messages, and let me go on
debugging...).

--

-- 
__Pascal Bourguignon__                     http://www.informatimago.com/

PUBLIC NOTICE AS REQUIRED BY LAW: Any use of this product, in any
(Continue reading)

Permalink | Reply | 0 comments |
headers
Jonathon McKitrick | 10 Jun 2006 22:06

Help with odd startup messages


Hi all,

I'm having this problem on 0.9.11 on a Linux box and an Intel Mac when I load
my asdf project:

Debugger invoked on a SIMPLE-ERROR:0
  overwriting old FUN-INFO
      #<SB-C::FUN-INFO :ATTRIBUTES (FOLDABLE FLUSHABLE
	  SB-C::UNSAFELY-FLUSHABLE)>
	    for ROTATE-BYTE

It shows three of these in a row, and selecting 0 for each is accepted and
runs.  I'd like to solve the problem correctly, though.  How may I do this?

--
My other computer is your Windows box.
Permalink | Reply | 1 comment |
headers
Jonathon McKitrick | 11 Jun 2006 16:43

Re: Help with odd startup messages

On Sun, Jun 11, 2006 at 06:57:59AM -0500, William Harold Newman wrote:
: On Sat, Jun 10, 2006 at 09:06:34PM +0100, Jonathon McKitrick wrote:
: > I'm having this problem on 0.9.11 on a Linux box and an Intel Mac when I load
: > my asdf project:
: > 
: > Debugger invoked on a SIMPLE-ERROR:0
: >   overwriting old FUN-INFO
: >       #<SB-C::FUN-INFO :ATTRIBUTES (FOLDABLE FLUSHABLE
: > 	  SB-C::UNSAFELY-FLUSHABLE)>
: > 	    for ROTATE-BYTE
: > 
: > It shows three of these in a row, and selecting 0 for each is accepted and
: > runs.  I'd like to solve the problem correctly, though.  How may I do this?
: 
: I do have one guess: that when you say "three of these in a row" you
: don't actually mean three of the same error message, but three similar
: error messages, one each for ROTATE-BYTE, %ROTATE-BYTE, and
: %UNSIGNED-32-ROTATE-BYTE. That is what you might see if your system

That's it exactly.  So how can I pinpoint where the second load is occurring?
I thought asdf handled this automatically.

Jonathon McKitrick
--
My other computer is your Windows box.
Permalink | Reply | 0 comments |
headers
Nikodemus Siivola | 11 Jun 2006 21:17
Gravatar

Re: Help with odd startup messages

Jonathon McKitrick <jcm <at> FreeBSD-uk.eu.org> writes:

> I'm having this problem on 0.9.11 on a Linux box and an Intel Mac when I load
> my asdf project:
>
> Debugger invoked on a SIMPLE-ERROR:0
>   overwriting old FUN-INFO
>       #<SB-C::FUN-INFO :ATTRIBUTES (FOLDABLE FLUSHABLE
> 	  SB-C::UNSAFELY-FLUSHABLE)>
> 	    for ROTATE-BYTE

Is any of the components doing (asdf:oos asdf:load-op :sb-rotate-byte :force t)
by any chance?

Cheers,

  -- Nikodemus              Schemer: "Buddha is small, clean, and serious."
                   Lispnik: "Buddha is big, has hairy armpits, and laughs."
Permalink | Reply | 0 comments |
headers
rif | 13 Jun 2006 18:27
Picon
Favicon

SBCL and foreign library paths


I'm using SBCL (0.9.13, x86 linux) and I'm observing something that
seems odd to me.  I load a foreign library and call a foreign function
that itself calls dlopen.  It seems that if I set LD_LIBRARY_PATH to
include the path to the internally dlopen'd library BEFORE I started
sbcl (in emacs with M-x setenv), everything works fine.  However, if I
instead wrap the C function setenv and use that to set LD_LIBRARY_PATH
from within SBCL, the internal call to dlopen fails --- it complains
that it can't find the library.  Any ideas what would cause this
phenomenon?  Right now, the workaround I have is to ensure that the
other library resolves via other system mechanisms (e.g. ldconfig), but
I'm curious as to what's going on.

rif
Permalink | Reply | 6 comments |
headers
Nikodemus Siivola | 13 Jun 2006 20:13
Gravatar

Re: SBCL and foreign library paths

rif <rif <at> MIT.EDU> writes:

> I'm using SBCL (0.9.13, x86 linux) and I'm observing something that
> seems odd to me.  I load a foreign library and call a foreign function
> that itself calls dlopen.  It seems that if I set LD_LIBRARY_PATH to
> include the path to the internally dlopen'd library BEFORE I started
> sbcl (in emacs with M-x setenv), everything works fine.  However, if I
> instead wrap the C function setenv and use that to set LD_LIBRARY_PATH
> from within SBCL, the internal call to dlopen fails --- it complains
> that it can't find the library.  Any ideas what would cause this
> phenomenon?  Right now, the workaround I have is to ensure that the
> other library resolves via other system mechanisms (e.g. ldconfig), but
> I'm curious as to what's going on.

Offhand, I'd guess that the dynamic loader (the Linux one) takes
a look at LD_LIBRARY_PATH when the program starts, and remembers it.

Just a guess, though.

Cheers,

  -- Nikodemus              Schemer: "Buddha is small, clean, and serious."
                   Lispnik: "Buddha is big, has hairy armpits, and laughs."
Permalink | Reply | 0 comments |
headers
Gisle Sælensminde | 14 Jun 2006 17:00
Picon
Picon

X86-64 memory problems


I'm writing/running a program that handles a large number of protein 
structures, each requiring a significant amount of memory. I don't want 
to read these structures into memory each time I need them, so I cache 
them in a hash table. Whenever the process pass just undre 800 MB 
(according to top), sbcl crash with the following error message.

mmap: Cannot allocate memory
fatal error encountered in SBCL pid 32008(tid 182901392576):
remap_free_pages: page moved, 0x0001d2d1 ==> 0x00000000
The system is too badly corrupted or confused to continue at the Lisp
level. If the system had been compiled with the SB-LDB feature, we'd drop
into the LDB low-level debugger now. But there's no LDB in this build, so
we can't really do anything but just exit, sorry.

Apparently it is running out of memory, but that don't make sense, since 
the machine is an x86-64 with 8GB of memory, and I'm using the 64-bit 
version of sbcl, that maps 8GB of memory (and top agree here), so 
effectively I should only be using about 10% of the available memory.

To continue debuging I'm told (according to the error message) to use 
SB-LDB, but according to the sbcl-internals wiki 
(http://sbcl-internals.cliki.net/ldb), I should add :sb-ldb to the file 
customize-target-features.lisp, but I can't find this file in the sbcl 
source tree. Furthermore, when trying with GDB, it stops time after time 
with "segmentation fault", so gdb alone is not useful for debuging. 
While I think to remember it mentioned somewhere that the "segfaults" is 
a part of the normal operation of the garbage collector (or similar), it 
don't help either.
(Continue reading)

Permalink | Reply | 4 comments |
headers
Juho Snellman | 14 Jun 2006 18:35
Picon
Picon
Favicon

Re: X86-64 memory problems


Gisle Sælensminde <Gisle.Salensminde <at> bccs.uib.no> writes:
> mmap: Cannot allocate memory
> fatal error encountered in SBCL pid 32008(tid 182901392576):
> remap_free_pages: page moved, 0x0001d2d1 ==> 0x00000000

This has been discussed on sbcl-devel a couple of times this year, you
should be able to find the discussions by searching for the error
message in the mailing list archives.

Alternatively CVS SBCL should give you an error message which explains
exactly what is wrong, and what you can do on your system to fix it.

--

-- 
Juho Snellman
Permalink | Reply | 2 comments |

Gmane