headers
Johan Ur Riise | 2 Nov 2009 10:26
Picon

Re: writing a 'unsigned-byte sequence to stdout

On Tue, Oct 20, 2009 at 05:08:41AM -0700, Ram Bhamidipaty wrote:
> Hi,
> 
> I want to write an unsigned byte sequence to stdout. How can I do that? When I
> try this: (write-sequence seq *standard-output*) I an error that says that
> stdout is not a binary-output-stream.
> 

This seems to work:

(let ((bytes (make-array 0 :element-type '(unsigned-byte 8) :fill-pointer 0 :adjustable t)))
  (loop for c across "Hello world!" do (vector-push-extend (char-code c) bytes))
  (with-open-file (*standard-output* "/dev/stdout"
                                     :direction :output
                                     :if-exists :append
                                     :element-type '(unsigned-byte 8))
    (write-sequence bytes *standard-output*)
    (write-byte #x0a *standard-output*))
  (values))

------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay 
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference
Permalink | Reply | 0 comments |
headers
Giovanni Gigante | 3 Nov 2009 13:36
Picon

memory troubles


Hello,
I tried starting sbcl with --dynamic-space-size 4000 (that is, 4GB; 
unfortunately I really need that much memory).
It refused to, saying "specified --dynamic-space-size too large". So is 
there an upper limit? (SBCL manual does not say it).

Also, trying with a lower value, 3000, sbcl does not complain in the 
same way, but still fails to allocate it ("mmap: Cannot allocate 
memory"), despite the fact that this machine has 6GB RAM of which 5.9GB 
are free, according to "free" command.

What's going on? Is there no way to use SBCL with a lot of memory?

I am on debian linux 5.0.3, kernel 2.6.26-2-686-bigmem, SBCL 1.0.29

Thanks,
Giovanni

------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay 
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference
Permalink | Reply | 5 comments |
headers
Michael Wood | 3 Nov 2009 13:46
Picon

Re: memory troubles

2009/11/3 Giovanni Gigante <giov <at> cidoc.iuav.it>:
>
> Hello,
> I tried starting sbcl with --dynamic-space-size 4000 (that is, 4GB;
> unfortunately I really need that much memory).
> It refused to, saying "specified --dynamic-space-size too large". So is
> there an upper limit? (SBCL manual does not say it).
>
> Also, trying with a lower value, 3000, sbcl does not complain in the
> same way, but still fails to allocate it ("mmap: Cannot allocate
> memory"), despite the fact that this machine has 6GB RAM of which 5.9GB
> are free, according to "free" command.
>
> What's going on? Is there no way to use SBCL with a lot of memory?
>
> I am on debian linux 5.0.3, kernel 2.6.26-2-686-bigmem, SBCL 1.0.29

That's 32 bit, right?

It won't work.  If you want a single process to use more than about
3GB of virtual memory it will need to be a 64 bit process (on a 64 bit
machine).

--

-- 
Michael Wood <esiotrot <at> gmail.com>

------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
(Continue reading)

Permalink | Reply | 0 comments |
headers
Sebastian Tennant | 3 Nov 2009 13:52

Re: memory troubles

Quoth Giovanni Gigante <giov <at> cidoc.iuav.it>:
> What's going on? Is there no way to use SBCL with a lot of memory?

SBCL is compiled with a upper limit.  To increase that limit you have to roll
your own.

See this article:

 http://article.gmane.org/gmane.lisp.steel-bank.general/2536/

If you need 4GiB (4096 MiB) you should set dynamic-space-end to #x109000000.

Hope this helps.

Seb
--

-- 
Emacs' AlsaPlayer - Music Without Jolts
Lightweight, full-featured and mindful of your idyllic happiness.
http://home.gna.org/eap

------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay 
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference
Permalink | Reply | 2 comments |
headers
Stas Boukarev | 3 Nov 2009 13:59
Picon

Re: memory troubles

Sebastian Tennant <sebyte <at> smolny.plus.com> writes:

> Quoth Giovanni Gigante <giov <at> cidoc.iuav.it>:
>> What's going on? Is there no way to use SBCL with a lot of memory?
>
> SBCL is compiled with a upper limit.  To increase that limit you have to roll
> your own.
>
> See this article:
>
>  http://article.gmane.org/gmane.lisp.steel-bank.general/2536/
>
> If you need 4GiB (4096 MiB) you should set dynamic-space-end to #x109000000.
>
Isn't that parameter changed by --dynamic-space-size? And the error is
not from sbcl, but from mmap failing to allocate memory.

--

-- 
With best regards, Stas.

------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay 
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference
Permalink | Reply | 0 comments |
headers
Sebastian Tennant | 3 Nov 2009 13:56

Re: memory troubles

Quoth Sebastian Tennant <sebyte <at> smolny.plus.com>:
> Quoth Giovanni Gigante <giov <at> cidoc.iuav.it>:
>> What's going on? Is there no way to use SBCL with a lot of memory?
>
> SBCL is compiled with a upper limit.  To increase that limit you have to roll
> your own.

Scratch that.  It's not an upper limit.  It's a default.  Looks like you're out
of luck unless you're on a 64 bit machine.

Seb
--

-- 
Emacs' AlsaPlayer - Music Without Jolts
Lightweight, full-featured and mindful of your idyllic happiness.
http://home.gna.org/eap

------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay 
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference
Permalink | Reply | 0 comments |
headers
Martin Cracauer | 3 Nov 2009 15:04

Re: memory troubles

Giovanni Gigante wrote on Tue, Nov 03, 2009 at 01:36:16PM +0100: 
> 
> Hello,
> I tried starting sbcl with --dynamic-space-size 4000 (that is, 4GB; 
> unfortunately I really need that much memory).
> It refused to, saying "specified --dynamic-space-size too large". So is 
> there an upper limit? (SBCL manual does not say it).
> 
> Also, trying with a lower value, 3000, sbcl does not complain in the 
> same way, but still fails to allocate it ("mmap: Cannot allocate 
> memory"), despite the fact that this machine has 6GB RAM of which 5.9GB 
> are free, according to "free" command.
> 
> What's going on? Is there no way to use SBCL with a lot of memory?

There is no way to use all that memory in a single 32 bit process.

> I am on debian linux 5.0.3, kernel 2.6.26-2-686-bigmem, SBCL 1.0.29

There was the 4GB/4GB VM split patch the Redhat hat in Fedore Core 2
and parts of 3 IIRC for a while.  That would allow you to go close to
4 GB.

Is there a specific reason why you can't go 64 bits?

Martin
--

-- 
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
Martin Cracauer <cracauer <at> cons.org>   http://www.cons.org/cracauer/
FreeBSD - where you want to go, today.      http://www.freebsd.org/
(Continue reading)

Permalink | Reply | 0 comments |
headers
Giovanni Gigante | 3 Nov 2009 15:41
Picon

Re: memory troubles


>>I am on debian linux 5.0.3, kernel 2.6.26-2-686-bigmem, SBCL 1.0.29
>>    
>>
>
>That's 32 bit, right?
>It won't work.  If you want a single process to use more than about
>3GB of virtual memory it will need to be a 64 bit process (on a 64 bit
>machine).
>
>  
>

My CPU is a Intel Q6600. I know that the right way would be a 64-bit 
kernel, but unfortunately at the time this machine was installed the 
wrong choice was made, and now it would be really painful to rebuild the 
whole system from scratch.
So, looking for a simpler solution, when I installed more RAM I switched 
to this 32-bit debian kernel "bigmem", which means it has PAE. I am no 
expert at these matters, but I had read that PAE allows for 64GB RAM 
maximum, with a limit of 4GB per process.
Which is why I tried --dynamic-space-size 4000. (a bit less than those 4GB).
Unfortunately, it doesn't seem to work.

So I still have two doubts:

1) what is the maximum theorical value for --dynamic-space-size in these 
conditions?

2) why mmap still fails under that value? (maybe it needs contiguous
(Continue reading)

Permalink | Reply | 4 comments |
headers
Martin Cracauer | 3 Nov 2009 15:48

Re: memory troubles

Giovanni Gigante wrote on Tue, Nov 03, 2009 at 03:41:15PM +0100: 
> 
> >>I am on debian linux 5.0.3, kernel 2.6.26-2-686-bigmem, SBCL 1.0.29
> >>    
> >>
> >
> >That's 32 bit, right?
> >It won't work.  If you want a single process to use more than about
> >3GB of virtual memory it will need to be a 64 bit process (on a 64 bit
> >machine).
> >
> >  
> >
> 
> My CPU is a Intel Q6600. I know that the right way would be a 64-bit 
> kernel, but unfortunately at the time this machine was installed the 
> wrong choice was made, and now it would be really painful to rebuild the 
> whole system from scratch.
> So, looking for a simpler solution, when I installed more RAM I switched 
> to this 32-bit debian kernel "bigmem", which means it has PAE. I am no 
> expert at these matters, but I had read that PAE allows for 64GB RAM 
> maximum, with a limit of 4GB per process.
> Which is why I tried --dynamic-space-size 4000. (a bit less than those 4GB).
> Unfortunately, it doesn't seem to work.

The PAE kernel does not extend the virtual memory available to a
single process in any way.

> So I still have two doubts:
>
(Continue reading)

Permalink | Reply | 3 comments |
headers
Giovanni Gigante | 3 Nov 2009 16:05
Picon

Re: memory troubles

Martin Cracauer wrote:

>With a 32 bit kernel it is the same regardless of how much RAM you
>have and regardless of whether you use PAE or not.
>
>  
>
>You issue at that point is not only the available space, but
>fragmentation.  
>  
>

Thanks for your replies.

So basically if I want a sbcl able to handle about 4GB of (mostly 
double-float) data, there is no way other than using a 64bit kernel.

But since (correct me if I am wrong) in 64bit the size of those objects 
will double,  that will become 8GB.

And accounting for the fragmentation problem, probably even more than 
that. Maybe 10, 12 GB...
Right?

(btw, for the fragmentation issue, is there any way to mitigate it, 
rather than just throwing more chips at it? increasing swap perhaps? 
tweaking something else?)

giovanni
(Continue reading)

Permalink | Reply | 2 comments |

Gmane