Waldek Hebisch | 1 Dec 2009 04:59
Picon

Re: Memory problems during IO

> On Sun, Nov 29, 2009 at 9:50 PM, Waldek Hebisch
> <hebisch@...> wrote:
> > I have problem reading modertely large (76 MB) input file
> > using FriCAS. ?I am geting ?the following message:
> >
> > Memory limit reached. Please jump to an outer pointer, quit program and
> > enlarge the memory limits before executing the program again.
> >
> > I am using CVS version from about 3 weeks ago. ?The machine is Core
> > 2 with 8 GB of ram running 64-bit Fedora 9.
> 
> The problem is not with I/O, but rather with the fact that you hit the
> memory limits creating a file with about 200Mb of non-eq strings. The
> memory eaten by a 64-bits system in that case is much larger than the
> one required for a 32-bits system. I just increased the limits to 1Gb
> for 32 bits and 4 Gb for 64 bits.
>

Thanks for increased limit.  But IO may be also to blame, loading
the test file from previous message a few times I see:

ECL (Embeddable Common-Lisp) 9.12.1
Copyright (C) 1984 Taiichi Yuasa and Masami Hagiya
Copyright (C) 1993 Giuseppe Attardi
Copyright (C) 2000 Juan J. Garcia-Ripoll
ECL is free software, and you are welcome to redistribute it
under certain conditions; see file 'Copyright' for details.
Type :h for Help.  
Top level.
> (load "iotst.lisp")
(Continue reading)

Juan Jose Garcia-Ripoll | 1 Dec 2009 08:55
Picon

Re: Memory problems during IO

On Tue, Dec 1, 2009 at 4:59 AM, Waldek Hebisch <hebisch <at> math.uni.wroc.pl> wrote:
The file has 122 MB, I compiled ECL without Unicode support, so
the strings should take similare space in memory as in a file.
So why there is so much consing?  For comparison sbcl needs
5.216 sec but conses only 13,360,144 bytes.  And for reading
sbcl is much faster, 2.559 sec and 492,137,056 bytes consed
(note that sbcl uses unicode strings, so the amount consed
is essentially the same as space needed to store strings).

ECL uses some buffers and maybe they are not as large as you need. When reading a string one reads until the buffer is full and then must adjust the size of the string dynamically. The strategy for doing that may condition how much memory is consed. And your file is not only large: it contains rather large strings (20000 characters) when compared to normal use.

Juanjo
 

--
Instituto de Física Fundamental, CSIC
c/ Serrano, 113b, Madrid 28006 (Spain)
http://juanjose.garciaripoll.googlepages.com
------------------------------------------------------------------------------
Join us December 9, 2009 for the Red Hat Virtual Experience,
a free event focused on virtualization and cloud computing. 
Attend in-depth sessions from your desk. Your couch. Anywhere.
http://p.sf.net/sfu/redhat-sfdev2dev
_______________________________________________
Ecls-list mailing list
Ecls-list@...
https://lists.sourceforge.net/lists/listinfo/ecls-list
Juan Jose Garcia-Ripoll | 1 Dec 2009 10:15

[ANN] ECL 9.12.1

Known issues
============

- The Mingw port does not work with the stable version of the Boehm-Weiser
  garbage collector. Users will have to download a more recent version and
  build and install it in an accessible location for ECL to compile.

- The OpenBSD port does not support multithreaded builds because the latest
  stable version that runs on it does not contain all the required
  functionality.

- --with-cxx is currently broken and ECL does not BUILD with a C++ compiler,
  but it can still be USED in C++ programs.

ECL 9.12.1:
===========

* Platform support:

 - Support for Itanium.

 - Support for Solaris and SunStudio compiler.

 - Support for Microsoft Visual Studio C++ 10 beta.

 - Support for Cygwin.

* Visible changes:

 - New function EXT:ARRAY-RAW-DATA returns a non-adjustable vector of type
   (UNSIGNED-BYTE 8) with the content of an array. The returned array overlaps
   with the original, so any change on one will affect the other.

 - LOAD works on special files (/dev/null and the like).

 - New command line option --version outputs the version number preceded
   by the implementation name ("ECL 9.11.1" in this particular release).

 - MULTIPLE-VALUE-BIND is optimally replaced with a LET form when the number
   of variables is 1.

 - ECL now accepts Windows' UNC pathnames.
    (with-open-file (s #P"//JUANJO-IMAC/Public Folder/index.html"
                       :direction :input)
      (loop for l = (read-line s nil nil)
            while l
            do (princ l)))

 - The default memory limits are increased up to 512Mb for 32 bit images and
   4Gb for 64-bits systems.

 - The bignums generated by RANDOM did not contain enough random bits.

* Bugs fixed:

 - In single-threaded builds, ECL did not properly restore the signal mask
   before jumping out of a signal handler.

 - Floating point exceptions were sometimes ignored, leading to infinite loops.

 - A new configuration flag is added, --with-dffi. This flag allows
   deactivating the foreign function interface when the compiler does not
   support inline assembly with the GCC syntax and libffi is not available.

 - In a multithreaded ECL, when handling a Ctr-C/SIGINT asynchronous interrupt,
   the CONTINUE restart was not always available.

 - In cygwin, LOAD was unable to load and execute binary files.

 - In cygwin, FASL files can not lack a file extension, because dlopen()
   then looks for a file ending in ".dll" and fails.

 - In cygwin, files that are dlopen'ed must have executable permissions.

 - ECL ignored the IGNORABLE declaration.

 - The IGNORE/IGNORABLE declarations accept (but ignore) arguments of the
   for (FUNCTION function-name).

 - SUBTYPEP caused a SIGSEGV when the input was a not finalized class.

 - Due to the SUBTYPEP bug, ECL could not compile DEFMETHOD forms whose
   arguments referenced non-finalized classes

 - When supplied an error value, (EXT:SAFE-EVAL form env &optional err-value)
   never returned the output of the evaluated form.

 - FIND-SYMBOL accepted string designators instead of just strings, as
   mandated by the ANSI specification.

 - APPEND copied also the last argument.

 - (LOG #C(x x)) now produces a better result when x == 0

 - (ATAN #C(0.0 1.0)) no longer produces an infinite recursion.

 - Solved a hard to hit bug in DEFCLASS's routine for detecting collisions in
   slot names

 - LOG and LOG1P did not work properly with NaNs under linux.

 - ECL had problems combining #. and #n=/#n# reader macros.

 - FDEFINITION and SYMBOL-FUNCTION caused an incorrect error condition when
   acting on NIL.

 - The optimizer for TYPEP did not work when passed three arguments.

* Clos:

 - CLOS:SET-FUNCALLABLE-INSTANCE-FUNCTION broke the value if SI:INSTANCE-SIG,
   preventing any further access to the instance slots.

 - The optimized slot accessors check that the instances are up to date.

 - The use of MAKE-INSTANCES-OBSOLETE now forces
UPDATE-INSTANCE-FOR-REDEFINED-CLASS
   to be invoked even if the slots did not change.

 - ENSURE-GENERIC-FUNCTION-USING-CLASS does not provide a default method class.

* Sockets:

 - The socket option TCP_NODELAY option has been fixed: it was improperly using
   the socket interface SOL_SOCKET instead of IPPROTO_TCP (Chun Tian)

 - sockopt-linger and (setf sockopt-linger) now work as they should, using
   the struct linger argument (M. Mondor)

* ASDF:

 - ASDF:MAKE-BUILD now accepts also a :PROLOGUE-CODE argument with code to
   be executed before all lisp files are run.

 - C:BUILDER's argument :PROLOGUE-CODE can now be a lisp form. In the case of
   standalone programs the prologue code is always executed after cl_boot() has
   been invoked.

 - QUIT did not work from standalone executables created with neither ASDF
   nor with C:BUILDER.

--

-- 
Instituto de Física Fundamental, CSIC
c/ Serrano, 113b, Madrid 28006 (Spain)
http://juanjose.garciaripoll.googlepages.com

------------------------------------------------------------------------------
Join us December 9, 2009 for the Red Hat Virtual Experience,
a free event focused on virtualization and cloud computing. 
Attend in-depth sessions from your desk. Your couch. Anywhere.
http://p.sf.net/sfu/redhat-sfdev2dev
Juan Jose Garcia-Ripoll | 1 Dec 2009 10:33

Re: ECL support in Closer projects

On Mon, Nov 30, 2009 at 3:19 PM, Pascal Costanza <pc@...> wrote:
> There is now full support for ECL in the Closer to MOP projects.

That's great!

> Thanks a lot to Alexander Gavrilov who provided a lot of very
> useful patches that I was able to incorporate, and to Juan
> Jose Garcia-Ripoll for fixing a lot of bugs in ECL to make this possible.

I would like to thank you for your patience in tracking and reporting
and testing all those bugs!
When I find time I will add Closer and the rest of the libraries to
the ECL test farm
  http://ecls.sourceforge.net/logs_lib.html

Juanjo

--

-- 
Instituto de Física Fundamental, CSIC
c/ Serrano, 113b, Madrid 28006 (Spain)
http://juanjose.garciaripoll.googlepages.com

------------------------------------------------------------------------------
Join us December 9, 2009 for the Red Hat Virtual Experience,
a free event focused on virtualization and cloud computing. 
Attend in-depth sessions from your desk. Your couch. Anywhere.
http://p.sf.net/sfu/redhat-sfdev2dev
Pascal Costanza | 1 Dec 2009 21:00

New 'random bug in 9.12.1

Hi,

I'm very sorry, but: What is this?

ECL (Embeddable Common-Lisp) 9.12.1
Copyright (C) 1984 Taiichi Yuasa and Masami Hagiya
Copyright (C) 1993 Giuseppe Attardi
Copyright (C) 2000 Juan J. Garcia-Ripoll
ECL is free software, and you are welcome to redistribute it
under certain conditions; see file 'Copyright' for details.
Type :h for Help.  
Top level in: #<process SI:TOP-LEVEL 00000001005c0f60>.
> (random 100.0)

68.830322
> (random 100)

Detected access to an invalid or protected memory address.

Available restarts:

1. (RESTART-TOPLEVEL) Go back to Top-Level REPL.

Broken at SI:BYTECODES. [Evaluation of: (RANDOM 100)] In: #<process SI:TOP-LEVEL 00000001005c0f60>.
>> 

Pascal

--

-- 
Pascal Costanza, mailto:pc@..., http://p-cos.net
Vrije Universiteit Brussel
Software Languages Lab
Pleinlaan 2, B-1050 Brussel, Belgium

------------------------------------------------------------------------------
Join us December 9, 2009 for the Red Hat Virtual Experience,
a free event focused on virtualization and cloud computing. 
Attend in-depth sessions from your desk. Your couch. Anywhere.
http://p.sf.net/sfu/redhat-sfdev2dev
Pascal Costanza | 1 Dec 2009 21:01

Re: New 'random bug in 9.12.1

I forgot to say: This stops a case in the ContextL test suite from working, so this bug wasn't there two days ago...

On 1 Dec 2009, at 21:00, Pascal Costanza wrote:

> Hi,
> 
> I'm very sorry, but: What is this?
> 
> ECL (Embeddable Common-Lisp) 9.12.1
> Copyright (C) 1984 Taiichi Yuasa and Masami Hagiya
> Copyright (C) 1993 Giuseppe Attardi
> Copyright (C) 2000 Juan J. Garcia-Ripoll
> ECL is free software, and you are welcome to redistribute it
> under certain conditions; see file 'Copyright' for details.
> Type :h for Help.  
> Top level in: #<process SI:TOP-LEVEL 00000001005c0f60>.
>> (random 100.0)
> 
> 68.830322
>> (random 100)
> 
> Detected access to an invalid or protected memory address.
> 
> Available restarts:
> 
> 1. (RESTART-TOPLEVEL) Go back to Top-Level REPL.
> 
> Broken at SI:BYTECODES. [Evaluation of: (RANDOM 100)] In: #<process SI:TOP-LEVEL 00000001005c0f60>.
>>> 
> 
> 
> 
> Pascal
> 
> -- 
> Pascal Costanza, mailto:pc@..., http://p-cos.net
> Vrije Universiteit Brussel
> Software Languages Lab
> Pleinlaan 2, B-1050 Brussel, Belgium
> 
> 
> 
> 
> 
> 
> 
> ------------------------------------------------------------------------------
> Join us December 9, 2009 for the Red Hat Virtual Experience,
> a free event focused on virtualization and cloud computing. 
> Attend in-depth sessions from your desk. Your couch. Anywhere.
> http://p.sf.net/sfu/redhat-sfdev2dev
> _______________________________________________
> Ecls-list mailing list
> Ecls-list@...
> https://lists.sourceforge.net/lists/listinfo/ecls-list

--

-- 
Pascal Costanza, mailto:pc@..., http://p-cos.net
Vrije Universiteit Brussel
Software Languages Lab
Pleinlaan 2, B-1050 Brussel, Belgium

------------------------------------------------------------------------------
Join us December 9, 2009 for the Red Hat Virtual Experience,
a free event focused on virtualization and cloud computing. 
Attend in-depth sessions from your desk. Your couch. Anywhere.
http://p.sf.net/sfu/redhat-sfdev2dev
Juan Jose Garcia-Ripoll | 1 Dec 2009 21:34

Re: New 'random bug in 9.12.1

No, a different bug related to RANDOM was reported yesterday afternoon
and I produced a fix that seems to work on the systems which I could
test. Obviously the sample was not large enough and it breaks for
64-bits machines. What I can't explain is why the test farm did not
reveal the problem this morning.

Juanjo

On Tue, Dec 1, 2009 at 9:01 PM, Pascal Costanza <pc@...> wrote:
> I forgot to say: This stops a case in the ContextL test suite from working, so this bug wasn't there two days ago...
>
> On 1 Dec 2009, at 21:00, Pascal Costanza wrote:
>
>> Hi,
>>
>> I'm very sorry, but: What is this?
>>
>> ECL (Embeddable Common-Lisp) 9.12.1
>> Copyright (C) 1984 Taiichi Yuasa and Masami Hagiya
>> Copyright (C) 1993 Giuseppe Attardi
>> Copyright (C) 2000 Juan J. Garcia-Ripoll
>> ECL is free software, and you are welcome to redistribute it
>> under certain conditions; see file 'Copyright' for details.
>> Type :h for Help.
>> Top level in: #<process SI:TOP-LEVEL 00000001005c0f60>.
>>> (random 100.0)
>>
>> 68.830322
>>> (random 100)
>>
>> Detected access to an invalid or protected memory address.
>>
>> Available restarts:
>>
>> 1. (RESTART-TOPLEVEL) Go back to Top-Level REPL.
>>
>> Broken at SI:BYTECODES. [Evaluation of: (RANDOM 100)] In: #<process SI:TOP-LEVEL 00000001005c0f60>.
>>>>
>>
>>
>>
>> Pascal
>>
>> --
>> Pascal Costanza, mailto:pc@..., http://p-cos.net
>> Vrije Universiteit Brussel
>> Software Languages Lab
>> Pleinlaan 2, B-1050 Brussel, Belgium
>>
>>
>>
>>
>>
>>
>>
>> ------------------------------------------------------------------------------
>> Join us December 9, 2009 for the Red Hat Virtual Experience,
>> a free event focused on virtualization and cloud computing.
>> Attend in-depth sessions from your desk. Your couch. Anywhere.
>> http://p.sf.net/sfu/redhat-sfdev2dev
>> _______________________________________________
>> Ecls-list mailing list
>> Ecls-list@...
>> https://lists.sourceforge.net/lists/listinfo/ecls-list
>
> --
> Pascal Costanza, mailto:pc@..., http://p-cos.net
> Vrije Universiteit Brussel
> Software Languages Lab
> Pleinlaan 2, B-1050 Brussel, Belgium
>
>
>
>
>
>
>
> ------------------------------------------------------------------------------
> Join us December 9, 2009 for the Red Hat Virtual Experience,
> a free event focused on virtualization and cloud computing.
> Attend in-depth sessions from your desk. Your couch. Anywhere.
> http://p.sf.net/sfu/redhat-sfdev2dev
> _______________________________________________
> Ecls-list mailing list
> Ecls-list@...
> https://lists.sourceforge.net/lists/listinfo/ecls-list
>

--

-- 
Instituto de Física Fundamental, CSIC
c/ Serrano, 113b, Madrid 28006 (Spain)
http://juanjose.garciaripoll.googlepages.com

------------------------------------------------------------------------------
Join us December 9, 2009 for the Red Hat Virtual Experience,
a free event focused on virtualization and cloud computing. 
Attend in-depth sessions from your desk. Your couch. Anywhere.
http://p.sf.net/sfu/redhat-sfdev2dev
Waldek Hebisch | 1 Dec 2009 22:54
Picon

Re: New 'random bug in 9.12.1

Juan Jose Garcia-Ripoll
> No, a different bug related to RANDOM was reported yesterday afternoon
> and I produced a fix that seems to work on the systems which I could
> test. Obviously the sample was not large enough and it breaks for
> 64-bits machines. What I can't explain is why the test farm did not
> reveal the problem this morning.
>

Maybe it _sometimes_ works?  Today on my machine I see the same problem
as Pascal Costanza.  After setting time back one day (random 100)
crashed but I was able to do

(random (expt 2 10000))

After setting time to correct value the above crashes.  As far as
I remember the testcase I reported worked yesterday using version
I pulled from git.  Today the same version has problem...

--

-- 
                              Waldek Hebisch
hebisch@... 

------------------------------------------------------------------------------
Join us December 9, 2009 for the Red Hat Virtual Experience,
a free event focused on virtualization and cloud computing. 
Attend in-depth sessions from your desk. Your couch. Anywhere.
http://p.sf.net/sfu/redhat-sfdev2dev
Juan Jose Garcia-Ripoll | 1 Dec 2009 23:23

Re: New 'random bug in 9.12.1

On Tue, Dec 1, 2009 at 10:54 PM, Waldek Hebisch
<hebisch@...> wrote:
> Juan Jose Garcia-Ripoll
>> No, a different bug related to RANDOM was reported yesterday afternoon
>> and I produced a fix that seems to work on the systems which I could
>> test. Obviously the sample was not large enough and it breaks for
>> 64-bits machines. What I can't explain is why the test farm did not
>> reveal the problem this morning.
>>
>
> Maybe it _sometimes_ works?  Today on my machine I see the same problem
> as Pascal Costanza.  After setting time back one day (random 100)
> crashed but I was able to do
>
> (random (expt 2 10000))
>
> After setting time to correct value the above crashes.  As far as
> I remember the testcase I reported worked yesterday using version
> I pulled from git.  Today the same version has problem...

I have changed the code and am running it in a 64-bits box as well. I
think I have identified the two problems that orginated the random
random bug. When I finish I will produce a patched tarball.

Sorry for the inconvenience, but please understand that these things
do not happen on purpose.

Juanjo

--

-- 
Instituto de Física Fundamental, CSIC
c/ Serrano, 113b, Madrid 28006 (Spain)
http://juanjose.garciaripoll.googlepages.com

------------------------------------------------------------------------------
Join us December 9, 2009 for the Red Hat Virtual Experience,
a free event focused on virtualization and cloud computing. 
Attend in-depth sessions from your desk. Your couch. Anywhere.
http://p.sf.net/sfu/redhat-sfdev2dev
Pascal Costanza | 2 Dec 2009 11:05

Re: New 'random bug in 9.12.1


On 1 Dec 2009, at 23:23, Juan Jose Garcia-Ripoll wrote:

> On Tue, Dec 1, 2009 at 10:54 PM, Waldek Hebisch
> <hebisch@...> wrote:
>> Juan Jose Garcia-Ripoll
>>> No, a different bug related to RANDOM was reported yesterday afternoon
>>> and I produced a fix that seems to work on the systems which I could
>>> test. Obviously the sample was not large enough and it breaks for
>>> 64-bits machines. What I can't explain is why the test farm did not
>>> reveal the problem this morning.
>>> 
>> 
>> Maybe it _sometimes_ works?  Today on my machine I see the same problem
>> as Pascal Costanza.  After setting time back one day (random 100)
>> crashed but I was able to do
>> 
>> (random (expt 2 10000))
>> 
>> After setting time to correct value the above crashes.  As far as
>> I remember the testcase I reported worked yesterday using version
>> I pulled from git.  Today the same version has problem...
> 
> I have changed the code and am running it in a 64-bits box as well. I
> think I have identified the two problems that orginated the random
> random bug. When I finish I will produce a patched tarball.
> 
> Sorry for the inconvenience, but please understand that these things
> do not happen on purpose.

Don't worry. Nobody claimed otherwise. (That would actually be cool, if you could produce _such_ bugs on
purpose. ;)

Pascal

--

-- 
Pascal Costanza, mailto:pc@..., http://p-cos.net
Vrije Universiteit Brussel
Software Languages Lab
Pleinlaan 2, B-1050 Brussel, Belgium

------------------------------------------------------------------------------
Join us December 9, 2009 for the Red Hat Virtual Experience,
a free event focused on virtualization and cloud computing. 
Attend in-depth sessions from your desk. Your couch. Anywhere.
http://p.sf.net/sfu/redhat-sfdev2dev

Gmane