Common Lisp ANSI compliance test suite ()

Ville Voutilainen | 8 Jan 2012 21:33

Does this work at all?

Forgot to reply to the list...

On 8 January 2012 22:16, Juan Jose Garcia-Ripoll
<juanjose.garciaripoll <at> googlemail.com> wrote:
> I would like to know on what basis you believe that the test is correct. It
> is producing an incorrect form, which is an assignment to a constant. The

Ah, now I see it: "If the store-value restart is invoked, its argument
becomes the new test-key, and is stored in keyplace as if by (setf
keyplace test-key). Then ctypecase starts over, considering each
clause anew."

The test didn't look like it's testing anything but the :good case,
but you are most certainly
correct that if the test for :keyword should fail, the test will do
funny things.

> not. A proper form for that test would be
> (PROGN
> (DEFPARAMETER *CTYPECASE-15* :FOO)
> (MACROLET ((%M (Z) Z))
>  (CTYPECASE (EXPAND-IN-CURRENT-ENV (%M *CTYPECASE-15*))
>    (INTEGER :BAD1)
>    (KEYWORD :GOOD)
>    (SYMBOL :BAD2)))
> )

Seems reasonable.

headers

Juan Jose Garcia-Ripoll | 26 Dec 2011 16:26

Wrong test?

(MACROLET ((%M (Z)

Z))

(CTYPECASE (EXPAND-IN-CURRENT-ENV (%M :FOO))

(INTEGER :BAD1)

(KEYWORD :GOOD)

(SYMBOL :BAD2)))

What is this supposed to check? In ECL this leads to the following code

(CTYPECASE :FOO

(INTEGER :BAD1)

(KEYWORD :GOOD)

(SYMBOL :BAD2)))

This is in my opinion wrong.. CTYPECASE is supposed to contain a form for a correctable error, including a replacement of the value of the form, in this case :FOO. This means CTYPECASE expands at some point to (SETF :FOO ...) which is not a valid code, given that :FOO is a constant.

I know that SBCL and probably other implementations do not complain about this until the assignment is performed, but this does not mean that the code is valid.

Juanjo

--
Instituto de Física Fundamental, CSIC
c/ Serrano, 113b, Madrid 28006 (Spain)
http://juanjose.garciaripoll.googlepages.com

_______________________________________________
ansi-test-devel mailing list
ansi-test-devel <at> common-lisp.net
http://lists.common-lisp.net/cgi-bin/mailman/listinfo/ansi-test-devel

headers

Juan Jose Garcia-Ripoll | 26 Dec 2011 16:22

Does this work at all?

I submitted an email to the list some time ago, with some patches for consideration, and received no answer so far.

I am worried because ECL now relies on this repository of the test suite for operations such as "make check" and I need the test suite to be updated -- it does not include flags to activate floating point exceptions and other stuff which is needed for compatibility.

If you guys are too busy, could I get some kind of write access to the repository? My user in common-lisp.net is jgarcia, but if the answer is positive I would also need some hints about the developer's access to svn repositories in common-lisp.net

Juanjo

--
Instituto de Física Fundamental, CSIC
c/ Serrano, 113b, Madrid 28006 (Spain)
http://juanjose.garciaripoll.googlepages.com

_______________________________________________
ansi-test-devel mailing list
ansi-test-devel <at> common-lisp.net
http://lists.common-lisp.net/cgi-bin/mailman/listinfo/ansi-test-devel

headers

Juan Jose Garcia-Ripoll | 10 Dec 2011 23:40

Re: Running tests out of place

I forward this again, given that the mailing list maintainers do not care to approve my previous post. The original attachments can be found here

https://docs.google.com/open?id=0B2brKGbNKZYDNTU0YjM4ZTYtMDk4OS00OWNlLTkzN2YtMGM0YmE0OGZmY2Vi

On Thu, Dec 8, 2011 at 9:07 PM, Juan Jose Garcia-Ripoll <juanjose.garciaripoll <at> googlemail.com> wrote:

Hi, I have been bothered several times by the inability of the ANSI test suite to run on a directory different from the one it resides in. This is a problem for various reasons:

* The source tree might be read-only. The test suite then is not allowed to create compiled files or temporary files.

* Some tests fail and leave temporary directories scattered around. This happens quite often. It is hard to clean the mess without keeping a list that differentiates the sources from the temporaries.

* If I want to keep state clean, I have to make a copy of the whole directory every time I want to run the tests. This happens quite often when I have to test multiple copies of ECL with a variety of settings.

Knowing this I have worked on the test suite, changing the tests so that they can be loaded from a different directory (what I call the "scratch" directory). I attach the changes which consist on this:

* The test suite defines some variables (see doit.lsp) which store the location of the sources and the location of the scratch directory in which the test suite is going to work.

* CLTEST: points to the scratch directory.

* CLTESTSRC: points to the sources.

* Various tests require some read-only file to be present. Formerly it used to be their own test file. I changed this to "ansi-aux.lsp" and ensured that such a file is present in the scratch directory.

* Some tests actually require entire files to be copied (see compile-file.lsp and load.lsp)

In addition, I attach an utility script which I use when I want to run just one set of tests. It has been customized to work also out of place.

Could you guys please have a look, run it with your favorite implementation (I tried SBCL and ECL) and hopefully incorporate the changes?

Juanjo

--
Instituto de Física Fundamental, CSIC
c/ Serrano, 113b, Madrid 28006 (Spain)
http://juanjose.garciaripoll.googlepages.com

--
Instituto de Física Fundamental, CSIC
c/ Serrano, 113b, Madrid 28006 (Spain)
http://juanjose.garciaripoll.googlepages.com

_______________________________________________
ansi-test-devel mailing list
ansi-test-devel <at> common-lisp.net
http://lists.common-lisp.net/cgi-bin/mailman/listinfo/ansi-test-devel

headers

Сергій Яковин | 17 Nov 2011 18:52

Fwd: Testing of x64 ECL for Windows

Hello ansi-test developers,

I have discovered the issue with ansi-test when using ECL with byte compiler (see the conversation below).

In order to fix it, I have applied a trivial patch (attached) locally. Could you please apply it to the svn repo?

Thanks,

Serhiy Y

---------- Forwarded message ----------
From: Juan Jose Garcia-Ripoll <juanjose.garciaripoll <at> googlemail.com>
Date: 2011/11/17
Subject: Re: Testing of x64 ECL for Windows
To: Сергій Яковин <syakovyn <at> gmail.com>

Hi Serhiy,

thanks again for your help.

On Wed, Nov 16, 2011 at 6:56 PM, Сергій Яковин <syakovyn <at> gmail.com> wrote:

I am testing ECL 64 bit Windows version using ecl-test.

First, I have found that using ignore-errors makes difficult to get errors, because of their hiding.

I have attached patch that replace ignore-errors with progn.

Thanks. I have applied it to my local repository. It will take two days until I can upload something again from home.

After applying it I got the following error:

;;; Loading #P"D:/Work/ECLS/ecl-test/ansi/doit.lsp"

;;; Loading "D:/Work/ECLS/ecl-test/ansi/gclload1.lsp"

An error occurred during initialization:

Cannot find the external symbol *SUPPRESS-COMPILER-WARNINGS* in #<"C" package>..

The ANSI test suite has become stale. There is a new development tree, but it is based on "git" and I have not yet migrated the Makefiles to use it. In the mean time, I keep a directory of patches ansi-patch/ where I will upload your fix. This directory gets applied on top of the ansi tes

If yes, will you send patch to ansi-tests or should I?

I would be very thankful if you did. But remember that ECL stil uses the old repository and not the live project which is now here http://www.common-lisp.net/project/ansi-test/

After applying the above change to gclload1.lsp I was able to get ansi-tests to COPY-TREE.1 before ECL crashed.

I am going to investigate that crash. I have attached the log file. It contains 25 fails. How much of those are expected?

EQUALP.7 and EQUALP.9 should not be there. This is the list of errors in my OS X computer.

30 out of 21703 total tests failed: COMMON-LISP-PACKAGE-NICKNAMES, COMMON-LISP-USER-PACKAGE-NICKNAMES,

BOUNDP.5, PROCLAIM.ERROR.7, CTYPECASE.15, CTYPECASE.16, EQUALP.5,

EQUALP.6, DEFGENERIC.ERROR.20, DEFGENERIC.ERROR.21,

CALL-NEXT-METHOD.ERROR.1, CALL-NEXT-METHOD.ERROR.2,

DEFMETHOD.ERROR.14, DEFMETHOD.ERROR.15, MAKE-CONDITION.4,

UPGRADED-ARRAY-ELEMENT-TYPE.8, FIND-ALL-SYMBOLS.2, /.ERROR.5,

/.ERROR.6, /.ERROR.7, /.ERROR.8, MAP.ERROR.11,

ALL-STANDARD-GENERIC-FUNCTIONS-ARE-INSTANCES-OF-THAT-CLASS,

ENSURE-DIRECTORIES-EXIST.8, FORMAT.LOGICAL-BLOCK.23,

SYNTAX.DOT-ERROR.1, COMPILE-FILE.6A, COMPILE-FILE.18, MISC.528,

MISC.529.

--
Instituto de Física Fundamental, CSIC
c/ Serrano, 113b, Madrid 28006 (Spain)
http://juanjose.garciaripoll.googlepages.com

Attachment (gclload1.lsp.patch): application/octet-stream, 465 bytes

_______________________________________________
ansi-test-devel mailing list
ansi-test-devel <at> common-lisp.net
http://lists.common-lisp.net/cgi-bin/mailman/listinfo/ansi-test-devel

headers

Erik Huelsmann | 25 Aug 2011 23:09

Local function binding shadows global macro

This week, I fixed an error in ABCL for which there seems to be no conformance test: a local function binding shadows global macros and compiler macros.

I'm not really sure how to set up a good test for it within the ansi tests framework, but this is the code which caused the issue in ABCL:

(defmacro var (name &optional value docs)

`(defparameter ,name , <at> (when value (list value))))

(defun make-var (the-name))

(defun foo (var-ref)

(flet ((var (x) (second x)))

`(make-var ,(var x))))

Any ideas about translating this into a good test to be committed?

Maybe we could come up with a similar test for compiler macros as well?

Bye,

Erik.

_______________________________________________
ansi-test-devel mailing list
ansi-test-devel <at> common-lisp.net
http://lists.common-lisp.net/cgi-bin/mailman/listinfo/ansi-test-devel

headers

Juan Jose Garcia-Ripoll | 24 Dec 2010 17:39

Problem with CTYPECASE.15

(MACROLET ((%M (Z)
                   Z))
        (CTYPECASE (EXPAND-IN-CURRENT-ENV (%M :FOO))
          (INTEGER :BAD1)
          (KEYWORD :GOOD)
          (SYMBOL :BAD2)))

Is this test really correct? CTYPECASE must be able to write into the keyval (in this case the form (EXPAND-IN-CURRENT-ENV (%M :FOO))). ECL properly detects that this is impossible, because the expression is constant, but this is not what the test suite expects.

Would the following make sense?

(MACROLET ((%M (Z)
                   Z))
        (CTYPECASE (LET ((X (EXPAND-IN-CURRENT-ENV (%M :FOO)))) X)
          (INTEGER :BAD1)
          (KEYWORD :GOOD)
          (SYMBOL :BAD2)))

Juanjo

--
Instituto de Física Fundamental, CSIC
c/ Serrano, 113b, Madrid 28006 (Spain)
http://juanjose.garciaripoll.googlepages.com

_______________________________________________
ansi-test-devel mailing list
ansi-test-devel <at> common-lisp.net
http://common-lisp.net/cgi-bin/mailman/listinfo/ansi-test-devel

headers

Sam Steingold | 27 Jul 2010 20:25

pprint-tabular.6 et al

Tests pprint-tabular.6 &ff are failed by the reference implementation in
http://www.lispworks.com/documentation/HyperSpec/Body/f_ppr_fi.htm
why? is there some pitfall which the standard authors created and then missed?

--

-- 
Sam Steingold <http://sds.podval.org>

headers

Raymond Toy | 21 Apr 2010 05:06

print-unreadable-object.2 and 3?

Does any Lisp actually pass these two tests?  I think
print-unreadable-object.2 expects a space after the type in all cases.
Test 3 expects a space before the address in all cases.

Does anything think that that is what the spec really wants?

If no Lisp passes this test, can we just change them or remove them?

Ray

headers

Sam Steingold | 20 Apr 2010 00:44

COMPILE-FILE.2 & COMPILE-FILE.2A

The aforementioned tests check that compile-file reports warnings and
style-warnings generated by the code being compiled (as opposed to the
compiler itself) as its secondary and tertiary values.

http://www.lispworks.com/documentation/HyperSpec/Body/f_cmp_fi.htm
The secondary value, warnings-p, is false if no conditions of type
error or warning were detected by the compiler, and true otherwise.
The tertiary value, failure-p, is false if no conditions of type error
or warning (other than style-warning) were detected by the compiler,
and true otherwise.

the tests assume that if the code being compile does something like

(eval-when (:compile-toplevel)
  (define-condition w (warning) nil))
  (warn (make-condition 'w)))

then the compiler must notice this and report this (second value being T).

It seems to me that the spec requires the 2nd and 3rd values to
reflect the signaling from within the compiler, not whatever is
signaled by the code being compiled.

do you agree with my interpretation of the spec?

--

-- 
Sam Steingold <http://sds.podval.org>

headers

Raymond Toy | 19 Apr 2010 01:53

Patch for syntax.escape test

I would like to propose the following patch for a few of the
syntax.escape tests.  The tests currently compares the symbol name
against the given string.  Given that cmucl converts the string to NFC
form, the tests fails in a few cases.  This patch changes the test to
compare the symbol name against (find-symbol string).  For all other
lisps that do not modify the string, this should not change anything.

A similar patch could be done for read-symbol tests, but I am not
proposing that; I want those tests to continue to fail on cmucl.

Ray

Index: syntax.lsp
===================================================================
--- syntax.lsp	(revision 2163)
+++ syntax.lsp	(working copy)
 <at>  <at>  -87,7 +87,7  <at>  <at> 
           for sym = (and c (read-from-string s))
           unless (or (not c)
                      (and (symbolp sym)
-                          (string= (symbol-name sym) s0)))
+                          (string= (symbol-name sym) (find-symbl s0))))
           collect (progn
                     (when (> (incf count) 100) (loop-finish))
                     (list i c s0 s sym))))
 <at>  <at>  -102,7 +102,7  <at>  <at> 
         repeat 1000
         unless (or (not c)
                    (and (symbolp sym)
-                        (string= (symbol-name sym) s0)))
+                        (string= (symbol-name sym) (find-symbol s0))))
         collect (list i c s0 s sym))
   nil)

 <at>  <at>  -128,7 +128,7  <at>  <at> 
           for sym = (and c (not bad) (read-from-string s))
           unless (or bad
                      (and (symbolp sym)
-                          (string= (symbol-name sym) s0)))
+                          (string= (symbol-name sym) (find-symbol s0))))
           collect (progn
                     (when (> (incf count) 100) (loop-finish))
                     (list c s0 s sym))))
 <at>  <at>  -144,7 +144,7  <at>  <at> 
         repeat 1000
         unless (or bad
                    (and (symbolp sym)
-                        (string= (symbol-name sym) s0)))
+                        (string= (symbol-name sym) (find-symbol s0))))
         collect (list c s0 s sym))
   nil)

_______________________________________________
ansi-test-devel mailing list
ansi-test-devel <at> common-lisp.net
http://common-lisp.net/cgi-bin/mailman/listinfo/ansi-test-devel

7	January 2012
9	December 2011
1	November 2011
1	September 2011
5	August 2011
2	February 2011
1	December 2010
1	July 2010
51	April 2010
2	November 2009
7	October 2009
6	September 2009
1	August 2009
7	July 2009
2	December 2008

Mon	Tue	Wed	Thu	Fri	Sat	Sun

		1	2	3	4	5
6	7	8	9	10	11	12
13	14	15	16	17	18	19
20	21	22	23	24	25	26
27	28	29	30	31