Tobias C Rittweiler | 24 May 11:13
Picon

Various issues

Hi there!

Thanks to the ASDF maintainers. I just visited ASDF's website
on common-lisp.net since a long time, and it makes a nicely
maintained impression! Well done.

I have the following issues:

  * The rather old ASDF version that I'm using ("2.010") does
    not seem to be able to cope with multiple :depends-on clauses.
    (Only one clause seems to be used.)

    Is this intended behaviour? Might it be fixed in more recent
    versions?

  * :STATIC-FILE (and :FILE for that matter) are not discussed
    or explained in the manual.

  * We generate a 'version.lisp' with a (DEFPARAMETER ...) form
    in it when using "make" to build a Lisp binary using buildapp.

    Is there a component that loads a file if it exists and
    if it does not exist just skips the clause?

Thanks,

T
Robert Goldman | 17 May 16:26

Tests on Mac OS X

I tested on the following lisps

ccl clisp sbcl ecl abcl allegro allegromodern

with success.  Haven't tried on CMUCL yet, because of prior problems.  I
will see about picking up a snapshot sometime.
Yuan Luo | 13 May 00:08
Picon

lisp source debugger

Hi,

This maybe an irrelevant question, if so, please point me to the best list to ask. But I am wondering whether you guys can recommend some source level debugger that can POINT ME PRECISELY to where the error was. For example, Allegro CL cannot because many times it tells me that there is an error in a function, but it doesn't tell me which line even if I set their debug flag to 3 the highest level and suppressed everything else, speed, optimization etc. (my function has hundreds of lines then I am frustrated ...)

Best,
Yuan

_______________________________________________
asdf-devel mailing list
asdf-devel@...
http://lists.common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel
Nikodemus Siivola | 7 May 06:22
Gravatar

:default-component-class and custom module classes

I was trying to make a custom module class, which would have provided
its own :DEFAULT-COMPONENT-CLASS via
:DEFAULT-INITARGS, but

        (setf (module-default-component-class ret)
              (or default-component-class
                  (and (typep parent 'module)
                       (module-default-component-class parent))))

means that does work as I hoped it would. The closest I can get is by
overriding ASDF::MODULE-DEFAULT-COMPONENT-CLASS method, but that seems
fairly icky.

Is there a better way?

Cheers,

 -- Nikodemus

_______________________________________________
asdf-devel mailing list
asdf-devel <at> common-lisp.net
http://lists.common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel
Stelian Ionescu | 6 May 21:27
Gravatar

Reloading a .asd

I'm using the elisp snippet below(courtesy of Stas Boukarev with some
modifications) to reload a .asd file. Is asdf::load-sysdef the best way
to trigger a reload(but not recompilation) ?

Use case: I want to work on a system that isn't in the ASDF path, it
being one of the many development trees I have around(e.g. CFFI), so I
want to be able to say that "this cffi.asd is the CFFI to use during
this session". Any ideas ?

(defvar slime-custom-file-loaders
  '(("asd"  . slime-load-asdf)
    ("lisp" . slime-load-file)))

(defun slime-load-asdf (filename)
  (slime-eval-with-transcript
   `(asdf::load-sysdef ,(file-name-sans-extension filename)
                       ,(slime-to-lisp-filename (expand-file-name filename)))))

(defun slime-custom-load-file ()
  (interactive)
  (unless buffer-file-name
    (error "Buffer %s is not associated with a file." (buffer-name)))
  (check-parens)
  (when (and (buffer-modified-p)
             (y-or-n-p (format "Save file %s? " (buffer-file-name))))
    (save-buffer))
  (let* ((type (file-name-extension buffer-file-name))
         (loader (cdr (assoc-string type slime-custom-file-loaders t))))
    (funcall (or loader
                 'slime-load-file)
             buffer-file-name)))

(define-key slime-mode-map (kbd "C-c C-l") 'slime-custom-load-file)

--

-- 
Stelian Ionescu a.k.a. fe[nl]ix
Quidquid latine dictum sit, altum videtur.
http://common-lisp.net/project/iolib

_______________________________________________
asdf-devel mailing list
asdf-devel@...
http://lists.common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel
Faré | 3 May 17:55
Picon
Gravatar

asdf-bundle

asdf-bundle is now working (hopefully) on SBCL and CCL as well as ECL,
by concatenating fasls. It won't work for systems where CFFI creates .so.

Try it!
(require :asdf)
(require :asdf-bundle)
(asdf:oos 'asdf::load-fasl-op :your-favorite-system)

It creates a your-favorite-system.system.fasl in the fasl cache.

Actually, I'm not sure about ECL anymore - load-fasl-op fails at the last minute
(after the fasl-op itself) because it tries doing something with a .fasb
even though I didn't enable the bytecode compiler. Since I leveraged that part
of the code from asdf's asdf-ecl.lisp, I suspect it is broken there already.

Juanjo: can you give a look when you have time?

Others: you're welcome to contribute support for your favorite implementations.

—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org
Don't have good ideas if you aren't willing to be responsible for them.
        — Alan Perlis

_______________________________________________
asdf-devel mailing list
asdf-devel <at> common-lisp.net
http://lists.common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel
Douglas Crosher | 25 Apr 05:39
Favicon

Tool for transcoding lisp source files and adding coding file options.


A tool to add the coding file option to lisp source files is now available.  This can also be used to transcode
all the files to
UTF-8, or to attempt transcoding to ISO-8859-1 or US-ASCII with loss in comments.  There is an example for
the current Quicklisp
releases with a few required exceptions.  This is experimental, particularly the transcoding to
ISO-8859-1 or US-ASCII.  Please
backup your code before giving this a try!  See:
http://www.scieneer.com/files/transcode.lisp

With the coding file option in place, the developmental version of ASDF is able to pass the correct
external-format to 'load and
'compile-file.

Note the lambda-reader is handled by substituting references to the lambda character back to "(lambda".

Since the :encoding file option should become unnecessary with the coding file option, and also because it
would be invalid upon any
transcoding, it is removed from .asd files.  I don't think it would be practical to try and update the
:encoding declarations.

With the coding file option on all the files, all that asdf-encodings needs to do is read this file option. 
The UTF-8 and other
detection could be removed and moved into a tools such as this.

If some of the CL implementations would add the reading of the file option as an external-format then the
coding file option could
be used by 'load and 'compile-file outside of ASDF - then for these CL implementations asdf-encodings
would not be necessary and
ASDF could pass on the encoding-file-option reading external-format which may well be the :default.  I
will look at implementing
this for the Scieneer CL, and perhaps take a look at CMUCL, but it would be best optimized for each CL implementation.

Regards
Douglas Crosher
Erik Pearson | 24 Apr 11:38
Favicon

Re: Faster loading for deployed systems?

I thought I'd report back briefly on some results.

First, I spent some time attempting to implement "perform with session state". This is the idea that when working with ASDF, it would be useful to be able to invoke or create specific behavior for my use case and also to be able to capture work session state without relying on globals. This was partially successful, but I broke something and ran out of patience. ASDF is some wonderful code, but takes a bit to grok it all.

So I took the crude approach of just installing a global and using it to accumulate loaded files from within a perform. I created an .asd file which defined the dependencies I wanted the image to have. The loaded it with load-system. Then at the end of load-system I inserted code to save a structure containing a list of the *defined-systems* as well as a list of the loaded files (in order of loading).

The next step was to create a little loader script which loads all of the files, and then populates *modules* via satisfies. There were a couple of glitches. For one, a couple of systems had hard coded asdf dependencies. After removing them, they compiled fine. Then another system required asdf-system-connections. I needed to remove that out of the file load list because it isn't necessary after things are loaded.

Finally the load list works, the image saves and starts just fine. I can use regular ol' REQUIRE to provide a sanity check, although I realize this may not be the best approach, at least it works without extra code. 

This saved about 10% on the image size compared to asdf-loading the same systems, and cleared out some unnecessary dependencies. It will take a while to work out the best "batteries included" image. I can always load ASDF for a given project if it needs to incorporate other systems, smaller projects can just load themselves.

Of course for development I'll still be using ASDF a hundred times a day!

Erik.

On Mon, Apr 23, 2012 at 2:10 PM, Robert Goldman <rpgoldman-+bU95EEGNjthl2p70BpVqQ@public.gmane.org> wrote:
On 4/23/12 Apr 23 -3:52 PM, Erik Pearson wrote:
> Hi Robert,
> What I was thinking is that OPERATE assembles the plan via TRAVERSE,
> then executes it via PERFORM-PLAN, which calls PERFORM to translate the
> abstraction of operation + component into an action with side effect. It
> seems to me that it is each PERFORM (or, rather, the ones that we are
> interested in) that needs to be captured, since it is only in the body
> of the PERFORM method that the abstractions are made concrete in into
> lisp forms which carry out the desired actions.
>
> What I am referring to as "recording" is to append the lisp forms
> produced by each PERFORM into a list or other structure. At present this
> structure would need to be a global, but ideally it would be a slot in a
> session object that threads through each PERFORM. It is because the
> PERFORMS are carried out serially by PERFORM-PLAN according to the plan
> assembled by TRAVERSE that they can be "played back" to recreate the
> actions of this asdf session. Now if there were parallel executions of
> PERFORMs by PERFORM-PLAN, that would be different...
>
> I hope that makes sense.

I suppose, but I don't know how you break into what PERFORM does.

A quick and dirty way, as I described above, does require breaking into PERFORM. But I think you could either create an alternate method and inject that at around the PERFORM-PLAN level, or use a SESSION object as an additional specializer argument to open up other possible PERFORM implementations.
 

I was thinking you would precompute and cache the plan, and then invoke
PERFORM-PLAN on that.

Or maybe another method altogether to take the place of perform, like SIMULATE-PLAN or RECORD-PLAN, or ASSEMBLE-PLAN. 
 
That assumes that the call to TRAVERSE is actually expensive enough that
this is worth the trouble.  I think you could confirm or disconfirm that
claim quickly with some timing experiments.

Yes, I think I did that -- but with a different concern. When I was concerned about ASDF mucking around inspecting files when I was doing a REQUIRE, I added a feature to cause a fully loaded module to be flagged as loaded via mark-operation-done, so that do-traverse would stop traversing at such a module. It was about 2.5 seconds to traverse normally, and about 0.5 with this optimization for a require after the first one (of course for a deployed system not development.)

In any case load time is not my only interest.  It is also a desire to be able to use ASDF more effectively for more use cases. 

Erik.


cheers,
r

_______________________________________________
asdf-devel mailing list
asdf-devel@...
http://lists.common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel
Dmitriy Ivanov | 23 Apr 09:55
Picon

ASDlite - light-weight excerpt from ASDF

Hello folks,

Excuse my bothering if this is not a correct place to announce.
Nonetheless...

ASDlite Design Goals

1. Small footprint.
2  Ease of embedding into applications and systems not related to
   "compile-and-load Lisp files" tasks.
   For example: YstokHelp (http://lisp.ystok.ru/yhelp/)
3. ASDF compatibility for basic needs.
4. Operation arguments specification in dependencies.

Observation and Rationale

 1. The ASDF built-in operation hierarchy is actually of two-level depth.
    Original ASDF code does not exploit operation inheritance much
    (though something can be found in asdf-ecl.lisp).

 2. The operation slots are rather useless:
    original-initargs
      Is only referred in print-object
    parent
      In principle, indicates the target operation that required this one.
      But due to reusing operation objects in make-sub-operation,
      this is not the case.
      Also used for marking along with other visiting slots during traverse
      but we follow another approach.

    Adding entirely new operations, i.e. on the first level, is fine.
    But there is comfortable way to refine existing operations: the easiest
way
    is to add slots to base operation classes as only those properties
    do get passed into dependency operations.

    There is a more simple way pass arguments from operate to
    operation functions - by means of key arguments!

 3. The :do-first initarg is actually ignored by ASDF - its always set to
     ((compile-op (load-op ,@depends-on))))

 4. Avoid inline methods, which are rather inelegant in ASDF:
    - they rely on eval,
    - ASDF tries to remove a method from a generic function of a different
name.
    Due to non-standard behavior of remove-method in LW 4.4, system
redefinition
    intrusively signals about this.

 5. Referring to features in component definition is more useful than for
    dependency rules.

Operations in ASDlite

  operation ::= keyword | operation-instance
  operation-type ::= keyword | type-symbol
  operation-designator ::= keyword | (keyword . plist)
       | type-symbol | operation-instance

Operations are passed to perform and other operation-specific  methods.
Operation designators can be used in the right-hand side of rules.

We encourage using simple keywords like :compile or :load.
For these, ASDlite defines corresponding methods with eql specializers.
The plist allows you to pass key arguments to the operation methods.
In the "normal" mode, ASDlite accepts only keyword-based forms.

If you feel these are not enough and need "full-fledged" ASDF operation
classes, you can switch to the ASDF compatibility mode as follows:
- push :asdf-compat into *features* before compiling asdlite.lisp,
- refer to asdf:compile-op, asdf:load-op and the like,
- define your own subclasses of the operation class
  etc.
In the compatibility mode, ASDlite accepts all the above forms of operations
and designators.

Downloads and Details
http://lisp.ystok.ru/asdlite/

--
Sincerely,
Dmitriy Ivanov
lisp.ystok.ru
Erik Pearson | 23 Apr 01:23
Favicon

Fwd: Faster loading for deployed systems?

Hi Robert,

On Sun, Apr 22, 2012 at 3:24 PM, Erik Pearson <erik-mJWlkWs08JlF6kxbq+BtvQ@public.gmane.org> wrote:
Hi Robert,

Thanks for the Sunday afternoon reply.

On Sun, Apr 22, 2012 at 2:31 PM, Robert Goldman <rpgoldman-+bU95EEGNjthl2p70BpVqQ@public.gmane.org> wrote:
On 4/22/12 Apr 22 -3:47 PM, Faré wrote:
> Dear Erik,
>

>> ASDF assumes that since the system def file has not changed, the system
>> hasn't been changed either (I know there has been discussion of this
>> assumption.) But lets assume that at least part of the reason for that is
>> because it is very time consuming to check the entire set of files every
>> time the developer wants to recompile and load just a single file.
>>
> I don't think that's true, but my head hurts just thinking about it

Sorry, this was me being confused by my own tracing of asdf operations, and not thinking before I typed. I had placed a debugging line in the call to "safe-file-write-date" to see how ofen this is called in during a load. I thought it was not being called on source files just .asd files, but this was incorrect. I had probably already changed something, or been confused about the multiple calls made on systems -- the first call will get the file write date on both source and compiled (if any), but subsequent calls only on the compiled.

Erik.

_______________________________________________
asdf-devel mailing list
asdf-devel@...
http://lists.common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel
Erik Pearson | 23 Apr 01:23
Favicon

Fwd: Faster loading for deployed systems?

Hi Robert,

Thanks for the Sunday afternoon reply.

On Sun, Apr 22, 2012 at 2:31 PM, Robert Goldman <rpgoldman-+bU95EEGNjthl2p70BpVqQ@public.gmane.org> wrote:
On 4/22/12 Apr 22 -3:47 PM, Faré wrote:
> Dear Erik,
>

>> ASDF assumes that since the system def file has not changed, the system
>> hasn't been changed either (I know there has been discussion of this
>> assumption.) But lets assume that at least part of the reason for that is
>> because it is very time consuming to check the entire set of files every
>> time the developer wants to recompile and load just a single file.
>>
> I don't think that's true, but my head hurts just thinking about it.

This is not an accurate understanding.  It has come up over and over.
The right thing to do would be to check systems recursively, but there
are a number of reasons why this does not work with the current state of
ASDF.  You can look in the archives for explanations; it has been
discussed many times [I am not trying to be rude; I just don't have the
energy to write up another copy of the explanation.  Probably we should
put it in as a FAQ in the manual...]

Don't worry. I've been hanging around Lisp long enough to have a high bar on "rude", and this doesn't even come close.

Thanks for the correction, I'll go back and revisit those threads.
 
>
>> Why not assume in a deployment mode that NO system will have changed if it
>> is already loaded in memory, and unless the application has asked that it be
>> reloaded? This would be for performance as well as for safety -- we really
>> don't want dependency checking to result in any filesystem checks, etc.
>>
> Because that's wrong? I don't have as many dependencies as you do when
> I reload a system with asdf, but usually, I do want the dependencies
> to be reloaded.

The deployment mode is an edge case and can already be handled very
efficiently by dumping an image with the (stable) system you want
already loaded (cf. Xach's buildapp, although I know that's for SBCL,
rather than CCL; and CCL:SAVE-APPLICATION --
http://ccl.clozure.com/manual/chapter4.9.html#Saving-Applications).  In
some sense you are right, ASDF is slow for this use case, but it's
really not the use case ASDF is looking for.

This is what I've been doing for several years now with no problems (none due to asdf, that is!)

I'm exploring now a method for deploying small systems which are based on a large-ish compiled, dumped image, but with small to medium sized systems to be loaded and run as batches. Yes, I could just load these files manually, but would at least like to be able to do run-time inquiries of module availability (ala require), which, it seems to me, is the purview of asdf.

Also, it is possible that I'd need to add a specific 3rd party lisp package to a deployment without incorporating it into the master runtime image, and integration via asdf also seems natural.

BTW the deployment mode, for me, is the purpose of my development, so it is kinda important. But I realize we all sit in a different place in the development process. The main object of my work is a lisp web server that sits under mod_lisp, but I also need to run lisp processes in batch, as well as plenty of lisp ad-hoc analytical work.

I have thought that perhaps ASDF could output a deployment structure, which is a set of instructions for constructing the running system. Not well worded, perhaps, but if you capture the actions of asf you should be able to play them back and create a functionaly equivalent image (albeit not necessarily with asdf loaded.) E.g. if you just emit the filenames that are loaded during (perform load-op cl-source-file), you can just place them into a list and load them in sequence using just simple common lisp machinery. Faster than asdf, but still slow compared to loading a big image.
 

And ASDF is already too liberal about assuming stability, in a way that
breaks a lot of people's development practice (see my earlier remarks);
the last thing I would want to do is make it even more likely to miss a
change and build an inconsistent system.

Best,
r

_______________________________________________
asdf-devel mailing list
asdf-devel <at> common-lisp.net
http://lists.common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel


_______________________________________________
asdf-devel mailing list
asdf-devel@...
http://lists.common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel

Gmane