Stephen C. Gilardi | 1 Sep 2008 02:12
Picon

remove-ns before loading lib?

One of the downsides of interactive development and testing using the REPL is the fact that an old definitions in a lib's namespace can linger in memory after its source code has been deleted from the lib. One can relaunch Clojure to be sure of clearing that situation, but that often loses some desirable context. I'm proposing a change to make relaunching Clojure for that purpose unnecessary.

With the new clojure/require and clojure/use, we've introduced a suggested, preferred, best-supported way of working with Clojure in which there's a one-to-one correspondence between a lib namespace and the lib that defines it.  The enclosed patch would ensure that any old definitions in a lib namespace are discarded before loading the lib: the namespace would be built up from scratch with each load. Putting the 'remove-ns call at the location I suggest would allow ":reload" and ":reload-all" to support this "from scratch" behavior as well. (:reload-all is especially useful from the REPL.)

There may still be situations where other code can hold onto an old definition across a reload but this change will cover the vast majority of cases easily and reliably.

Would this be a good change?

--Steve

Index: boot.clj
===================================================================
--- boot.clj (revision 1013)
+++ boot.clj (working copy)
<at> <at> -3045,6 +3045,7 <at> <at>
   namespace exists after loading. If require, records the load so any
   duplicate loads can be skipped."
   [lib need-ns require]
+  (remove-ns lib)
   (load-resources (root-resource lib))
   (throw-if (and need-ns (not (find-ns lib)))
             "namespace '%s' not found after loading '%s'"
 

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Clojure" group.
To post to this group, send email to clojure <at> googlegroups.com
To unsubscribe from this group, send email to clojure+unsubscribe <at> googlegroups.com
For more options, visit this group at http://groups.google.com/group/clojure?hl=en
-~----------~----~----~----~------~----~------~--~---

Rich Hickey | 1 Sep 2008 04:04
Picon
Gravatar

Re: Discrepancies between boot.clj and API Docs


On Aug 31, 6:05 pm, Meikel Brandmeyer <m... <at> kotka.de> wrote:
> Hello Rich,
>
> I noticed a difference between boot.clj and the API Docs. I know that
> there are internal definitions, which are not supposed to be used by the
> user. New functions might also take some time, before they show up in
> the docs.
>
> But I'm missing for example delay and force. I found myself implementing
> a Promise class to do exactly this! While working on VimClojure I
> found out
> by accident that there is a force and hence a delay. That could have
> saved
> me some time...
>
> I understand that you don't want the docs to be open for public editing.
> But maybe we can install a process so that people can easily access the
> source for the docs (eg. via SVN repo), write a patch and submit it to
> you. You can review it and push it finally to the website if you agree
> or request changes or reject the patch completely.
>
> This would be a compromise between you doing all the work and public
> editing. (And it would free some time for you to concentrate on Clojure
> itself. :))
>
> I think clojure.org should stay a moderated and authoritative resource
> for the docs. But it should be up-to-date even when the available
> functions still change.
>

The API docs are generated from the source, so there's nothing there
you can't get from the source. Generally, the site corresponds to a
release. Sometimes things change between releases and I don't want to
make interim promises in the API docs that might not be stable/final.

So, when working from SVN, you should use functions like find-doc, doc
and ns-publics.

Rich
Rich Hickey | 1 Sep 2008 04:11
Picon
Gravatar

Re: remove-ns before loading lib?


On Aug 31, 8:12 pm, "Stephen C. Gilardi" <squee... <at> mac.com> wrote:
> One of the downsides of interactive development and testing using the
> REPL is the fact that an old definitions in a lib's namespace can
> linger in memory after its source code has been deleted from the lib.
> One can relaunch Clojure to be sure of clearing that situation, but
> that often loses some desirable context. I'm proposing a change to
> make relaunching Clojure for that purpose unnecessary.
>
> With the new clojure/require and clojure/use, we've introduced a
> suggested, preferred, best-supported way of working with Clojure in
> which there's a one-to-one correspondence between a lib namespace and
> the lib that defines it.  The enclosed patch would ensure that any old
> definitions in a lib namespace are discarded before loading the lib:
> the namespace would be built up from scratch with each load. Putting
> the 'remove-ns call at the location I suggest would allow ":reload"
> and ":reload-all" to support this "from scratch" behavior as well.
> (:reload-all is especially useful from the REPL.)
>
> There may still be situations where other code can hold onto an old
> definition across a reload but this change will cover the vast
> majority of cases easily and reliably.
>
> Would this be a good change?
>

No, because code that used the namespace will then be bound to
orphaned vars. New/fixed definitions under those names will create new
vars, and thus not update referencing code.

So, only namespaces that are not referenced by anything else can be
safely removed - it's a very special-purpose operation.

Rich
Stephen C. Gilardi | 1 Sep 2008 05:00
Picon

Re: remove-ns before loading lib?


On Aug 31, 2008, at 10:11 PM, Rich Hickey wrote:
On Aug 31, 8:12 pm, "Stephen C. Gilardi" <squee... <at> mac.com> wrote:
There may still be situations where other code can hold onto an old
definition across a reload but this change will cover the vast
majority of cases easily and reliably.

Or, as it turns out not the vast majority at all. Whoops!

No, because code that used the namespace will then be bound to
orphaned vars. New/fixed definitions under those names will create new
vars, and thus not update referencing code.

I understand, thanks. I recall now we've discussed "transactional loading" in the past and you decided to pass.

Would it work to compare the set of ns-publics for the namespace before and after the load and unbind-root the vars that were present before but are not present after? I'm not proposing it, just trying to understand how things work.

--Steve


--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Clojure" group.
To post to this group, send email to clojure <at> googlegroups.com
To unsubscribe from this group, send email to clojure+unsubscribe <at> googlegroups.com
For more options, visit this group at http://groups.google.com/group/clojure?hl=en
-~----------~----~----~----~------~----~------~--~---

Parth Malwankar | 1 Sep 2008 05:48
Picon

avoiding duplication using multi-methods


Hello,

I have a situation in which different multi-methods
need similar behavior with different arguments.

So they call the same function (one-or-two below)
with the arguments. E.g.

    (defmulti what-num (fn [x] x))

    (defn one-or-two [x] (println "saw one or two: " x))

    (defmethod what-num :one [x] (one-or-two x))
    (defmethod what-num :two [x] (one-or-two x))
    (defmethod what-num :three [x] (println "saw 3"))
    (defmethod what-num :four [x] (println "saw 4"))

    user=> (what-num :one)
    saw one or two:  :one
    nil
    user=> (what-num :two)
    saw one or two:  :two
    nil
    user=> (what-num :three)
    saw 3

This works fine. However, I was wondering if its at all
possible to return duplication of creating the
methods 'what-num :one [x]' and 'what-num :two [x]'.

Can we have something to the effect of
'what-num (or :one :two) [x]'? This would help get
rid of one-or-two and I would be able to define a single
what-num for this.

I could have 'defmulti what-num' conditionally return
something like :one-or-two but I am not sure if thats such
a good idea as I might end up with a big 'cond' over
time.

What would be a good way to do something like this?

Thanks.
Parth

darren.austin | 1 Sep 2008 09:26
Picon
Gravatar

Re: avoiding duplication using multi-methods


On Aug 31, 8:48 pm, Parth Malwankar <parth.malwan... <at> gmail.com> wrote:
> Hello,
>
> I have a situation in which different multi-methods
> need similar behavior with different arguments.
>
> So they call the same function (one-or-two below)
> with the arguments. E.g.
>
>     (defmulti what-num (fn [x] x))
>
>     (defn one-or-two [x] (println "saw one or two: " x))
>
>     (defmethod what-num :one [x] (one-or-two x))
>     (defmethod what-num :two [x] (one-or-two x))
>     (defmethod what-num :three [x] (println "saw 3"))
>     (defmethod what-num :four [x] (println "saw 4"))
>
>     user=> (what-num :one)
>     saw one or two:  :one
>     nil
>     user=> (what-num :two)
>     saw one or two:  :two
>     nil
>     user=> (what-num :three)
>     saw 3
>
> This works fine. However, I was wondering if its at all
> possible to return duplication of creating the
> methods 'what-num :one [x]' and 'what-num :two [x]'.
>
> Can we have something to the effect of
> 'what-num (or :one :two) [x]'? This would help get
> rid of one-or-two and I would be able to define a single
> what-num for this.
>
> I could have 'defmulti what-num' conditionally return
> something like :one-or-two but I am not sure if thats such
> a good idea as I might end up with a big 'cond' over
> time.
>
> What would be a good way to do something like this?

I don't know if it is a "good" way, but here is what I used for a
similar problem:

(defmacro defmethod-for-values
  [multifn dispatch-values-seq & fn-tail]
  `(let [pvar# (var ~multifn)
          method# (fn ~ <at> fn-tail)]
         (doseq dispatch-val# ~dispatch-values-seq
           (. pvar# (commuteRoot (fn [#^clojure.lang.MultiFn mf#]
                                                           (. mf#
(assoc dispatch-val# method#))))))))

Which is just a wrapper around the defmethod macro that allows you to
specify a list of values instead of a single value that will be mapped
to the method.

This would allow you to define your above example with:

(defmulti what-num (fn [x] x))

(defmethod-for-values what-num '(:one :two) [x] (println "saw one or
two: " x))
(defmethod what-num :three [x] (println "saw 3"))
(defmethod what-num :four [x] (println "saw 4"))

See this thread for more details:

http://groups.google.com/group/clojure/browse_thread/thread/eed9015c2456e336/8822ecd0d7f54b8b

Hope this helps,
--Darren

Parth Malwankar | 1 Sep 2008 10:31
Picon

Re: avoiding duplication using multi-methods


On Sep 1, 12:26 pm, darren.aus... <at> gmail.com wrote:
> On Aug 31, 8:48 pm, Parth Malwankar <parth.malwan... <at> gmail.com> wrote:
>
>
>
> > Hello,
>
> > I have a situation in which different multi-methods
> > need similar behavior with different arguments.
>
> > So they call the same function (one-or-two below)
> > with the arguments. E.g.
>
> >     (defmulti what-num (fn [x] x))
>
> >     (defn one-or-two [x] (println "saw one or two: " x))
>
> >     (defmethod what-num :one [x] (one-or-two x))
> >     (defmethod what-num :two [x] (one-or-two x))
> >     (defmethod what-num :three [x] (println "saw 3"))
> >     (defmethod what-num :four [x] (println "saw 4"))
>
> >     user=> (what-num :one)
> >     saw one or two:  :one
> >     nil
> >     user=> (what-num :two)
> >     saw one or two:  :two
> >     nil
> >     user=> (what-num :three)
> >     saw 3
>
> > This works fine. However, I was wondering if its at all
> > possible to return duplication of creating the
> > methods 'what-num :one [x]' and 'what-num :two [x]'.
>
> > Can we have something to the effect of
> > 'what-num (or :one :two) [x]'? This would help get
> > rid of one-or-two and I would be able to define a single
> > what-num for this.
>
> > I could have 'defmulti what-num' conditionally return
> > something like :one-or-two but I am not sure if thats such
> > a good idea as I might end up with a big 'cond' over
> > time.
>
> > What would be a good way to do something like this?
>
> I don't know if it is a "good" way, but here is what I used for a
> similar problem:
>
> (defmacro defmethod-for-values
>   [multifn dispatch-values-seq & fn-tail]
>   `(let [pvar# (var ~multifn)
>           method# (fn ~ <at> fn-tail)]
>          (doseq dispatch-val# ~dispatch-values-seq
>            (. pvar# (commuteRoot (fn [#^clojure.lang.MultiFn mf#]
>                                                            (. mf#
> (assoc dispatch-val# method#))))))))
>

Thanks Darren.
Along the lines of your macro I created the one
below that avoids clojure.lang.*.
I keep forgetting this is lisp and we can write a macro :P

Here is what I could come up with.

    (defmulti what-num (fn [x] x))

    (defmacro defmethod-in [n dval-coll args & body]
      (let [get-method (fn [d] `(defmethod ~n ~d ~args ~ <at> body))]
        `(do ~ <at> (map get-method dval-coll))))

    (defmethod-in what-num [:one :two] [x] (println "1 or 2" x))
    (defmethod what-num :three [x] (println "saw 3"))
    (defmethod what-num :four [x] (println "saw 4"))
    (defmethod what-num :default [x] :default)

In case this can be improved further, comments welcome.
Seems to be working fine so far:

    user=> (what-num :four)
    saw 4
    nil
    user=> (what-num :one)
    1 or 2 :one
    nil
    user=> (what-num :two)
    1 or 2 :two
    nil
    user=> (what-num :three)
    saw 3
    nil
    user=> (what-num :five)
    :default

Parth

> Which is just a wrapper around the defmethod macro that allows you to
> specify a list of values instead of a single value that will be mapped
> to the method.
>
> This would allow you to define your above example with:
>
> (defmulti what-num (fn [x] x))
>
> (defmethod-for-values what-num '(:one :two) [x] (println "saw one or
> two: " x))
> (defmethod what-num :three [x] (println "saw 3"))
> (defmethod what-num :four [x] (println "saw 4"))
>
> See this thread for more details:
>
> http://groups.google.com/group/clojure/browse_thread/thread/eed9015c2...
>
> Hope this helps,
> --Darren
Apurva Sharan | 1 Sep 2008 05:46
Picon

Re: with-meta usage


Thanks Meikel!

That makes it clearer...

Regards,
Apurva

----- Original Message -----
From: "Meikel Brandmeyer" <mb <at> kotka.de>
To: clojure <at> googlegroups.com
Sent: Sunday, August 31, 2008 8:46:01 PM GMT +05:30 Chennai, Kolkata, Mumbai, New Delhi
Subject: Re: with-meta usage

Hello,

> I was wondering why the first scenario didn't work but couldn't find
> details on this. Can someone please explain?
I also stumbled over this issue. Please read this sentence from the
"Metadata" section on clojure.org (http://clojure.org/metadata)

> "Symbols and collections support metadata"

I also missed this. It's pretty subtle. Since 10 is neither a Symbol nor
a Collection. You get the error. You can set the metadata of the Var:

---8<---
user=> (def #^{:Some :MetaData} v 10)
#'user/v
user=> (meta #'v)
{:Some :MetaData, :file "NO_SOURCE_FILE", :name v, :ns #<Namespace:  
user>, :line 1}
---8<---

Hope this helps.

Sincerely
Meikel

Michael Reid | 1 Sep 2008 15:15
Picon

Re: Local recursive lazy sequences


Hi Achim,

On Sun, Aug 31, 2008 at 1:58 PM, Achim Passen <achim.passen <at> gmail.com> wrote:

> Now suppose you want to define a function which returns the nth
> fibonacci number, but you down want "fibs" around globally. My guess
> was:
>
>        (defn fib [n]
>          (let [fibs2 (lazy-cat '(0 1) (map + fibs2 (drop 1 fibs2)))]
>            (nth fibs2 n)))
>
> But this yields "Unable to resolve symbol: fibs2 in this context", so
> let doesn't allow being self-referential the way def does. I don't
> quite understand why the second arg to lazy-cat is being evaluated in
> the first place. Shouldn't evaluation be delayed until needed?
>
> Does anyone know how to get around this?

You can create locally recursive functions by writing an inline (fn
..) form with a name. So your example above becomes:

(defn fib [n]
  (let [fibs2 (fn fibs2 [] (lazy-cat '(0 1) (map + (fibs2) (drop 1 (fibs2)))))]
    (nth (fibs2) n)))

So instead of defining fibs2 as a lazy-cat directly, we wrap it in a
function that when called returns the lazy-cat you are trying to
define. Note that the inline (fn) form has a symbol inserted before
its argument list:

 (fn fibs2 [] (lazy-cat ...)))

This allows an anonymous function to refer to itself. Then we just
need to replace each use of fibs2 with a call to the wrapper instead
of a direct reference to the lazy-cat--each reference to fibs2 becomes
a call to (fibs2).

Results:

user> (fib 0)
0
user> (fib 1)
1
user> (fib 2)
1
user> (fib 3)
2
user> (fib 4)
3
user> (fib 5)
5

There might be a cleaner way to do this, I'm not sure. This was the
first idea that popped into my head :).

/mike.

Achim Passen | 1 Sep 2008 16:08
Picon
Gravatar

Re: Local recursive lazy sequences


Hi Mike,

thank you for your reply!

On Sep 1, 3:15 pm, "Michael Reid" <kid.me... <at> gmail.com> wrote:
> (defn fib [n]
>   (let [fibs2 (fn fibs2 [] (lazy-cat '(0 1) (map + (fibs2) (drop 1 (fibs2)))))]
>     (nth (fibs2) n)))

Dodgy. ;-) I wasn't aware of named anonymous fns – nice.

But i suspect disguising lazy seqs as fns prevents memoization from
happening, which is the benefit of using lazy seqs over naive
recursion. So on each (fibs2) call, a completely new fibs sequence is
built, without reusing values computed earlier, it seems:

With

(def fibs (lazy-cat '(0 1) (map + fibs (drop 1 fibs))))

(defn fib [n]
  (let [fibs2 (fn fibs2 [] (lazy-cat '(0 1) (map + (fibs2) (drop 1
(fibs2)))))]
    (nth (fibs2) n)))

freshly defined, (nth fibs 10000) finishes in a couple of seconds,
while (fib 10000) might run for ages …

Any ideas very much appreciated!

Thanks again,

kind regards,
achim


Gmane