Leslie P. Polzer | 2 Aug 2008 08:49
Picon
Gravatar

Re: Deferred indexing


> It would be nice to be able to say:
>
> (defpclass banana () (:index t))
>
>   (setf banana (make-instance 'banana))
>
> ; do stuff with banana, then later optionally:
>
>   (store-indexed-object banana)
>
> We could then do without proxy objects in Weblocks.

(defun make-deferred-instance (classname &rest initargs)
  (with-transaction ()
    (let ((obj (apply #'make-instance classname initargs)))
      (remove-kv (slot-value obj 'elephant::%oid)
                 (find-class-index classname)))))

(defun activate-deferred-instance (obj)
  (let ((oid (slot-value obj 'elephant::%oid))
        (idx (find-class-index (class-of obj))))
  (if (get-value oid idx)
    (warn "Attempted to activate Object ~S which is already indexed!" obj)
    (setf (get-value oid idx) obj))))

This seems to work nicely, even with secondary indices.
Haven't tested derived, though.

  Leslie
(Continue reading)

Alex Mizrahi | 2 Aug 2008 13:54
Picon
Favicon

Re: Deferred indexing

 ??>> We could then do without proxy objects in Weblocks.
 LPP> This seems to work nicely, even with secondary indices.

that looks like an ugly hack, though. what problem are you trying to solve?
hide some instances from enumeration? why not just make slot called
"visible" so you can just filter out instances that are not visible?

Leslie P. Polzer | 2 Aug 2008 14:15
Picon
Gravatar

Re: Re: Deferred indexing


> what problem are you trying to solve?
> hide some instances from enumeration? why not just make slot called
> "visible" so you can just filter out instances that are not visible?

That would be another solution, but it's got a few features
I don't like compared to the other one.

As long as the ugly (actually not that ugly IMHO) hack
doesn't have any negative side effects I'm fine.

But thanks; I always value your opinion, Alex.

  Leslie

Leslie P. Polzer | 9 Aug 2008 12:10
Picon
Gravatar

NIL values in secondary indices


(defpclass artichoke ()
  ((slot1 :accessor slot1 :initform nil :index t)))
-> #<PERSISTENT-METACLASS ARTICHOKE>

(make-instance 'artichoke)
-> #<ARTICHOKE oid:53249>

(get-instances-by-range 'artichoke nil nil)
-> NIL

Is there any sense behind this result? Are NIL values somehow
special to the secondary indexing mechanism?

How should I find the missing instances with
GET-INSTANCES-BY-RANGE?

  Leslie

Alex Mizrahi | 9 Aug 2008 13:18
Picon
Favicon

Re: NIL values in secondary indices

 LPP> Is there any sense behind this result? Are NIL values somehow
 LPP> special to the secondary indexing mechanism?

it is somehow special in map-index, and inherently in 
GET-INSTANCES-BY-RANGE --
if start is NIL it starts from beginning, it end is NIL it iterates to the 
end, so (NIL, NIL)
range should mean all the sequence. behaviour you're observing is wrong, i 
guess.

 LPP> How should I find the missing instances with
 LPP> GET-INSTANCES-BY-RANGE?

question is _why_ should you use GET-INSTANCES-BY-RANGE when
you can do this via GET-INSTANCES-BY-VALUE?

Leslie P. Polzer | 9 Aug 2008 14:14
Picon
Gravatar

Re: Re: NIL values in secondary indices


Hi Alex,

>  LPP> Is there any sense behind this result? Are NIL values somehow
>  LPP> special to the secondary indexing mechanism?
>
> it is somehow special in map-index, and inherently in
> GET-INSTANCES-BY-RANGE --
> if start is NIL it starts from beginning, it end is NIL it iterates to the
> end, so (NIL, NIL)
> range should mean all the sequence.

That's the behaviour per the documentation as I understand it, yes.

>  LPP> How should I find the missing instances with
>  LPP> GET-INSTANCES-BY-RANGE?
>
> question is _why_ should you use GET-INSTANCES-BY-RANGE when
> you can do this via GET-INSTANCES-BY-VALUE?

Because I need sorting... ideally the NIL values should just be
clustered at the end or beginning of the sorted space.

  Leslie

Alex Mizrahi | 9 Aug 2008 16:59
Picon
Favicon

Re: NIL values in secondary indices

??>> question is _why_ should you use GET-INSTANCES-BY-RANGE when
??>> you can do this via GET-INSTANCES-BY-VALUE?
LPP> Because I need sorting... ideally the NIL values should just be
LPP> clustered at the end or beginning of the sorted space.

it seems you're using db-postmodern, i have bad news for you -- postmodern
backend supports correct sorting _only_ if all values are either interger,
or string. if you ever insert NIL there, sorting will be random. yep, i know
it sucks, but it's a price we pay for using SQL directly.

actually supporting slot type (OR STRING NULL) is easier than supporting
arbitrary type combinations -- we can translate NIL to SQL NULL, so NIL
values
will be clustered, and rest will be sorted, but it's still considerable
amount of work, and it's tricky to support all cases..

      LPP>>  Is there any sense behind this result? Are NIL values somehow
      special to the secondary indexing mechanism?

i've found why it's such -- there are actually checks in btree code,
introduced by patch:

[db-postmodern removed possiblity of using NIL as a key in btrees
Henrik Hjelte<henrik@...>**20071124163828]

it works in pretty nasty way -- if key is NIL it silently ignores
insert into btree. a problem is if NIL wasn't ignored it will
upgrade btree key type to "object" -- that means that
 sorting won't work. iirc Henrik was annoyed with this behaviour,
so he've just banned all NILs. this means GET-INSTANCES-BY-VALUE
(Continue reading)

Leslie P. Polzer | 9 Aug 2008 21:35
Picon
Gravatar

Re: Re: NIL values in secondary indices


> iirc Henrik was annoyed with this behaviour,
> so he've just banned all NILs. this means GET-INSTANCES-BY-VALUE
> won't work either -- those object simply won't be in indices.
>
> this appears to be pretty bad, i'll check if i
> can fix this..

That would be a good start.

I can sort out the objects with NIL values later on the Lisp
side -- that's acceptable. But those objects not showing up
at all in the index is pretty bad indeed!

  Leslie

Leslie P. Polzer | 10 Aug 2008 10:13
Picon
Gravatar

Re: Re: NIL values in secondary indices


While we're at it, I remember running into a similar problem
with float slots.

Are there any hidden difficulties in implementing float sort
support?

Alex Mizrahi | 11 Aug 2008 15:26
Picon
Favicon

Re: NIL values in secondary indices

 LPP> While we're at it, I remember running into a similar problem
 LPP> with float slots.

yep, if it sees float it will upgrade column to "object" type and
sorting will be lost

 LPP> Are there any hidden difficulties in implementing float sort
 LPP> support?

i think there are no, we just didn't get to it as floats aren't used
as frequently as integers and strings are. i think it would be
enough to modify functions  postgres-format and postgres-value-to-lisp
to support floats and it will work. 


Gmane