Counter-intuitive API for cxml:parse on LWM?

... and another

Re: Counter-intuitive API for cxml:parse on LWM?

Hi,

Quoting Marco Antoniotti (marcoxa <at> cs.nyu.edu):
> here is the example on LWM:

what is LWM?

> CL-USER 17 > (cxml:parse #P"CellCycle-1991Tys-2.xml" (cxml:make-whitespace-normalizer
(make-instance 'sax:default-handler)) :validate nil)
> NIL
> 
> CL-USER 18 > (with-open-file (f #P"CellCycle-1991Tys-2.xml" :direction :input)
>               (cxml:parse f (cxml:make-whitespace-normalizer (make-instance 'sax:default-handler)) :validate nil))
> 
> Error: Binary operation READ-BYTE attempted on character stream #<STREAM::LATIN-1-FILE-STREAM /Users/marcoxa/Projects/Genomics/biolosyst/sbmlambda/tests/CellCycle-1991Tys-2.xml>.
>   1 (abort) Return to level 0.
>   2 Return to top loop level 0.
[...]
>    Parse an XML document from  <at> var{input}, which can be a string, pathname,
>    octet vector, or stream.
[...]
> It appears that you need a 'xstream', but this is against the doc string content (and IMHO, against common sense).

Not an xstream, no (those are internal to cxml and would only be used
very rarely by user code).

What you need is a _binary_ stream though (and the docstring could
indeed be improved to say "octet vector, or octet stream" rather than
just "octet vector, or stream".

In other words, add :element-type '(unsigned-byte 8) to the
with-open-file.

d.

Re: Counter-intuitive API for cxml:parse on LWM?

On Apr 21, 2011, at 15:06 , David Lichteblau wrote:

> Hi,
> 
> Quoting Marco Antoniotti (marcoxa <at> cs.nyu.edu):
>> here is the example on LWM:
> 
> what is LWM?

Lispworks Mac

> 
>> CL-USER 17 > (cxml:parse #P"CellCycle-1991Tys-2.xml" (cxml:make-whitespace-normalizer
(make-instance 'sax:default-handler)) :validate nil)
>> NIL
>> 
>> CL-USER 18 > (with-open-file (f #P"CellCycle-1991Tys-2.xml" :direction :input)
>>              (cxml:parse f (cxml:make-whitespace-normalizer (make-instance 'sax:default-handler))
:validate nil))
>> 
>> Error: Binary operation READ-BYTE attempted on character stream #<STREAM::LATIN-1-FILE-STREAM /Users/marcoxa/Projects/Genomics/biolosyst/sbmlambda/tests/CellCycle-1991Tys-2.xml>.
>>  1 (abort) Return to level 0.
>>  2 Return to top loop level 0.
> [...]
>>   Parse an XML document from  <at> var{input}, which can be a string, pathname,
>>   octet vector, or stream.
> [...]
>> It appears that you need a 'xstream', but this is against the doc string content (and IMHO, against common sense).
> 
> Not an xstream, no (those are internal to cxml and would only be used
> very rarely by user code).
> 
> What you need is a _binary_ stream though (and the docstring could
> indeed be improved to say "octet vector, or octet stream" rather than
> just "octet vector, or stream".
> 
> In other words, add :element-type '(unsigned-byte 8) to the
> with-open-file.

Ok.  But the file is not a binary file.

Marco

> 
> 
> d.
> 

--
Marco Antoniotti

Re: Counter-intuitive API for cxml:parse on LWM?

Quoting Marco Antoniotti (marcoxa <at> cs.nyu.edu):
> Ok.  But the file is not a binary file.

The fundamental reason XML files need to be processed using binary
streams instead of character streams in (portable) Common Lisp is that
an XML parser needs to read the first few bytes before it can know which
external format to parse the file as.

Implemented using a Common Lisp character stream, that would require a
switch of the external format in the middle, using (setf
stream-external-format).  But that function doesn't exist in portable
Lisp.  (One of the few Lisps that supports it is Allegro with its
simple-streams, but we aim to support more Lisps than just Allegro.  We
also aim for more speed than typical gray streams-based implementations
of this idea offer.)

d.

Re: ... and another

Quoting Marco Antoniotti (marcoxa <at> cs.nyu.edu):
> Always on LWM.  I assume this has to do with UNICODE or not support, but the example in the docs is misleading.
> 
> 
> CL-USER 37 > (defparameter *source* (cxml:make-source "<example>text</example>"))

That example should work in any implementation with Unicode support.

Backtrace?
Value of *features*?

d.

Re: ... and another

Re: ... and another

Re: ... and another

Quoting Marco Antoniotti (marcoxa <at> cs.nyu.edu):
> 
> On Apr 21, 2011, at 16:54 , David Lichteblau wrote:
> 
> > Quoting Marco Antoniotti (marcoxa <at> cs.nyu.edu):
> >> Always on LWM.  I assume this has to do with UNICODE or not support, but the example in the docs is misleading.
> >> 
> >> 
> >> CL-USER 37 > (defparameter *source* (cxml:make-source "<example>text</example>"))
> > 
> > That example should work in any implementation with Unicode support.
> > 
> > Backtrace?
> > Value of *features*?
> > 
> 
> 
> Here they are.  I suspect that LWM does not support UNICODE.

It supports Unicode, but LispWorks has slighly weird subtypes of
CHARACTER, and sometimes code insists on one subtype over the other.

In this case, I'm afraid the argument to MAKE-SOURCE needs to be a
string made up of LW:SIMPLE-CHAR rather than CHARACTER.

Things to try:

;; returns T on other Lisps, but might go wrong on LispWorks:
(typep "<example>text</example>" '(vector runes:rune))

;; possible workaroud
(cxml:make-source (coerce "<example>text</example>"
                          '(simple-array runes:rune)))

Sorry about that, but I gave up on trying to fix all of these little
issues related to lw:simple-char a long time ago.  We have to assume
lw:simple-char at some point (for a presumably good reason which I can't
recall any more), and then that assumption trickles down.

So: "the example works in any implementation with Unicode support [IF
the object behind the printed representation of the string has the right
shape]".  Not ideal, I admit that.

d.

Re: ... and another