Faré | 29 Jan 2013 19:01
Picon
Gravatar

Patch for ASDF 3

The way you look for the path of catalog.dtd doesn't work in ASDF 3.
Here is a supported way that works with all versions of ASDF 2 and later.

I'll try to improve backward compatibility, but you may nonetheless
want to apply the patch.
Also, I believe that three years afterwards, and
with each and every implementation providing ASDF 2,
it is safe to remove the #-asdf2 code around your codebase.

—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org
Work harder! Millions on welfare depend on you.
Camille Troillard | 5 Dec 2012 16:46
Favicon
Gravatar

Possible bug in rune-dom::adjust-vector-exponentially

Hi,

I have a process that is stuck in the following function:

;; from dom-impl.lisp
(defun adjust-vector-exponentially (vector new-dimension set-fill-pointer-p)
  (let ((d (array-dimension vector 0)))
    (when (< d new-dimension)
      (loop
	  do (setf d (* 2 d))
	  while (< d new-dimension))
      (adjust-array vector d))
    (when set-fill-pointer-p
      (setf (fill-pointer vector) new-dimension))))

Apparently, adjust-vector-exponentially assumes d > 0, but this function has been called from (METHOD
SAX:CHARACTERS (RUNE-DOM::DOM-BUILDER T)) with an empty string (d = 0).  The loop obviously shows it will
never exit the function.

Adding the following before the loop statement fixes the problem:

      (when (= d 0)
        (incf d))

Best Regards,
Camille

Camille Troillard | 5 Dec 2012 17:11
Favicon
Gravatar

Incorrect character with code 12 in output?

Hello,

I have a database with text data which sometimes is not proper to valid XML.
For example, there is a field sometimes containing the character new-page or ^L in emacs (ASCII code = 12).

While CXML does not complain and outputs cmxl:text as it was given, another XML parser complains that this
is not valid XML.

	PCDATA invalid Char value 12

I am expecting CXML to produce valid output, is there something I missed to make it more strict?

I am using LispWorks 64 bits.

Thanks!

Best Regards,
Cam

Andreas Fuchs | 10 Sep 2012 02:33
Gravatar

Klacks bug with &# entities on CCL

Hi there,

The following code breaks in CCL (Version 1.8-r15286M  (DarwinX8632))
with the most recent cxml from quicklisp (cxml-20110619-git):

(klacks:with-open-source (source (cxml:make-source "<value>&#32;987lkjasd</value>"))
  (klacks:consume source)
  (klacks:consume source)
  (klacks:consume source))

Here's the SLIME backtrace I get from evaluating that form:

  0: ((SETF RUNES:%RUNE) #\  "" 0)
  1: (CXML::READ-TOKEN-3 #S(CXML::ZSTREAM :TOKEN-CATEGORY NIL :TOKEN-SEMANTIC NIL :INPUT-STACK ...))
  2: (CXML::PEEK-TOKEN #S(CXML::ZSTREAM :TOKEN-CATEGORY NIL :TOKEN-SEMANTIC NIL :INPUT-STACK ...))
  3: (CXML::KLACKS/CONTENT #<CXML::CXML-SOURCE #xD7DE306> #S(CXML::ZSTREAM :TOKEN-CATEGORY NIL
:TOKEN-SEMANTIC NIL :INPUT-STACK ...) #<CCL:COMPILED-LEXICAL-CLOSURE (:INTERNAL ..))
  4: (CXML::KLACKS/ELEMENT-2 #<CXML::CXML-SOURCE #xD7DE306> #S(CXML::ZSTREAM :TOKEN-CATEGORY NIL
:TOKEN-SEMANTIC NIL :INPUT-STACK ...) (("") ("xmlns" . "http://www.w3.org/2000/xmlns/") ..) #<CCL:COMPILED-L..
  5: (CXML::FILL-SOURCE #<CXML::CXML-SOURCE #xD7DE306>)
  6: (#<STANDARD-METHOD KLACKS:CONSUME (CXML::CXML-SOURCE)> #<CXML::CXML-SOURCE #xD7DE306>)
  7: (#<Anonymous Function #xD7E474E>)
  8: (CCL::CHEAP-EVAL (KLACKS:WITH-OPEN-SOURCE (SOURCE (MAKE-SOURCE
"<value>&#32;987lkjasd</value>")) (KLACKS:CONSUME SOURCE) (KLACKS:CONSUME SOURCE)
(KLACKS:CONSUME SOURCE)))

That same form works on SBCL, so I assume there's some
implementation-specific charset-related weirdness going on. I'm happy to
help debug this, but sadly, my code-diving time is a bit limited at the
moment.
(Continue reading)

Bart Botta | 29 Aug 2012 00:58
Picon

Bug in DTD parsing?

(flex:with-input-from-sequence (s (babel:string-to-octets
                                   "<![IGNORE[ ]]><!-- -->"
                                   :encoding :utf-8))
  (cxml:parse-dtd-stream s))

errors with
Document not well-formed: Unexpected character #\!.
Context:
  Line 1, column 17 in NIL

   [Condition of type CXML:WELL-FORMEDNESS-VIOLATION]

but parses OK with whitespace between ]]> and <!--

(for a real example, see line 396 in http://docbook.org/xml/4.3/dbpoolx.mod)

it looks like P/IGNORE-SECT in cxml/xml/xml-parse.lisp should unread 1
rune after matching ]]> (or exit the loop sooner so it doesn't read an
extra).

Raymond Wiker | 9 Jul 2012 20:39
Picon

Inconsistency in creating %children vector

I was scratching my head for a couple of hours today, because of an apparent failure of
cxml-dom::make-space - it did not increase the size of the %children vector. On closer examination, it
turned out that the reason was that the %children vector was not adjustable, at that point.

The only places in cxml-stp where %children is set up are in %unchecked-insert-child (parent-node.lisp)
and normalize-text-nodes! in xpath.lisp. The first of these has (make-array ... :adjustable t ...),
while the other does not. My guess is that normalize-text-nodes! might be called as part of an xpath
evaluation, and if this happens in code that is actually building the cxml-stp structure, the result
might be a situation where the %children vector is not adjustable.

I'm not sure if this is a valid use of cxml-stp/xpath, but there is at least one documented case where it has
been done :-)

I could (trivially) change my code to not use xpath as part of the construction process, but adding
:adjustable t to the call to make-array in xpath.lisp makes my (current) code work, and I guess it also
makes cxml-stp slightly more robust.

Steven Nunez | 11 Jun 2012 05:50
Picon
Favicon

Real-world example: with attachment

Greetings again,

Perhaps a good example would be a real one. I'm starting with some simple functions, such as obtaining the model name, algorithm, etc, from the attached XML example. Can anyone provide an exemplary method for doing this? Below is my naive attempt, which are all failing miserably:

(defun algorithm-name (model)
  "Returns the algorithm for the model"
  (stp:filter-recursively (stp:of-name "algorithmName" (stp:namespace-uri model)) model))

I know this one will (if it were working) return a list, but since I can't get past the namespace problem haven't bothered to go any further. That function, I assume, is a rather trivial one for someone with some good examples or knowledge to work from. Here's a slightly more difficult function:

(defun model-coefficient-list (model)
  "Returns a nested list of the following structure: (<model-name>-coefficients (coefficient1 coefficient1-value) (coefficient1 coefficient2-value) ... (coefficientn coefficientn-value))"
  ...)

Something like this would be good to put on the STP website.

Regards,
- Steve
Steven Nunez | 11 Jun 2012 04:46
Picon
Favicon

Real-world Example

Greetings again,

Perhaps a good example would be a real one. I'm starting with some simple functions, such as obtaining the model name, algorithm, etc, from the attached XML example. Can anyone provide an exemplary method for doing this? Below is my naive attempt, which are all failing miserably:

(defun algorithm-name (model)
  "Returns the algorithm for the model"
  (stp:filter-recursively (stp:of-name "algorithmName" (stp:namespace-uri model)) model))

I know this one will (if it were working) return a list, but since I can't get past the namespace problem haven't bothered to go any further. That function, I assume, is a rather trivial one for someone with some good examples or knowledge to work from. Here's a slightly more difficult function:

(defun model-coefficient-list (model)
  "Returns a nested list of the following structure: (<model-name>-coefficients (coefficient1 coefficient1-value) (coefficient1 coefficient2-value) ... (coefficientn coefficientn-value))"
  ...)

Something like this would be good to put on the STP website.

Regards,
- Steve




Steven Nunez | 11 Jun 2012 04:27
Picon
Favicon

namespace-uri usage

Gents,

Can anyone tell me why namespace-uri can't give me the namespace for this test file?

<PMML version="4.0" xsi:schemaLocation="http://www.dmg.org/PMML-4_0 http://www.dmg.org/v4-0/pmml-4-0.xsd" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://www.dmg.org/PMML-4_0">
  <Header copyright="Steve Nunez" description="Sample for Namespace">
    <Timestamp>2012-06-11T08:17:10.8</Timestamp>
  </Header>
</PMML>

* (defparameter *ns-test*
     (cxml:parse #p"ns-test.xml" (stp:make-builder)))

* (stp:namespace-uri *ns-test*)

This, and a few variations, give "There is no applicable method for the generic function". 

I suspect the answer is because I'm asking for the namespace of a document, whereas STP wants it for an element. That's fine, but different enough from the typical ways of XML processing to make figuring out how to use it rather difficult.

Does anyone have any working examples of processing documents typically found 'in the wild'? I've read the test.lisp file, but it simply programatically creates elements and then tests for them. Some real-world examples, common idioms and best practices would be very helpful.

Cheers,
    - SteveN


Steven Nunez | 10 Jun 2012 09:51
Picon
Favicon

cxml-stp usage

Greetings all,

I'm a bit stumped by some of the STP functions and usage. The documentation is, well, a bit sparse, so it's not clear whether there really is a bug or I'm mis-interpreting what I'm reading. Here is a simple function that should return all the child nodes of an element with the local-name of 'RegressionModel':

(defun function-name (document)
  "Return the PMML model type of the document"
  (stp:filter-recursively (stp:of-name "RegressionModel") (stp:first-child document)))

According to the documentation of filter-recursively:
Return a list of descendant nodes of node in pre-order, from which nodes that do not satisfy predicate have been removed.
and that for of-name:
This function creates a test function for nodes of this name.

The function returned will return T if the argument is an instance of attribute or element and has the specified local-name and namespace URI, and will return NIL otherwise.

If local-name is nil, only the namespace URI is considered for comparison.

and here's the local snippet of the xmls that I'm trying to filter for:

#.(CXML-STP:ELEMENT
    #| :PARENT of type ELEMENT |#
    :ATTRIBUTES '(#.(CXML-STP:ATTRIBUTE
                     #| :PARENT of type ELEMENT |#
                     :VALUE "PolynomialRegression"
                     :LOCAL-NAME "algorithmName")
                  #.(CXML-STP:ATTRIBUTE
                     #| :PARENT of type ELEMENT |#
                     :VALUE "regression"
                     :LOCAL-NAME "functionName")
                  #.(CXML-STP:ATTRIBUTE
                     #| :PARENT of type ELEMENT |#
                     :VALUE "KNIME Polynomial Regression"
                     :LOCAL-NAME "modelName")
                  #.(CXML-STP:ATTRIBUTE
                     #| :PARENT of type ELEMENT |#
                     :VALUE "s_s_temp"
                     :LOCAL-NAME "targetFieldName"))
    :BASE-URI "file://+/Users/nunez/src/lisp/pmml/polynomial-regression.xml"
    :CHILDREN '(#.(CXML-STP:TEXT
                   #| :PARENT of type ELEMENT |#
                   :DATA "
    ")
So, there's an attribute with a local-name that matches the value in of-name. Why is this function always returning NIL?

Is stp stable enough for production use? It's been a while since it's been updated. I'm new to XML with LISP, so if there's a better documented or easier way to process these documents, happy to take a different route.

Finally, does anyone know why I need to pass the function the first child? Seem cxml-stp is getting tripped up on the first line of the xml file: <?xml version="1.0" encoding="UTF-8"?> and forcing me to manually grab the second element. Is there a convention or idiom I'm missing?

Sample file attached.

Cheers,
    - Steve
Raymond Wiker | 25 May 2012 12:28
Picon

Exported symbols [was: Re: Possible bug in cxml-stp]

On Wed, May 23, 2012 at 12:02 AM, David Lichteblau <david <at> lichteblau.com> wrote:
Quoting Raymond Wiker (rwiker <at> gmail.com):
> Would it be an idea to add functionality similar to what
> plexippus-xpath provides via with-namespaces? I.e, a special variable
> that holds the current set of namespace tag/uri mappings, and have
> some some additional constructors that use split-qname to extract the
> tag and look-up the url from the current set of mappings.

Possibly...  cxml currently has such a macro only for its serialization
API, i.e.

CL-USER> (cxml:with-xml-output (stp:make-builder)
          (cxml:with-namespace ("a" "http://a")
            (cxml:with-element "a:b"
              (cxml:text "xyz"))))
#.(CXML-STP-IMPL::DOCUMENT
  :CHILDREN '(#.(CXML-STP:ELEMENT
                 #| :PARENT of type DOCUMENT |#
                 :CHILDREN '(#.(CXML-STP:TEXT
                                #| :PARENT of type ELEMENT |#
                                :DATA
                                "xyz"))
                 :LOCAL-NAME "b"
                 :NAMESPACE-PREFIX "a"
                 :NAMESPACE-URI "http://a")))

That API can't really modify STP in place though; the builder only knows
how to set up a full document.

[Subject: changed, as it was getting embarassingly inappropriate.]

I ended up modifying my add-element function, based on the code that you included earlier. It now looks like this:

(defun add-element (parent element-name attributes &optional element-value)
  (let ((element-name (if (listp element-name) (first element-name) element-name))
        (namespace-uri (if (listp element-name) (second element-name)
                          (cxml-stp:find-namespace (cxml::split-qname element-name) parent))))
    (let ((new-element (cxml-stp:make-element element-name namespace-uri)))
      (cxml-stp:append-child parent new-element)
      (loop for (name value) on attributes by #'cddr
            do (cxml-stp:add-attribute
                new-element
                (cxml-stp:make-attribute
                 (princ-to-string value)
                 name
                 (cxml-stp-impl::find-attribute-namespace
                  (cxml::split-qname name) new-element))))
      (when element-value
        (cxml-stp:append-child new-element (cxml-stp:make-text (princ-to-string element-value))))
      new-element)))

This seems to work, but I'm not 100% sure of the code that handles namespaces for attributes.

Note that the code uses two un-exported symbols: cxml::split-name and cxml-stp-impl::find-attribute-namespace. Should these be exported?

Best Regards,
Raymond Wiker

Gmane