Clark C. Evans | 2 Dec 2005 18:24
Gravatar

Re: JSON and YAML - a minor incompatibility

On Fri, Dec 02, 2005 at 05:31:47AM -0800, Douglas Crockford wrote:
| How are you doing with the ', ' problem?

I've been hammered /w day job work lately; so I've made sparse
progress. This is going to be a bit tough -- not with ", " but with ": ".
Quite a few existing Perl serializations use Package::Module items
without quoting; by contrast only a few documents use unquoted values
like 3,493.  I think there is hope, however, since these existing
serializations mostly use the indented, aka "block" style.

I think we can bring consistency by forbidding the comma and colon in
unquoted values found in the in-line style; but still admitting them in
the more verbose indented style.  This should get us JSON compatibility
without huge amount of existing document breakage.  However, it will (as
such things typically do) complicate the productions even more.  Fun.

On the very positive side, Why has gracefully hinted that he'd make any
necessary changes in his implementation, Syck.  Then it is a matter of
getting the other implementations on-board.

Best,

Clark

-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click
(Continue reading)

Burt Harris | 3 Dec 2005 03:34
Picon
Favicon

Missing commas in example 4.90

I’m trying to use Example 4.90 to debug something, but I think it may be missing some commas.

 

Here’s what I transcribed:

 

{

 ?  : value # Empty Key

 ? explicit

  key: value,

 simple key  : value

  [ collection, simple, key ] : value

}

 

Would I be right in thinking that the first and third key-value pair should have comma termination?

 

{

 ?  : value, # Empty Key

 ? explicit

  key: value,

 simple key  : value,

  [ collection, simple, key ] : value

}

 

 

Burt Harris | 3 Dec 2005 04:52
Picon
Favicon

Example 4.86?

Example 4.86 in the Yaml 1.1 draft:  

 

The input has a token “three”, but it does not appear in the output.

 

 

Brad Baxter | 3 Dec 2005 14:28
Picon

Re: Example 4.86?

Burt,

I believe that this has been reported, at least by me and I think by
others, too.  I suspect that tuits are likely in short supply.  See also

http://wiki.galib.uga.edu/bin/view/Main/YamlSpecsMods

for my own list of questions/observations regarding the specs.

Regards,

Brad

On 12/2/05, Burt Harris <Burt.Harris-0li6OtcxBFHby3iVrkZq2A@public.gmane.org> wrote:

Example 4.86 in the Yaml 1.1 draft:  

 

The input has a token "three", but it does not appear in the output.

 

 


Rich Marisa | 3 Dec 2005 21:44
Picon
Favicon

Perl / Ruby compatability

Hello,

I've been having a couple of issues transfering data via YAML between 
Perl and Ruby.

1.  YAML.pm produces null tags which yaml.rb doesn't like:

        title: ! >-
          Analysis of the Computer Security Act of 1987 and Its 
Consequences for the
          Freedom of Information Act

2: yaml.rb creates sequence-in-mapping-shortcuts which YAML.pm doesn't like.

Are there ways to turn off these features?
yaml.rb 1.9.2.10
YAML.pm 0.35 or 0.39

Thanks,
  Rich Marisa
  Cornell University

-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click
Burt Harris | 4 Dec 2005 22:49
Picon
Favicon

JSON == YAML 0.5 ?

While the similarities between JSON and YAML are quite tempting, I thinkYAML's treatment comma and colon
are extremely valuable for human-authored YAML. 
Real-world types that could be adversely impacted by manipulating the YAML spec to make JSON a strict
subset include:  

  - URLs 
  - IPV6 Addresses
  - Windows/DOS/DEC style file paths
  - Perl identifiers

Rather than modifying the current YAML spec, I'll suggest it would be best to treat JSON as a back-dated
version of YAML, perhaps YAML 0.5.   

Clearly a YAML/JSON dumper would need some state indicating if the output should be in JSON or YAML. 
Assuming JSON == YAML 0.5 allows a continuum of versions to be specified by the caller, including 0.5, 1.0
and 1.1.

A parser needs such state as well, but in almost all cases it can tell the version by reading the first few
tokens in the input stream.   By default, a parser could start out with _version_ = 0.0.   As soon as it hits any
of the YAML-only constructs (including a directive, document separator, or even a plain scalar starting
with an non-numeric character) it would set _version_ = 1.0 (or higher if token is a %YAML directive).  

Very few cases where an ambiguity would remain by a parser encountered a comma or colon while parsing a plain
scalar.   Perhaps a list of numbers    

   [ 1,2,3 ]

or some really strange cases like:

   { true:1 }

Following this logic, if a parser were to hit one of these cases while the _version_ == 0.0, it should set
_version_ = 0.5 and parse it as JSON.   Then if a YAML-only construct is encountered while _version_ == 0.5,
this should be treated as an error.

One other plain scalar that needs special treatment would be "null", which needs special treatment for
JSON, but for YAML.   A YAML application concerned with this error could pre-set _version_ = 1.0.   An
application wanting to accept only JSON could pre-set _version_ = 0.5. 

-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://ads.osdn.com/?ad_idv37&alloc_id865&op=click
Burt Harris | 5 Dec 2005 01:12
Picon
Favicon

Simplifying the specification with semantics

Chapter 4 of the YAML 1.1 draft dated 2004-12-28 seems to go to extremes to try to reduce YAML semantics to
productions in the grammar.   It occurs to me that the specification could be simplified substantially by
mixing grammar-based syntax with some semantics specified in tables and pseudo-code.  

Take for example section 4.1.6 Escape Sequences.  Wouldn't it make sense to use syntax rules simply to
determine the extent of a double-quoted string, and then specify escape-sequence substitution as
semantic processing with double-quotes?

Another example might be section 4.5.3 Block Scalar Styles.  A lot of rules seem to be devoted to specifying
folding and chomping in terms of syntax.   Instead, I think it would be simpler to deal with all block literal
bodies with a simpler set of rules, and then apply some semantic processing to the parsed value to achieve
(un)folding, chomping, and processing of an explicit indentation indicator?

I still haven't fully internalized the idea of the "Context: c" described in section 4.2.1.   Might this also
be dealt with more understandably outside the formal syntax?

-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://ads.osdn.com/?ad_idv37&alloc_id865&op=click
Burt Harris | 5 Dec 2005 04:20
Picon
Favicon

A proposal for extensible YAML directives

The YAML 1.1 working draft dated 2004-12-28 in section 4.3.1 Directives, says:

    "There is no way to define private directives.   This is intentional."

but then goes on to say:

    "A YAML processor should ignore unknown directives with an appropriate warning."

Previous experience suggests is a dangerous combination!   Long term, it is safer to forbid an extension
mechanism entirely, or to allow for it and define a mechanism for identifying which extensions
directives may be safely ignored.   My preference would be for the later.

Here we can borrow from tagged data formats, such as PNG http://www.libpng.org/pub/png/
<http://www.libpng.org/pub/png/> , which use an initial lower-case letters to indicate if a tagged
value can be safely ignored.   YAML should do the same, but since we aren't limited to PNG's 4-byte chunk
types, YAML directive names should be URIs!   A YAML parser could safely ignore (and/or pass to
higher-levels) any directive that started with a lower-case letter, for example:

    %YAML 1.1
    %title The "Lost" Numbers
    ---
    [ 4, 8, 15, 16, 23, 42 ]

where the %title directive could be treated as a metatadata, or simply a tagged comment.   On the other hand, a
YAML processor should treat as an error any unrecognized directive starting with an upper-case letter,
for example:

    %YAML 1.1
    %HTTP://WWW.CONTOSO.COM/ATSIGN_INCLUDE
    ---
    [ 4, 8, 15,  <at> last3numbers.yaml ]

where the private directive might (for example) indicate a proprietary YAML extension which gives some
specific meaning to the otherwise-reserved ' <at> ' indicator.   Since the first character of the directive
name ('H') is not lower-case, any YAML processor unsure of its meaning should refuse to process the document.

Treating directive names as URIs allows use of any of the available URI mechanisms to ensure uniqueness.  
Directives without a URI scheme delimiter (':') would be treated as relative URIs based on
"tag:yaml.org,2005:", where we would maintain a "directives repository" parallel to the existing
"types catalog".   Determination  if a directive is ignorable or not should be done prior to relative URI processing.

I would further propose directive names be reserved in the tag:yaml.org,2005: "directive repository":

  # The YAML 1.1 directives MUST be understood

  - YAML <int>.<int>     
  - TAG  <handle> <uri-prefix>   

  # Ignorable metadata directive to identify the 
  # software name and version which generated 
  # the YAML document, expressed as a URI.

  - generator <uri>

  # Ignorable metadata based on the Dublin Core, 
  # see http://dublincore.org/documents/2004/12/20/dces/

  - title <scalar>
  - creator <scalar>
  - subject <scalar>
  - description <scalar>
  - publisher <scalar>
  - contributor <scalar>
  - date <iso 8601>
  - type <uri>
  - format <mime-type> 
  - identifier <scalar>
  - source <scalar>
  - language <rfc-3066_iso639>
  - relation <scalar>
  - coverage <scalar>
  - rights <scalar>

There are at least a couple open issues in my mind:

  Folding of parameters:   
    It may be desirable to allow directive parameters to be 
    folded using indentation, rather than limited to a single 
    line, but this could cause compatibility issues.

  Context where directives may appear:
    I believe the current draft only allows directives at the 
    beginning of a document.   I could imagine cases where such an 
    extension mechanism could be valuable inside a YAML document.

I hope this proposal is clear and useful.  I look forward to your feedback...

-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://ads.osdn.com/?ad_idv37&alloc_id865&op=click
Burt Harris | 5 Dec 2005 19:48
Picon
Favicon

Apologies for the formatting...

Sorry about the formatting of the 3 messages I sent yesterday, I sent them from home and wasn’t aware that the web-mail client wasn’t compatible with the mailing list.   I can re-send anything that’s not clear…

 

Adrian Klaver | 12 Dec 2005 03:57
Picon

PyYAML

I have tried to access the pyyaml.org site but keep getting site not found 
errors. Is the site up or is there another URL for the pyyaml code?

Thank you,
--

-- 
Adrian Klaver	
aklaver@...

-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click

Gmane