Andrey Somov | 2 Nov 10:46 2009
Picon

[ANN] SnakeYAML-1.5: Improve usage of generic collections in JavaBeans

==========================
 Announcing SnakeYAML-1.5
==========================

A new release of SnakeYAML is now available:

    http://code.google.com/p/snakeyaml/

This release delivers better support for usage of generic collections
in JavaBeans,
as well as other minor changes and bug fixes. It is backwards
compatible with the 1.4 release.

Changes
========
 * Improve usage of generic collections
 * Performance improvements
 * Add possibility to define a custom Class Loader

The complete list of changes is here:
http://snakeyamlrepo.appspot.com/releases/1.5/site/changes-report.html

Resources
==========

SnakeYAML homepage: http://code.google.com/p/snakeyaml/
SnakeYAML documentation: http://code.google.com/p/snakeyaml/wiki/Documentation

JAR package: http://snakeyamlrepo.appspot.com/repository/org/yaml/snakeyaml/1.5/snakeyaml-1.5.jar
Reports: http://snakeyamlrepo.appspot.com/
(Continue reading)

William Spitzak | 20 Nov 22:11 2009

Possible bug with null document in libyaml

libyaml treats a file containing nothing but zero or more newlines as 
being completely empty and parses nothing out of it. However this 
appears to be inconsistent:

The text "---\nx\n..." and "x" produce the same result: a start, "x" 
scalar, and an end token.

But the text "---\n\n..." and "" produce different results. The first 
produces start, a "" scalar, and an end. The second produces nothing.

In addition writing a single "" scalar using libyaml without the 
document start/end produces a file containing only a newline and thus is 
not inverted by the parser.

This may seem trivial but I am using yaml internally to store serialized 
values of small pieces of data, and blank strings were breaking because 
of this. I had to modify libyaml to quote zero-length strings but I do 
not think that is the desired solution.

My recommendation is that a completely blank yaml file be parsed as a 
single "" scalar. This may seem dangerous, because this file will 
"vanish" if concatenated with other data, but concatenation is not 
supposed to work anyway if the surrounding "---" and "..." are missing.

Any opinions or have I missed anything?

Thanks
Bill Spitzak
Rhythm & Hues software department.

(Continue reading)

William Spitzak | 21 Nov 04:01 2009

Re: Possible bug with null document in libyaml

Ok, sounds reasonable.

In that case the bug is that libyaml output when told to write a single 
"" scalar, does not write the correct thing, instead writing text that 
looks the same as an empty document.

It is true that if you ask for document-start to be written it fixes it 
but there does not seem to be a requirement for that, and in fact I 
would prefer not to as the documents I am writing are so short that the 
extra 8 bytes are a significant fraction.

> It has always been the intent of YAML, that a Stream may contain zero or 
> more Documents. This implies that there needs to be a way to serialize 
> zero objects.
> 
> BTW, a stream containing just comments is another example of something 
> that will parse as having zero documents.
> 
> It may not be written like this in the spec, but one way to think of it 
> is as follows:
> 
>    1. Every YAML Document in a YAML Stream begins with '---'.
>    2. If the first thing a parser sees in a stream, after skipping all
>       ignorable whitespace (including comments), is not '---', nor a
>       YAML directive, the parser should assume it saw "---\n".
>    3. If, after skipping all ignorable whitespace, the parser reaches
>       the End Of Stream, it should report an STREAM_END event, without
>       reporting any documents.
> 
> Being able to serialize zero documents is important. For a real use 
(Continue reading)

Ingy dot Net | 21 Nov 03:51 2009
Picon

Re: Possible bug with null document in libyaml



On Fri, Nov 20, 2009 at 1:11 PM, William Spitzak <spitzak <at> rhythm.com> wrote:
libyaml treats a file containing nothing but zero or more newlines as
being completely empty and parses nothing out of it. However this
appears to be inconsistent:

The text "---\nx\n..." and "x" produce the same result: a start, "x"
scalar, and an end token.

But the text "---\n\n..." and "" produce different results. The first
produces start, a "" scalar, and an end. The second produces nothing.

In addition writing a single "" scalar using libyaml without the
document start/end produces a file containing only a newline and thus is
not inverted by the parser.

This may seem trivial but I am using yaml internally to store serialized
values of small pieces of data, and blank strings were breaking because
of this. I had to modify libyaml to quote zero-length strings but I do
not think that is the desired solution.

My recommendation is that a completely blank yaml file be parsed as a
single "" scalar. This may seem dangerous, because this file will
"vanish" if concatenated with other data, but concatenation is not
supposed to work anyway if the surrounding "---" and "..." are missing.

Any opinions or have I missed anything?

It has always been the intent of YAML, that a Stream may contain zero or more Documents. This implies that there needs to be a way to serialize zero objects.

BTW, a stream containing just comments is another example of something that will parse as having zero documents.

It may not be written like this in the spec, but one way to think of it is as follows:
  1. Every YAML Document in a YAML Stream begins with '---'.
  2. If the first thing a parser sees in a stream, after skipping all ignorable whitespace (including comments), is not '---', nor a YAML directive, the parser should assume it saw "---\n".
  3. If, after skipping all ignorable whitespace, the parser reaches the End Of Stream, it should report an STREAM_END event, without reporting any documents.
Being able to serialize zero documents is important. For a real use case, imagine a service listening for YAML documents over a web socket. If the socket is closed after receiving no data, the service should *not* parse that as one document containing an empty string.

After testing all these cases against libyaml, IMHO it gets things right every time.

Cheers, Ingy
 
Thanks
Bill Spitzak
Rhythm & Hues software department.

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now.  http://p.sf.net/sfu/bobj-july
_______________________________________________
Yaml-core mailing list
Yaml-core-5NWGOfrQmnetEtDZOKyKiw@public.gmane.orgrge.net
https://lists.sourceforge.net/lists/listinfo/yaml-core

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day 
trial. Simplify your report design, integration and deployment - and focus on 
what you do best, core application coding. Discover what's new with
Crystal Reports now.  http://p.sf.net/sfu/bobj-july
_______________________________________________
Yaml-core mailing list
Yaml-core@...
https://lists.sourceforge.net/lists/listinfo/yaml-core
Osamu TAKEUCHI | 21 Nov 05:22 2009
Picon

Re: Possible bug with null document in libyaml

Bill and Ingy,

Thanks Ingy. I finally understood the importance of 
providing a way to serialize zero objects by reading 
your post. Before that, I was thinking as the same as Bill.

Then, according to the spec, a YAML serializer can encode 
an empty string (in YAML 1.1 document) or a null object (in 
YAML 1.2 document) as an empty node in all the cases except 
for the one where the whole YAML stream becomes empty.
I think an explicit warning should be given in the spec 
to clarify this point with some explanation of the
importance of providing a way to serialize zero objects.
It will be very much helpful.

In addition, I point out that any empty string should not
be encoded into empty nodes in YAML 1.1 document in 
any sense because it will not be compatible with YAML 1.2. 
In YAML 1.2 spec, an empty node represents a null object.
So, anyways, all the YAML 1.1 serializers should be patched 
to encode all the empty strings in quoted styles "" or ''.

BTW, I do not think a YAML 1.1 library will accept "x" as a 
valid YAML stream if the library strictly implements the 
specification. In YAML 1.1, a plain scalar node must be 
followed by s-l-comments that contains at least one line 
break. So, "x\n" is a valid document but "x" is not.

This is the same problem pointed out for YAML 1.2 spec, first by 
Joshua Choi,

http://sourceforge.net/mailarchive/forum.php?thread_name=200e06280904252219i1da33755h200bdac14058ba36%40mail.gmail.com&forum_name=yaml-core

and then by me.

http://sourceforge.net/mailarchive/forum.php?thread_name=1254356424.5043.224.camel%40nero&forum_name=yaml-core

Probably, most of the real implementations ignore the spec 
to be compatible with hand-written documents possibly without 
a line break at the end. But, at least, a YAML serializer 
should not output a YAML document without a line break at the 
end.

Best,
Osamu Takeuchi

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day 
trial. Simplify your report design, integration and deployment - and focus on 
what you do best, core application coding. Discover what's new with
Crystal Reports now.  http://p.sf.net/sfu/bobj-july
Ingy dot Net | 21 Nov 07:20 2009
Picon

Re: Possible bug with null document in libyaml



On Fri, Nov 20, 2009 at 7:01 PM, William Spitzak <spitzak <at> rhythm.com> wrote:
Ok, sounds reasonable.

In that case the bug is that libyaml output when told to write a single "" scalar, does not write the correct thing, instead writing text that looks the same as an empty document.

That sounds like a bug. A YAML processor should not output invalid YAML (or in this case, valid YAML that means the wrong thing).
 
It is true that if you ask for document-start to be written it fixes it but there does not seem to be a requirement for that, and in fact I would prefer not to as the documents I am writing are so short that the extra 8 bytes are a significant fraction.

By 8, you mean 4 ("---\n") + 4 ("...\n")?

If you have more than one document in a stream, then you need "---\n" anyway for all but the first document. You really don't need "...\n" at all, unless timing is an issue. If you are writing single document streams to disk, then certainly you are consuming more space for the inode and file name etc.

I don't know what your exact use case is though. The shortest YAML serialization for an empty string is 2 single quotes, and libyaml should support that. (Actually it probably adds a newline as well which is a sane thing for a general purpose library.)

--Ingy

It has always been the intent of YAML, that a Stream may contain zero or more Documents. This implies that there needs to be a way to serialize zero objects.

BTW, a stream containing just comments is another example of something that will parse as having zero documents.

It may not be written like this in the spec, but one way to think of it is as follows:

  1. Every YAML Document in a YAML Stream begins with '---'.
  2. If the first thing a parser sees in a stream, after skipping all

     ignorable whitespace (including comments), is not '---', nor a
     YAML directive, the parser should assume it saw "---\n".
  3. If, after skipping all ignorable whitespace, the parser reaches

     the End Of Stream, it should report an STREAM_END event, without
     reporting any documents.

Being able to serialize zero documents is important. For a real use case, imagine a service listening for YAML documents over a web socket. If the socket is closed after receiving no data, the service should *not* parse that as one document containing an empty string.

After testing all these cases against libyaml, IMHO it gets things right every time.

Cheers, Ingy
 
   Thanks
   Bill Spitzak
   Rhythm & Hues software department.

   ------------------------------------------------------------------------------
   Let Crystal Reports handle the reporting - Free Crystal Reports 2008
   30-Day
   trial. Simplify your report design, integration and deployment - and
   focus on
   what you do best, core application coding. Discover what's new with
   Crystal Reports now.  http://p.sf.net/sfu/bobj-july
   _______________________________________________
   Yaml-core mailing list
   Yaml-core-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org <mailto:Yaml-core-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org>

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day 
trial. Simplify your report design, integration and deployment - and focus on 
what you do best, core application coding. Discover what's new with
Crystal Reports now.  http://p.sf.net/sfu/bobj-july
_______________________________________________
Yaml-core mailing list
Yaml-core@...
https://lists.sourceforge.net/lists/listinfo/yaml-core
Ingy dot Net | 21 Nov 07:38 2009
Picon

Re: Possible bug with null document in libyaml

On Fri, Nov 20, 2009 at 8:22 PM, Osamu TAKEUCHI <osamu-X7JyfprlRQA@public.gmane.org> wrote:

Bill and Ingy,

Thanks Ingy. I finally understood the importance of providing a way to serialize zero objects by reading your post. Before that, I was thinking as the same as Bill.

Then, according to the spec, a YAML serializer can encode an empty string (in YAML 1.1 document) or a null object (in YAML 1.2 document) as an empty node in all the cases except for the one where the whole YAML stream becomes empty.
I think an explicit warning should be given in the spec to clarify this point with some explanation of the
importance of providing a way to serialize zero objects.
It will be very much helpful.

In addition, I point out that any empty string should not
be encoded into empty nodes in YAML 1.1 document in any sense because it will not be compatible with YAML 1.2. In YAML 1.2 spec, an empty node represents a null object.

To put a finer point on it, an empty node parses as a plain, (unquoted) empty string, scalar event. YAML loaders are encouraged (but not required) to construct a null object from this event.

To be honest, I was not aware that this changed between 1.1 and 1.2. Can you point me to the appropriate part of the spec that says this?

--Ingy
 
So, anyways, all the YAML 1.1 serializers should be patched to encode all the empty strings in quoted styles "" or ''.


BTW, I do not think a YAML 1.1 library will accept "x" as a valid YAML stream if the library strictly implements the specification. In YAML 1.1, a plain scalar node must be followed by s-l-comments that contains at least one line break. So, "x\n" is a valid document but "x" is not.

This is the same problem pointed out for YAML 1.2 spec, first by Joshua Choi,

http://sourceforge.net/mailarchive/forum.php?thread_name=200e06280904252219i1da33755h200bdac14058ba36%40mail.gmail.com&forum_name=yaml-core

and then by me.

http://sourceforge.net/mailarchive/forum.php?thread_name=1254356424.5043.224.camel%40nero&forum_name=yaml-core

Probably, most of the real implementations ignore the spec to be compatible with hand-written documents possibly without a line break at the end. But, at least, a YAML serializer should not output a YAML document without a line break at the end.

Best,
Osamu Takeuchi


------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day 
trial. Simplify your report design, integration and deployment - and focus on 
what you do best, core application coding. Discover what's new with
Crystal Reports now.  http://p.sf.net/sfu/bobj-july
_______________________________________________
Yaml-core mailing list
Yaml-core@...
https://lists.sourceforge.net/lists/listinfo/yaml-core
Osamu TAKEUCHI | 21 Nov 10:35 2009
Picon

Re: Possible bug with null document in libyaml

Ingy,

>     In addition, I point out that any empty string should not
>     be encoded into empty nodes in YAML 1.1 document in any sense
>     because it will not be compatible with YAML 1.2. In YAML 1.2 spec,
>     an empty node represents a null object.
> 
> 
> To put a finer point on it, an empty node parses as a plain, (unquoted) 
> empty string, scalar event. YAML loaders are encouraged (but not 
> required) to construct a null object from this event.
> 
> To be honest, I was not aware that this changed between 1.1 and 1.2. Can 
> you point me to the appropriate part of the spec that says this?

According to the spec, this change was done for JSON compatibility.
A YAML 1.2 processor seems to be required to construct a null object 
from an empty node.

In this point, YAML 1.2 is completely incompatible from YAML 1.1.
Please compare example 7.3 in YAML 1.2 spec with example 8.13 in 
YAML 1.1 spec.

*** YAML 1.2 spec

Status of this Document
http://www.yaml.org/spec/1.2/spec.html

 >>> this is a minor revision.
 >>> ...
 >>>
 >>> We have removed unique implicit typing rules and have updated 
 >>> these rules to align them with JSON's productions. 

7.2. Empty Nodes
http://www.yaml.org/spec/1.2/spec.html#id2786563

 >>> Nodes with empty content are interpreted as if they were plain 
 >>> scalars with an empty value. Such nodes are commonly resolved
 >>> to a "null" value. 

 Example 7.3. Completely Empty Flow Nodes

 >>> {
 >>>   ? foo :,
 >>>   : bar,
 >>> }

 >>> %YAML 1.2
 >>> ---
 >>> !!map {
 >>>   ? !!str "foo" : !!null "",
 >>>   ? !!null ""   : !!str "bar",
 >>> }

10.2.1.1. Null
http://www.yaml.org/spec/1.2/spec.html#id2803362

 >>> Note that a null is different from an empty string.

*** YAML 1.1 spec

8.5.1. Flow Nodes
http://yaml.org/spec/1.1/#id902924

 >>> A node with empty content is considered to be an empty plain scalar. 

 Example 8.13. Completely Empty Flow Nodes

 >>> {
 >>>   ? foo :,
 >>>   ? : bar,
 >>> }

 >>> %YAML 1.1
 >>> ---
 >>> !!map {
 >>>   ? !!str "foo"
 >>>   : !!str "",
 >>>   ? !!str "",
 >>>   : !!str "bar",
 >>> }

Both the spec says an empty node is considered to be an empty plain scalar.
This seems to be the excuse how the spec describe YAML 1.2 is almost always
compatible to YAML 1.1. However, an empty plain scalar is considered as an 
empty string in YAML 1.1 but as a null object in YAML 1.2.

Actually, in the type repository for _YAML1.1_ also specifies an empty
plain scalar value to be resolved as !!null type. It seems incompatible
to the YAML 1.1 spec. I imagine this is due to some historical reason.

*** Null Language-Independent Type for YAML Version 1.1 

http://yaml.org/type/null.html

 >>> Shorthand:
 >>> 
 >>>     !!null
 >>> 
 >>>  Regexp:
 >>> 
 >>>      ~ # (canonical)
 >>>     |null|Null|NULL # (English)
 >>>     | # (Empty)

This point was more clearly stated in this mailing list.
http://sourceforge.net/mailarchive/forum.php?thread_name=1248226963.15646.1326179237%40webmail.messagingengine.com&forum_name=yaml-core

I think it's better to point out this incompatibility more clearly in the
YAML 1.2 spec as well.

Best,
Osamu Takeuchi

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day 
trial. Simplify your report design, integration and deployment - and focus on 
what you do best, core application coding. Discover what's new with
Crystal Reports now.  http://p.sf.net/sfu/bobj-july
Kirill Simonov | 21 Nov 18:24 2009
Picon

Re: Possible bug with null document in libyaml

Hi Osamu,

> Then, according to the spec, a YAML serializer can encode 
> an empty string (in YAML 1.1 document) or a null object (in 
> YAML 1.2 document) as an empty node in all the cases except 
> for the one where the whole YAML stream becomes empty.
> I think an explicit warning should be given in the spec 
> to clarify this point with some explanation of the
> importance of providing a way to serialize zero objects.
> It will be very much helpful.

I believe all existing yaml processors interpret an empty plain scalar 
as the null value by default.

> 
> In addition, I point out that any empty string should not
> be encoded into empty nodes in YAML 1.1 document in 
> any sense because it will not be compatible with YAML 1.2. 
> In YAML 1.2 spec, an empty node represents a null object.
> So, anyways, all the YAML 1.1 serializers should be patched 
> to encode all the empty strings in quoted styles "" or ''.
> 
> 
> BTW, I do not think a YAML 1.1 library will accept "x" as a 
> valid YAML stream if the library strictly implements the 
> specification. In YAML 1.1, a plain scalar node must be 
> followed by s-l-comments that contains at least one line 
> break. So, "x\n" is a valid document but "x" is not.

I don't think any pre-YAML 1.2 processors uses the grammar described in 
the spec as the basis for the parser.

Thanks,
Kirill

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day 
trial. Simplify your report design, integration and deployment - and focus on 
what you do best, core application coding. Discover what's new with
Crystal Reports now.  http://p.sf.net/sfu/bobj-july
Andrey Somov | 21 Nov 21:52 2009
Picon

permission to use 'org.yaml' as the group name for SnakeYAML

Hi all,

I am Andrey Somov, developer of SnakeYAML. SnakeYAML gets more and
more attention and developers would like to put it to the central
Maven repository.
More information about Maven is here: http://maven.apache.org/

All the artifacts are defined in the hierarchy of groups (practically
folders on a file system). General agreement is that the group name is
the same as the domain in URL.
This is my request:
https://issues.sonatype.org/browse/OSSRH-128

I have chosen the group 'org.yaml' for SnakeYAML a few versions ago and I
am not in favour to change it since it would cause some changes for
developers.

Dear 'org.yaml' owners, may I ask you to use 'org.yaml' as the group
name for SnakeYAML?
(It affects only Maven users in Java.)

Then I can ask Maven admins to proceed with the migration to central
Maven repository.

-
Andrey

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day 
trial. Simplify your report design, integration and deployment - and focus on 
what you do best, core application coding. Discover what's new with
Crystal Reports now.  http://p.sf.net/sfu/bobj-july

Gmane