RANDRIAMASY, SABINE (SABINE | 28 Oct 18:03 2014

Updates on Multi-Cost ALTO

Hi all,

 

A new version of the Multi-Cost ALTO draft by Wendy Roome and myself  is available at

http://tools.ietf.org/id/draft-randriamasy-alto-multi-cost-09.txt

The revision relates to discussions in Toronto and before on the possibility to request Cost Values on a set C1 of Cost Types with  AND/OR constraints on a different set C2 of cost types.  

The draft  proposes a light way to do it:

-it introduces a member “testable cost-types” in the IRD capabilities of the Filtered Multi-Cost Map and Endpoint Multi-Cost Map

- it adapts the constraints syntax rules accordingly.

The draft provides related transaction examples and refers to previous work that dealt with requests on cost-types with AND-constraints on different cost-types, see draft-lee-alto-app-net-info-exchange.   

Feedback on this proposal is more than welcome,

Thanks,

 

Sabine and Wendy

 

 

<div>
<div class="WordSection1">
<p class="MsoNormal">Hi all,<p></p></p>
<p class="MsoNormal"><p>&nbsp;</p></p>
<p class="MsoNormal">A new version of the Multi-Cost ALTO draft by Wendy Roome and myself &nbsp;is available at
<p></p></p>
<p class="MsoNormal"><a href="http://tools.ietf.org/id/draft-randriamasy-alto-multi-cost-09.txt">http://tools.ietf.org/id/draft-randriamasy-alto-multi-cost-09.txt</a>
<p></p></p>
<p class="MsoNormal">The revision relates to discussions in Toronto and before on the possibility to request Cost Values on a set C1 of Cost Types with &nbsp;AND/OR constraints on a different set C2 of cost types. &nbsp;<p></p></p>
<p class="MsoNormal">The draft &nbsp;proposes a light way to do it: <p></p></p>
<p class="MsoNormal">-it introduces a member &ldquo;testable cost-types&rdquo; in the IRD capabilities of the Filtered Multi-Cost Map and Endpoint Multi-Cost Map<p></p></p>
<p class="MsoNormal">- it adapts the constraints syntax rules accordingly. <p></p></p>
<p class="MsoNormal">The draft provides related transaction examples and refers to previous work that dealt with requests on cost-types with AND-constraints on different cost-types, see
<a href="http://tools.ietf.org/html/draft-lee-alto-app-net-info-exchange">draft-lee-alto-app-net-info-exchange</a>. &nbsp;&nbsp;<p></p></p>
<table class="MsoNormalTable" border="0" cellpadding="0"><tr>
<td></td>
<td></td>
</tr></table>
<p class="MsoNormal">Feedback on this proposal is more than welcome,<p></p></p>
<p class="MsoNormal">Thanks,<p></p></p>
<p class="MsoNormal"><p>&nbsp;</p></p>
<p class="MsoNormal">Sabine and Wendy<p></p></p>
<p class="MsoNormal"><p>&nbsp;</p></p>
<p class="MsoNormal"><p>&nbsp;</p></p>
</div>
</div>
Y. Richard Yang | 27 Oct 17:38 2014
Picon

Fwd: New Version Notification for draft-shi-alto-yang-json-00.txt

Hi all,

Related with the previous email. This document looks at not only ALTO but other protocols that use JSON objects in message body. We look at if one may use YANG to specify it (note, only the message body). It turns out ALTO JSON objects cannot be generated from a YANG specification. The main issue, as already discussed in some previous email messages appeared in both ALTO/netmod, is key-value store. 

Any comments are appreciated, again!

Richard

---------- Forwarded message ----------
From: <internet-drafts <at> ietf.org>
Date: Mon, Oct 27, 2014 at 11:31 AM
Subject: New Version Notification for draft-shi-alto-yang-json-00.txt
To: "Y. Richard Yang" <yang.r.yang <at> gmail.com>, Xiao Shi <xiao.shi <at> yale.edu>



A new version of I-D, draft-shi-alto-yang-json-00.txt
has been successfully submitted by Xiao Shi and posted to the
IETF repository.

Name:           draft-shi-alto-yang-json
Revision:       00
Title:          Modeling JSON Messages Using YANG
Document date:  2014-10-27
Group:          Individual Submission
Pages:          23
URL:            http://www.ietf.org/internet-drafts/draft-shi-alto-yang-json-00.txt
Status:         https://datatracker.ietf.org/doc/draft-shi-alto-yang-json/
Htmlized:       http://tools.ietf.org/html/draft-shi-alto-yang-json-00


Abstract:
   JavaScript Object Notation (JSON) has been a popular choice as the
   message encoding for many network protocols.  Meanwhile, there are
   broad interests in the networking community to use the YANG data
   modeling language [RFC6020] to define data store and protocol
   messages, so that one can use YANG related tools such as the
   OpenDayLight Controller.  Although YANG itself is XML based, there
   have been efforts to model JSON content using YANG
   [draft-ietf-netmod-yang-json-01]

   This document explores the conditions under which the messages of a
   JSON based protocol can have a syntactically equivalent and hence
   interoperable YANG model.  In particular, this document shows that
   any JSON protocol message with stand-alone non-object JSON values,
   certain JSON arrays of elements of mixed types, or non-keyword keys
   in key-value pairs cannot have a syntactically equivalent YANG model.
   It also applies these conditions to the ALTO and CDNi protocol
   messages as examples.




Please note that it may take a couple of minutes from the time of submission
until the htmlized version and diff are available at tools.ietf.org.

The IETF Secretariat




--
Richard
<div><div dir="ltr">
<span>Hi all,</span><div><br></div>
<div>Related with the previous email. This document looks at not only ALTO but other protocols that use JSON objects in message body. We look at if one may use YANG to specify it (note, only the message body). It turns out ALTO JSON objects cannot be generated from a YANG specification. The main issue, as already discussed in some previous email messages appeared in both ALTO/netmod, is key-value store.&nbsp;</div>
<div><br></div>
<div>Any comments are appreciated, again!</div>
<div><br></div>
<div>Richard</div>
<div><br></div>
<div class="gmail_quote">---------- Forwarded message ----------<br>From:  <span dir="ltr">&lt;<a href="mailto:internet-drafts <at> ietf.org">internet-drafts <at> ietf.org</a>&gt;</span><br>Date: Mon, Oct 27, 2014 at 11:31 AM<br>Subject: New Version Notification for draft-shi-alto-yang-json-00.txt<br>To: "Y. Richard Yang" &lt;<a href="mailto:yang.r.yang <at> gmail.com">yang.r.yang <at> gmail.com</a>&gt;, Xiao Shi &lt;<a href="mailto:xiao.shi <at> yale.edu">xiao.shi <at> yale.edu</a>&gt;<br><br><br><br>
A new version of I-D, draft-shi-alto-yang-json-00.txt<br>
has been successfully submitted by Xiao Shi and posted to the<br>
IETF repository.<br><br>
Name:&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;draft-shi-alto-yang-json<br>
Revision:&nbsp; &nbsp; &nbsp; &nbsp;00<br>
Title:&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; Modeling JSON Messages Using YANG<br>
Document date:&nbsp; 2014-10-27<br>
Group:&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; Individual Submission<br>
Pages:&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 23<br>
URL:&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <a href="http://www.ietf.org/internet-drafts/draft-shi-alto-yang-json-00.txt" target="_blank">http://www.ietf.org/internet-drafts/draft-shi-alto-yang-json-00.txt</a><br>
Status:&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;<a href="https://datatracker.ietf.org/doc/draft-shi-alto-yang-json/" target="_blank">https://datatracker.ietf.org/doc/draft-shi-alto-yang-json/</a><br>
Htmlized:&nbsp; &nbsp; &nbsp; &nbsp;<a href="http://tools.ietf.org/html/draft-shi-alto-yang-json-00" target="_blank">http://tools.ietf.org/html/draft-shi-alto-yang-json-00</a><br><br><br>
Abstract:<br>
&nbsp; &nbsp;JavaScript Object Notation (JSON) has been a popular choice as the<br>
&nbsp; &nbsp;message encoding for many network protocols.&nbsp; Meanwhile, there are<br>
&nbsp; &nbsp;broad interests in the networking community to use the YANG data<br>
&nbsp; &nbsp;modeling language [RFC6020] to define data store and protocol<br>
&nbsp; &nbsp;messages, so that one can use YANG related tools such as the<br>
&nbsp; &nbsp;OpenDayLight Controller.&nbsp; Although YANG itself is XML based, there<br>
&nbsp; &nbsp;have been efforts to model JSON content using YANG<br>
&nbsp; &nbsp;[draft-ietf-netmod-yang-json-01]<br><br>
&nbsp; &nbsp;This document explores the conditions under which the messages of a<br>
&nbsp; &nbsp;JSON based protocol can have a syntactically equivalent and hence<br>
&nbsp; &nbsp;interoperable YANG model.&nbsp; In particular, this document shows that<br>
&nbsp; &nbsp;any JSON protocol message with stand-alone non-object JSON values,<br>
&nbsp; &nbsp;certain JSON arrays of elements of mixed types, or non-keyword keys<br>
&nbsp; &nbsp;in key-value pairs cannot have a syntactically equivalent YANG model.<br>
&nbsp; &nbsp;It also applies these conditions to the ALTO and CDNi protocol<br>
&nbsp; &nbsp;messages as examples.<br><br><br><br><br>
Please note that it may take a couple of minutes from the time of submission<br>
until the htmlized version and diff are available at <a href="http://tools.ietf.org" target="_blank">tools.ietf.org</a>.<br><br>
The IETF Secretariat<br><br>
</div>
<br><br clear="all"><div><br></div>-- <br>Richard
</div></div>
Y. Richard Yang | 27 Oct 17:32 2014
Picon

Fwd: New Version Notification for draft-shi-alto-yang-model-00.txt

Hi all,

In a setting of providing ALTO into ODL, we started to specify ALTO services defined in RFC7285 using YANG. It turns to be a quite interesting experience. Below is an draft specifying our approaches and first two models. The third model, in the context of considering NETCONF/RESTCONF, is slightly more complex and we will post in a next revision.

Any comments are appreciated!

Richard


---------- Forwarded message ----------
From: <internet-drafts <at> ietf.org>
Date: Mon, Oct 27, 2014 at 12:11 PM
Subject: New Version Notification for draft-shi-alto-yang-model-00.txt
To: Michael Scharf <michael.scharf <at> alcatel-lucent.com>, "Y. Richard Yang" <yang.r.yang <at> gmail.com>, Xiao Shi <xiao.shi <at> yale.edu>



A new version of I-D, draft-shi-alto-yang-model-00.txt
has been successfully submitted by Xiao Shi and posted to the
IETF repository.

Name:           draft-shi-alto-yang-model
Revision:       00
Title:          A YANG Data Model for Base ALTO Data and Services
Document date:  2014-10-27
Group:          Individual Submission
Pages:          66
URL:            http://www.ietf.org/internet-drafts/draft-shi-alto-yang-model-00.txt
Status:         https://datatracker.ietf.org/doc/draft-shi-alto-yang-model/
Htmlized:       http://tools.ietf.org/html/draft-shi-alto-yang-model-00


Abstract:
   The Application-Layer Traffic Optimization (ALTO) protocol [RFC7285]
   defines a set of network information services, including the network-
   map service, the cost-map service, the filtered map services, the
   endpoint property service, and the endpoint cost service.  A meta
   service, called the information resource directory (IRD) service,
   allows an ALTO server to provide ALTO clients with meta information
   (e.g., the access URI) about each resource and service it provides.
   [RFC7285] uses a RESTful design and encodes client request parameters
   and server responses using JSON.  One may consider that most of these
   services are based on data maintained at an ALTO server.  Hence, in
   this document, we explore how one may use the data modeling language
   YANG [RFC6020] to specify the services defined in [RFC7285].  We
   first define two YANG models for RPC specification and data instance
   description of of ALTO services.  We then discuss the "standard
   operations" defined in NETCONF/RESTCONF to evaluate potential
   integration.




Please note that it may take a couple of minutes from the time of submission
until the htmlized version and diff are available at tools.ietf.org.

The IETF Secretariat




--
Richard
<div><div dir="ltr">Hi all,<div><br></div>
<div>In a setting of providing ALTO into ODL, we started to specify ALTO services defined in RFC7285 using YANG. It turns to be a quite interesting experience. Below is an draft specifying our approaches and first two models. The third model, in the context of considering NETCONF/RESTCONF, is slightly more complex and we will post in a next revision.</div>
<div><br></div>
<div>Any comments are appreciated!</div>
<div><br></div>
<div>Richard</div>
<div><br></div>
<div>
<br><div class="gmail_quote">---------- Forwarded message ----------<br>From:  <span dir="ltr">&lt;<a href="mailto:internet-drafts <at> ietf.org">internet-drafts <at> ietf.org</a>&gt;</span><br>Date: Mon, Oct 27, 2014 at 12:11 PM<br>Subject: New Version Notification for draft-shi-alto-yang-model-00.txt<br>To: Michael Scharf &lt;<a href="mailto:michael.scharf <at> alcatel-lucent.com">michael.scharf <at> alcatel-lucent.com</a>&gt;, "Y. Richard Yang" &lt;<a href="mailto:yang.r.yang <at> gmail.com">yang.r.yang <at> gmail.com</a>&gt;, Xiao Shi &lt;<a href="mailto:xiao.shi <at> yale.edu">xiao.shi <at> yale.edu</a>&gt;<br><br><br><br>
A new version of I-D, draft-shi-alto-yang-model-00.txt<br>
has been successfully submitted by Xiao Shi and posted to the<br>
IETF repository.<br><br>
Name:&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;draft-shi-alto-yang-model<br>
Revision:&nbsp; &nbsp; &nbsp; &nbsp;00<br>
Title:&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; A YANG Data Model for Base ALTO Data and Services<br>
Document date:&nbsp; 2014-10-27<br>
Group:&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; Individual Submission<br>
Pages:&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 66<br>
URL:&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <a href="http://www.ietf.org/internet-drafts/draft-shi-alto-yang-model-00.txt" target="_blank">http://www.ietf.org/internet-drafts/draft-shi-alto-yang-model-00.txt</a><br>
Status:&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;<a href="https://datatracker.ietf.org/doc/draft-shi-alto-yang-model/" target="_blank">https://datatracker.ietf.org/doc/draft-shi-alto-yang-model/</a><br>
Htmlized:&nbsp; &nbsp; &nbsp; &nbsp;<a href="http://tools.ietf.org/html/draft-shi-alto-yang-model-00" target="_blank">http://tools.ietf.org/html/draft-shi-alto-yang-model-00</a><br><br><br>
Abstract:<br>
&nbsp; &nbsp;The Application-Layer Traffic Optimization (ALTO) protocol [RFC7285]<br>
&nbsp; &nbsp;defines a set of network information services, including the network-<br>
&nbsp; &nbsp;map service, the cost-map service, the filtered map services, the<br>
&nbsp; &nbsp;endpoint property service, and the endpoint cost service.&nbsp; A meta<br>
&nbsp; &nbsp;service, called the information resource directory (IRD) service,<br>
&nbsp; &nbsp;allows an ALTO server to provide ALTO clients with meta information<br>
&nbsp; &nbsp;(e.g., the access URI) about each resource and service it provides.<br>
&nbsp; &nbsp;[RFC7285] uses a RESTful design and encodes client request parameters<br>
&nbsp; &nbsp;and server responses using JSON.&nbsp; One may consider that most of these<br>
&nbsp; &nbsp;services are based on data maintained at an ALTO server.&nbsp; Hence, in<br>
&nbsp; &nbsp;this document, we explore how one may use the data modeling language<br>
&nbsp; &nbsp;YANG [RFC6020] to specify the services defined in [RFC7285].&nbsp; We<br>
&nbsp; &nbsp;first define two YANG models for RPC specification and data instance<br>
&nbsp; &nbsp;description of of ALTO services.&nbsp; We then discuss the "standard<br>
&nbsp; &nbsp;operations" defined in NETCONF/RESTCONF to evaluate potential<br>
&nbsp; &nbsp;integration.<br><br><br><br><br>
Please note that it may take a couple of minutes from the time of submission<br>
until the htmlized version and diff are available at <a href="http://tools.ietf.org" target="_blank">tools.ietf.org</a>.<br><br>
The IETF Secretariat<br><br>
</div>
<br><br clear="all"><div><br></div>-- <br>Richard
</div>
</div></div>
Vijay K. Gurbani | 28 Oct 14:47 2014

Draft agenda posted

Folks: A draft agenda for ALTO meeting in HI has been posted.

Please see https://datatracker.ietf.org/meeting/91/agenda/alto/

Let the chairs know of any changes.

Meeting materials (slides) are due on November 9, 2014.  Please
convert your slides to PDF and email them to the chairs.

Thanks,

- vijay
--

-- 
Vijay K. Gurbani, Bell Laboratories, Alcatel-Lucent
1960 Lucent Lane, Rm. 9C-533, Naperville, Illinois 60563 (USA)
Email: vkg <at> {bell-labs.com,acm.org} / vijay.gurbani <at> alcatel-lucent.com
Web: http://ect.bell-labs.com/who/vkg/  | Calendar: http://goo.gl/x3Ogq

Wendy Roome | 27 Oct 19:44 2014

Revised Incremental Update draft

Richard Yang, Xiao Shi & I submitted a new proposal for incremental
updates to ALTO resources. See
http://datatracker.ietf.org/doc/draft-roome-alto-incr-update-sse/

This version uses Server-Sent Events (SSE) and JSON Merge Patch, and is a
substantial change from my previous proposal.

We would like to present this version at IETF ¹91. I expect there will be
a lot of discussion, so we might need 25 minutes or more.

	- Wendy Roome

邓灵莉/Lingli Deng | 27 Oct 05:19 2014

FW: New Version Notification for draft-deng-alto-p2p-ext-04.txt

Hi all,

We just uploaded a new revision of the draft on extended endpoint properties for ALTO.
Your review and comments are welcome.

Lingli

> -----Original Message-----
> From: internet-drafts <at> ietf.org [mailto:internet-drafts <at> ietf.org]
> Sent: Monday, October 27, 2014 12:05 PM
> To: Haibin Song; Qin Wu; Wenson Wu; Richard Yang; Haibin Song; Deng Lingli;
> Sebastian Kiesel; Yang Yang; Lingli Deng; Sebastian Kiesel
> Subject: New Version Notification for draft-deng-alto-p2p-ext-04.txt
> 
> 
> A new version of I-D, draft-deng-alto-p2p-ext-04.txt
> has been successfully submitted by Lingli Deng and posted to the
> IETF repository.
> 
> Name:		draft-deng-alto-p2p-ext
> Revision:	04
> Title:		Extended Endpoint Properties for Application Layer Traffic
> Optimization
> Document date:	2014-10-26
> Group:		Individual Submission
> Pages:		16
> URL:
> http://www.ietf.org/internet-drafts/draft-deng-alto-p2p-ext-04.txt
> Status:         https://datatracker.ietf.org/doc/draft-deng-alto-p2p-ext/
> Htmlized:       http://tools.ietf.org/html/draft-deng-alto-p2p-ext-04
> Diff:           http://www.ietf.org/rfcdiff?url2=draft-deng-alto-p2p-ext-04
> 
> Abstract:
>    The purpose of the ALTO protocol is to provide better-than-random
>    peer selection for P2P networks. The base ALTO protocol focuses,
>    however, only on providing network topological location information
>    (i.e., network maps and cost maps). However, the peer selection
>    method of an endpoint may also use other properties such as
>    geographic location. This document defines a framework and an
>    extended set of endpoint properties to extend the base ALTO protocol.
> 
> 
> 
> 
> Please note that it may take a couple of minutes from the time of submission
> until the htmlized version and diff are available at tools.ietf.org.
> 
> The IETF Secretariat

13xinwang | 24 Oct 16:42 2014
Picon

Fw: Incremental Update/SSE Key Design Issue 2: sending full resources at the beginning of stream

Hi Xiao,


My suggestion is that if we can adopt your window design into the most costly solution to make all version can be acceptable. We assume V0 is the first version and Vkn is the last version, and the server keeps V0->Vk, Vk->V2k,..., V(n-1)k->Vkn.

The problem here i think is the size of patch between Vi->Vj and the sum of Vi->V(i+1), V(i+1)->V(i+2),... V(j-1)->Vj. We set the first one is S1 and the second one is S2: 1. if each patch does not overlap with others, S1 = S2; 2. else, S1 < S2.

However, as versions change, the difference between Vi and Vj can be huge (the size of Vi->Vj is also huge correspondingly), which makes Vi->Vj is not suitable as a patch. So, i think if we can get the best value of k, we can make server not reject any request while maintaining high performance.

Is this right?

Thanks,
Xin 

From: xiao.shi <at> yale.edu
Date: Thu, 23 Oct 2014 06:37:12 -0400
To: alto <at> ietf.org
CC: w.roome <at> alcatel-lucent.com
Subject: [alto] Incremental Update/SSE Key Design Issue 2: sending full resources at the beginning of stream

Hi folks,

Another design question is whether we should include the full resource (e.g. full network map/cost map) when the SSE stream is first initiated.

The advantage of doing so is that the server does not need to keep any old version of the resources. The full map the server first sends to the client will always be updated, and the updates come when they become available.

It also has the following disadvantage:
(1) Modularity: SSE streams are for incremental updates, it carries a different functionality than full resource requests;
(2) Space efficiency: to reduce the payload for the SSE stream, it would be nice not to do that. It is conceivable that the client fetches the full resource somewhere else and then ask for incremental updates via HTTP SSE stream.


If we do not send the full resource via SSE, the client needs to request for incremental updates with resource-id and version-tag. there are two possible extreme solutions:
(1) the laziest: The server accepts the request that have the most recent version-tag; if the version-tag is not the most update version, the server simply rejects this request.
(2) the most costly: The server remembers all history of all resources, when the client requests with a resource-id and vtag, the server accepts the connection, and immediately send out patches to "catch the client up."


Example:
Notate version tags as vi, e.g. v0, v1, v2, etc. And the patches that updates vi to vj as patch(vi->vj) where vi is the old tag and vj is the new tag.

Say the current network map is v3. Client 1 requests incr updates with {"resource-id":"my-networkmap", "tag":"v0"} and Client 2 requests incr updates with {"resource-id":"my-networkmap", "tag":"v3"}. The lazy server would reject the request from client 1 and accepts that from client 2. The costly server would accept both request and send patch(v0->v3)


Another possible solution is a hybrid of the two extremes. The server keeps a patch that can update the resource from vi to vj, and accepts requests that has vtag vk such that i<=k<=j.

Because of the JSON merge-patch replacement property, applying patch(vi->vj) would update vk to vj. No matter what vtag vk the request comes in with, the server accepts if i<=k<=j, and immediately sends patch(vi->vj) via the stream. This will update the client's resource to the most updated version. The subsequent updates will be pushed as they become available.The window of [i, j] can be determined by the server, according to time or storage constraints.

You comments would be much appreciated. Thank you!

Best,
Xiao

_______________________________________________ alto mailing list alto <at> ietf.org https://www.ietf.org/mailman/listinfo/alto

<!-- .hmmessage P { margin:0px; padding:0px } body.hmmessage { font-size: 12pt; font-family:微软雅黑 } -->
_______________________________________________
alto mailing list
alto <at> ietf.org
https://www.ietf.org/mailman/listinfo/alto

_______________________________________________
alto mailing list
alto <at> ietf.org
https://www.ietf.org/mailman/listinfo/alto

Xiao SHI | 23 Oct 11:34 2014
Picon

Incremental Update/SSE Key Design Issue 1: Event subscription name in SSE

Hi all,

Wendy, Richard, and I are considering the incremental updates subscription model for ALTO using Server-Sent Event (SSE) streams. In this model, the client subscribes to an event stream, and the server push updates as events via the stream.

We use an "event" field to encode the unique identifier of the resource that the event should update, which should be a subset of {resource-id, resource-parameters, input-parameters}. (The client also has the IRD in its possession so that it can do certain look-ups).

We would like a __minimal__ unique identifier of each resource and each resource object. One resource could have multiple objects, e.g. when the "accepts" field is non-empty (i.e. they have input params), or they could have filters etc. Below are the proposal solutions:

For full network/cost map objects, resource-id is sufficient.
object {
  ResourceID resource-id;
} NetworkMapEventName;

object {
  ResourceID resource-id;
} CostMapEventName;

For resource objects that have non-empty "accepts" field or filters (i.e. input params) in the IRD, there are multiple solutions:

Solution 1. put the the input params in the events field
For  filtered network map objects, resource-id coupled with the pids for the filter:
object {
  ResourceID resource-id;
  PIDName pids<0..*>;
  [AddressType address-types<0..*>;]
} FilteredNetworkMapEventName;

For filtered cost map objects, we need to identify the costmap objects and the filter:
object {
  ResourceID resource-id;
  CostType cost-type;
  [JSONString constraints<0..*>;]
  [PIDFilter  pids;]
} CostMapEventName;

e.g. the event for a filtered network map would start with:
event: {"resource-id":"my-default-network-map", "pids":["PID2", "PID4"]}\n

Solution 2. add the input params information as meta information for an update.
We put the merge-patch object with the meta information together to form the update object:
object {
  ResponseMeta update-meta;
  JSONObject merge-patch;
} ALTOUpdate;

What do people think?

Best,
Xiao
<div><div dir="ltr">Hi all,<div><br></div>
<div>Wendy, Richard, and I are considering the incremental updates subscription model for ALTO using Server-Sent Event (SSE) streams. In this model, the client subscribes to an event stream, and the server push updates as events via the stream.</div>
<div><br></div>
<div>We use an "event" field to encode the unique identifier of the resource that the event should update, which should be a subset of {resource-id, resource-parameters, input-parameters}. (The client also has the IRD in its possession so that it can do certain look-ups).</div>
<div><br></div>
<div>We would like a __minimal__ unique identifier of each resource and each resource object. One resource could have multiple objects, e.g. when the "accepts" field is non-empty (i.e. they have input params), or they could have filters etc. Below are the proposal solutions:</div>
<div><br></div>
<div>For full network/cost map objects, resource-id is sufficient.<br>
</div>
<div>object {</div>
<div>&nbsp; ResourceID resource-id;</div>
<div>} NetworkMapEventName;</div>
<div><br></div>
<div>object {</div>
<div>&nbsp; ResourceID resource-id;</div>
<div>} CostMapEventName;</div>
<div><br></div>
<div>For resource objects that have non-empty "accepts" field or filters (i.e. input params) in the IRD, there are multiple solutions:</div>
<div><br></div>
<div>Solution 1. put the the input params in the events field</div>
<div>For &nbsp;filtered network map objects, resource-id coupled with the pids for the filter:</div>
<div>
<div>object {</div>
<div>&nbsp; ResourceID resource-id;</div>
<div>
<div>&nbsp; PIDName pids&lt;0..*&gt;;</div>
<div>&nbsp; [AddressType address-types&lt;0..*&gt;;]</div>
</div>
<div>} FilteredNetworkMapEventName;</div>
</div>
<div><br></div>
<div>For filtered cost map objects, we need to identify the costmap objects and the filter:</div>
<div>
<div>object {</div>
<div>&nbsp; ResourceID resource-id;</div>
<div>&nbsp; CostType cost-type;</div>
<div>
<div>&nbsp; [JSONString constraints&lt;0..*&gt;;]</div>
<div>&nbsp; [PIDFilter &nbsp;pids;]</div>
</div>
<div>} CostMapEventName;</div>
</div>
<div><br></div>
<div>e.g. the event for a filtered network map would start with:</div>
<div>event: {"resource-id":"my-default-network-map", "pids":["PID2", "PID4"]}\n</div>
<div><br></div>
<div>Solution 2. add the input params information as meta information for an update.</div>
<div>We put the merge-patch object with the meta information together to form the update object:</div>
<div>object {</div>
<div>&nbsp; ResponseMeta update-meta;</div>
<div>&nbsp; JSONObject merge-patch;</div>
<div>} ALTOUpdate;</div>
<div><br></div>
<div>What do people think?</div>
<div><br></div>
<div>Best,</div>
<div>Xiao</div>
</div></div>
Vijay K. Gurbani | 20 Oct 15:10 2014

Agenda requests for IETF 91

Folks: Please send me, Enrico and Jan the agenda requests for the
upcoming meeting in HI.

We have a 2.5 hour session on Thu, Nov 13 2014 1640-1910 local time.

As usual, agenda time will be prioritized according to item discussion
on the mailing list.

When sending a request, please indicate title, relevant drafts,
presenter, and time requested.

Thanks,

- vijay
--

-- 
Vijay K. Gurbani, Bell Laboratories, Alcatel-Lucent
1960 Lucent Lane, Rm. 9C-533, Naperville, Illinois 60563 (USA)
Email: vkg <at> {bell-labs.com,acm.org} / vijay.gurbani <at> alcatel-lucent.com
Web: http://ect.bell-labs.com/who/vkg/  | Calendar: http://goo.gl/x3Ogq

Xiao SHI | 20 Oct 08:05 2014
Picon

Modeling JSON based protocols using YANG

Hi folks,


As a few of us were working on modeling the ALTO protocol using YANG, we were pondering on a more general question: can YANG model all JSON based protocols? What is the condition for a JSON based protocol (at least the message format) to have an syntactically equivalent, hence interoperable, YANG model with JSON encoding? Alternatively, in order to interoperate, semantic equivalence might be sufficient, is there any condition for semantic equivalence?


tl;dr: A JSON based protocol carried by HTTP can have syntactically equivalent YANG model, if and only if all the keys in the key-value pairs in the JSON message are pre-defined keywords in the protocol.


We've come up with a few ideas, and a few things below are work in progress, but we would love your feedback!


Thank you!

Xiao


======================

1. Introduction.

JavaScript Object Notation (JSON) has been a popular choice as the message encoding for many network protocols such as the Application Layer Traffic Optimization (ALTO) protocol, the Content Delivery Networks Interconnection (CDNi) protocol, etc.


Meanwhile, there are broad interests in the networking community to use YANG to define data model so that one can use YANG related tools such as OpenDayLight controller. Although YANG itself is XML based, there have been efforts to model JSON content using YANG [draft-ietf-netmod-yang-json-01].


A natural question rises: can YANG model all JSON based protocols? What is the condition for a JSON based protocol (at least the message format) to have an syntactically equivalent, hence interoperable, YANG model with JSON encoding? Alternatively, in order to interoperate, semantic equivalence might be sufficient, is there any condition for semantic equivalence?


2. Claim

A JSON based protocol carried by HTTP can have syntactically equivalent YANG model, if and only if:

(1) the message encoding condition is met;

(2) the uri condition is met.


2.1. The message encoding condition

The JSON message encoding MUST not contain a variable as a key in a JSON object key-value pair. In other words, the keys in the JSON message must be an already defined constant (or keyword) in the message format of the protocol.


For example, if I have a protocol where “network-map”, “src”, and “dsts” are defined keywords, JSON text A meets the condition whereas B does not because “PID1” and “PID2” are not protocol keywords (they are resource IDs).

A:

{

 “network-map”: [

   {

     “src”: “PID1”,

     “dsts”: [“PID2”, “PID3”]

   },

   {

     “src”: “PID2”,

     “dsts”: [“PID3”, “PID4”]

   }

 ]

}

B:



{

 “network-map”: [

   “PID1: [“PID2”, “PID3”],

   “PID2”: [“PID3”, “PID4”]

 ]

}


From this we know that since ALTO protocol uses encoding B, there cannot be a syntactically equivalent YANG model.


2.2 The URI condition

Some of the YANG-related protocols might have URI constraints, e.g. RESTCONF. For now, we assume either that the JSON-based protocol URI could be conformed to RESTCONF compliant uri, or that the server could have a routing mapping between the protocol compliant uri and the RESTCONF compliant uri, hence this condition would not be an issue, which allows us to focus on the message encoding condition.


3. Proof

3.1. The message encoding condition is necessary.

We first note that this condition is a necessary condition for a JSON based protocol to have a syntactically equivalent YANG model by proving its contrapositive.


If one of the keys in the key-value pair in the JSON document is not pre-defined, the corresponding XML tags will not be pre-defined keywords. Therefore, it would not possible to model it in YANG without using YANG’s anyxml statement (which allows arbitrary XML content).


However, using the anyxml statement would defeat our purpose of modeling the data as it allows arbitrary XML content, and will not be helpful in the subsequent parsing process.


3.2. The message encoding condition is sufficient.

We prove this by providing a translation procedure from a JSON message that is compliant with the protocol we are trying to model, to a custom java class that can be used for Jackson data binding, then to a YANG model.


We note that the middle step of translating to and from the custom parser class is not necessary, but it will be useful.


3.2.1. motivation

JSON data binding is the process of binding data structures (objects, arrays, etc.) in JSON to the appropriate data structures in the server (e.g. java classes, database tables, etc.) in parsing the JSON text. In order to process JSON messages in a meaningful manner, data binding is necessary. Even if the binding is not explicit, the server would need to do it eventually. For example, one can read JSON content in a stream without binding it to the java classes, but eventually in order to make sense of the data, the server would eventually have to organize it, which is roughly analogous to data binding upfront. Popular choices for JSON parsing and data binding include jackson and gson.


We use Jackson full data binding as our approach. Full data binding binds JSON content into plain old java objects (POJOs), i.e. this custom parser class can neither extend nor implement any other class. Jackson uses ObjectMapper with the custom parser class to parse JSON content into this class.


3.2.2. The message encoding condition

The message encoding condition is that all keys in each key-value pair in the JSON text must be pre-defined keywords. As the keys will become either class names and instance variable names, or be keys in the java maps, it is easy to see that the condition is equivalent to “there exists a full data binding in Jackson custom parser class without using any map structures (Map<String, ?>).”


3.2.3. Proof

We provide a recursive binding process from a JSON object to the Jackson custom parser class to be used by Jackson ObjectMapper.


Type determine_type(value) {

 if (value is of a primitive type, i.e. string, number, boolean, null) {

   return the corresponding java primitive type;

 }

 if (value is a JSON object) {

   return build_parser_class(value).class;

 }

 if (value is an array) {

   return ArrayList<T> where T=determine_type(value[0]);

 }

 // should not reach here.

}


Class build_parser_class(JSONObject obj) {

 create custom class C;

 for each key/value pair in the obj {

   add instance variable v in C;

   the name of variable v <- key; // (__known__ a/c to our assumption)

   the type of variable v <- determine_type(value);

 }

 return C.class;

}


Naming:

--change everything into CamelCase (i.e. remove dashes, etc.)

--for instance variables, use “my” prefix, (e.g. myVariable, myNetworkMap, etc.)

--for the custom class name, if the object is an element of the array, use “Element” suffix.


This is just one convention so that the next step proceeds smoothly. As long as this naming translation is consistent with the naming stage in the next step, it will work just fine.


Example (a modified ALTO protocol network map example):

JSON object:

{

 “meta”: {

   “resource-id”: “my-default-map”,

   “tag”: “aab875ef69c87d012”

 },

 “network-map”: [

   {

     “src”: “PID1”,

     “dsts”: [“PID1”, “PID2”, “PID3”]

   },

   {

     “src”: “PID2”,

     “dsts”: [“PID1”, “PID3”]

   },

   {

     “src”: “PID3”,

     “dsts”: [“PID2”, “PID3”]

   }

 ]

}


Result of build_parser_class(obj):

Class JSONObject {

 Meta myMeta;

 ArrayList<NetworkMapElement> myNetworkMap;

}

Class Meta {

 String myResourceId;

 String myTag;

}

Class NetworkMapElement {

 String mySrc;

 ArrayList<String> myDsts;

}


Now given the Jackson Parser Java Class, to get a syntactically equivalent YANG model:

YANGModel build_yang_model(Class C) {

 for each instance variable (Type, Name) {

   if (Type is primitive type: string, number, boolean, null) {

     add the following to the YANG module:

     “leaf Name { type <YANG equivalent of Type>; }”

   }

   if (Type is an ArrayList<TypeElement>) {

     if (TypeElement is primitive type) {

       add the following to the YANG module:

       “leaf-list Name { type <YANG equivalent of TypeElement>; }”

     } else {

       // TypeElement is a custom parser class

       add the following to the YANG module:

       “list Name { <result from build_yang_model<TypeElement.class>> }”

     }

   }

   if (Type is a custom parser class) {

     add the following to the YANG module:

     “container Name { <result from build_yang_model<Type.class>> }”        

   }

 }

}


Result from the previous example:

container meta {

 leaf resource-id {

   type string;

 }

 leaf tag {

   type string;

 }

}

list network-map {

 leaf src {

   type string;

 }

 leaf-list dsts {

   type string;

 }

}


This does validate the JSON document with XML-JSON encoding. For your reference, this is the XML document which validates:

<?xml version="1.0" encoding="UTF-8" ?>

<meta>

 <resource-id>my-default-map</resource-id>

 <tag>aab875ef69c87d012</tag>

</meta>

<network-map>

 <src>PID1</src>

 <dsts>PID1</dsts>

 <dsts>PID2</dsts>

 <dsts>PID3</dsts>

</network-map>

<network-map>

 <src>PID2</src>

 <dsts>PID1</dsts>

 <dsts>PID3</dsts>

</network-map>

<network-map>

 <src>PID3</src>

 <dsts>PID2</dsts>

 <dsts>PID3</dsts>

</network-map>


This proves that the message encoding condition is a sufficient condition for the JSON object to have a YANG model.


Note the model generated is very crude and lose almost all constraints and all inheritance features (if any), because it focuses on the syntax and is essentially converted from an JSON object compliant with a protocol instead of from the protocol itself. Hence this result is more useful in determining which JSON based protocols cannot have a syntactically equivalent YANG model, than in generating a good YANG model.


3.3. Conclusion

Our claim holds. A JSON based protocol carried by HTTP can have syntactically equivalent YANG model, if and only if all the keys in the key-value pairs in the JSON message are pre-defined keywords.


4. Semantic equivalence

For JSON based protocols that don’t satisfy the message encoding condition, it is still possible to have a semantically equivalent YANG model. All that is required for the protocol compliant clients and the YANG model compliant server to interoperate is an adapter which does the following:


1) translate FROM YANG server compliant response msg TO alto compliant response msg

2) translate FROM alto compliant request msg TO YANG server compliant request msg


4.1. Claim

This adapter needs to be protocol-aware.


Ideally, given any YANG model, we would like to be able to automatically (or at least mechanically) generate this message adapter, which means not looking at the protocol or its compliant msgs. However, without knowing the specific protocol that we are working with (i.e. human intervention, i.e. looking at the protocol compliant msgs), such an adapter cannot be auto-generated.


4.2. Proof by Indistinguishability

Suppose both the YANG server compliant msg m_y and the actually protocol compliant msg m_p are in JSON (or have been encoded into JSON). Looking at the differences between the two messages, call these differences {d1, d2, ..., dn}. The goal for the auto-generated adapter would be to identify and eliminate these differences. Construct a new JSON msg m' where all but one difference di is the same as m_p and di is the same as the m_y. Without looking at the protocol (or m_p), the auto-generated adapter would not be able to distinguish between m' and m_p in its translation process, which means, it won't be able to tell whether it should change di or not. Hence, such an adapter must be protocol-aware.


A good example is the dependent-vtag in the ALTO protocol:

"dependent-vtag" : [

 {

   "resource-id" : "my-network-map",

   "tag" : "abcd1234"

 }

]


It was specified this way in the alto protocol. However, it could conceivably be the case that it was originally the following map structure, and was converted into the above encoding because of the map->list+key issue. (This case is actually one of the few differences in the m_y and m_p where the adapter does not need to convert it back to a map structure.)

"dependent-vtag" : {

 "my-network-map" : {

   "tag" : "abcd1234"

 }

}


Without knowing the protocol, there is no way to tell.


5. Ramifications

We now understand the basic condition for a JSON based protocol to have a YANG Model. For the protocols that don’t meet this condition, there can be a semantic equivalent YANG model, but there won’t be a generic process of generating the adapter for all such protocols.


<div><div dir="ltr">
<span><p><span>Hi folks,</span></p>
<p dir="ltr"><span><br></span></p>
<p><span>As a few of us were working on modeling the ALTO protocol using YANG, we were pondering on a more general question: can YANG model all JSON based protocols? What is the condition for a JSON based protocol (at least the message format) to have an syntactically equivalent, hence interoperable, YANG model with JSON encoding? Alternatively, in order to interoperate, semantic equivalence might be sufficient, is there any condition for semantic equivalence?</span><br></p>
<p><br></p>
<p><span>tl;dr: </span><span>A JSON based protocol carried by HTTP can have syntactically equivalent YANG model, if and only if all the keys in the key-value pairs in the JSON message are pre-defined keywords in the protocol.</span></p>
<p><br></p>
<p><span>We've come up with a few ideas, and a few things below are work in progress, but we would love your feedback!</span><span><br></span></p>
<p><span><br></span></p>
<p><span>Thank you!</span></p>
<p><span>Xiao</span></p>
<p><span><br></span></p>======================</span><div><span><p dir="ltr"><span>1. Introduction.</span></p>
<p dir="ltr"><span>JavaScript Object Notation (JSON) has been a popular choice as the message encoding for many network protocols such as the Application Layer Traffic Optimization (ALTO) protocol, the Content Delivery Networks Interconnection (CDNi) protocol, etc.</span></p>
<br><p dir="ltr"><span>Meanwhile, there are broad interests in the networking community to use YANG to define data model so that one can use YANG related tools such as OpenDayLight controller. Although YANG itself is XML based, there have been efforts to model JSON content using YANG [draft-ietf-netmod-yang-json-01]. </span></p>
<br><p dir="ltr"><span>A natural question rises: can YANG model all JSON based protocols? What is the condition for a JSON based protocol (at least the message format) to have an syntactically equivalent, hence interoperable, YANG model with JSON encoding? Alternatively, in order to interoperate, semantic equivalence might be sufficient, is there any condition for semantic equivalence?</span></p></span></div>
<div><span><br><p dir="ltr"><span>2. Claim</span></p>
<p dir="ltr"><span>A JSON based protocol carried by HTTP can have syntactically equivalent YANG model, if and only if:</span></p>
<p dir="ltr"><span>(1) the message encoding condition is met;</span></p>
<p dir="ltr"><span>(2) the uri condition is met.</span></p>
<br><p dir="ltr"><span>2.1. The message encoding condition</span></p>
<p dir="ltr"><span>The JSON message encoding MUST not contain a variable as a key in a JSON object key-value pair. In other words, the keys in the JSON message must be an already defined constant (or keyword) in the message format of the protocol.</span></p>
<br><p dir="ltr"><span>For example, if I have a protocol where &ldquo;network-map&rdquo;, &ldquo;src&rdquo;, and &ldquo;dsts&rdquo; are defined keywords, JSON text A meets the condition whereas B does not because &ldquo;PID1&rdquo; and &ldquo;PID2&rdquo; are not protocol keywords (they are resource IDs).</span></p>
<p dir="ltr"><span>A:</span></p>
<p dir="ltr"><span>{</span></p>
<p dir="ltr"><span> &nbsp;&ldquo;network-map&rdquo;: [</span></p>
<p dir="ltr"><span> &nbsp;&nbsp;&nbsp;{</span></p>
<p dir="ltr"><span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&ldquo;src&rdquo;: &ldquo;PID1&rdquo;,</span></p>
<p dir="ltr"><span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&ldquo;dsts&rdquo;: [&ldquo;PID2&rdquo;, &ldquo;PID3&rdquo;]</span></p>
<p dir="ltr"><span> &nbsp;&nbsp;&nbsp;},</span></p>
<p dir="ltr"><span> &nbsp;&nbsp;&nbsp;{</span></p>
<p dir="ltr"><span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&ldquo;src&rdquo;: &ldquo;PID2&rdquo;,</span></p>
<p dir="ltr"><span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&ldquo;dsts&rdquo;: [&ldquo;PID3&rdquo;, &ldquo;PID4&rdquo;]</span></p>
<p dir="ltr"><span> &nbsp;&nbsp;&nbsp;}</span></p>
<p dir="ltr"><span> &nbsp;]</span></p>
<p dir="ltr"><span>}</span></p>
<p dir="ltr"><span>B:</span></p>
<br><br><p dir="ltr"><span>{</span></p>
<p dir="ltr"><span> &nbsp;&ldquo;network-map&rdquo;: [</span></p>
<p dir="ltr"><span> &nbsp;&nbsp;&nbsp;&ldquo;PID1: [&ldquo;PID2&rdquo;, &ldquo;PID3&rdquo;],</span></p>
<p dir="ltr"><span> &nbsp;&nbsp;&nbsp;&ldquo;PID2&rdquo;: [&ldquo;PID3&rdquo;, &ldquo;PID4&rdquo;]</span></p>
<p dir="ltr"><span> &nbsp;]</span></p>
<p dir="ltr"><span>}</span></p>
<br><p dir="ltr"><span>From this we know that since ALTO protocol uses encoding B, there cannot be a syntactically equivalent YANG model.</span></p>
<br><p dir="ltr"><span>2.2 The URI condition</span></p>
<p dir="ltr"><span>Some of the YANG-related protocols might have URI constraints, e.g. RESTCONF. For now, we assume either that the JSON-based protocol URI could be conformed to RESTCONF compliant uri, or that the server could have a routing mapping between the protocol compliant uri and the RESTCONF compliant uri, hence this condition would not be an issue, which allows us to focus on the message encoding condition.</span></p>
<br><p dir="ltr"><span>3. Proof</span></p>
<p dir="ltr"><span>3.1. The message encoding condition is necessary.</span></p>
<p dir="ltr"><span>We first note that this condition is a necessary condition for a JSON based protocol to have a syntactically equivalent YANG model by proving its contrapositive.</span></p>
<br><p dir="ltr"><span>If one of the keys in the key-value pair in the JSON document is not pre-defined, the corresponding XML tags will not be pre-defined keywords. Therefore, it would not possible to model it in YANG without using YANG&rsquo;s </span><span>anyxml </span><span>statement (which allows arbitrary XML content).</span></p>
<br><p dir="ltr"><span>However, using the anyxml statement would defeat our purpose of modeling the data as it allows arbitrary XML content, and will not be helpful in the subsequent parsing process.</span></p>
<br><p dir="ltr"><span>3.2. The message encoding condition is sufficient.</span></p>
<p dir="ltr"><span>We prove this by providing a translation procedure from a JSON message that is compliant with the protocol we are trying to model, to a custom java class that can be used for Jackson data binding, then to a YANG model.</span></p>
<br><p dir="ltr"><span>We note that the middle step of translating to and from the custom parser class is not necessary, but it will be useful.</span></p>
<br><p dir="ltr"><span>3.2.1. motivation</span></p>
<p dir="ltr"><span>JSON data binding is the process of binding data structures (objects, arrays, etc.) in JSON to the appropriate data structures in the server (e.g. java classes, database tables, etc.) in parsing the JSON text. In order to process JSON messages in a meaningful manner, data binding is necessary. Even if the binding is not explicit, the server would need to do it eventually. For example, one can read JSON content in a stream without binding it to the java classes, but eventually in order to make sense of the data, the server would eventually have to organize it, which is roughly analogous to data binding upfront. Popular choices for JSON parsing and data binding include jackson and gson.</span></p>
<br><p dir="ltr"><span>We use Jackson full data binding as our approach. Full data binding binds JSON content into plain old java objects (POJOs), i.e. this custom parser class can neither extend nor implement any other class. Jackson uses ObjectMapper with the custom parser class to parse JSON content into this class.</span></p>
<br><p dir="ltr"><span>3.2.2. The message encoding condition</span></p>
<p dir="ltr"><span>The message encoding condition is that all keys in each key-value pair in the JSON text must be pre-defined keywords. As the keys will become either class names and instance variable names, or be keys in the java maps, it is easy to see that the condition is equivalent to &ldquo;there exists a full data binding in Jackson custom parser class without using any map structures (Map&lt;String, ?&gt;).&rdquo; </span></p>
<br><p dir="ltr"><span>3.2.3. Proof</span></p>
<p dir="ltr"><span>We provide a recursive binding process from a JSON object to the Jackson custom parser class to be used by Jackson ObjectMapper.</span></p>
<br><p dir="ltr"><span>Type determine_type(value) {</span></p>
<p dir="ltr"><span> &nbsp;if (value is of a primitive type, i.e. string, number, boolean, null) {</span></p>
<p dir="ltr"><span> &nbsp;&nbsp;&nbsp;return the corresponding java primitive type;</span></p>
<p dir="ltr"><span> &nbsp;}</span></p>
<p dir="ltr"><span> &nbsp;if (value is a JSON object) {</span></p>
<p dir="ltr"><span> &nbsp;&nbsp;&nbsp;return build_parser_class(value).class;</span></p>
<p dir="ltr"><span> &nbsp;}</span></p>
<p dir="ltr"><span> &nbsp;if (value is an array) {</span></p>
<p dir="ltr"><span> &nbsp;&nbsp;&nbsp;return ArrayList&lt;T&gt; where T=determine_type(value[0]);</span></p>
<p dir="ltr"><span> &nbsp;}</span></p>
<p dir="ltr"><span> &nbsp;// should not reach here.</span></p>
<p dir="ltr"><span>}</span></p>
<br><p dir="ltr"><span>Class build_parser_class(JSONObject obj) {</span></p>
<p dir="ltr"><span> &nbsp;create custom class C;</span></p>
<p dir="ltr"><span> &nbsp;for each key/value pair in the obj {</span></p>
<p dir="ltr"><span> &nbsp;&nbsp;&nbsp;add instance variable v in C; </span></p>
<p dir="ltr"><span> &nbsp;&nbsp;&nbsp;the name of variable v &lt;- key; // (__known__ a/c to our assumption)</span></p>
<p dir="ltr"><span> &nbsp;&nbsp;&nbsp;the type of variable v &lt;- determine_type(value);</span></p>
<p dir="ltr"><span> &nbsp;}</span></p>
<p dir="ltr"><span> &nbsp;return C.class;</span></p>
<p dir="ltr"><span>}</span></p>
<br><p dir="ltr"><span>Naming:</span></p>
<p dir="ltr"><span>--change everything into CamelCase (i.e. remove dashes, etc.)</span></p>
<p dir="ltr"><span>--for instance variables, use &ldquo;my&rdquo; prefix, (e.g. myVariable, myNetworkMap, etc.)</span></p>
<p dir="ltr"><span>--for the custom class name, if the object is an element of the array, use &ldquo;Element&rdquo; suffix.</span></p>
<br><p dir="ltr"><span>This is just one convention so that the next step proceeds smoothly. As long as this naming translation is consistent with the naming stage in the next step, it will work just fine.</span></p>
<br><p dir="ltr"><span>Example (a </span><span>modified</span><span> ALTO protocol</span><span> </span><span>network map example):</span></p>
<p dir="ltr"><span>JSON object:</span></p>
<p dir="ltr"><span>{</span></p>
<p dir="ltr"><span> &nbsp;&ldquo;meta&rdquo;: {</span></p>
<p dir="ltr"><span> &nbsp;&nbsp;&nbsp;&ldquo;resource-id&rdquo;: &ldquo;my-default-map&rdquo;,</span></p>
<p dir="ltr"><span> &nbsp;&nbsp;&nbsp;&ldquo;tag&rdquo;: &ldquo;aab875ef69c87d012&rdquo;</span></p>
<p dir="ltr"><span> &nbsp;},</span></p>
<p dir="ltr"><span> &nbsp;&ldquo;network-map&rdquo;: [</span></p>
<p dir="ltr"><span> &nbsp;&nbsp;&nbsp;{</span></p>
<p dir="ltr"><span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&ldquo;src&rdquo;: &ldquo;PID1&rdquo;,</span></p>
<p dir="ltr"><span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&ldquo;dsts&rdquo;: [&ldquo;PID1&rdquo;, &ldquo;PID2&rdquo;, &ldquo;PID3&rdquo;]</span></p>
<p dir="ltr"><span> &nbsp;&nbsp;&nbsp;},</span></p>
<p dir="ltr"><span> &nbsp;&nbsp;&nbsp;{</span></p>
<p dir="ltr"><span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&ldquo;src&rdquo;: &ldquo;PID2&rdquo;,</span></p>
<p dir="ltr"><span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&ldquo;dsts&rdquo;: [&ldquo;PID1&rdquo;, &ldquo;PID3&rdquo;]</span></p>
<p dir="ltr"><span> &nbsp;&nbsp;&nbsp;},</span></p>
<p dir="ltr"><span> &nbsp;&nbsp;&nbsp;{</span></p>
<p dir="ltr"><span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&ldquo;src&rdquo;: &ldquo;PID3&rdquo;,</span></p>
<p dir="ltr"><span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&ldquo;dsts&rdquo;: [&ldquo;PID2&rdquo;, &ldquo;PID3&rdquo;]</span></p>
<p dir="ltr"><span> &nbsp;&nbsp;&nbsp;}</span></p>
<p dir="ltr"><span> &nbsp;]</span></p>
<p dir="ltr"><span>}</span></p>
<br><p dir="ltr"><span>Result of build_parser_class(obj):</span></p>
<p dir="ltr"><span>Class JSONObject {</span></p>
<p dir="ltr"><span> &nbsp;Meta myMeta;</span></p>
<p dir="ltr"><span> &nbsp;ArrayList&lt;NetworkMapElement&gt; myNetworkMap;</span></p>
<p dir="ltr"><span>}</span></p>
<p dir="ltr"><span>Class Meta {</span></p>
<p dir="ltr"><span> &nbsp;String myResourceId;</span></p>
<p dir="ltr"><span> &nbsp;String myTag;</span></p>
<p dir="ltr"><span>}</span></p>
<p dir="ltr"><span>Class NetworkMapElement {</span></p>
<p dir="ltr"><span> &nbsp;String mySrc;</span></p>
<p dir="ltr"><span> &nbsp;ArrayList&lt;String&gt; myDsts;</span></p>
<p dir="ltr"><span>}</span></p>
<br><p dir="ltr"><span>Now given the Jackson Parser Java Class, to get a syntactically equivalent YANG model:</span></p>
<p dir="ltr"><span>YANGModel build_yang_model(Class C) {</span></p>
<p dir="ltr"><span> &nbsp;for each instance variable (Type, Name) {</span></p>
<p dir="ltr"><span> &nbsp;&nbsp;&nbsp;if (Type is primitive type: string, number, boolean, null) {</span></p>
<p dir="ltr"><span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;add the following to the YANG module:</span></p>
<p dir="ltr"><span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&ldquo;leaf Name { type &lt;YANG equivalent of Type&gt;; }&rdquo;</span></p>
<p dir="ltr"><span> &nbsp;&nbsp;&nbsp;}</span></p>
<p dir="ltr"><span> &nbsp;&nbsp;&nbsp;if (Type is an ArrayList&lt;TypeElement&gt;) {</span></p>
<p dir="ltr"><span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;if (TypeElement is primitive type) {</span></p>
<p dir="ltr"><span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;add the following to the YANG module:</span></p>
<p dir="ltr"><span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&ldquo;leaf-list Name { type &lt;YANG equivalent of TypeElement&gt;; }&rdquo;</span></p>
<p dir="ltr"><span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;} else {</span></p>
<p dir="ltr"><span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;// TypeElement is a custom parser class</span></p>
<p dir="ltr"><span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;add the following to the YANG module:</span></p>
<p dir="ltr"><span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&ldquo;list Name { &lt;result from build_yang_model&lt;TypeElement.class&gt;&gt; }&rdquo;</span></p>
<p dir="ltr"><span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;}</span></p>
<p dir="ltr"><span> &nbsp;&nbsp;&nbsp;}</span></p>
<p dir="ltr"><span> &nbsp;&nbsp;&nbsp;if (Type is a custom parser class) {</span></p>
<p dir="ltr"><span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;add the following to the YANG module:</span></p>
<p dir="ltr"><span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&ldquo;container Name { &lt;result from build_yang_model&lt;Type.class&gt;&gt; }&rdquo; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span></p>
<p dir="ltr"><span> &nbsp;&nbsp;&nbsp;}</span></p>
<p dir="ltr"><span> &nbsp;}</span></p>
<p dir="ltr"><span>}</span></p>
<br><p dir="ltr"><span>Result from the previous example:</span></p>
<p dir="ltr"><span>container meta {</span></p>
<p dir="ltr"><span> &nbsp;leaf resource-id {</span></p>
<p dir="ltr"><span> &nbsp;&nbsp;&nbsp;type string;</span></p>
<p dir="ltr"><span> &nbsp;}</span></p>
<p dir="ltr"><span> &nbsp;leaf tag {</span></p>
<p dir="ltr"><span> &nbsp;&nbsp;&nbsp;type string;</span></p>
<p dir="ltr"><span> &nbsp;}</span></p>
<p dir="ltr"><span>}</span></p>
<p dir="ltr"><span>list network-map {</span></p>
<p dir="ltr"><span> &nbsp;leaf src {</span></p>
<p dir="ltr"><span> &nbsp;&nbsp;&nbsp;type string;</span></p>
<p dir="ltr"><span> &nbsp;}</span></p>
<p dir="ltr"><span> &nbsp;leaf-list dsts {</span></p>
<p dir="ltr"><span> &nbsp;&nbsp;&nbsp;type string;</span></p>
<p dir="ltr"><span> &nbsp;}</span></p>
<p dir="ltr"><span>}</span></p>
<br><p dir="ltr"><span>This does validate the JSON document with XML-JSON encoding. For your reference, this is the XML document which validates:</span></p>
<p dir="ltr"><span>&lt;?xml version="1.0" encoding="UTF-8" ?&gt;</span></p>
<p dir="ltr"><span>&lt;meta&gt;</span></p>
<p dir="ltr"><span> &nbsp;&lt;resource-id&gt;my-default-map&lt;/resource-id&gt;</span></p>
<p dir="ltr"><span> &nbsp;&lt;tag&gt;aab875ef69c87d012&lt;/tag&gt;</span></p>
<p dir="ltr"><span>&lt;/meta&gt;</span></p>
<p dir="ltr"><span>&lt;network-map&gt;</span></p>
<p dir="ltr"><span> &nbsp;&lt;src&gt;PID1&lt;/src&gt;</span></p>
<p dir="ltr"><span> &nbsp;&lt;dsts&gt;PID1&lt;/dsts&gt;</span></p>
<p dir="ltr"><span> &nbsp;&lt;dsts&gt;PID2&lt;/dsts&gt;</span></p>
<p dir="ltr"><span> &nbsp;&lt;dsts&gt;PID3&lt;/dsts&gt;</span></p>
<p dir="ltr"><span>&lt;/network-map&gt;</span></p>
<p dir="ltr"><span>&lt;network-map&gt;</span></p>
<p dir="ltr"><span> &nbsp;&lt;src&gt;PID2&lt;/src&gt;</span></p>
<p dir="ltr"><span> &nbsp;&lt;dsts&gt;PID1&lt;/dsts&gt;</span></p>
<p dir="ltr"><span> &nbsp;&lt;dsts&gt;PID3&lt;/dsts&gt;</span></p>
<p dir="ltr"><span>&lt;/network-map&gt;</span></p>
<p dir="ltr"><span>&lt;network-map&gt;</span></p>
<p dir="ltr"><span> &nbsp;&lt;src&gt;PID3&lt;/src&gt;</span></p>
<p dir="ltr"><span> &nbsp;&lt;dsts&gt;PID2&lt;/dsts&gt;</span></p>
<p dir="ltr"><span> &nbsp;&lt;dsts&gt;PID3&lt;/dsts&gt;</span></p>
<p dir="ltr"><span>&lt;/network-map&gt;</span></p>
<br><p dir="ltr"><span>This proves that the message encoding condition is a sufficient condition for the JSON object to have a YANG model.</span></p>
<br><p dir="ltr"><span>Note the model generated is very crude and lose almost all constraints and all inheritance features (if any), because it focuses on the syntax and is essentially converted from an JSON object compliant with a protocol instead of from the protocol itself. Hence this result is more useful in determining which JSON based protocols cannot have a syntactically equivalent YANG model, than in generating a good YANG model.</span></p>
<br><p dir="ltr"><span>3.3. Conclusion</span></p>
<p dir="ltr"><span>Our claim holds. A JSON based protocol carried by HTTP can have syntactically equivalent YANG model, if and only if all the keys in the key-value pairs in the JSON message are pre-defined keywords.</span></p>
<br><p dir="ltr"><span>4. Semantic equivalence</span></p>
<p dir="ltr"><span>For JSON based protocols that don&rsquo;t satisfy the message encoding condition, it is still possible to have a semantically equivalent YANG model. All that is required for the protocol compliant clients and the YANG model compliant server to interoperate is an adapter which does the following:</span></p>
<br><p dir="ltr"><span>1) translate FROM YANG server compliant response msg TO alto compliant response msg</span></p>
<p dir="ltr"><span>2) translate FROM alto compliant request msg TO YANG server compliant request msg</span></p>
<br><p dir="ltr"><span>4.1. Claim</span></p>
<p dir="ltr"><span>This adapter needs to be protocol-aware.</span></p>
<br><p dir="ltr"><span>Ideally, given any YANG model, we would like to be able to automatically (or at least mechanically) generate this message adapter, which means not looking at the protocol or its compliant msgs. However, without knowing the specific protocol that we are working with (i.e. human intervention, i.e. looking at the protocol compliant msgs), such an adapter cannot be auto-generated.</span></p>
<br><p dir="ltr"><span>4.2. Proof by Indistinguishability</span></p>
<p dir="ltr"><span>Suppose both the YANG server compliant msg m_y and the actually protocol compliant msg m_p are in JSON (or have been encoded into JSON). Looking at the differences between the two messages, call these differences {d1, d2, ..., dn}. The goal for the auto-generated adapter would be to identify and eliminate these differences. Construct a new JSON msg m' where all but one difference di is the same as m_p and di is the same as the m_y. Without looking at the protocol (or m_p), the auto-generated adapter would not be able to distinguish between m' and m_p in its translation process, which means, it won't be able to tell whether it should change di or not. Hence, such an adapter must be protocol-aware.</span></p>
<br><p dir="ltr"><span>A good example is the dependent-vtag in the ALTO protocol:</span></p>
<p dir="ltr"><span>"dependent-vtag" : [</span></p>
<p dir="ltr"><span> &nbsp;{</span></p>
<p dir="ltr"><span> &nbsp;&nbsp;&nbsp;"resource-id" : "my-network-map",</span></p>
<p dir="ltr"><span> &nbsp;&nbsp;&nbsp;"tag" : "abcd1234"</span></p>
<p dir="ltr"><span> &nbsp;}</span></p>
<p dir="ltr"><span>]</span></p>
<br><p dir="ltr"><span>It was specified this way in the alto protocol. However, it could conceivably be the case that it was originally the following map structure, and was converted into the above encoding because of the map-&gt;list+key issue. (This case is actually one of the few differences in the m_y and m_p where the adapter does not need to convert it back to a map structure.)</span></p>
<p dir="ltr"><span>"dependent-vtag" : {</span></p>
<p dir="ltr"><span> &nbsp;"my-network-map" : {</span></p>
<p dir="ltr"><span> &nbsp;&nbsp;&nbsp;"tag" : "abcd1234"</span></p>
<p dir="ltr"><span> &nbsp;}</span></p>
<p dir="ltr"><span>}</span></p>
<br><p dir="ltr"><span>Without knowing the protocol, there is no way to tell. </span></p>
<br><p dir="ltr"><span>5. Ramifications</span></p>
<p dir="ltr"><span>We now understand the basic condition for a JSON based protocol to have a YANG Model. For the protocols that don&rsquo;t meet this condition, there can be a semantic equivalent YANG model, but there won&rsquo;t be a generic process of generating the adapter for all such protocols.</span></p>
<div><span><br></span></div></span></div>
</div></div>
Y. Richard Yang | 15 Oct 06:32 2014
Picon

Incremental updates transport using SSE?

Dear all,

Given the potential progress in incremental update messaging format (e.g., merge-patch or tree-replace), it is time to consider the next item: transport. In particular, we want decoupling of update message format from transport: the payload can be carried by not only HTTP Patch, but other methods, say client pull using HTTP POST, or SSE.

In particular, consider a design where an ALTO server pushes incremental updates to an ALTO client using SSE. How about the following straw-man mapping design, for client subscription and server push content indication:

event name: ALTO resource-id

id: 
  if initial resource (full content), tag as full alto resource
  if new-tag_old-tag

data: {
data: "Content-Type": "application/merge-patch+json" | rfc6902 patch
data: "Content": 
data: {
data: <op object here, need escape to avoid ambiguity, i.e., no extra \ndata:>
data: }
data: }\n\n

Make sense, by SSE experts?

Richard
<div><div dir="ltr">
<div>Dear all,</div>
<div><br></div>
<div>Given the potential progress in incremental update messaging format (e.g., merge-patch or tree-replace), it is time to consider the next item: transport. In particular, we want decoupling of update message format from transport: the payload can be carried by not only HTTP Patch, but other methods, say client pull using HTTP POST, or SSE.</div>
<div><br></div>
<div>In particular, consider a design where an ALTO server pushes incremental updates to an ALTO client using SSE. How about the following straw-man mapping design, for client subscription and server push content indication:</div>
<div><br></div>
<div>event name: ALTO resource-id</div>
<div><br></div>
<div>id:&nbsp;</div>
<div>&nbsp; if initial resource (full content), tag as full alto resource</div>
<div>&nbsp; if new-tag_old-tag</div>
<div><br></div>
<div>data: {</div>
<div>data: "Content-Type": "application/merge-patch+json" | rfc6902 patch</div>
<div>data: "Content":&nbsp;</div>
<div>data: {</div>
<div>data: &lt;op object here, need escape to avoid ambiguity, i.e., no extra \ndata:&gt;</div>
<div>data: }</div>
<div>data: }\n\n</div>
<div><br></div>
<div>Make sense, by SSE experts?</div>
<div><br></div>
<div>Richard</div>
</div></div>

Gmane