Fred Baker (fred | 1 Apr 03:11 2015

Re: Discussion: IPv4 as a Service

Since they might be among the early folks I would ask to write something, I asked Xing Li, Akira, and Suprita to comment specifically on the outline: “if you were to write such a document, how would this outline work for you”.

Suprita replied as follows. I have changed the proposed outline in a manner similar to her suggestion.

On Mar 31, 2015, at 2:41 PM, suprita <> wrote:

I tried to first re-write the "Outlined hierarchy" used  and then make any modification to it, as per ease of answering them. 
I did not have to do many changes to the one already used by you, with minor adjustments in naming convention and placements..

Experience Gathering : Heading

Major Motivation(s)
v4 as-a-service Requirements
Exactly which v4 Service Types
Architecture and Methodology
Major Design Considerations
Regulatory Considerations
Security Considerations
Operational Considerations
End-User Experience Considerations
Observations and Experiences
Effects on End-User
Effects on Internal Staff
Planning & Design
Effects on Business
Summary: Post-mortem Report
Deviations from RFC's/Drafts/Standards
Worked Well
Did not work well

On Wed, Apr 1, 2015 at 1:56 AM, suprita <> wrote:
Dear Fred,

I am yet to apply newly learnt skill from you of applying filters for the directly addressed e-mails :).
I will sure go through the doc in detail and comment. 

Thank you,

On Tue, Mar 31, 2015 at 2:04 AM, Fred Baker (fred) <> wrote:

> On Mar 30, 2015, at 12:50 PM, Fred Baker <> wrote:
> Over the weekend, we crowd-sourced a number of pretty good questions. I took the liberty, this morning, of reorganizing the text a bit. The "list of technologies" and "list of questions" remain, but I have added "possible (classes of) documents to be developed" and a first cut at a proposed outline, with the questions from above pulled in to make it clear what I might hope the section might comment on.
> Collecting comments and thoughts. I guarantee there is something we haven’t thought of (as I was reorganizing, several points popped out, and I’m sure I’m not that smart).

The first thing I am wondering about is whether the proposed outline “works”. Would you be willing to spend an hour thinking through the outline and wondering how you would respond to it for Reliance? What questions are missing? Would another organization work better?

v6ops mailing list
Fred Baker (fred | 31 Mar 07:35 2015

IPv4 trajectory

In the course of the Google Doc “towards IPv4 as a service” discussion, Brian Carpenter wrote

> Scope question: do we need to distinguish edge networks from Tier 3 provider networks? Are we making the
assumption that transit and core carriers remain dual-stack? Listing LISP suggests that we might not be
making that assumption exactly.

Let me give you my thought process. Happy to be told I am wrong.

As I read RFC 4213, the transition process was envisioned as taking three steps:

1) IPv4-only
2) IPv4+IPv6 Dual Stack
3) IPv6-only

Had the transition started five or ten years ago, that might have been reasonable. Where things stand,
though, I don’t think it is. From my perspective, the process over some amount of time into the future
(amount of time is anybody’s guess) will take six steps:

1) IPv4-only network
2) IPv4 with bits of IPv6 here and there, in some combination of overlay and native
3) IPv4+IPv6 (native) dual stack network
4) IPv6+IPv4 non-native, translated or overlay (e.g., as a service)
5) IPv6 with little bits of IPv4 here and there; your printer might be an example
6) IPv6-only

In steps 1 and 2, it is an IPv4 network, possibly with some toys in it playing IPv6. We might have toys playing
AppleTalk and DECNet too. IPv4 is the workhorse, and toys is toys.

In step 3, every host in the network has an IPv4 address and an IPv6 address, or constructively could have if
it wants one, and every router is carrying IPv4 routing as well as IPv6 routing. It’s likely
multiplexing IPv4 addresses, and IPv4 is increasingly broken. But the big question is what applications
and providers support IPv6; as long as a big one (Skype?) is not making the transition, people have a
business reason to keep IPv4 going. As applications become dual stack, traffic levels approach a 50/50
distribution, barring some specific reason they would be forced to IPv4 or IPv6.

Right now, most networks are still in step 1, and those that have a clue are somewhere between step 2 and step
3. But some are moving, or at least experimenting, with step 4.

steps 4-6 are fundamentally about an IPv6 network.

In step 4, the network is optimizing configurations and costs by making IPv4 a translation or an overlay,
but IPv4 remains an important aspect of the network. Obvious examples include anything listed. IPv4
might reach the host one way or another, or be translated some distance away, but a client using IPv4 can
reach an IPv4-only server, and it can reply, and part of the path in between is IPv6-only.

In step 5, IPv4 has taken a place comparable to AppleTalk; yes, there’s some there, but where, precisely?
And in step 6, we have finally turned IPv4 off entirely, and with it probably Bisync and X.25.

Now, what assumptions am I making? I believe that an IPv4 packet can make it end to end until step 5; it might
spend part of that as a translated IPv6 packet or in a tunnel of some kind, but it can get there. As of step 5,
that is no longer the case. Exactly how that happens is Not My Department. Note that many core networks
don’t describe themselves as IPv4, IPv6, or dual stack networks; they describe themselves as MPLS
networks that carry IP as a payload.

Listing LISP... to me, LISP is yet another encapsulation, a tunnel broker, and one that is particularly
complex. It can carry IPv6 across an IPv4 network or IPv4 over an IPv6 network. Someone suggested it should
be on the list, and it’s on the list. One thing I actually expect is that the list of technologies we might
invite deployment reports regarding is longer than the list of technologies people actually use. We
might learn something from that.

Is anyone’s head exploding yet?
v6ops mailing list
Francis Dupont | 30 Mar 23:43 2015

IPv6 is 20 year old!

Exactly 20 years ago the first interconnection between two independent
IPv6 implementations succeeded between and



PS: IPv4-compatible IPv6 addresses were used (I thought this defunct
type of addresses was only used for this kind of early tests), the
first "native" IPv6 (and Neighbor Discovery) was during the 33th IETF
at Stockholm in July 1995.
PPS: I can give or put somewhere details, including the typescript
of the second connection I kept for this kind of occasions, just ask.

v6ops mailing list

Fred Baker (fred | 26 Mar 23:04 2015

Verification on the list

We have three drafts relevant to scalable use of translation in a datacenter environment:
  "Explicit Address Mappings for Stateless IP/ICMP Translation", tore,
  "SIIT-DC: Stateless IP/ICMP Translation for IPv6 Data Centre
  Environments", tore, 2014-12-18
  "SIIT-DC: Dual Translation Mode", tore, 2015-01-27,

In discussion today in v6ops, we agreed to
1) adopt draft-anderson-v6ops-siit-eam as draft-ietf-v6ops-siit-eam
2) excise that algorithm from draft-ietf-v6ops-siit-dc and instead insert a reference
3) similarly refer to -eam in draft-ietf-v6ops-siit-dc-2xlat

The question for the house is “do you agree with the above?”. The chairs will presume that we agree, but
invite people to disabuse us of the notion.

Tore is looking for review of each and hopefully a co-author for each. Please post reviews to the list.

One comment raised in the WG meeting is that this is a minor update to either RFC 6145 (giving the option of
specifying a mapping between an IPv4 and an IPv6 address as opposed to using an RFC 6052 IPv6 address) or RFC
6146 (which already has the option of a static mapping, but is fairly insistent on per-flow state as
opposed to per-address preconfiguration). It should not be construed as yet another mechanism, calling
for a blizzard of evaluation and guidance documents. If RFC 6145 is updated at some point in the future,
this document and any other updates and errata would be incorporated into the revision.
v6ops mailing list
Fred Baker | 26 Mar 00:48 2015

Discussion: IPv4 as a Service

Lee and I opened a discussion in the v6ops meeting today on a topic that I believe that the IETF, in
cooperation with the *NOGs, needs to address. My slide is at Sunset4 and softwire
chairs, please feel free to forward this to your mailing lists.

In short, we are starting to see the deployment of IPv6-only networks. In most cases, these are existing
IPv4 networks that have become dual stack, and are somewhere in the process of shutting IPv4 down. These
include Facebook, which (regardless of whether you view it as an enterprise or a content network) has been
public about the fact, several mobile networks that treat IPv6-capable telephones as an IPv6-only
network and use 464xlat as an operational methodology dealing with that, and some research networks,
notably CERNET2, which has been running IPv6-only for ten years now. We also see green-field networks,
such as Reliance JIO Infocomm Ltd; Suprita Sah presented on that topic today.

I am of the opinion, and there seemed to be agreement in the room, that the Internet community would be well
served if we collected deployment experience with these technologies, and the considered advice of
deploying operators. We have tried to guide the deployment of IPv6, and gathered experience and
mitigations for that. Now, as IPv4 begins to fade, we need to take a leadership role in that.

I have opened a document on Google Docs, and given folks the ability to view it. That is at I have
included an initial list of relevant technologies, and an initial list of the questions that we would want
discussed in experience documents. I guarantee that there is an important question not asked, and while I
hope otherwise, there may be some technology I have overlooked. So I invite discussion to flesh out that
initial phase.

v6ops mailing list

Ian Farrer | 25 Mar 20:17 2015

Existing work on Softwire Deployment Experiences


As mentioned at the mike in today’s session, Softwire has already produced a couple of drafts on deployment experience:


It would be good to get any input to these from the operators who presented their implementations today.

v6ops mailing list
Ca By | 19 Mar 22:27 2015

The need for local-ipv4 socket transition solutions -- NAT64/DNS64 remains insufficient

Hi Folks,

The point of this message is to discuss the use of IPv4 literals and how this use requires transition solutions such as MAP, DS-Lite, and 464XLAT -- all of which provide local IPv4 socket support to the host.  If the primary operating environment of the host is the "internet", and the "web" is a highly valuable part of the that operating environment, then local IPv4 socket support is still required.

Specifically, this note builds the case that NAT64/DNS64 alone is still (at this time) not a sufficient solution for consumer Internet access by creating an IPv6-only network as described in

Empirically, it is commonly know that IPv4 addresses are used on the Internet, despite the guidance in RFC1958 (  One common example is enterprise VPN services.

Quantitatively, it is easy to generate information about how frequently IPv4 address literals are used on the web.

Several years ago, draft-wing-behave-http-ip-address-literals-02 shared a simple method to gathering the data and reported that 2.38% of the top 1 million web pages have IPv4 literals in their homepage.  I re-ran a similar test this week that yielded 1.39% of the top 1 million web pages having IPv4 literals.   My test, like Dan's, only looks at the HTML on the home page and thus under-counts literals in deeper links or loaded via CSS, XML, or Javascript. 

Through other manual testing  i have seen catastrophic failure in Amazon's video streaming service with them passing literals in XML.  Facebook passes IPv4 literals in Javascript, but is not impacting the page load.

I posted IPv4 literals found in the HTML homepage of these 2,220 of the top 100,000 Alexa domains here

My conclusion is that it is not feasible for a network operator to deploy a purely IPv6-only socket solution.  As has been previously explored, about 20% of the apps in the Google and Apple "app store" cannot function on IPv6-only + NAT64/DNS64.  Even if these "app stores" were policed to force all applications to be Address Family agnostic, it is not possible to police the world wide web or enterprise VPN or SIP Phones.  Any network operator that attempt to do this would have to accept at least 1.39% failure to the total web and perhaps accept failure to some major services (like Amazon streaming).

I believe we are beyond the point of blaming the world for using IPv4 literals / referrals.  We just need to be realistic that internet service requires IPv4 sockets, and thus RFC6144 #2.1 is not today a consumer grade service.  A wild guess is that it this situation will not meaningfully improve (risk reduced to ~0) for 10+ years.  



v6ops mailing list
Fernando Gont | 17 Mar 17:24 2015

Non-dest ASes dropping EH-enabledIPv6 packets


Since the question as been asked, hereby I'm providing a heads-up about
our finding regarding transit ASes that are filtering IPv6 EH-enabled
packets. -- sme folks my be willing to double-check with the relevant
people, and possibly inquire the reasoning behind the filtering decision.

Here we are just consider "best-case scenarios" (please see
draft-gont-ipv6-ehs-in-real-world for a definition of what we mean by that):

* Folks filtering packets with Dest Options EHs

             AS  2119 (TELENOR-NEXTEL Telenor Norge AS,NO)
             AS   286 (KPN KPN International / KPN Eurorings,NL)
             AS  3356 (LEVEL3 - Level 3 Communications, Inc.,US)

* Folks filtering packets with HBH EHs:
    (0.31%): AS   174 (COGENT-174 - Cogent Communications,US)
    (1.89%): AS   286 (KPN KPN International / KPN Eurorings,NL)
    (1.13%): AS  1273 (CW Cable and Wireless Worldwide plc,GB)
    (14.42%): AS  1299 (TELIANET TeliaSonera International Carrier,SE)
    (0.31%): AS  1764 (NEXTLAYER-AS next layer
Telekommunikationsdienstleistungs- GmbH,AT)
    (0.38%): AS  3257 (TINET-BACKBONE Tinet SpA,DE)
    (81.50%): AS  3356 (LEVEL3 - Level 3 Communications, Inc.,US)
    (0.31%): AS  5588 (GTSCE T-Mobile Czech Republic a.s.,CZ)
    (0.31%): AS  6695 (DECIX-AS DE-CIX Management GmbH,DE)
    (0.94%): AS  8218 (NEO-ASN Neo Telecoms S.A.S.,FR)
    (0.31%): AS  8447 (TELEKOM-AT A1 Telekom Austria AG,AT)
    (0.31%): AS  8767 (MNET-AS M-net Telekommunikations GmbH, Germany,DE)
    (0.31%): AS  9063 (SAARGATE-AS VSE NET GmbH,DE)
    (0.19%): AS 12389 (ROSTELECOM-AS OJSC Rostelecom,RU)
    (0.19%): AS 16086 (DNA DNA Oy,FI)
    (0.63%): AS 20676 (QSC-1 QSC AG,DE)
    (0.19%): AS 29470 (RETNNET-AS CJSC _RetnNet_,RU)
    (0.31%): AS 36623 (HGTLD - VeriSign Global Registry Services,US)

* Folks filtering Fragments

 AS  6939 (HURRICANE - Hurricane Electric, Inc.,US)
 AS 34779 (T-2-AS T-2, d.o.o.,SI)

FWIW, this post should not be considered a "wall of shame", but, if
anything, a "wall of reality".



Fernando Gont
SI6 Networks
e-mail: fgont@...
PGP Fingerprint: 6666 31C6 D484 63B2 8FB1 E3C4 AE25 0D55 1D4E 7492

v6ops mailing list

Tim Chown | 13 Mar 18:23 2015

Re: IPv6 EHs in the Real World (Fwd: New Version Notification for draft-gont-v6ops-ipv6-ehs-in-real-world-02.txt)

> On 13 Mar 2015, at 15:26, Nick Hilliard <nick <at>> wrote:
> On 13/03/2015 12:54, Fernando Gont wrote:
>> For the results we've published in the I-D, the tests were performed
>> from host Slovenia. Same results were obtained when redoing the tests
>> from Brazil.
>> And measuring things from the UK, and even (distributed) with the RIPE
>> probes gives the same results.  -- this is not surprising, since the
>> drops do not happen close to the origin, but rather at the Dest AS or
>> transit ASes.
> ok great.  I had assumed it was from multiple start points.  The source
> ASNs and any filtering between the source hosts and the upstream links
> (even if there is none) should be mentioned in the draft because it's an
> important part of the experiment methodology.

I agree. And a very fair point. There may be commonalities to certain points of origin. That said, it is
possible to benchmark local filtering by tests to specific external services that can be seen to work.

I’d add that not all results are black and white. As an example, you can see drop rates increase as certain
header chain lengths increase. 

Perhaps we can further clarify the methodology in response to comments so that we can have people run the
tests from a wider set of observation points. And/or release the code used.

We have held bar BoFs between those running the experiments at previous IETFs, e.g. London. We can do more of
that too, if people are interested.

v6ops mailing list
v6ops <at>
Fernando Gont | 13 Mar 14:26 2015

Re: IPv6 EHs in the Real World (Fwd: New Version Notification for draft-gont-v6ops-ipv6-ehs-in-real-world-02.txt)

Hi, Silvia,

On 03/13/2015 08:28 AM, Silvia Hagen wrote:
> I totally understand all those issues, limitations, practices and
> concerns. I am concerned about a tendency in the discussions to try
> to make up and adjust standards for vendor limitations, configuration
> issues, hardware bugs etc. And I think that is the wrong way to go. I
> do believe that when it comes to architectural things like EHs the
> working groups should stick to the design and the architecture and
> help and expect the market to implement it properly. It is not a
> working group/IETF problem if hardware have limitations, if firewall
> administrators decide to drop EHs etc.... We should stick to the
> original design and let this be their issue to solve by developing
> mechanisms which support the future protocol and its architecture. 

I sympathize with the idea. However, I should note that there are areas
where "proper implementation" doesn't seem to be on the horizon.
Specific example: support of IPv6 header chains of arbitrary lengths (at
least a packet full of EHs). Most of the architectures I recall checking
enforce a limit on theIPv6 header chain. -- and it seems to be a
"chicken and egg" problem: It's hard to believe that a router vendor
will put money into something that is currently not required by any
protocols (in practice). And of course you cannot relly on such
full-sized EH chains if there's no support for them.

FWIW, I'm not in favor of "banning" EHs, but I'm rather all for
improving the current state of affairs.


Fernando Gont
e-mail: fernando@... || fgont@...
PGP Fingerprint: 7809 84F5 322E 45C7 F1C9 3945 96EE A9EF D076 FFF1

v6ops mailing list

Tore Anderson | 13 Mar 10:36 2015

Stateful NAT64: Strange translation of ICMPv4 error source addresses


Stateful NAT64 (RFC6146) requires that whenever an ICMPv4 error is
received from a router in the IPv4 network, the (IPv4-converted) IPv6
source address of the translated ICMPv6 error must be set to the
(IPv4-converted) address of the destination address in the embedded
ICMPv4 payload, i.e., the original destination of the packet causing
the ICMPv4 error. I find this rather counter-intuitive - I would expect
the (IPv4-converted) address of the IPv4 router originating the ICMPv4
error to be used instead. Is my reading of RFC6146 faulty, or if not,
is this truly the intented behaviour (why?), or is an erratum warranted?

How does actual Stateful NAT64 implementations behave? [Cc: v6ops]

Longer version:

While testing Jool, an implementation of Stateful NAT64, I noticed that
traceroutes started from the IPv6 network towards IPv4 destinations
were looking decidedly odd. Some traceroute programs (e.g., mtr) would
report that any traceroute target (including definitively unreachable
destinations such as a martian) would be shown as successfully reached
on the hop immediately after the Stateful NAT64 router, while others
(e.g., tracroute6) would report each hop beyond the Stateful NAT64 as
having the same address of the traceroute target.

Examples of these odd traceroute outputs can be found in the bug report
I submitted about this error:

It turns out that first class of traceroute programs considers any
[ICMPv6] packet received, including "time exceeded" errors, originating
from the targeted address as a overall success. The second class of
traceroute programs will on the other hand continue to transmit packets
with an increasing Hop Limit until a non-ICMPv6 error (ICMPv6
echo-request, TCP SYN/ACK, etc. depending on type of traceroute) has
been receeived from the target.

After closer reading of RFC6146, both the Jool developers and I agree
that it appears that Jool is not buggy, as it faithfully implements the
algorithm in RFC6146. See below of a walk-through of our understanding
of the algorithm. However the resulting outcome makes absolutely no
sense from an operational point of view; I'd expect the IPv4 hops to be
represented using their IPv4-converted IPv6 address, in other words by
having the RFC6052 translation prefix prepended. This would have the
added benefit of causing the correct hostnames for each hop to be
reported if DNS64 is in use.

Towards the end of the initial message of the Github issue there is an
example traceroute output that looks more like what I'd expect, but on
the other hand that's not using a true Stateful NAT64 (RFC6146)
implementation, but rather the bastard child of TAYGA (a SIIT-ish
IPv4/IPv6 translator) plus NAPT44 (Linux IPTables), which might explain
that its outcome isn't RFC6146 compliant.

Walkthrough of Stateful NAT64 processing, re-using the example
addresses used in section 1.2.2 of RFC6146:

1) A TCP packet is sent from 2001:db8::1,1500 towards
64:ff9b::,80 through a Stateful NAT64 (whose IPv4 address is

The NAT64 picks an available source port (2000) and saves the following

(2001:db8::1,1500) <--> (,2000)

The resulting translated IPv4 packet gets the following addresses:


2) This packet ends up being discarded by the IPv4 router
for some reason (say, TTL exceeded). It originates an ICMPv4 error
packet that looks like this:

Src embedded in ICMPv4 payload:,2000
Dst embedded in ICMPv4 payload:,80

3) Upon receipt of the ICMPv4 error packet, the NAT64 determines an
"Incoming Tuple" according to section 3.4:

> If the incoming IP packet contains a complete (un-fragmented) ICMP
> error message containing a UDP or a TCP packet, then the incoming
> 5-tuple is computed by extracting the appropriate fields from the IP
> packet embedded inside the ICMP error message.  However, the role of
> source and destination is swapped when doing this: the embedded
> source IP address becomes the destination IP address in the incoming
> 5-tuple, the embedded source port becomes the destination port in the
> incoming 5-tuple, etc.

Thus, now we have a complete Incoming Tuple, containing the following:

SrcPort: 80
DstPort: 2000
Protocol: 6 (TCP)

Note that at this point the IPv4 source address of the IPv4 router
originating the ICMPv4 error ( is already lost, but I'll
complete the walkthrough for completeness anyway...

4) The NAT64 proceeds to compute the "Outgoing Tuple" for this packet,
according to section 3.6.1:

> The transport protocol in the outgoing 5-tuple is always the same as
> that in the incoming 5-tuple.  When translating from IPv4 ICMP to
> IPv6 ICMP, the protocol number in the last next header field in the
> protocol chain is set to 58 (IPv6-ICMP).

These two sentences do seem to me to contradict each other, but my
assumption is the latter sentences are to be understood as an exception
to the first sentence (otherwise the Protocol of the Outgoing Tuple
ends up being TCP, which would make zero sense). Thus the Outgoing
Tuple at this point contains:

Protocol: 58 (IPv6-ICMP)

> When translating in the IPv4 --> IPv6 direction, let the source and
> destination transport addresses in the incoming 5-tuple be (S,s) and
> (D,d), respectively.  The outgoing source transport address is
> computed as follows:

In other words, at this point we (temporarily) have:

(S,s) = (,80)
(D,d) = (,2000)

> The outgoing source transport address is generated from S using
> the address transformation algorithm described in Section 3.5.4.

Section 3.5.4 essentially says «see RFC6052», so:

Replacement S = 64:ff9b::

> The BIB table is searched for an entry (X',x) <--> (D,d), and if
> one is found, the outgoing destination transport address is set to
> (X',x).

This lookup will find the entry that was saved in step #1. Thus:

Replacement (D,d) = (2001:db8::1,1500)

So now we have a complete Outgoing Tuple:

SrcIP: 64:ff9b::
SrcPort: 80
DstIP: 2001:db8::1
DstPort: 1500
Protocol: 48 (IPv6-ICMP)

5) The Stateful NAT64 builds an IPv6 packet according to section 3.7:

> o  When translating an IP header (Sections 4.1 and 5.1 of [RFC6145]),
>    the source and destination IP address fields are set to the source
>    and destination IP addresses from the outgoing tuple as determined
>    in Section 3.6.


> o  When the protocol following the IP header is ICMP and it is an
>    ICMP error message, the source and destination transport addresses
>    in the embedded packet are set to the destination and source
>    transport addresses from the outgoing 5-tuple (note the swap of
>    source and destination).

Thus we end up with the final translated ICMPv6 packet:

Src: 64:ff9b:: <-- I'd expect 64:ff9b:: here!
Dst: 2001:db8::1 <-- makes sense
Src embedded in ICMPv6 payload: 2001:db8::1,1500 <-- makes sense
Dst embedded in ICMPv6 payload: 64:ff9b::,80 <-- makes sense


v6ops mailing list
v6ops <at>