Lisa's Apps Area Activity for July 2007

Fwd: Last Call: draft-saintandre-rfc4622bis (Internationalized Resource Identifiers (IRIs) and Uniform Resource Identifiers (URIs) for the Extensible Messaging and Presence Protocol (XMPP)) to Proposed Standard

[Fwd: Notes from VCardDAV BOF]

Dear all,

Attached minutes that Lisa took for the vcarddav BOF at IETF 69.  The 
BoF chairs would like to thank those who participated, and in 
particular, the presenters.  If you attended, could you please take a 
moment to review Lisa's notes?  These will be the basis for the BoF minutes.

Regards,

Eliot

From: Lisa Dusseault <lisa <at> osafoundation.org>
Subject: Notes from VCardDAV BOF
Date: 2007-07-26 21:11:43 GMT

Re: Straw-man charter for http-bis

http://www.w3.org/Protocols/HTTP/1.1/rfc2616bis/issues/#i74

On 10/06/2007, at 6:05 PM, Martin Duerst wrote:

> - RFC 2616 prescribes that headers containing non-ASCII have to use
>   either iso-8859-1 or RFC 2047. This is unnecessarily complex and
>   not necessarily followed. At the least, new extensions should be
>   allowed to specify that UTF-8 is used.

--
Mark Nottingham     http://www.mnot.net/

Character encodings in headers [i74][was: Straw-man charter for http-bis]

On 10/06/2007, at 6:05 PM, Martin Duerst wrote:
> - RFC 2616 prescribes that headers containing non-ASCII have to use
>   either iso-8859-1 or RFC 2047. This is unnecessarily complex and
>   not necessarily followed. At the least, new extensions should be
>   allowed to specify that UTF-8 is used.

My .02;

I'm concerned about allowing UTF-8; it may break existing  
implementations.

I'd like to see the text just require that the actual character set  
be 8859-1, but to allow individual extensions to nominate encodings  
*like* 2047,without being restricted to it. For example, the encoding  
specified in 3987 is appropriate for URIs. However, it *has* to be  
explicit; I've heard some people read this requirement and think that  
they need to check *every* header for 2047 encoding.

So, I think this means;

1) Change
   "Words of *TEXT MAY contain characters from character sets other  
than ISO-8859-1 [22] only when encoded according to the rules of RFC  
2047 [14]."
to
   "Words of *TEXT MUST NOT contain characters from character sets  
other than ISO-885901 [22]."
and,

2) Identify headers that may have non-8859 content and explicitly say  
how to encode them (IRI, 2047, whatever; the existing ones will have  
to be 2047, I believe), modifying their BNF to suit.

3) When we document extensibility, require new headers to nominate  
any encoding explicitly.

--
Mark Nottingham     http://www.mnot.net/

Re: Character encodings in headers [i74][was: Straw-man charter for http-bis]

Mark Nottingham wrote:
> On 10/06/2007, at 6:05 PM, Martin Duerst wrote:
>> - RFC 2616 prescribes that headers containing non-ASCII have to use
>>   either iso-8859-1 or RFC 2047. This is unnecessarily complex and
>>   not necessarily followed. At the least, new extensions should be
>>   allowed to specify that UTF-8 is used.
>
> My .02;
>
> I'm concerned about allowing UTF-8; it may break existing
> implementations.
concur.  though at least it is possible to distinguish utf-8 from 8859-1. 

also, I'll note that supporting utf-8 in a way that is backward
compatible with existing implementations is almost certainly more
complex (and thus more costly, error-prone, etc) than supporting rfc 2047.
>
> I'd like to see the text just require that the actual character set be
> 8859-1, but to allow individual extensions to nominate encodings
> *like* 2047,without being restricted to it. For example, the encoding
> specified in 3987 is appropriate for URIs. However, it *has* to be
> explicit; I've heard some people read this requirement and think that
> they need to check *every* header for 2047 encoding.
2047 was specifically not intended for use with protocol elements that
have meaning to protocol engines.  how many HTTP headers contain text
that is intended solely for human use?

Re: Character encodings in headers [i74][was: Straw-man charter for http-bis]

--On Monday, 20 August, 2007 13:40 +1000 Mark Nottingham
<mnot <at> mnot.net> wrote:

> On 10/06/2007, at 6:05 PM, Martin Duerst wrote:
>> - RFC 2616 prescribes that headers containing non-ASCII have
>> to use either iso-8859-1 or RFC 2047. This is unnecessarily
>>   complex and not necessarily followed. At the least, new
>>   extensions should be allowed to specify that UTF-8 is used.
> 
> My .02;
> 
> I'm concerned about allowing UTF-8; it may break existing
> implementations.

And whatever is done about it should be consistent with the EAI
work.  Otherwise, we are likely to find ourselves in big trouble
going down the line.

> I'd like to see the text just require that the actual
> character set be 8859-1, but to allow individual extensions to
> nominate encodings *like* 2047,without being restricted to it.
> For example, the encoding specified in 3987 is appropriate for
> URIs. However, it *has* to be explicit; I've heard some people
> read this requirement and think that they need to check
> *every* header for 2047 encoding.

Sigh.  My own sense is that, going forward, we need to lose
8859-N, not make it the default (or only) character set for more
protocols.  It is, to put it mildly, a little Euro-centric (and
not even completely suitable for Europe).  Much of the advantage
of Unicode is that one does not need to designate/ nominate a
particular CCS or encoding and then maintain state for it... and
that is a fairly large advantage.  See also
draft-klensin-unicode-escapes-03.txt(probably expired, but you
should be able to find a copy somewhere -- I'll get back to it
sometime soon) for a discussion of issues in ASCII encoding of
multioctet character sets.   The IRI spec may constrain things
to encoding of octets, but that doesn't make it a good idea.

If we are going to consider changes in this area, let's make
them improvements.  Locking in 8859-1 is not an improvement: it
would, IMO, be better to deprecate its use and require explicit
charset designation always if that is the only choice.

     john

Re: Character encodings in headers [i74][was: Straw-man charter for http-bis]

John C Klensin said:
> If we are going to consider changes in this area, let's make
> them improvements.  Locking in 8859-1 is not an improvement: it
> would, IMO, be better to deprecate its use

+1

--

-- 
Clive D.W. Feather  | Work:  <clive <at> demon.net>   | Tel:    +44 20 8495 6138
Internet Expert     | Home:  <clive <at> davros.org>  | Fax:    +44 870 051 9937
Demon Internet      | WWW: http://www.davros.org | Mobile: +44 7973 377646
THUS plc            |                            |

Re: Character encodings in headers [i74][was: Straw-man charter for http-bis]

Hello Mark,

Thanks for giving this an issue number.

At 12:40 07/08/20, Mark Nottingham wrote:
>On 10/06/2007, at 6:05 PM, Martin Duerst wrote:
>> - RFC 2616 prescribes that headers containing non-ASCII have to use
>>   either iso-8859-1 or RFC 2047. This is unnecessarily complex and
>>   not necessarily followed. At the least, new extensions should be
>>   allowed to specify that UTF-8 is used.
>
>My .02;
>
>I'm concerned about allowing UTF-8; it may break existing  
>implementations.
>
>I'd like to see the text just require that the actual character set  
>be 8859-1, but to allow individual extensions to nominate encodings  
>*like* 2047,without being restricted to it.

What do you mean by "encodings *like* 2047"? And why do you think
that UTF-8 may break existing implementations? UTF-8 has virtually
the same footprint in terms of bytes as ISO-8859-1: All bytes
above 0x7F may be used. Implementations that have to deal with
ISO-8859-1 usually do this by just being 8-bit-transparent;
that works for UTF-8, too.

If your opinion is that UTF-8 cannot be allowed at all, then
that's going to be a problem for cases where it's already in
use, see e.g. earlier posts in the http list.

It's easy to say "may break existing implementations",
but in over 10 years of being involved in the Web, I haven't
heard about that happening. If you or anybody have, please
speak up.

>For example, the encoding  
>specified in 3987 is appropriate for URIs.

As one of the authors of RFC 3987, I know what you mean, but
"the encoding specified in 3987" wouldn't be enough for a spec.
Also, it's not really very well suited to the job, because
%hh-encoding is used to escape any bytes, not only UTF-8,
and there is a considerably length increase for some scripts.

>However, it *has* to be  
>explicit; I've heard some people read this requirement and think that  
>they need to check *every* header for 2047 encoding.

I have read it that way, too. If it can be safely argued that
it was never intended that way, and that no harm is produced
if this is restricted, then I'd befine with restricting it,
because checking everything for 2047 is indeed tough, but
I'd really like to make sure that this isn't creating problems.
(i.e. that a careful examination of the various headers makes
some reasonably conservative assumptions).

>So, I think this means;
>
>1) Change
>   "Words of *TEXT MAY contain characters from character sets other  
>than ISO-8859-1 [22] only when encoded according to the rules of RFC  
>2047 [14]."
>to
>   "Words of *TEXT MUST NOT contain characters from character sets  
>other than ISO-885901 [22]."
>and,
>
>2) Identify headers that may have non-8859

There are many parts to ISO-8859, not just ISO-8859-1.

>content and explicitly say  
>how to encode them (IRI, 2047, whatever; the existing ones will have  
>to be 2047, I believe), modifying their BNF to suit.
>
>3) When we document extensibility, require new headers to nominate  
>any encoding explicitly.

If that includes UTF-8, I'd be fine with it. If it excludes
UTF-8, I think that would be a problem.

Regards,    Martin.

#-#-#  Martin J. Du"rst, Assoc. Professor, Aoyama Gakuin University
#-#-#  http://www.sw.it.aoyama.ac.jp       mailto:duerst <at> it.aoyama.ac.jp     

Re: Character encodings in headers [i74][was: Straw-man charter forhttp-bis]

At 15:56 07/08/20, Keith Moore wrote:
>Mark Nottingham wrote:
>> On 10/06/2007, at 6:05 PM, Martin Duerst wrote:
>>> - RFC 2616 prescribes that headers containing non-ASCII have to use
>>>   either iso-8859-1 or RFC 2047. This is unnecessarily complex and
>>>   not necessarily followed. At the least, new extensions should be
>>>   allowed to specify that UTF-8 is used.
>>
>> My .02;
>>
>> I'm concerned about allowing UTF-8; it may break existing
>> implementations.
>concur.  though at least it is possible to distinguish utf-8 from 8859-1. 

In practice indeed this can be done with high reliability; please
see http://www.ifi.unizh.ch/mml/mduerst/papers/PDF/IUC11-UTF-8.pdf
for details. For iso-8859-1, see in particular p. 21.

>also, I'll note that supporting utf-8 in a way that is backward
>compatible with existing implementations is almost certainly more
>complex (and thus more costly, error-prone, etc) than supporting rfc 2047.

Well, if "backwards compatible" means also supporting RFC 2047,
then that's a tautology. If the choice is between UTF-8 and RFC 2047,
however, then I'd take UTF-8 any time, because RFC 2047 includes
UTF-8 as well as many other encodings.

Regards,    Martin.

#-#-#  Martin J. Du"rst, Assoc. Professor, Aoyama Gakuin University
#-#-#  http://www.sw.it.aoyama.ac.jp       mailto:duerst <at> it.aoyama.ac.jp