4409 to STD

Hi, how about moving RFC 4409 to STD ?  Chapter 8.1 has to be
updated, at the moment it says:

| The MSA MAY add or replace the 'Sender' field, if the
| identity of the sender is known and this is not given
| in the 'From' field.

| The MSA MUST ensure that any address it places in a 'Sender'
| field is in fact a valid mail address.

Proposal:

: The MSA MAY add or replace the 'Sender' or 'Resent-Sender'
: field, if the identity of the sender is known and this is
: not given in the 'From' field.

: For a mail without 'Resent-From' field the MSA adds or
: replaces a 'Sender' field to reflect the known identity.

: For a mail with one 'Resent-From' field the MSA adds or
: replaces a 'Resent-Sender' field, if the 'Resent-From'
: field does not already reflect the known identity.

: For a mail with two or more 'Resent-From' fields the MSA
: verifies that the first (top-down) 'Resent-From' field
: reflects the known identity.  Where that's not the case
: it replaces the first 'Resent-Sender' field before the
: second 'Resent-From' field, if that exists.  Otherwise
: it adds a 'Resent-Sender' field reflecting the known

SMTP headers

MIME part header (was: SMTP headers)

PDeRochambeau <at> iht.com wrote:

> I have noticed that the former sends "attachment" headers
> which don't contain the attachments' contents :

> ...
> --------14507F4DC80F4D6511158149340B40B01
> Content-Disposition: attachment; filename="test.txt"
> Content-Transfer-Encoding: 7bit
> 
> Content-Type: Text/Plain
> 
> --------14507F4DC80F4D6511158149340B40B01
> ...

Sounds like a bug, for starters there should be no empty line
before the Content-Type.  

> --------14507F4DC80F4D6511158149340B40B01
> Content-Disposition: attachment; filename="test.txt"
> Content-Transfer-Encoding: 7bit
> 
> Content-Type: Text/Plain
> 
> This is the contents of the file
> --------14507F4DC80F4D6511158149340B40B01

Same issue here, but you said it works for you, so maybe the
bogus empty line is a problem with your UA.

> Does anyone know how to force RTEs or MTAs such as sendmail
> to send "attachment" headers of the second kind?

For a plain text US-ASCII part you generally need no header
fields at all (for the default inline disposition).  As you
said you want it as attachment, the minimal header would be:

--------14507F4DC80F4D6511158149340B40B01
Content-Disposition: attachment; filename="test.txt"

This is the contents of the file
--------14507F4DC80F4D6511158149340B40B01
Content-Disposition: attachment; filename="verbose.txt"
Content-Transfer-Encoding: 7bit
Content-Type: text/plain; charset=us-ascii

This is another file after a more verbose header
--------14507F4DC80F4D6511158149340B40B01

Frank

Re: SMTP headers

At 09:38 14-09-2006, PDeRochambeau <at> iht.com wrote:
>we currently use a RTE (Reliable Transport Engine) to send emails 
>containing attachments. Although the attachments appear within the 
>e-mails' bodies as icons, in "heavy" mail clients, such as Lotus 
>Notes, Apple's Mail, etc., they don't show up in "light" mail 
>clients such as Web mail clients (www.excite.com, www.free.fr, etc.) 
>By comparing SMTP headers sent by our RTE and those sent by Mutt on 
>Linux, I have noticed that the former

That should be MIME headers.

>sends "attachment" headers which don't contain the attachments' contents :
>
>...
>--------14507F4DC80F4D6511158149340B40B01
>Content-Disposition: attachment; filename="test.txt"
>Content-Transfer-Encoding: 7bit
>
>Content-Type: Text/Plain

There shouldn't be a new line before the Content-Type line.  There 
file content was not "attached" in this version.

>Does anyone know how to force RTEs or MTAs such as sendmail to send 
>"attachment" headers of the second kind?

This is not a MTA or sendmail problem.  Sendmail doesn't send 
"attachment" headers.  Your RTE may not be formatting the message 
correctly before sending it to the MTA.  Refer to MIME for how the 
message body should be formatted.

Regards,
-sm 

Re: MIME part header (was: SMTP headers)

Re: MIME part header (was: SMTP headers)

On Thu September 14 2006 17:03, Philippe de Rochambeau wrote:

> --------14506F0F080F4D6511158082800B418B1
> Content-Disposition: attachment; filename="test.txt"
> Content-Transfer-Encoding: 7bit
> Content-Type: Text/Plain
> 
> 
> --------14506F0F080F4D6511158082800B418B1
> Content-Disposition: inline
> Content-Transfer-Encoding: 7bit
> Content-Type: Text/Plain
> 
> 
> 
> *****
> THIS IS AN AUTOMATIC EMAIL.
> ****
> 
> 
> 
> 
> --------14506F0F080F4D6511158082800B418B1--
> 
> 
> 
> Do the contents of email attachment always appear in the headers?

No, never (by definition).  The header is a sequence of lines which precedes
body (or MIME-part) content, separated from that content by an empty line.

> Why   
> are the contents of "test.txt" not showing up above the second boundary?

Some possibilities:
1. A bug in the generating program
2. An empty file

[...]
> Why does the last boundary have two dashes at the end?

It is a closing delimiter.  See RFC 2046 section 5.1.1.

> Is this some   
> kind of bug?

No.

Re: MIME part header (was: SMTP headers)

On Thu September 14 2006 13:47, Frank Ellermann wrote:

> For a plain text US-ASCII part you generally need no header
> fields at all (for the default inline disposition).

There is no such thing as "default inline disposition".

A MIME-part has inline disposition if and only if it contains a
Content-Disposition MIME-part header field specifying inline disposition.

In the absence of a Content-Disposition field, quoting RFC 2183:
"   Content-Disposition is an optional header field. In its absence, the
   MUA may use whatever presentation method it deems suitable.
"

N.B. the default behavior is a characteristic of the specific MUA in use, not
of the MIME-part.

Also note that specified but unrecognized dispositions are treated as
equivalent to "attachment", again quoting RFC 2183:
"  Unrecognized disposition
   types should be treated as `attachment'. The choice of `attachment'
   for unrecognized types is made because a sender who goes to the
   trouble of producing a Content-Disposition header with a new
   disposition type is more likely aiming for something more elaborate
   than inline presentation.
"

On a related note, a MIME-part with no corresponding Content-Disposition
header field does not have semantics of "attachment", regardless of what
'KANA Response' and some other MUAs may label them.  More pointedly, a
MIME-part explicitly labeled as "inline" disposition most certainly does not
have semantics of "attachment", such broken MUAs' claims to the contrary
notwithstanding.

Re: MIME part header

Bruce Lilly wrote:

>> For a plain text US-ASCII part you generally need no header
>> fields at all (for the default inline disposition).

> There is no such thing as "default inline disposition".

> A MIME-part has inline disposition if and only if it contains
> a Content-Disposition MIME-part header field specifying
> inline disposition.

I'd consider a MUA not displaying US-ASCII text/plain as plain
text as broken, YMMV.

>| a sender who goes to the trouble of producing a
>| Content-Disposition header with a new disposition type is
>| more likely aiming for something more elaborate than inline
>| presentation.

Yes, you'd need a Content-Disposition: attachment if you don't
want the normal behaviour for "no Content header field at all".

> a MIME-part with no corresponding Content-Disposition
> header field does not have semantics of "attachment",
> regardless of what 'KANA Response' and some other MUAs may
> label them.

+100 (factor 100 for "KANA", I recall that from the SWEN-times
before MyDoom)

Frank

disposition vs. media type (Was Re: MIME part header)

On Mon September 18 2006 03:29, Frank Ellermann wrote:
> 
> Bruce Lilly wrote:

> > A MIME-part has inline disposition if and only if it contains
> > a Content-Disposition MIME-part header field specifying
> > inline disposition.
> 
> I'd consider a MUA not displaying US-ASCII text/plain as plain
> text as broken, YMMV.

Media type (text/plain, etc.) and disposition (inline, attachment, etc.)
are distinct characteristics of a MIME-part.

Incidentally, RFC 2049 specifies that MIME-conforming UAs must treat
text/plain (and all other media types) with an unrecognized transfer
encoding as application/octet-stream [sect. 2, paragraph labeled "(3)"].
The same section, paragraph "(6)" states that if an unrecognized charset
is specified, text media types should be treated as application/octet-stream
(RFC 2046 states that behavior only if the subtype is also unrecognized).

So an MUA which displays US-ASCII text/plain with an unrecognized
content transfer encoding as application/octet-stream (rather than
text/plain) is MIME-conforming, and one which displays it as text/plain
is not MIME-conforming.  Perhaps you consider MIME-conforming behavior
to be broken?

So while there is interaction between transfer encoding and media type,
and between charset and media type for text subtypes, disposition and
media type are orthogonal.  Mind you, the expected behavior for
application/octet-stream and for disposition "attachment" is similar;
moreover application/octet-stream with a disposition of "inline" doesn't
make much sense.

Getting back you your statement, Frank, US-ASCII text/plain with a
specified disposition of "attachment" should never be displayed inline
by a 2183-conforming UA.  US-ASCII text/plain with no specified
disposition can be displayed inline or not by an MUA.  One which does
not display it inline would be unusual, but also unusually safe, as it is
not above the ethics of spammers, virus writers and other miscreants to
mislabel other content as text/plain (and it apparently is beyond the
sensibility of some UA programmers to avoid decoding and displaying
mislabeled HTML, including scripting content, putatively on the grounds
that users want to have their computers infected with viruses, worms, etc.),
or to imbed ECMA 48 escape sequences, etc. in text.

Re: disposition vs. media type

Bruce Lilly wrote:

>>> A MIME-part has inline disposition if and only if it
>>> contains a Content-Disposition MIME-part header field
>>> specifying inline disposition.

>> I'd consider a MUA not displaying US-ASCII text/plain as
>> plain text as broken, YMMV.

> Media type (text/plain, etc.) and disposition (inline,
> attachment, etc.) are distinct characteristics of a MIME-part

Where they are specified.  Your objection was to my remark
about a part with no Content header field at all.  In that
case it's either the default message/rfc822 for a part of
multipart/digest, or default text/plain US-ASCII 7bit for
all other multiparts (RFC 2046 5.1.1)

For a simple message/rfc822 without Content header fields
that's ordinary mail, no tricks, no traps, no attachment -
unless you and your MUA wish to look for say UUE, that's
your private business.

> Incidentally, RFC 2049 specifies that MIME-conforming UAs
> must treat text/plain (and all other media types) with an
> unrecognized transfer encoding as application/octet-stream

An unrecognized CTE isn't the same as no explicit CTE.  The
default is 7bit (RFC 2045 6.1).  For unrecognized subtypes
of text the default type is text/plain (RFC 2046 4.1.4), and
any charset other than US-ASCII must be specified (4.1.2).

 From that I guess that UAs could be free to panic if they
find  an 8bit octet in wannabe-default text/plain US-ASCII,
and enter "unrecognized CTE mode".  But that's not what you
found in RFC 2049, where "unrecognized" IMO means "the UA
doesn't know the explicitly specified CTE", e.g. CTE x-yenc.

Clearly there must be an explicit CTE as soon as there are
8bit bytes in mail (RFC 2046 4.1.2 on page 8)

> The same section, paragraph "(6)" states that if an
> unrecognized charset is specified, text media types should
> be treated as application/octet-stream

Yes, I'll test it with this message, let's see what our
UAs do.

> RFC 2046 states that behavior only if the subtype is
> also unrecognized

Again 4.1.4, I see it.  With that text/foo; charset=bar
is a convoluted way to get application/octet-stream.

If that's an intentional difference from RFC 2049 I miss
the point.  An attempt: text/html; charset=bar  For a
recognized text subtype there could be a more elaborated
fallback, e.g. Latin-1 in that case.  Because RFC 2049
discusses "minimal conformance" such cases are out of
scope.

> Perhaps you consider MIME-conforming behavior to be broken?

I just don't confuse unrecognized and unspecified... <beg>

> while there is interaction between transfer encoding and
> media type, and between charset and media type for text
> subtypes, disposition and media type are orthogonal.

If specified.  Otherwise there are sound defaults with the
unsurprising effect to display plain text ASCII as plain
text ASCII.

> application/octet-stream with a disposition of "inline"
> doesn't make much sense.

That's up to the UA, it could start a hex. viewer <shrug />

In reality we want it to ask if it should save this as a
file, if we care to name an application to do something
with it, or if we want to just ignore / skip / trash it.

And ideally (in my parallel universe) it's never accepted
by the MSA or MX, let the sender figure out how to get it
right.

> US-ASCII text/plain with a specified disposition of
> "attachment" should never be displayed inline by a
> 2183-conforming UA.

Certainly it shouldn't without asking, if it supports
RFC 2183 (not covered by RFC 2049 for obvious reasons).

That's again a case of a _specified_ disposition, and
your objection was to my remark about no Content header
field at all.

Apparently that's the definition of an "attachment" in
RFC 2183:  no automatic display without asking.  Display
after asking is okay.  Makes sense, e.g. for a text/html
attachment you could decide to display it as plain text.
For some attached text/plain source code you could wish
to look at it before you save it as file.  For ordinary
mail worms checking if the raw B64 starts with TV or UE
(= boring, delete) is also okay.

> US-ASCII text/plain with no specified disposition can
> be displayed inline or not by an MUA.  One which does
> not display it inline would be unusual, but also
> unusually safe, as it is not above the ethics of
> spammers, virus writers and other miscreants to mislabel
> other content as text/plain

That's IMO not unusually safe but unusually paranoid.

If an UA runs on top of a terminal emulation where weird
escape sequences can do bad things, then the "security
considerations" for the spammers would be "if you try
this add an explicit Content-Disposition: inline".  Zero
advantage for the users of this paranoid UA, but more
gibberish annoying users of pre-MIME UAs and/or modems.

> it apparently is beyond the sensibility of some UA
> programmers to avoid decoding and displaying mislabeled
> HTML, including scripting content

True, but my vision of displaying plain text is "as is".
If possible without crashing at NUL bytes (my UA does
this).  Wrt NO-WS-CTL we discussed this already for some
years "elsewhere" (BTW, get your LC comments in shape 

Frank