Ned Freed | 17 Aug 16:33 2014

Re: utf8 messages

> I haven’t been very involved with EAI, so please indulge my attempt to
> restate the problem to see if I understand it correctly: a 6532 message is
> *implicitly* of type message/global, whereas a non-6532 message is implicitly
> of type message/rfc822.  That implicit state is specified is inferred from the
> presence or absence of 8-bit characters.

With the proviso that those characters have to be valid utf-8 and have
to appear in specific parts of a header.

It's also the case that the SMTPUTF8 marker provided by EAI is not a reliable
indicator of a RFC 6532 message: It may be set for a message/rfc822 message
that happens to have had EAI addresses in the envelope at some point, and it
may be clear for a random 8bit message that happens to meet RFC 6532 criteria.
Indeed, the SMTPUTF8 transport flag's purpose is to try and keep EAI material
from getting into places where it won't be understood, not to identify RFC 6532
messages.

It's even possible for nonconformant messages containing non-utf-8 8bit to
arrive with the SMTPUTF8 bit set. For example someone might forward such a p	
message to someone with an EAI address.

>  Brandon and others have problems because there are non-conformant messages
> with 8-bit characters that are not 6532 messages, most often because they
> use Windows-1250 instead of UTF-8.  Is that a correct restatement?

> How does Google (or anyone else) tell that a message is cp-1250 instead of
> UTF-8?  Can we specify a clear algorithm for detection?

I can't speak to how Google does it, but the way we handle it is to apply a set
of heuristics. Obvious ones include a check for utf-8 syntax validity (the
(Continue reading)

Brandon Long | 11 Aug 22:45 2014
Picon

utf8 messages

In our recent launch of support for EAI, we noticed an issue with 6532 "utf8" messages.

As near as I can tell, there is nothing about a 6532 message which tells you it is such a message... except the existence of 8bit characters in the headers.  Ie, 7bit -> 5322, 8bit -> 6532.

Our problem is that this isn't actually true in practice.  Prior to launching support for 6532 messages, we've already had to support widespread use of 8bit messages that were not always in utf8.  Since these typically didn't specify which charset they were in, we used a variety of techniques including direct charset detection on such messages.

The problem we're having with 6532 messages, is that we moved from explicitly identified charsets via 2047/etc mechanisms, to "its just utf8"... and sometimes we mis-detect the utf8 as cp1250 or other encodings.

Now, we can work on improving our detection and maybe start biasing it to utf8 or even just assuming utf8 for any 8bit message which is in interchange valid utf8.  Anything we do there will result in some potential for mistakes, of course.

This would all be solved if 6532 messages were actually denoted as such, and I recall seeing at least one such X header used by another service we've been interoperability testing with:
X-CM-HeaderCharset: UTF-8
CM no doubt standing for CoreMail, which is the software used:
X-Mailer: Coremail Webmail Server Version XT3.0.4 build
 20140526(27182.6409.6185) Copyright (c) 2002-2014 www.mailtech.cn coremail

Thoughts?  It looks like there was a i-Email/Header-Type originally, but was removed early in the utf8smtp timeframe: http://www.ietf.org/mail-archive/web/ima/current/msg01358.html
The general consensus for removal seemed to be "you'll know because it was specified at SMTP time", "just look for 8bit" and "its bad to duplicate data between the envelope and the headers".

Looks like it goes nearly to the beginning of the utf8smtp time frame:

It seems that the pre-existence of 8bit messages was not considered by those who felt it wasn't necessary, as least as far as I've read in the discussions (wow do I wish the mhonarc had been updated with an easier to explore/read model)

Now, as hinted at in the consensus to remove such a marker from the draft, we can certainly add such a header when composing 6532 messages or when we receive any message via SMTPUTF8 for our own utility, but I would think there would be some utility in such a marker being mutually understood and shared.

Brandon
_______________________________________________
ietf-822 mailing list
ietf-822 <at> ietf.org
https://www.ietf.org/mailman/listinfo/ietf-822
Douglas Otis | 8 Jun 03:05 2014
Picon

Re: WSJ/gmail/ML, was a permission to...


On May 5, 2014, at 1:19 AM, Michael Richardson <mcr+ietf <at> sandelman.ca> wrote:

> John Levine <johnl <at> taugh.com> wrote:
> [...]
>> Remember that there are other things that DMARC broke, that do not
>> involve forwarding.  That's what "WSJ/gmail" in the subject line are
>> about.
> 
> Yes, I understand.  I'm not sure that web sites sending on my behalf
> is ultimately distinguishable from spam without some cryptographic route
> through the browser/webserver into my MUA.
> 
> I can imagine such a channel, but I don't think we want to talk about that.
> I think that we should list the various problems that we have discovered and
> it may be that some we can fix, and some we can not.
> 
> Ultimately, in the space of end-to-end SMTP, lists are a form of
> intermediary.  DMARC is a form of BCP38...

Dear Michael,

Cryptography will help secure many things, perhaps none better than SMTP-DANE.  But can a sending domain
make request without providing information necessary to avoid exposure of private exchanges by way of
insecure DMARC feedback. Talk about a major security hole. 

Email is currently being accepted based on fairly weak validation schemes.  DKIM does not even indicate who
sent the message nor is it expected to indicate to whom it was intended.  It seems that when there are strong
methods to validate email sources, there should be a way for a DMARC strategy to make authoritative
exceptions. DKIM Delegate seems to assume sending domains know which destinations can be trusted to the
point of offering a weak signature.  Although it does not leverage DNS it also can not address important use
cases and further weakens already weak protections that even failed to exclude invalid header field from
producing PASS.  Since DMARC is expecting others to act on their behalf, it should also directly convey
which non-aligned domains have been used by their domain's u
 sers.

This communication can be an authorization of validated sources.  The granularity of the authorization
can be by domain.  Any further resolution of the source will depend on trusting message handling. 
Authorization might include a requirement  an Original Authentication Results header be included, for example.

Regards,
Douglas Oits

_______________________________________________
ietf-822 mailing list
ietf-822 <at> ietf.org
https://www.ietf.org/mailman/listinfo/ietf-822

Brandon Long | 6 May 19:34 2014
Picon

Re: WSJ/gmail/ML, was a permission to...




On Mon, May 5, 2014 at 7:03 AM, John R Levine <johnl <at> taugh.com> wrote:
Those two problems can be solved in different ways.  Gmail could use a
third party's submission server just like they use its pop/imap one.

Gmail does allow you to use a third party submission server, and it looks
like we may have to encourage its use even more in the future.

I'm not sure how realistic that is in practice for users who aren't uber-nerds.

To set up to use Yahoo's submission server from Gmail, I tried to configure it in the popup Gmail provided, which failed with an error message that told me to go log in at Yahoo.  I did, didn't help.  After some poking around I found a message in my Yahoo inbox that suggested I needed an app specific password.  (How many people will realize that Yahoo considers Gmail to be an app?)  It provided a link to the place in their credential server to create such a password, which is otherwise not easy to find.  So I finally found it, and made a password for Gmail, and then went back to Gmail, and used it, and indeed it worked.

But how many people without CS degrees are going to be able to go through all that?

Yes, it runs up against that other problem, that username/passwords are no longer near useful enough.  In Gmail's case, you would only need an application specific password if you have one-time passwords enabled on your account.  The error message when trying to use your password would give you to this url: https://support.google.com/accounts/answer/185833 which should be enough for some non-CS types to figure it out.  Its less than ideal, however.

But, even then, we have http://googleonlinesecurity.blogspot.com/2014/04/new-security-measures-will-affect-older.html which means that trying to use a password on an account isn't going to work all that well in the future.  OAUTH2 SASL is almost an RFC, but using it still has scaling issues for clients, in that there is no discovery/registration protocol yet.  Theoretically, once all that is accomplished and implemented, trying to authorize smtp-msa from one account to another via the web would be as simple as an ACL pop-up that you can agree to.

Clearly, that level of interop is a bit further away than we'd want any solution to the DMARC issue.

Brandon

_______________________________________________
ietf-822 mailing list
ietf-822 <at> ietf.org
https://www.ietf.org/mailman/listinfo/ietf-822
Paul Smith | 6 May 12:12 2014
Picon

Re: WSJ/gmail/ML, was a permission to...


On 05/05/2014 07:01, Brandon Long wrote:
>
>
> Gmail does allow you to use a third party submission server, and it 
> looks like we may have to encourage its use even more in the future.

We're thinking that as well, or just stopping forwarding services at all.

The problem with forwarders using third party services' submission 
servers is that the user would need to give out their login details - 
which is a big issue. Maybe users should be able to create extra sets of 
authentication details for their submission servers, but ONLY for mail 
going to their own email addresses, for forwarding servers to use. These 
could still let the messages be blocked by a spam filter, but would let 
the receiving server know that the messages are coming from a forwarding 
server, so (a) the forwarding server's reputation should not be affected 
by spam from there, and (b) some checks like DKIM might have to be 
ignored/loosened.

I suppose that a scheme like this could potentially be extended to 
mailing lists - when you subscribe, you give authentication details as 
well as an email address.

-

Paul Smith Computer Services
Tel: 01484 855800
Vat No: GB 685 6987 53

_______________________________________________
ietf-822 mailing list
ietf-822 <at> ietf.org
https://www.ietf.org/mailman/listinfo/ietf-822

Miles Fidelman | 5 May 20:42 2014
Picon

Re: WSJ/gmail/ML, was a permission to... (off-topic)

S Moonesamy wrote:
> Hi Miles,
> At 04:51 04-05-2014, Miles Fidelman wrote:
>> No. If I have a secretary make 20 copies of a letter, and drop it off 
>> at the post office, I'm still the author of the message.  A 
>> secretary, forwarder, mailing list, what have you, is acting on my 
>> behalf, as my agent.  The letter is from me, and the return address 
>> should be mine.
>
> Which MUA would you recommend for the secretary to send a message on 
> behalf of an author?
>

Well, I was using physical mail handling as an analogy, but... Outlook 
supports delegated access rights for sending on behalf of, as well as 
scheduling meetings on behalf of another mail user. Would a recommend 
Outlook - that's a separate question :-)

_______________________________________________
ietf-822 mailing list
ietf-822 <at> ietf.org
https://www.ietf.org/mailman/listinfo/ietf-822

Brandon Long | 5 May 20:32 2014
Picon

Re: one can re-sign without a permission to re-sign header




On Mon, May 5, 2014 at 1:50 AM, Paul Smith <paul <at> pscs.co.uk> wrote:
On 02/05/2014 15:42, John R Levine wrote:
I don't see any replay protection in here at all. Nothing that says to keep the signature expiration relatively short, and nothing which a mailing list recipient could not subsequently use to send spam. The first issue just needs a mention. It's the second issue that needs to be addressed IMO:

Yeah, that occurred to me about five minutes after I posted it. Here's a tweaked version where the mf tag is now mf=list.domain, with handwaving about how a may-forward signature doesn't count unless there's also a signature from the list domain.  Given lengthy discussions about how little abuse comes from real mailing lists, that'd probably be adequate.

http://datatracker.ietf.org/doc/draft-levine-may-forward/
Could this be 'extended' to include message-ids in the MF signature?

That would provide some replay protection, especially if the forwarder checks for duplicate message-ids (the recipient could also check for dupes). Without it, I could see one of your messages on a list, then send messages to everyone on the list, pretending to be you.

I was wondering if we wanted a new canonicalization which would allow for the Subject to be included but stripped of "standard" MLM subject prefixes, ie something similar to the reply_regexp in mutt (http://www.mutt.org/doc/devel/manual.html#reply-regexp)

Brandon

_______________________________________________
ietf-822 mailing list
ietf-822 <at> ietf.org
https://www.ietf.org/mailman/listinfo/ietf-822
John Levine | 6 May 04:01 2014

Re: one can re-sign without a permission to re-sign header

>That would provide some replay protection, especially if the forwarder 
>checks for duplicate message-ids (the recipient could also check for 
>dupes). Without it, I could see one of your messages on a list, then 
>send messages to everyone on the list, pretending to be you.

You could, but now we're back to whether we believe that list managers
act to keep crud out of their lists.  In general, I observe that they
do, so I don't see any point to adding features that assume that
managers will just sit there and allow subscribers to abuse their
lists.

Anecdote: I am on one non-technical list where there is an extremely
obnoxious person who chronically poisons the dialog, yet the list
managers are unwilling to eject.  (My guess is that he is a large
donor to the organization.)  I have dealt with him by bozo filtering
his mail in procmail, but other people keep writing to me and asking
isn't there list software that will let subscribers decide which other
subscribers' mail to get, i.e. do the bozo filtering in the list
software.  As far as I can tell, no, because this is not a problem
that many lists have.

R's,
John

_______________________________________________
ietf-822 mailing list
ietf-822 <at> ietf.org
https://www.ietf.org/mailman/listinfo/ietf-822

Pete Resnick | 18 Apr 20:43 2014

Mailing lists - assumptions

[Bcc'ing dmarc, but directed to ietf-822, since that's where we appear 
to be having the discussion for the moment.]

These ideas about mailing lists have been rattling around in my head 
these past couple of days, and they're based on a bunch of design 
assumptions. So I figured I'd post my list of assumptions and see if 
anybody thought any of them were off in space.

1. The mailing list itself is going to have to participate in this in 
some way. There's no point in trying to design something for mailing 
lists that simply will not make any modifications.

2. In the end, we want mailing lists to be able to send messages that 
say "From: user <at> originatingdomain.example.com" and not have to say 
"From: list <at> listdomain.example.net".

3. If an originator sends mail to a mailing list, the originator is 
implicitly giving permission for the list to re-distribute the message 
"From:" the originator.

4. If an originating site allows its users sending mail to mailing lists 
at all, the site is OK with *any* mailing list re-distributing mail from 
its users. so long as the mailing list received the mail directly from 
the originating user through the originating site. That is, originating 
sites don't care about pre-vetting mailing lists; they just care that 
the mail sent by mailing lists came directly from their users.

5. For a recipient of mailing list mail, their site cares about whether 
the message they got came directly from the mailing list site, cares 
that the mailing list got the mail directly from the originating user's 
site, and cares that the mailing list got the mail relatively recently. 
For the most part, the recipient's site doesn't care how much has 
changed about the content of the message. The eventual recipient might 
care if the changes are in the extreme, but from a "is this spoofed 
spam" perspective, that really doesn't matter.

6. The mailing list cares about whether it got the message directly from 
the originating user's site.

7. An originating site would be willing to query (through a DNS lookup 
or otherwise) the first hop recipient for any message and stick 
something in the message that indicates, "This message came from 
originating user's site and was sent to recipient at such-and-so time", 
in order to facilitate #4 and #5.

8. The mechanism we use might need to chain: If I send to a mailing list 
A, which itself sends to another mailing list B, the eventual recipient 
will be able to see that the message it got came directly from B, which 
it got from A, which it got from me.

Anything I screwed up there? Any assumption I'm missing?

pr

--

-- 
Pete Resnick<http://www.qualcomm.com/~presnick/>
Qualcomm Technologies, Inc. - +1 (858)651-4478

_______________________________________________
ietf-822 mailing list
ietf-822 <at> ietf.org
https://www.ietf.org/mailman/listinfo/ietf-822

John Levine | 18 Apr 04:19 2014

A permission to re-sign header

As I understand it, the original sender puts a hard to forge single
use token in the message, which the forwarder can include in the
signed message.

Since I am lazy, I will reuse DKIM key records and invent a new
May-Resign header something like this:

May-Resign: f=marissam <at> yahoo.com; r=ietf.org; s=foo; a=rsa-sha256; \
   t=1397786669; b=hashhashhash

This is a permission to re-sign for a message From:
marissam <at> yahoo.com, to be re-signed by a mailing list at ietf.org. The
s= and a= and t= are the same as DKIM, the b= is a signature of a hash
of the M-R header, similar to the b= signature in a DKIM-Signature.

The relay includes the M-R header in the DKIM signature.  So now
we modify DMARC to say that

IF there is a M-R header with f= that matches the From: line address,

AND the M-R header is included in a DKIM signature that is signed with
d= that matches the M-R r=

AND the M-R signature validates using the s= selector and f= domain

AND the t= isn't too old (for some meaning of too old)

THEN the message is considered to be aligned.

Is that the general idea?

You could put an M-R header on anything, but if you want to limit it
to mail to addresses that claim to be mailing lists, you could use the
same name convention as the DANE S/MIME draft, with hashed mailboxes,
e.g.:

<hash of ietf-822>._mayresign.ietf.org TXT "v=MR1; d=ietf.org"

That says the ietf-822 <at> ietf.org list is signed with d=ietf.org.  If a
domain contains only mailing lists, you can use a wildcard

*._mayresign.lists.iecc.com TXT "v=MR1; d=lists.iecc.com"

R's,
John

_______________________________________________
ietf-822 mailing list
ietf-822 <at> ietf.org
https://www.ietf.org/mailman/listinfo/ietf-822

Vasiliy Faronov | 18 Feb 15:15 2014
Picon

List-Id on the receiving end

Hi,

I'm looking for information on what receiving software does anything with the List-Id header as defined in RFC 2919. I'm especially interested in whether (and how) they process the list description phrase (section 3) if it's included.

Thanks for any pointers. (Not subscribed so please Cc.)


--
_______________________________________________
ietf-822 mailing list
ietf-822 <at> ietf.org
https://www.ietf.org/mailman/listinfo/ietf-822

Gmane