Re: Review of draft-melnikov-smtp-priority-14
Alexey Melnikov <alexey.melnikov <at> isode.com>
2012-06-01 19:59:05 GMT
Hi Ned,
On 01/06/2012 17:18, Ned Freed wrote:
> A few comments are in order here.
>
>> > particularly important during emergencies for first responders and
>> > for environments such as military messaging, where messages have
>> high
>> > operational significance, and the consequences of extraneous delay
>> > can be significant.
>> >
>> > In order for an SMTP receiver to be able to send higher priority
>
>> send -> relay
>
> Given the imprecise nature of this document overall, it's a stretch to
> believe
> that being precise here is really necessary, but if it is, this fixes
> only part
> of the problem. The message may have been received via SMTP, SUBMIT, or
> something else. And the higher *or* *lower* priority may apply to
> whatever the
> next step is: SMTP, LMTP, or something else.
>
> If you want to be precise here, it needs to say something like:
>
> In order for an MSA, MTA, or MDA to be able to relay, deliver, or
> otherwise process higher priority messages first, ...
Yeah, I think this is less readable than what I have. I am sure readers
understand the meaning of what is being said.
>
>> > messages first, there needs to be a mechanism to communicate
>> (during
>> > both Message Submission [RFC6409] and Message Transfer
>> [RFC5321]) the
>> > priority of each message. This specification defines this
>> mechanism
>> > by specification of an SMTP [RFC5321] extension.
>> >
>> > Implementors of this document might also consider implementing
>> > [PRIORITY-TUNNELING] which talks about tunneling of Message
>> Transfer
>> > Priority information through Message Transfer Agents (MTAs) that do
>> > not support this extension, using a new message header field
>> > [RFC5322].
>
>> Consider replacing above paragraph with:
>
>> In order to permit end-to-end use of this extension across email
>> infrastructure that does not support it, a companion tunneling mechanism
>> is defined in [PRIORITY-TUNNELING] through use of a new message header
>> field [RFC5322].
>
> This is better. These specifications are not just for implementors.
Yes, already used in -15.
>> > 2. Conventions Used in This Document
>> >
>> > The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
>> > "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in
>> this
>> > document are to be interpreted as described in [RFC2119] when they
>> > appear in ALL CAPS. These words also appear in this document in
>> > lower case as plain English words, absent their normative meanings.
>
>> "when they appear in ALL CAPS". sigh. In other words, this document is
>> modifying RFC2119.
>
> On the contrary, all this is doing is being specific about the way RFC
> 2119 has
> actually been used ever since it was created. I can easily produce a
> long list
> of RFCs containing instances of these words in lower case where, even
> though it
> wasn't stated, the intent *clearly* was for lower case versions of
> these words
> to have their regular meanings.
>
> And even if I were to accept the notion that this document specifies a
> novel
> derivation of RFC 2119 (the notion that it *modifies* it being patently
> absurd), that ship has also sailed. I can also easily point to RFCs
> that do
> things like specify ALLS CAPS versions of these words have one
> meaning, Mixed
> Case another, and lower case yet another.
>
>> Does anyone seriously expect this new nuance of usage to be read and
>> remembered by average readers of this document?
>
> The better question is does anyone seriously expect anyone to assume,
> given
> abundant evidence to the contrary, that the lower case versions of
> these words
> will be taken to have RFC 2119 meaning?
>
>> FWIW, it took me about 3 minutes to make the relevant changes, below.
>> Case-sensitivity in semantics invites comprehension problems here.
>> Worse, there is absolutely no need or benefit in creating the problem
>> for this document. At a minimum, please do not try to modify IETF
>> document writing policy on the fly.
>
> You're the one who is arguing for a modification to the *actual* IETF
> document
> writing policy here.
Agreeing with that.
>> > The formal syntax use the Augmented Backus-Naur Form (ABNF)
>> [RFC5234]
>> > notation including the core rules defined in Appendix B of RFC 5234
>> > [RFC5234].
>> >
>> > In examples, "C:" and "S:" indicate lines sent by the client and
>> > server respectively. Line breaks that do not start with a new "C:"
>> > or "S:" exist for editorial reasons and are not a part of the
>> > protocol.
>> >
>> > This document uses the term "priority" specifically in relation to
>> > the internal treatment of a message by the server: messages with
>> > higher priorities may be given expedited handling, and those with
>
>> may -> can
>
>
>> > lower priorities may be handled only as resources become available.
>
>> may -> can
>
> Again, given the loosey-goosey nature of this document overall, I
> don't think
> precision is really required, but if it is, both of these changes are
> somewhat
> objectionable.
>
> Like it or not, "may" and "can" are *not* synonyms. "Can" is more
> assertive,
> and carries with it a flavor that's closer to "should", whereas "may" is
> expresses a possibilit with no preference.
>
> We're talking about prioritization methodology here, specifically
> expedited
> handling and deferred processing. There can be overhead associated
> with both of
> these, and we should not be encourage the use of these techniques when
> it isn't
> absolutely necessary.
> And yes, I'm being very pedantic here. That's what you're going to get
> when
> you elect to go down this path.
>
>> >
>> > 3. Definition of the Priority SMTP Extension
>> >
>> > The Priority SMTP service extension is defined as follows:
>> >
>> >
>> >
>> >
>> > Melnikov & Carlberg Expires November 25, 2012
>> [Page 3]
>> >
>> > Internet-Draft Message Transfer Priority SMTP Extension May
>> 2012
>> >
>> >
>> > 1. The textual name of this extension is "Priority Message
>> > Handling".
>> >
>> > 2. The EHLO keyword value associated with this extension is "MT-
>> > PRIORITY".
>> >
>> > 3. The EHLO keyword has no parameters. Parameters are reserved
>> for
>> > possible future extensions and MUST be ignored by clients that
>> > don't understand them.
>> >
>> > 4. No additional SMTP verbs are defined by this extension.
>> >
>> > 5. One optional parameter ("MT-PRIORITY") is added to the MAIL
>> FROM
>> > command. The value associated with this parameter is a decimal
>> > integer number from -9 to 9 (inclusive) indicating the priority
>> > of the email message. The syntax of the MT-PRIORITY
>> parameter is
>> > described by the <priority-value> ABNF non-terminal defined in
>> > Section 6. Higher numbers mean higher priority.
>
>> As a minor point, the form of -9 to 9 seems more complicated than
>> useful. Is there a need for 19 values? I'd think that 0 to 9 would be
>> sufficient.
>
> Well, to be fair, the specification only requires support for six
> distinct
> values:
>
> A message priority is a decimal integer in the range from -9 to 9
> (inclusive). SMTP servers compliant with this specification are not
> required to support all 19 distinct priority levels (i.e. to treat
> each priority value as a separate priority), but they MUST implement
> at least the following 6 distinct priority levels: -4, -2, 0, 2, 4,
> 9. I.e. an implementation that only supports the 6 priority levels
> will internally round up a syntactically valid level that isn't
> supported to the next higher supported number. For example, such an
> implementation will treat priority values below and including -4 as
> priority -4, priority -3 as priority -2, and all priorities starting
> from 5 are treated as priority 9. An SMTP server MAY support more
> than 6 different priority levels. Decision about which levels to
> support in addition to the 6 mentioned above is a local matter.
>
> Now, I have no idea what it actually means to "support" six values, but I
> assume it means that if you only make a processing distinction between
> five or
> fewer values total, if you support only one "lower than normal"
> priority, or
> you support fewer than three "higher than normal" priorities, you cannot
> implement this specification. (Note I'm assuming 0 is "normal" here - see
> below.)
More or less, yes.
>
> Of course there's a trivial way around this: You simply state that, as
> a matter
> of implementation policy, you only support X number and here's the
> mapping.
> This would allow, say, a system that only does the old X.400 non-urgent,
> normal, urgent thing to claim support.
Right. And there is already such a mapping in Appendix B.
>> For that matter, why are many values needed? What is the basis for
>> choosing to have a range of more than a few values?
>
> IMO this is one of the rare cases where X.400 was spot-on correct.
>
>> This appears to be the only place the provides semantics to the priority
>> number. As such, it should be more thorough. The critical issue in
>> assigning the number, I think, is trying to get independent originators
>> to assign numbers according roughly the same criteria. For example, if
>> I label "average" messages I send as as 0 and you label them as 1, then
>> your messages get preferential treatment over mine, although that was
>> not your intent. (Assuming you're not trying to game the service...)
>
>> This issue goes to the core of the usage model for the feature: It
>> really needs an environment tailored to its use, with all participants
>> not only within the same trust system, but sharing the same priority
>> assignment model/policy. It's probably good for this document to permit
>> a range of assignment models, but should note that an operational
>> requirement for use of the extension is that an assignment policy be
>> defined for all participants.
>
>> This document might, then, offer a candidate model, to be use by default
>> and absent one specific to an environment.
>
> Well, this goes to the problem I have with the entire document - I
> have no real
> idea what the policy associated with these priorities is supposed to
> be. We
> have a prioritization capability in our product and I can tell you
> what it
> does, but s to whether or not it satisfies the intended target of this
> specification, I realy couldn't say.
I hope I clarified this in -16. Also look at Appendix A and Appendix B.
> More generally, in a modern, high performance MTA that's capable of
> processing
> large numbers of messages simultaneously, there's a significant problem
> actually implementing mechanisms that are guaranteed to preferentially
> process
> certain messages.
>
> For example, suppose I have several dozen threads/processes/whatever
> connected
> up to a given system, all busy transferring large messages. A higher
> priority
> message comes in. If I shove this message on the thread that is the
> closest to
> being done, there may still be a significant delay. OTOH, if I start a
> new
> connection to handle it, what if I hit the limit on the number of
> connections
> the remote MTA will accept from me?
>
> Put another way, if you assume a fairly strict policy is needed, your
> point
> about this only being applicable in an environment that's specifically
> tailored
> for it is, if anything, an massive understatement. For this to provide
> a real
> priority boost with high reliability it's actually necessary to closely
> coordinate operational policies across all involved ADMDs. And not to
> put too
> fine a point on it, the level of technical acumen needed to actually
> do this
> sort of thing is something I've observed to be the exception and not
> the rule.
>
> All that said, I think this actually argues for putting in some
> statements
> as to what this document does *not* do. It calls for best effort to
> provide preferential processing, nothing more. It does *not* provide
> any sort of guarantees.
Fair enough, I will add some text about that. Best effort was certainly
the intent.
> And if that's not the case, then this needs to be experimental.
>
>> > 6. The maximum length of a MAIL FROM command line is increased
>> by 15
>> > octets by the possible addition of a space, the MT-PRIORITY
>> > keyword and value.
>> >
>> > 7. The MT-PRIORITY extension is valid for the submission service
>> > [RFC6409] and LMTP [RFC2033].
>> >
>> > 4. Handling of messages received via SMTP
>> >
>> > This section describes how a conforming SMTP server should
>> handle any
>> > messages received via SMTP.
>> >
>> > 4.1. Handling of the MT-PRIORITY parameter by the receiving SMTP
>> server
>> >
>> > The following rules apply to SMTP transactions in a server that
>> > supports the MT-PRIORITY parameter:
>> >
>> > 1. A conforming SMTP server MUST NOT refuse a MAIL FROM command
>> > based on the absence of the MT-PRIORITY parameter.
>
>> hmmm. For some operational environments, it will be essential to have
>> /all/ handling conform to the priority policy model in force for the
>> environment. In those cases, the absence of the parameter would be a
>> violation of the spec.
>
> Interesting point. I'm not sure what, if anything, to do about it.
>
>> > 2. If any of the associated <esmtp-value>s (as defined in Section
>> > 4.1.2 of [RFC5321]) are not syntactically valid, or if there is
>> > more than one MT-PRIORITY parameter in a particular MAIL FROM
>> > command, the server MUST return an error, for example "501
>> syntax
>> > error in parameter" (with 5.5.2 Enhanced Status Code
>> [RFC2034]).
>> >
>> > 3. When inserting a Received header field as specified in Section
>> > 4.4 of [RFC5321], the compliant MTA/MSA SHOULD include the
>> > "PRIORITY" clause whose syntax is specified in Section 6.
>> >
>> >
>> >
>> > Melnikov & Carlberg Expires November 25, 2012
>> [Page 4]
>> >
>> > Internet-Draft Message Transfer Priority SMTP Extension May
>> 2012
>> >
>> >
>> > 4. If the sending SMTP client specified the MT-PRIORITY
>> parameter to
>> > the MAIL FROM command, then the value of this parameter is the
>> > message priority.
>> >
>> > 5. If no priority has been determined by the above, the server may
>
>> may -> can
>
>
>> > use its normal policies to set the message's priority. By
>> > default, each message has priority 0.
>
>> ouch. This chooses a particular model for meaning of the numbers, and
>> without having defined the model beforehand.
>
> I for one was assuming 0 was the normal, default priority. But now
> that you
> mention it, at least this needs to be made explicit.
I believe the document already says that, but I will double check.
>> That is, I think the problem with this default is that it really is
>> assuming a particular policy for meaning of the values, namely that
>> average mail is mid-priority, assuming the numbers have linear
>> semantics.
>
> That sure was what I was assuming.
Yes.
>> > The SMTP server MUST NOT allow upgraded priorities from untrusted
>> > (e.g. unauthenticated) or insufficiently trusted sources. (One
>> > example of an "insufficiently trusted source" might be an SMTP
>> sender
>> > which authenticated using SMTP AUTH, but which is not explicitly
>> > whitelisted to use the SMTP MT-PRIORITY service.) The server MAY,
>
>> It is good that this issue is raised, but it really requires an earlier,
>> separate and careful discussion of the need for this extension to be
>> used within an administered trust environment. Given such an
>> introductory section, the above text would be sufficient.
>
> Agreed.
Ok.
>> In its absence, the reference to a whitelist is based on the very large
>> assumption that participants in the extension are whitelisted. This is
>> not document elsewhere.
>
> True.
Hmm, Ok.
>> > however, allow an untrusted source to lower its own message's
>> > priorities -- consider, for example, an email marketer that
>> > voluntarily sends its marketing messages at low priority.
>
>> To beat the point a bit deader: this assumes a particular policy for
>> the meaning of the priority values. Worse, it also appears to
>> contradict the earlier default of 0, but perhaps I've misunderstood
>> that.
>
> I think that's a misunderstanding.
I already replied in another message that it is.
> I generally agree with the remainder of your comments, although I
> think some
> of them become less important as long as it's clear this doesn't provide
> guaranteees, but they still should be addressed.
I believe I addressed most of them in -15 and -16. I might need to add a
couple of clarifying sentences as per above.