headers
Hector Santos | 1 Aug 2010 01:10

Re: Yahoo Mail Filtering


Very odd. I don't see how yahoo's "shred" logic or deferrals promoting 
10+ retries before it finally accepts, and they all seem to finally 
make it, helps anyone.

Thanks for the link.

-- 
Sincerely

Hector Santos
http://www.santronics.com

Laura Atkins wrote:
> 
> On Jul 31, 2010, at 11:22 AM, Hector Santos wrote:
> 
>> J.D. Falk wrote:
>>
>>> It's not the wikipedia definition of greylisting, but they do use 4xx replies to control the flow of
inbound mail.
>> Ok, now I am starting to see 45x "grey listings", here is snap shot of a mail queue:
> 
> <snip>
> 
>> There are multiple retries spread across different machines.
>>
>> Do you think they need the same exact IP or they support a class C
>> machine distribution?
>
(Continue reading)

Permalink | Reply | 0 comments |
headers
Dave CROCKER | 10 Aug 2010 02:10

Processing after the end of DATA


G'day.

It's Monday.  I'm feeling aggressive...

Once upon a time, it was deemed important to respond after crlf.crlf as quick as 
possible.  Pausing for additional processing of varying complexity and delay was 
deemed unacceptable.

There seems to be an emerging constituency that thinks this stricture is not 
applicable to modern email services.

I did a 10-second scan of 5321 and didn't find the relevant text, but I'd like 
to resolve -- and preferably squash -- this issue quickly.

Can folks offer pointers to normative language?

Thanks.

d/

ps.  Yes, there is a differential to my aggressiveness.
--

-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net
Permalink | Reply | 84 comments |
headers
Steve Atkins | 10 Aug 2010 02:25

Re: Processing after the end of DATA


On Aug 9, 2010, at 5:10 PM, Dave CROCKER wrote:

> 
> G'day.
> 
> It's Monday.  I'm feeling aggressive...
> 
> Once upon a time, it was deemed important to respond after crlf.crlf as quick as possible.  Pausing for
additional processing of varying complexity and delay was deemed unacceptable.
> 
> There seems to be an emerging constituency that thinks this stricture is not applicable to modern email services.
> 
> I did a 10-second scan of 5321 and didn't find the relevant text, but I'd like to resolve -- and preferably
squash -- this issue quickly.
> 
> Can folks offer pointers to normative language?

Tail end of section 6.1 of 5321:

"To avoid receiving duplicate messages as the result of timeouts, a
   receiver-SMTP MUST seek to minimize the time required to respond to
   the final <CRLF>.<CRLF> end of data indicator. "

However the same section also says:

  "if there is a delivery failure after acceptance of a message, the
   receiver-SMTP MUST formulate and mail a notification message"

If you're doing any useful level of spam / virus filtering then your
(Continue reading)

Permalink | Reply | 78 comments |
headers
Dave CROCKER | 10 Aug 2010 02:30
Favicon

Re: Processing after the end of DATA


On 8/9/2010 5:25 PM, Steve Atkins wrote:
>> Can folks offer pointers to normative language?
>
> Tail end of section 6.1 of 5321:
>
> "To avoid receiving duplicate messages as the result of timeouts, a
>     receiver-SMTP MUST seek to minimize the time required to respond to
>     the final<CRLF>.<CRLF>  end of data indicator. "
>
> However the same section also says:

Thanks!

And, yeah, it might be worth reviewing both normative directives.

d/

--

-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net
Permalink | Reply | 50 comments |
headers
Tony Hansen | 10 Aug 2010 02:36
Picon
Favicon

Re: Processing after the end of DATA


On the flip side is this:

4.5.3.2.6.  DATA Termination: 10 Minutes.

    This is while awaiting the "250 OK" reply.  When the receiver gets
    the final period terminating the message data, it typically performs
    processing to deliver the message to a user mailbox.  A spurious
    timeout at this point would be very wasteful and would typically
    result in delivery of multiple copies of the message, since it has
    been successfully sent and the server has accepted responsibility for
    delivery.  See Section 6.1 for additional discussion.

     Tony Hansen
     tony <at> att.com

On 8/9/2010 8:30 PM, Dave CROCKER wrote:
>
>
>
> On 8/9/2010 5:25 PM, Steve Atkins wrote:
>>> Can folks offer pointers to normative language?
>>
>> Tail end of section 6.1 of 5321:
>>
>> "To avoid receiving duplicate messages as the result of timeouts, a
>>     receiver-SMTP MUST seek to minimize the time required to respond to
>>     the final<CRLF>.<CRLF>  end of data indicator. "
>>
>> However the same section also says:
(Continue reading)

Permalink | Reply | 48 comments |
headers
Dave CROCKER | 10 Aug 2010 02:40
Favicon

Re: Processing after the end of DATA


On 8/9/2010 5:36 PM, Tony Hansen wrote:
> On the flip side is this:
>
> 4.5.3.2.6. DATA Termination: 10 Minutes.

I can understand thinking that this is a 'flip' side, but as I recall, there was 
different intent that drove the two-three rules.

The core problem with doing any serious processing after crlf.crlf is that it 
can take an unbounded amount of time, plus Internet delays can get interesting. 
  All of which can run things up to the timeout when that's not intended.

A big timeout is not meant to mean that one can take lots of time...

d/

--

-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net
Permalink | Reply | 3 comments |
headers
Carl S. Gutekunst | 10 Aug 2010 02:55

Re: Processing after the end of DATA


Tony Hansen wrote:
>
> On the flip side is this:
>
> 4.5.3.2.6.  DATA Termination: 10 Minutes.
>
>    This is while awaiting the "250 OK" reply.

There's a world of difference between the obligation of the server to 
emit a timely response, and the obligation of the client to not give up 
early.

Dave, unfortunately I'm not good at the normative references, but I am 
long on experience here. My observation at Postini was that most clients 
give up after a much shorter time. 5 minutes was common, but some were 
as little as 30 seconds. Postini uses a proxy, so the amount of time 
after the dot was the sum of all latencies, including the receiver and 
all of the network delays from the sender to the proxy, and from the 
proxy to the receiver.

As a result, duplicates were such a staggering huge problem. To mitigate 
that, Postini computed a SHA-1 hash for every message, and if there was 
any evidence that the receiver had sent the 250 OK, but the sender never 
got it, the hash was saved in a database. If the sender resent the 
message, the proxy (knowing it had already been delivered) quietly ate it.

When that database broke, the customers noticed immediately. The effect 
was that significant.
(Continue reading)

Permalink | Reply | 43 comments |
headers
John C Klensin | 10 Aug 2010 03:11

Re: Processing after the end of DATA


--On Monday, August 09, 2010 17:40 -0700 Dave CROCKER
<dcrocker <at> bbiw.net> wrote:

> 
> 
> 
> On 8/9/2010 5:36 PM, Tony Hansen wrote:
>> On the flip side is this:
>> 
>> 4.5.3.2.6. DATA Termination: 10 Minutes.
> 
> I can understand thinking that this is a 'flip' side, but as I
> recall, there was different intent that drove the two-three
> rules.
> 
> The core problem with doing any serious processing after
> crlf.crlf is that it can take an unbounded amount of time,
> plus Internet delays can get interesting.   All of which can
> run things up to the timeout when that's not intended.
> 
> A big timeout is not meant to mean that one can take lots of
> time...

That is certainly how I've always understood it.

   john
Permalink | Reply | 0 comments |
headers
John Levine | 10 Aug 2010 03:18

Re: Processing after the end of DATA


>Tail end of section 6.1 of 5321:
>
>"To avoid receiving duplicate messages as the result of timeouts, a
>   receiver-SMTP MUST seek to minimize the time required to respond to
>   the final <CRLF>.<CRLF> end of data indicator. "

That language is taken verbatim from 2821.  In both RFCs the following
sentence refers to RFC 1047.

RFC 1047 describes a problem Craig Partridge observed when an SMTP
server did lots of work after the dot, and the client gave up and went
away in the meantime.  The client assumed the delivery failed and it
had to resend later, but the server eventually said OK, so the message
got delivered twice.

I believe this was a problem on CSNET when he wrote the RFC in 1988.
It is my impression that MTAs have gotten someone more robust in the
ensuing two decades.  Is it still a problem now?

For a while I used a very aggressive greylister which soft failed
after the dot, kept a hash of the message, and wanted to see an
exact copy with the same hash to let the message through.  I stopped
using it, but not because of SMTP problems.  (A surprising number of
ESPs reconstruct the message rather than retransmitting it, so the
hash didn't match.)

Anyone got actual experience with delays at the dot?  It wouldn't be
hard for me to stick a module into my SMTP daemon that inserts extra
delays and see what breaks.
(Continue reading)

Permalink | Reply | 26 comments |
headers
Dave CROCKER | 10 Aug 2010 03:36

Re: Processing after the end of DATA


On 8/9/2010 6:18 PM, John Levine wrote:
> I believe this was a problem on CSNET when he wrote the RFC in 1988.

No, John, it was not just CSNet, nor did CSNet over the SMTP net display 
differential behavior.

> It is my impression that MTAs have gotten someone more robust in the
> ensuing two decades.  Is it still a problem now?

cf, Carl's note.

d/

--

-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net
Permalink | Reply | 0 comments |

Gmane