RE: How reliable is it to block/reject on SPF fail?

-----Original Message-----
From: Stuart D. Gathman [mailto:stuart <at> bmsi.com] 
Sent: Tuesday, 1 December 2009 6:52 AM
To: spf-discuss <at> v2.listbox.com
Subject: Re: [spf-discuss] How reliable is it to block/reject on SPF
fail?

While this might be ok for a casual user, I don't want my mail
traversing
google or any other provider in plaintext.  I suppose I could use S/MIME
or GPG, but TLS is no effort privacy measure that doesn't involve trying
to
tell non-geek correspondents how to create a key pair.  When a
correspondent has a small office mail server supporting TLS, adequate
privacy
is automatic.  
---------

This is OT, but the REAL solution to THAT problem is Federated Identity
Management. No Passwords, certificates only.

Resolving MFROM/HELO conflicts

Here is a little nit that wasn't addressed in RFC4408.  If HELO SPF says 
to reject, but SPF for MAIL FROM says Pass, which takes precedence?  For 
spfv1, I think we are stuck with "receiver policy" (especially since 
checking HELO is optional).  Should we specify a precedence for spfv3?
Make HELO check a MUST?  Or keep HELO optional, but give precedence to 
MFROM? 

Here is an example for the latter.  Set SPF for HELO to "v=spfv3sdg -all".  
This has the effect of saying "this server only legitimately sends
MFROM with SPF" (with MFROM taking precedence).

We would probably need to specify the whole matrix of MFROM vs HELO
SPF results.

--

-- 
	      Stuart D. Gathman <stuart <at> bmsi.com>
    Business Management Systems Inc.  Phone: 703 591-0911 Fax: 703 591-6154
"Confutatis maledictis, flammis acribus addictis" - background song for
a Microsoft sponsored "Where do you want to go from here?" commercial.

Re: Resolving MFROM/HELO conflicts

Stuart D. Gathman wrote:
> Here is a little nit that wasn't addressed in RFC4408.  If HELO SPF says 
> to reject, but SPF for MAIL FROM says Pass, which takes precedence?  For 
> spfv1, I think we are stuck with "receiver policy" (especially since 
> checking HELO is optional).  Should we specify a precedence for spfv3?
> Make HELO check a MUST?  Or keep HELO optional, but give precedence to 
> MFROM? 
>
> Here is an example for the latter.  Set SPF for HELO to "v=spfv3sdg -all".  
> This has the effect of saying "this server only legitimately sends
> MFROM with SPF" (with MFROM taking precedence).
>
> We would probably need to specify the whole matrix of MFROM vs HELO
> SPF results.
>
>   
The HELO check should be mandatory, and should take precedence over the 
MFROM check.  There is no "forwarding problem" (or any other excuse for 
failure) with the HELO check.  Furthermore, all the "bells and whistles" 
in an SPF record should not apply to the HELO check.  It should be a 
simple Pass/Fail, with an immediate SMTP REJECT on Fail.

This is the only way I can see SPF will ever fulfill its original 
promise of eliminating domain name forgery.

--

-- 
************************************************************     *
* David MacQuigg, PhD    email: macquigg at ece.arizona.edu   *  *
* Research Associate                phone: USA 520-721-4583   *  *  *
* ECE Department, University of Arizona                       *  *  *

Re: Resolving MFROM/HELO conflicts

At 18:00 13/01/2010  Wednesday, Stuart D. Gathman wrote:
>Here is a little nit that wasn't addressed in RFC4408.  If HELO SPF says 
>to reject, but SPF for MAIL FROM says Pass, which takes precedence?  For 
>spfv1, I think we are stuck with "receiver policy" (especially since 
>checking HELO is optional).  Should we specify a precedence for spfv3?
>Make HELO check a MUST?  Or keep HELO optional, but give precedence to 
>MFROM? 
>
>Here is an example for the latter.  Set SPF for HELO to "v=spfv3sdg -all".  
>This has the effect of saying "this server only legitimately sends
>MFROM with SPF" (with MFROM taking precedence).
>
>We would probably need to specify the whole matrix of MFROM vs HELO
>SPF results.

I hate to say it but on this we could take a leaf from M$

ie in spf3
add manditory /scope

v=spf3/helo rest of record...
or
v=spf3/mfrom rest of record...

or for those wanting to send from and helo as the same domain
v=spf3/helo/mfrom or spf3/mfrom/helo

but i would suggest not allowing that and  forcing either
v=spf3/helo
or/and

Re: Resolving MFROM/HELO conflicts

> At 18:00 13/01/2010  Wednesday, Stuart D. Gathman wrote:
> >Here is a little nit that wasn't addressed in RFC4408.  If HELO SPF says 
> >to reject, but SPF for MAIL FROM says Pass, which takes precedence?  For 

On Wed, 13 Jan 2010, alan wrote:
> add manditory /scope
> 
> v=spf3/helo rest of record...
> or
> v=spf3/mfrom rest of record...
> ... additional useful discussion that misses the original issue

Your proposal doesn't change the fact that HELO and MFROM may give
opposing results.  The scope might be a good idea, but HELO != MFROM
domain in most cases anyway, so it is not an issue for this thread.

MacQuig suggests making HELO mandatory with precedence.  (MUST check HELO,
if result is anything other than Pass, overall SPF result is Fail.)

My suggestion was to make HELO optional with MFROM having precedence.

Here is one possible matrix:

    		MFROM
HELO		None	Neutral	Softfail Fail	PermErr	TempErr
None		None	Neutral	Softfail Fail	PermErr	TempErr
Neutral		Fail	Fail	Fail	 Fail	PermErr	TempErr
SoftFail	Fail	Fail	Fail	 Fail	PermErr	TempErr
Fail		Fail	Fail	Fail	 Fail	PermErr	TempErr
PermErr		PermErr	PermErr	Softfail Fail	PermErr	TempErr

what is problem with this list ? I am receiving 1/5 messages maybe

Re: Resolving MFROM/HELO conflicts

At 22:39 13/01/2010  Wednesday, Stuart D. Gathman wrote:
>> At 18:00 13/01/2010  Wednesday, Stuart D. Gathman wrote:
>> >Here is a little nit that wasn't addressed in RFC4408.  If HELO SPF says 
>> >to reject, but SPF for MAIL FROM says Pass, which takes precedence?  For 
>
>On Wed, 13 Jan 2010, alan wrote:
>> add manditory /scope
>> 
>> v=spf3/helo rest of record...
>> or
>> v=spf3/mfrom rest of record...
>> ... additional useful discussion that misses the original issue
>
>Your proposal doesn't change the fact that HELO and MFROM may give
>opposing results.  The scope might be a good idea, but HELO != MFROM
>domain in most cases anyway, so it is not an issue for this thread.
>
>MacQuig suggests making HELO mandatory with precedence.  (MUST check HELO,
>if result is anything other than Pass, overall SPF result is Fail.)
>
>My suggestion was to make HELO optional with MFROM having precedence.

well heres the part of the {yes it was long} mail that stated my preference

"and it would be hoped that all spfv3 checking functions would always check helo
if fail thats it!
if pass or assumed pass due to no record then continue on to check the mfrom details"

my justification is if the owner of the HELO domain explicitly claims it is forged,
who cares if the ratware is within the ISP of the forged envelope-sender 
{as the envelope-senders often have to allow large sweeps of IP's due to not having direct control of which
IPS their ISP might designate to outgoing mail service}

>Here is one possible matrix:
>
>                MFROM
>HELO            None    Neutral Softfail Fail   PermErr TempErr  Pass
>None            None    Neutral Softfail Fail   PermErr TempErr  ?
>Neutral Fail    Fail    Fail    Fail    PermErr TempErr  ?
>SoftFail        Fail    Fail    Fail    Fail    PermErr TempErr  ?
>Fail            Fail    Fail    Fail    Fail    PermErr TempErr  ?
>PermErr PermErr PermErr Softfail Fail   PermErr TempErr  ?
>TempErr TempErr TempErr Softfail Fail   PermErr TempErr  ?
>Pass            Neutral Neutral Softfail Fail   PermErr TempErr  ?

so to re-use your matrix i would suggest {if using scopes and not allowing ~all ?all in helo scope}
                MFROM
                None    Neutral Softfail Fail   PermErr TempErr  pass
HELO
None            None    Neutral Softfail Fail   PermErr TempErr  pass
Fail            Fail    Fail    Fail     Fail   Fail    Fail     fail
PermErr PermErr PermErr PermErr Fail    PermErr TempErr  pass
TempErr TempErr TempErr TempErr Fail    TempErr TempErr  pass
Pass            Neutral Neutral Softfail Fail   PermErr TempErr  pass

but personally i would see no reason for 1 result for two entirely {logically} unrelated tests
I currently report to users MUA's HELO-SPF-PASS and HELO-SPF-NONE and reject at rcpt-to if HELO-SPF-FAIL
{without ever looking-up the mail-from}
I only process the mail-from identity SPF if the HELO is already non-fail {i do not fail on neutral or
softfail but in 3 years have never seen one {and i have a special logwatcher looking}}
this is reported to the user as a standard SPF result header {few users reject on mfrom-spf-fail due to badly
setup spfs/forwarders/and invite spam {that they seem to want}

but with helo domain its fairly cut n dry it either is one of the outbound ip's of that server/cluster or it isn't

thats why i think separate the two results, even separate the utility/syntax for checking/reporting them.
thus if helo spf fails, receivers can mark the stream for later reject with out any more overhead 
{of looking up mail-froms mx's spf rhdnsbl etc}
but in the case of a known false negative result {say incompetent ISP}, the receiver can learn of it and
whitlist this ip against helo spf checks

then later the mfrom spf checks are done
like now in spfv1 allow mfrom through macros and redirects to succeed/fail based on helo value 
{but if the spf record doesn't reference the helo value directly it dosn't effect the result of the mfrom-spf-check}

then {for those receivers who reject nothing} have a standard header reporting the HELO-spf status FAIL/PASS/NONE/PE/TE
and the mfrom-spf-status of FAIL/PASS/SOFTFAIL/NEUTRAL/NONE/PE/TE

but as never should the helo success/pass result be dependant on anything but its ip 
my server name doesn't become a forgery just because an unexpected envelope-sender appears on the email
conversely a forgery of my server name doesn't become legit because an envelope-senders SPF

>-- 
>              Stuart D. Gathman <stuart <at> bmsi.com>
>    Business Management Systems Inc.  Phone: 703 591-0911 Fax: 703 591-6154
>"Confutatis maledictis, flammis acribus addictis" - background song for
>a Microsoft sponsored "Where do you want to go from here?" commercial.
>
>
>-------------------------------------------
>Sender Policy Framework: http://www.openspf.org [http://www.openspf.org]
>Modify Your Subscription: http://www.listbox.com/member/ [http://www.listbox.com/member/]
>
>Archives: https://www.listbox.com/member/archive/735/=now
>RSS Feed: https://www.listbox.com/member/archive/rss/735/
>Powered by Listbox: http://www.listbox.com

Re: what is problem with this list ? I am receiving 1/5 messages maybe

Its likely you won't get this one too

A because your doing senderID pra checks {on mail from a mailinglist !}

B because apparently something is broke in your senderID implementation but i can's see what

>Wed 2010-01-13 18:22:32: Performing Sender ID lookup (alandoherty.net / 64.74.157.82)

correct

>Wed 2010-01-13 18:22:33: *  Policy: spf2.0/pra redirect=%{l}._sid-pra.%{d2}._mail.%{d2}

correct

>Wed 2010-01-13 18:22:33: *  Evaluating redirect=%{l}._sid-pra.%{d2}._mail.%{d2}: 

meaning lookup
spfdiscuss._sid-pra.alandoherty.net._mail.alandoherty.net

which i lookup
> set type=txt
> spfdiscuss._sid-pra.alandoherty.net._mail.alandoherty.net
Server:  ns1.ns.esat.net
Address:  192.111.39.1

Non-authoritative answer:
spfdiscuss._sid-pra.alandoherty.net._mail.alandoherty.net       text =

        "spf2.0/pra redirect=_sid-pra-restrictive.%{d3}"

>Wed 2010-01-13 18:22:33: *  Evaluating redirect=%{l}._sid-pra.%{d2}._mail.%{d2}: performing lookup
>Wed 2010-01-13 18:22:35: *    Policy: spf2.0/pra exp=_msg-nosuch2.%{d3} -all

crazy this is the result for *._sid-pra.alandoherty.net._mail.alandoherty.net
so somehow your library isn't adding the %{l} == spfdiscuss to its lookup
so its getting the result for senders using utter-bollix <at> alandoherty.net

you can verify this by manual querying your own dns servers for the legit records

$ORIGIN _sid-pra.alandoherty.net._mail.alandoherty.net.
*                       TXT     "spf2.0/pra exp=_msg-nosuch2.%{d3} -all"
alan                    TXT     "spf2.0/pra redirect=_sid-pra-non-restrictive.%{d3}"
error-tracker           TXT     "spf2.0/pra redirect=_sid-pra-restrictive.%{d3}"
postmaster              TXT     "spf2.0/pra redirect=_sid-pra-restrictive.%{d3}"
spam-l                  TXT     "spf2.0/pra redirect=_sid-pra-restrictive.%{d3}"
spfdiscuss              TXT     "spf2.0/pra redirect=_sid-pra-restrictive.%{d3}"
spider-trap             TXT     "spf2.0/pra redirect=_sid-pra-restrictive.%{d3}"

>Wed 2010-01-13 18:22:35: *    Evaluating exp=_msg-nosuch2.%{d3}: deferred
>Wed 2010-01-13 18:22:35: *    Evaluating -all: match
>Wed 2010-01-13 18:22:35: *  Result: fail
>Wed 2010-01-13 18:22:35: ---- End Sender ID results
>Wed 2010-01-13 18:22:35: --> 550 This address is never valid in from: header as nouser sends with this
address from this domain, it may yet be a valid receive-only address
>Wed 2010-01-13 18:22:35: SMTP session terminated (Bytes in/out: 7088/574)
> 
>
>Sender Policy Framework: <http://www.openspf.org>http://www.openspf.org
>Modify Your Subscription:
<http://www.listbox.com/member/>http://www.listbox.com/member/
><https://www.listbox.com/member/archive/735/=now>Archives<https://www.listbox.com/member/archive/rss/735/> 

Re: Resolving MFROM/HELO conflicts

On Thu, 14 Jan 2010, alan wrote:

> but as never should the helo success/pass result be dependant on anything but
> its ip my server name doesn't become a forgery just because an unexpected
> envelope-sender appears on the email conversely a forgery of my server name
> doesn't become legit because an envelope-senders SPF

A good point.  Which leads back to receiver policy as to whether to reject
for either/both.

Rejecting on HELO fail has caused the most ire.  One of my clients lost
a customer because that customer was sending mail with HELO fail,
and got mad when their email was rejected (used a CNAME):

mail.incompetent.com	IN CNAME incompetent.com.
incompetent.com		IN TXT "v=spf1 a mx -all"
incompetent.com		IN A  1.2.3.4

And of course, the IP of the MTA using mail.incompetent.com is not 1.2.3.4 or
any of the mxes.

--

-- 
	      Stuart D. Gathman <stuart <at> bmsi.com>
    Business Management Systems Inc.  Phone: 703 591-0911 Fax: 703 591-6154
"Confutatis maledictis, flammis acribus addictis" - background song for
a Microsoft sponsored "Where do you want to go from here?" commercial.

Re: what is problem with this list ? I am receiving 1/5 messages maybe