headers
Arthur Dent | 12 Apr 2010 10:09
Picon
Favicon

Filter Oddity

Hello All,

On my small family mailserver on a Fedora 11 machine /etc/procmailrc
deals with incoming mail collected by fetchmail. Anything unmolested by
clamav or spamassassin gets passed on to the small number of users.

In my user area I further filter into (mbox) folders for more organised
reading. This all works just fine.... ...except for one rule.

:0:
* ^From .*(dickensnetwork.org.uk|(b2e-(resourcing|solutions)))
Misc/Agencies

In this I get mail from 2 agencies. In one all mail comes from contacts
with a someone <at> dickensnetwork.org.uk address, and in the other they have
either a someone <at> b2e-resourcing.co.uk or a
someonelse <at> b2e-solutions.co.uk address.

The Dickens one works fine, but anything from B2e just falls through to
my Inbox.

Now, here's the odd thing...

I have a sandbox set up based on Sean Straw's (OK, shamelessly copied
exactly from Sean Straw's) sandbox concept, and when I run a message
from B2e through that, it hits the rule!

procmail: No match on "(^((Original-)?(Resent-)?(To|Cc|Bcc)|(X-Envelope|Apparently(-Resent)?)-To):(.*[^-a-zA-Z0-9_.])?)console-users"
procmail: No match on "(^((Original-)?(Resent-)?(To|Cc|Bcc)|(X-Envelope|Apparently(-Resent)?)-To):(.*[^-a-zA-Z0-9_.])?)archivemail"
procmail: No match on "^From .*(linuxformat.co|admin <at> fedoraforum.org)"
(Continue reading)

Permalink | Reply | 16 comments |
headers
Charles Gregory | 12 Apr 2010 15:59

Re: Filter Oddity

On Mon, 12 Apr 2010, Arthur Dent wrote:
> The Dickens one works fine, but anything from B2e just falls through to
> my Inbox.
> ..... when I run a message from B2e through that, it hits the rule!

This is the symptom of an 'encoding' mechanism, such as 'quoted printable'
being used on the headers. The program you use to read or forward the mail 
is 'decoding' the header, so you only see the plain text.

Use a text editor to view the original (unforwarded) mail in your inbox
see what the 'From:' header *really* looks like....

- C
Permalink | Reply | 12 comments |
headers
Professional Software Engineering | 12 Apr 2010 15:53

Re: Filter Oddity

At 09:09 2010-04-12 +0100, Arthur Dent wrote:
>I have a sandbox set up based on Sean Straw's (OK, shamelessly copied
>exactly from Sean Straw's)

I publish the sandbox for a reason - so people can use it.  There's no 
problem with "shamelessly copying it".  Besides, the sandbox itself itself 
is formed from a collection of practices of others over the years on this list.

>sandbox concept, and when I run a message from B2e through that, it hits 
>the rule!

It is sounding as if the copy of the message you first receive from 
fetchmail may have a slightly different From_ than the ones you're 
reprocessing.  I can't say WHY, but it's seeming like it might.

>And this is true whether I am using a copy of the filter or pointing the
>sandbox directly at the live filter.

Good test - this generally rules out the possibility of a typo in the test 
copy.

>Thanks in advance for any ideas or suggestions...

If would be helpful to see the logged delivery summary showing the From_ 
line of a native failed copy of the message in question.  Enable logging if 
you don't already in your /etc/procmailrc (note that you could enable it 
just before the recipe that is failing, so you're not innundated with 
logging from everything else).

---
(Continue reading)

Permalink | Reply | 2 comments |
headers
Professional Software Engineering | 12 Apr 2010 16:23

Re: Filter Oddity

At 09:59 2010-04-12 -0400, Charles Gregory wrote:

>This is the symptom of an 'encoding' mechanism, such as 'quoted printable'
>being used on the headers. The program you use to read or forward the mail 
>is 'decoding' the header, so you only see the plain text.

On an ENVELOPE SENDER line (From ) ?  I know it happens all the time with 
other header fields, such as From:, To:, and Subject:, but I can't say that 
I've ever noted an instance of it with the From_

---
  Sean B. Straw / Professional Software Engineering

  Procmail disclaimer: <http://www.professional.org/procmail/disclaimer.html>
  Please DO NOT carbon me on list replies.  I'll get my copy from the list.
Permalink | Reply | 7 comments |
headers
Arthur Dent | 12 Apr 2010 16:23
Picon
Favicon

Re: Filter Oddity

On Mon, 2010-04-12 at 06:53 -0700, Professional Software Engineering
wrote:
> At 09:09 2010-04-12 +0100, Arthur Dent wrote:
> 
> >Thanks in advance for any ideas or suggestions...
> 
> If would be helpful to see the logged delivery summary showing the From_ 
> line of a native failed copy of the message in question.  Enable logging if 
> you don't already in your /etc/procmailrc (note that you could enable it 
> just before the recipe that is failing, so you're not innundated with 
> logging from everything else).

Ah! - I didn't know you could do that. So my "live" filter would look
something like:

...various other rules

VERBOSE=ON
:0:
* ^From .*(dickensnetwork.org.uk|(b2e-(resourcing|solutions)))
Misc/Agencies
VERBOSE=OFF

...more rules

Is that right?

I will try that. The only problem is that emails from this source are
sporadic. I might get three in one day and then none for two weeks, but
that is what makes this such a good suggestion. I certainly don't want
(Continue reading)

Permalink | Reply | 1 comment |
headers
Arthur Dent | 12 Apr 2010 16:36
Picon
Favicon

Re: Filter Oddity

On Mon, 2010-04-12 at 09:59 -0400, Charles Gregory wrote:
> On Mon, 12 Apr 2010, Arthur Dent wrote:
> > The Dickens one works fine, but anything from B2e just falls through to
> > my Inbox.
> > ..... when I run a message from B2e through that, it hits the rule!
> 
> This is the symptom of an 'encoding' mechanism, such as 'quoted printable'
> being used on the headers. The program you use to read or forward the mail 
> is 'decoding' the header, so you only see the plain text.
> 
> Use a text editor to view the original (unforwarded) mail in your inbox
> see what the 'From:' header *really* looks like....

Hmmm... Thanks for that. It *IS* 'quoted printable'. These (below) are
the headers as seen in Mutt. Is there a better way to view the plain
text?

Given that, if this is the problem what can I do?

Thanks for your help so far...

Mark

#############################################################################

(Apologies for line-wraps)

Return-Path: esc1103287837504_1101248844445_5153 <at> in.constantcontact.com
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mydomain.com
X-Spam-Level:
(Continue reading)

Permalink | Reply | 3 comments |
headers
Professional Software Engineering | 12 Apr 2010 17:02

Re: Filter Oddity

At 15:23 2010-04-12 +0100, Arthur Dent wrote:
>Ah! - I didn't know you could do that. So my "live" filter would look
>something like:
>
>...various other rules
>
>VERBOSE=ON
>:0:
>* ^From .*(dickensnetwork.org.uk|(b2e-(resourcing|solutions)))
>Misc/Agencies
>VERBOSE=OFF
>
>...more rules
>
>Is that right?

Yes.  Though verbosity won't add a whole lot in the case of this recipe - 
you just want to make sure that logging is on.  The delivery summary - the 
From_, size, subject, and delivery target are sufficient.

---
  Sean B. Straw / Professional Software Engineering

  Procmail disclaimer: <http://www.professional.org/procmail/disclaimer.html>
  Please DO NOT carbon me on list replies.  I'll get my copy from the list.
Permalink | Reply | 0 comments |
headers
Professional Software Engineering | 12 Apr 2010 17:07

Re: Filter Oddity

At 15:36 2010-04-12 +0100, Arthur Dent wrote:
> > Use a text editor to view the original (unforwarded) mail in your inbox
> > see what the 'From:' header *really* looks like....
>
>Hmmm... Thanks for that. It *IS* 'quoted printable'.

>Content-Type: text/plain; charset=ISO-8859-1
>Content-Transfer-Encoding: quoted-printable

These headers describe the CONTENT of the message itself, not the encoding 
of the headers.

>  These (below) are
>the headers as seen in Mutt. Is there a better way to view the plain
>text?

'more' from a shell prompt.  Try:

         grep -10 o3B8O6li010788 mailboxfilename | less

That 'o3B8O6li010788' bit is an SMTP id seen in the header of your 
example.  Adjust the -10 according to how many lines before and after (or 
use -Bnn and -Ann for accomplish similar) the matched line.  This is 
generally sufficient to get a "window" into the mailbox.  No decoding will 
be performed, so you should see it nice and raw.

---
  Sean B. Straw / Professional Software Engineering

  Procmail disclaimer: <http://www.professional.org/procmail/disclaimer.html>
(Continue reading)

Permalink | Reply | 2 comments |
headers
Charles Gregory | 12 Apr 2010 18:09

Re: Filter Oddity

On Mon, 12 Apr 2010, Professional Software Engineering wrote:
>> This is the symptom of an 'encoding' mechanism, such as 'quoted printable'
>> being used on the headers. The program you use to read or forward the mail 
>> is 'decoding' the header, so you only see the plain text.
>
> On an ENVELOPE SENDER line (From ) ?  I know it happens all the time with 
> other header fields, such as From:, To:, and Subject:, but I can't say that 
> I've ever noted an instance of it with the From_

There is no guarantee that the sender's envelope is actually the same as 
the "From:" header. And there isn't even any guarantee that the envelope 
sender is a 'From' line. On my postfix it is 'Return-Path'....

So in my experience (YYMV) any time a simple rule fails to trigger
when the text 'obviously' matches, it is the fault of encoding like QP.

The OP says the mail 'fell through' to his inbox. so that pretty much 
eliminates the chance of a match on another rule earlier in the list
(which would likely have been revealed through the secondary testing).

- C
Permalink | Reply | 6 comments |
headers
Arthur Dent | 12 Apr 2010 19:12
Picon
Favicon

Re: Filter Oddity

On Mon, 2010-04-12 at 08:07 -0700, Professional Software Engineering
wrote:
> At 15:36 2010-04-12 +0100, Arthur Dent wrote:
> > > Use a text editor to view the original (unforwarded) mail in your inbox
> > > see what the 'From:' header *really* looks like....
> >
> >Hmmm... Thanks for that. It *IS* 'quoted printable'.
> 
> >Content-Type: text/plain; charset=ISO-8859-1
> >Content-Transfer-Encoding: quoted-printable
> 
> These headers describe the CONTENT of the message itself, not the encoding 
> of the headers.
> 
> >  These (below) are
> >the headers as seen in Mutt. Is there a better way to view the plain
> >text?
> 
> 'more' from a shell prompt.  Try:
> 
>          grep -10 o3B8O6li010788 mailboxfilename | less
> 
> That 'o3B8O6li010788' bit is an SMTP id seen in the header of your 
> example.  Adjust the -10 according to how many lines before and after (or 
> use -Bnn and -Ann for accomplish similar) the matched line.  This is 
> generally sufficient to get a "window" into the mailbox.  No decoding will 
> be performed, so you should see it nice and raw.

Well to my untrained eye, I can see very little difference here. But
you're right. This one fails to match on the test. What have I missed?
(Continue reading)

Permalink | Reply | 1 comment |

Gmane