Re: Q: RCPT Check by acting as a mailproxy?

Kyle Wheeler wrote:
> On Friday, May 30 at 07:20 PM, quoth Hugo Monteiro:
>> I also noticed Soffian took down both the patch and the docs for the 
>> external RCPTCHECK program use. If you have to docs to his patch, 
>> please set them up on your site.
>
> I don't have the original documentation (I wish I did), otherwise I 
> would set it up. But I just added a quick summary of how to use it to 
> my qmail page (http://www.memoryhole.net/qmail/#rcptcheck), which 
> should do for now.
>
> One of these days I'll write a few example perl scripts that could be 
> used with the patch, for example to do three-tuple greylisting (I keep 
> meaning to, since it's trivial, but don't have the time).
>

Kyle and list,

I'm not sure you're familiar with the envelope scanner patch for qmail. 
Either way, it pretty much does the same than Soffian's patch, but 
passes the data to the external checker as parameters in the call, 
rather than using env vars.

I've beem using both of them, although they could reduced to a single 
system call, having the envelope scanning working with a simple helper i 
wrote called qenvscan-policyd to interact with policyd. (policyd 
provides an impressive set of possible checks, just take a peep here 
http://policyd.org/features.html)

Another thing that i've been wanting to do also, while using the ESMTP 

Re: Q: RCPT Check by acting as a mailproxy?

On Sunday, June  1 at 10:22 PM, quoth Hugo Monteiro:
> I'm not sure you're familiar with the envelope scanner patch for 
> qmail. Either way, it pretty much does the same than Soffian's 
> patch, but passes the data to the external checker as parameters in 
> the call, rather than using env vars.

True; Kelly deserves credit (she published her patch earlier), I just 
found Soffian's patch first when I was googling for a solution once 
upon a time. :)

> I've beem using both of them, although they could reduced to a 
> single system call, having the envelope scanning working with a 
> simple helper i wrote called qenvscan-policyd to interact with 
> policyd. (policyd provides an impressive set of possible checks, 
> just take a peep here http://policyd.org/features.html)

Interesting... I may have to look into this.

> - Alter Soffian's patch to set two other env vars. HELO and MSGSIZE 
> (or similar)

Sounds reasonable.

> - Write a program that does the recipient checking in the usual 
> places (~qmail/users/assign, /etc/aliases.cdb - a la fastforward, 
> dot qmail files in ~alias), custom cdb file, ldap and mysql. The 
> program could be configured to return success on the first match or 
> go through all the defined backends (multiple backends would be 
> supported). More details on this one below.

Re: svc don't stop smtpd

Re: svc don't stop smtpd

On Monday 02 June 2008 10:47, West wrote:
> exec \
>
>         tcpserver -u $QDUID -g $QDGID -v -HRl $ME

Looks like a blank line between your "exec \" and tcpserver line: If 
so, remove it, see if that helps.

cheers,

Andrew Richards.
--

-- 
======================================================================
   * Custom email solutions * Systems Administration * Networking

		http://www.acrconsulting.co.uk/

====================================================================

Re: svc don't stop smtpd

Moved to FAO

I will be out of the office starting  01/03/2008 and will not return until
28/02/2009.

I am now working with FAO/TCE (Room B-655) and have the following email:
Pedro.Figueiredo <at> fao.org.  My WFP email account will still be active but I
will only monitor it occasionally.  My new contacts are: +39 06 57056042
(landline); +39 349 2269876 (cell); 1301 51 56042 (Food Sat)

-----------------------------------------

The information contained in this electronic message and any
attachments are
intended for specific individuals or entities, and may be
confidential, proprietary or
privileged. If you are not the intended recipient, please notify
the sender
immediately, delete this message and do not disclose, distribute or
copy it to any
third party or otherwise use this message. The content of this
message does not
necessarily reflect the official position of the World Food
Programme. Electronic
messages are not secure or error free and may contain viruses or
may be
delayed, and the sender is not liable for any of these occurrences.
The sender
reserves the right to monitor, record and retain electronic
messages.

Re: Q: RCPT Check by acting as a mailproxy?

On 2008-06-01, at 1722, Hugo Monteiro wrote:
>
> All that said, i'm starting to think about:
>
> - Alter Soffian's patch to set two other env vars. HELO and MSGSIZE  
> (or similar)

i've just added it to my combined patch. i took your suggestion and  
added a HELO variable to my version- it was literally one line added  
to the patch. thanks for the idea.

http://qmail.jms1.net/patches/combined-details.shtml#7.07

however, i don't think a MSGSIZE variable would be possible (unless  
i'm not understanding what you mean.) remember that the program is  
called after the client sends a RCPT command. at this point in the  
SMTP conversation, the message itself hasn't been sent so the size is  
not known (unless the client added a SIZE= extension to their MAIL  
command, and that number may or may not be accurate.)

> - Write a program that does the recipient checking in the usual  
> places (~qmail/users/assign, /etc/aliases.cdb - a la fastforward,  
> dot qmail files in ~alias), custom cdb file, ldap and mysql. The  
> program could be configured to return success on the first match or  
> go through all the defined backends (multiple backends would be  
> supported). More details on this one below.

so... in order to save one syscall within qmail-smtpd, you're adding  
fork/exec to run another program, plus however many syscalls that  
other program does to read all of these files, while qmail-smtpd is  
waiting for the external program to finish.

to me it doesn't sound like you're actually saving any CPU cycles.

you list "custom cdb file" as one of your options... why not have that  
as your ONLY option, and only spend the CPU cycles needed for checking  
all of these other sources, when you rebuild that cdb file?

personally, the only reason i added the RCPTCHECK patch to my own is  
that i plan to set up 3-tuple greylisting with it. and while i could  
easily add a validrcptto.cdb check to such a program, i still think it  
makes more sense to do the check within qmail-smtpd, since it's only a  
stat() and open() when the program starts, and no more than two seek()/ 
read() calls per RCPT command.

plus as kyle said, it opens the doors to so many other potential uses  
in the future...

--------------------------------------------------------
| John M. Simpson  --  KG4ZOW  --  Programmer At Large |
| http://www.jms1.net/                 <jms1 <at> jms1.net> |
--------------------------------------------------------
|   Hope for America  --  http://www.ronpaul2008.com/  |
--------------------------------------------------------

Re: Q: RCPT Check by acting as a mailproxy?

John Simpson wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On 2008-06-01, at 1722, Hugo Monteiro wrote:
>>
>> All that said, i'm starting to think about:
>>
>> - Alter Soffian's patch to set two other env vars. HELO and MSGSIZE 
>> (or similar)
>
> i've just added it to my combined patch. i took your suggestion and 
> added a HELO variable to my version- it was literally one line added 
> to the patch. thanks for the idea.
>
> http://qmail.jms1.net/patches/combined-details.shtml#7.07
>

I'm very happy to see you decided to include Soffian's patch in yours, 
plus the the HELO setting and stdout redirection.  =)

> however, i don't think a MSGSIZE variable would be possible (unless 
> i'm not understanding what you mean.) remember that the program is 
> called after the client sends a RCPT command. at this point in the 
> SMTP conversation, the message itself hasn't been sent so the size is 
> not known (unless the client added a SIZE= extension to their MAIL 
> command, and that number may or may not be accurate.)
>

I should have clarified that i was in fact referring to the SIZE= ESMTP 
extension. I'm aware that the message SIZE advertisement is voluntary 
and might not even be correct. However, I believe that if the client 
decides to advertise such value, it's the clients responsability if the 
message doesn't get accepted because it's providing wrong information. 
The goal in incorporating that information, in my perspective, is to 
save server resources in the case that the client wants to cooperate and 
the message is actually larger that the "allowed" size.
>
>> - Write a program that does the recipient checking in the usual 
>> places (~qmail/users/assign, /etc/aliases.cdb - a la fastforward, dot 
>> qmail files in ~alias), custom cdb file, ldap and mysql. The program 
>> could be configured to return success on the first match or go 
>> through all the defined backends (multiple backends would be 
>> supported). More details on this one below.
>
> so... in order to save one syscall within qmail-smtpd, you're adding 
> fork/exec to run another program, plus however many syscalls that 
> other program does to read all of these files, while qmail-smtpd is 
> waiting for the external program to finish.
>
> to me it doesn't sound like you're actually saving any CPU cycles.
>
> you list "custom cdb file" as one of your options... why not have that 
> as your ONLY option, and only spend the CPU cycles needed for checking 
> all of these other sources, when you rebuild that cdb file?
>
> personally, the only reason i added the RCPTCHECK patch to my own is 
> that i plan to set up 3-tuple greylisting with it. and while i could 
> easily add a validrcptto.cdb check to such a program, i still think it 
> makes more sense to do the check within qmail-smtpd, since it's only a 
> stat() and open() when the program starts, and no more than two 
> seek()/read() calls per RCPT command.
>
> plus as kyle said, it opens the doors to so many other potential uses 
> in the future...
>
>

Indeed the cdb file lookup could (and should) be done inside 
qmail-smtpd. The main reason why i said the external program would 
support such lookups is because i would also like be see the "usual 
places" looked up, such as:

1 - system users - this can be easily done with a getpwnam() call and a 
3 or 4 more lines for logic.

2 - /var/qmail/users/cdb (built from the assign file) - The cdb lookup 
code could be used here also, with some minor tweaks since the key 
layout isn't exactly the as values to match. One could also add the 
check for the referenced ~alias/.qmail files, for sanity checking, but i 
think that it might not be worth the trouble.

3 - /etc/aliases.cdb (used with fastforward) - Again the cdb lookup code 
and again with some minor changes since the key layout also differs.

If you feel confortable adding support for this in your patch, I won't 
have second thoughts about starting using it =)

As for Kyle's suggestion regarding mailman lists. That could either be 
done by stat()'ing the list directory name in the mailman installation 
(eg. /var/lib/mailman/lists/). The problem is that there are a lot of 
other associated addresses, such as -owner, -subscribe, etc.

There is already an easy way for doing this, i think. In my 
qmail+mailman installations i always found easier to pipe -default 
entries in the ~qmail/users/assign file to the mailman 
qmail-to-mailman.py script. That way lists are automagically set up once 
i just define something like in the assign file.

+info:alias:64010:65534:/var/qmail/alias:-:mlists:
+sales:alias:64010:65534:/var/qmail/alias:-:mlists:
+lusers:alias:64010:65534:/var/qmail/alias:-:mlists:

And have a general ~alias/.qmail-mlists like

|preline /usr/bin/python /usr/lib/mailman/bin/qmail-to-mailman.py

Comments are most welcomed.

Regards,

Hugo Monteiro.

--

-- 
ci.fct.unl.pt:~# cat .signature

Hugo Monteiro
Email	 : hugo.monteiro <at> fct.unl.pt
Telefone : +351 212948300 Ext.15307

Centro de Informática
Faculdade de Ciências e Tecnologia da
		   Universidade Nova de Lisboa
Quinta da Torre   2829-516 Caparica   Portugal
Telefone: +351 212948596   Fax: +351 212948548
www.ci.fct.unl.pt	      apoio <at> fct.unl.pt

ci.fct.unl.pt:~# _

Re: Q: RCPT Check by acting as a mailproxy?

On Tuesday, June  3 at 12:24 PM, quoth Hugo Monteiro:
> As for Kyle's suggestion regarding mailman lists. That could either 
> be done by stat()'ing the list directory name in the mailman 
> installation (eg. /var/lib/mailman/lists/).

The primary problem with this approach is virtual domain support. In 
order to figure out which domain owns a given list, your checker has 
to have *write* permissions on the list innards (mailman insists on it 
for some bizarre reason, if you're going to use a tool like 
list_lists). You basically have to write your own setuid binary just 
for checking list existence accurately.

Periodically generating a database of mailman lists makes much more 
sense, but is not exactly standardized.

> The problem is that there are a lot of other associated addresses, 
> such as -owner, -subscribe, etc.

Meh, these are easy to identify via a script:

     case "$localpart" in
         *-owner)     listname=${localpart%-owner} ;;
         *-subscribe) listname=${localpart%-subscribe} ;;
         ...
     esac

But that's only useful for doing recipient checking. And since mailman 
makes it difficult to handle virtualdomains by checking on-the-fly, it 
probably makes more sense to use cron to generate a database that can 
be checked.

> There is already an easy way for doing this, i think. In my 
> qmail+mailman installations i always found easier to pipe -default 
> entries in the ~qmail/users/assign file to the mailman 
> qmail-to-mailman.py script. That way lists are automagically set up 
> once i just define something like in the assign file.
>
> +info:alias:64010:65534:/var/qmail/alias:-:mlists:
> +sales:alias:64010:65534:/var/qmail/alias:-:mlists:
> +lusers:alias:64010:65534:/var/qmail/alias:-:mlists:
>
> And have a general ~alias/.qmail-mlists like
>
> |preline /usr/bin/python /usr/lib/mailman/bin/qmail-to-mailman.py

I have entire domains devoted to GNU Mailman, without any "real" 
users, e.g. lists.domain.tld. Thus, adding lists doesn't require 
modifying the qmail configuration, because the entire domain is fed to 
a single .qmail-default script that does all the heavy lifting.

But that's beside the point as far as recipient verification goes.

~Kyle
--

-- 
The truth is that there are no things for which men will make such 
herculean efforts as the things of which they know they are unworthy.
                                                    -- G. K. Chesterton

RCPTCHECK support in your combined patch

Hello John,

I just visited the page in the section for the RCPTCHECK support, and 
there was something that i didn't quite understand. You say

"The validrcptto.cdb file (if any) will be checked before the RCPTCHECK 
program (if any) runs. If the validrcptto.cdb mechanism rejects a 
recipient, the RCPTCHECK program will not be run for that recipient."

How does the validrcptto.cdb mechanism reject a recipient? Doesn't it 
only check for the existance, authorizing if it does exist and rejecting 
if it doesn't? If so, what's the point of supporting the external 
RCPTCHECK? Is it only to provide a way for greylisting (and friends)?

I realy think that the external RCPTCHECK should be able to be done, 
since it could also provide the original purpose of doing recipient 
checking not supported natively by the (altered) qmail installation. If 
your mind is really set to what you have described, maybe you could 
consider allowing the further recipient checking to be done if another 
env var was set. (EXTENDED_RCPTCHECK?)

Regards,

Hugo Monteiro.

--

-- 
ci.fct.unl.pt:~# cat .signature

Hugo Monteiro
Email	 : hugo.monteiro <at> fct.unl.pt
Telefone : +351 212948300 Ext.15307

Centro de Informática
Faculdade de Ciências e Tecnologia da
		   Universidade Nova de Lisboa
Quinta da Torre   2829-516 Caparica   Portugal
Telefone: +351 212948596   Fax: +351 212948548
www.ci.fct.unl.pt	      apoio <at> fct.unl.pt

ci.fct.unl.pt:~# _