headers
Charlie Brady | 1 Feb 2006 03:08
Picon
Picon

Re: tmp dir and logs


On Tue, 31 Jan 2006, John Peacock wrote:

>> I'm struggling to think of any case where qpsmtpd needs to preserve 
>> content. The content is always available elsewhere - either in the 
>> backend, which has acknowledged receiving it, or in the sender's system, 
>> which will keep it for resending until it receives a permanent status 
>> code. Or both, if the backend has provided a response which hasn't yet 
>> been relayed to the sender.
>
> Personally, I sometimes like to save copies of virus infected messages which 
> were only caught by the second scanner and not the first.  I like to submit 
> those to the primary AV package so that they can add the signature to their 
> next update.

Sure, but you can do that, and still use an "anonymous" file for your 
spool. Just copy from the file descriptor into a named file when you 
decide to keep a copy.

Note that I said "needs to preserve". I don't think there's any time when 
it needs to, but I don't dispute that there are times when you might want 
to.
Permalink | Reply | 0 comments |
headers
Charlie Brady | 1 Feb 2006 03:13
Picon
Picon

Re: tmp dir and logs


On Tue, 31 Jan 2006, John Peacock wrote:

> My next SWAG is that the undeleted files are exclusively from the cases where 
> the remote MTA ended the connection prematurely (or timed-out). This would 
> seem to bear out the contents of the files I find, which frequently include 
> only headers, or maybe part of the message body but it ends in the middle of 
> an HTML tag (for example).

You'd see the same if the process died - e.g. segfault, or killed by OOM 
condition. You are sure that the remote MTA timed out?

I still think that anonymous files are the right spool to use, precisely 
for these conditions.
Permalink | Reply | 3 comments |
headers
Ask Bjørn Hansen | 1 Feb 2006 09:16
Gravatar

Re: tmp dir and logs


On Jan 31, 2006, at 6:13 PM, Charlie Brady wrote:

> You'd see the same if the process died - e.g. segfault, or killed  
> by OOM condition. You are sure that the remote MTA timed out?
>
> I still think that anonymous files are the right spool to use,  
> precisely for these conditions.

The original logic for not doing the immediate unlink was to keep it  
as a tool to track down bugs in the error handling.

If qpsmtpd crashes or gets killed (maybe except for by OOM) then it's  
not bad that it leaves behind a tell-tale.

  - ask

--

-- 
http://askask.com/  - http://develooper.com/
Permalink | Reply | 2 comments |
headers
John Peacock | 1 Feb 2006 17:01
Favicon

Re: tmp dir and logs

Ask Bjørn Hansen wrote:
> The original logic for not doing the immediate unlink was to keep it as 
> a tool to track down bugs in the error handling.

And I think we've done that now (mostly, see below), so maybe it's time 
to revisit that decision.

> If qpsmtpd crashes or gets killed (maybe except for by OOM) then it's 
> not bad that it leaves behind a tell-tale.

The problem is that the spooled message isn't usually enough to go on to 
track down _why_ it crashed/was killed, so I don't think that the slow 
accretion of garbage (which is what I was seeing).

I have had exactly one "orphan" file in my spool_dir since I last 
cleaned them out, oddly enough from this morning.  The file was just the 
headers of a message (no body).

I looked at the log entries for that item and here are the last two log 
lines for that item:

2006-02-01 10:32:41.108221500 23825 spooling message to disk
2006-02-01 10:52:41.588052500 23825 Connection Timed Out

When I first looked, it was before the "Connection Timed Out" and the 
file was still in the tmp/ directory.  After the 20 minutes had expired, 
the code correctly cleaned up the spooled message, so qpsmtpd seems to 
be doing what we want.

However, when I originally surveyed the contents of the tmp/ directory,
(Continue reading)

Permalink | Reply | 1 comment |
headers
Mark Powell | 1 Feb 2006 19:19
Picon

Re: New Patch: MaxLoad/MaxConnection handler

On Fri, 27 Jan 2006, Ed McLain wrote:

> Heh.. Sadly, I never thought of that ;)  But that does tend to make more
> sense.

What about:

http://search.cpan.org/~clintdw/Sys-CpuLoad-0.03/

Cheers.

>
> Ed
>
>
> On Thu, 26 Jan 2006 23:55:58 -0600, Peter Eisch wrote:
>
>> On 1/26/06 10:42 PM, "Robert Spier" <rspier <at> pobox.com> wrote:
>>
>>> I'm sure there is a module that
>>> someone has written that abstracts this all away.
>>
>> Sorry for the text inline, I'm somewhat encumbered at the moment:
>>
>> #!/usr/bin/perl
>>
>> =head1 NAME
>>
>> loadcheck
>>
(Continue reading)

Permalink | Reply | 4 comments |
headers
John Peacock | 1 Feb 2006 19:52
Favicon

Re: New Patch: MaxLoad/MaxConnection handler

Mark Powell wrote:
> What about:
> 
> http://search.cpan.org/~clintdw/Sys-CpuLoad-0.03/

That gets us FreeBSD/OpenBSD, as well as Linux /proc support, and a 
fallback to /usr/bin/uptime (which should be everywhere *nix).  But it 
requires an external dependency, of which there are few in the core.

John
Permalink | Reply | 3 comments |
headers
Mark Powell | 1 Feb 2006 19:56
Picon

Re: New Patch: MaxLoad/MaxConnection handler

On Wed, 1 Feb 2006, John Peacock wrote:

> Mark Powell wrote:
>> What about:
>> 
>> http://search.cpan.org/~clintdw/Sys-CpuLoad-0.03/
>
> That gets us FreeBSD/OpenBSD, as well as Linux /proc support, and a fallback 
> to /usr/bin/uptime (which should be everywhere *nix).  But it requires an 
> external dependency, of which there are few in the core.

Is there another way to call getloadavg() from perl under BSD?
   Cheers.

--

-- 
Mark Powell - UNIX System Administrator - The University of Salford
Information Services Division, Clifford Whitworth Building,
Salford University, Manchester, M5 4WT, UK.
Tel: +44 161 295 4837  Fax: +44 161 295 5888  www.pgp.com for PGP key
Permalink | Reply | 2 comments |
headers
John Peacock | 1 Feb 2006 20:51
Favicon

Re: New Patch: MaxLoad/MaxConnection handler

Mark Powell wrote:
> Is there another way to call getloadavg() from perl under BSD?

Nope (well, Inline::C, but that still requires a compiler).  I'd be 
inclined to just try /proc and uptime (in that order) and don't worry 
about OS specific methods.

John
Permalink | Reply | 0 comments |
headers
Ed McLain | 1 Feb 2006 20:44
Favicon

Re: New Patch: MaxLoad/MaxConnection handler

After looking at using a connect plugin I realized that for my situation
that just wouldn't work so well.  Since I have 5 boxes in a load balanced
config I need the load balancer to be able to determine if a box is able
to take more connections.  Since a plugin would have to load after the
connection has been accepted that just won't work, the only way I can see
to do it, and what seems to be working here, is the way I implemented it
before. One thing that could be done is to use Sys-CpuLoad and make it an
option that if you want support for that feature you can install the dep,
otherwise it just won't check for load.

On Wed, 01 Feb 2006 18:56:29 +0000, Mark Powell wrote:

> On Wed, 1 Feb 2006, John Peacock wrote:
> 
>> Mark Powell wrote:
>>> What about:
>>> 
>>> http://search.cpan.org/~clintdw/Sys-CpuLoad-0.03/
>>
>> That gets us FreeBSD/OpenBSD, as well as Linux /proc support, and a fallback 
>> to /usr/bin/uptime (which should be everywhere *nix).  But it requires an 
>> external dependency, of which there are few in the core.
> 
> Is there another way to call getloadavg() from perl under BSD?
>    Cheers.
Permalink | Reply | 0 comments |
headers
Peter J. Holzer | 1 Feb 2006 16:46

[perl #38397] qpsmtpd-forkserver doesn't like PERL_UNICODE

# New Ticket Created by  Peter J. Holzer 
# Please include the string:  [perl #38397]
# in the subject line of all future correspondence about this issue. 
# <URL: https://rt.perl.org/rt3/Ticket/Display.html?id=38397 >

I accidentally restarted qpsmtpd-forkserver while having 
the following locale-related environment variables set:

    LC_COLLATE=POSIX
    LANG=en_US.UTF-8
    PERL_UNICODE=SDAL

The result was that all mails which were not decodable as UTF-8 
mere mangled. I.e., mails with a content-transfer-encoding of
quoted-printable or base64 were fine (because then they only contain
ASCII characters and ASCII is a subset of UTF-8) as were mails with
charset UTF-8 and a content-transfer-encoding of 8bit. But mails with a
different charset (iso-8859-*) and 8bit content-transfer-encoding were
mangled.

A simple solution is to explicitely unset PERL_UNICODE (and maybe also
LANG and LC_*) in the startup scripts. I will do that in the next
release of the RPMs. 

A real fix would probably be to explicitely call binmode ":raw" on all
filehandles, which need to read/write octets, not characters (I.e., at
least all the sockets, but probably also temporary files, pipes, etc.
(Sounds like a lot of work).

	hp
(Continue reading)

Permalink | Reply | 1 comment |

Gmane