headers
RW | 1 Feb 01:07

Re: Burst of large messages sometimes causes spamd to lock up

On Tue, 31 Jan 2012 15:17:34 -0800 (PST)
John Hardin wrote:

> On Tue, 31 Jan 2012, Kris Deugau wrote:

> > But most of the time, there are 6-8 children running.
> >
> >>  ...to answer my own question, max-spare overrides it?
> >
> > Hm, I wondered if that might be the case.
> 
> Can't see what else would.

What's happening is that spamd has a test to stop it deleting
children at or below min-children, but it's missing the test to add them
based on min-children. It initializes with max-spare children, the
number montonically increases to satisfy min-spare until it
reaches min-children, and thereafter behaves as expected.

I think this is probably a bug, although it might conceivably be
considered a feature.

As regards the original question I just remembered that something like
this has come up before. IIRC there's a problem with timing-out a spamd
child while it's inside a regex, so a single bad regex can cause serious
problems. It's most likely  a rawbody, or possibly a body rule.
Permalink | Reply | 7 comments |
headers
Martin Gregorie | 1 Feb 11:56
Favicon

Re: Burst of large messages sometimes causes spamd to lock up

On Tue, 2012-01-31 at 15:17 -0800, John Hardin wrote:
> On Tue, 31 Jan 2012, Kris Deugau wrote:
> 
> > John Hardin wrote:
> >>  On Tue, 31 Jan 2012, John Hardin wrote:
> >> >  You posted this command line:
> >> > 
> >> >  /usr/local/bin/spamd -d -x -q -r /var/run/spamd.pid --min-children=59
> >> >  --min-spare=1 --max-spare=1 --max-conn-per-child=100 -m 60 -s local1
> >> >  -u spamd --timeout-child=60 -i 0.0.0.0 -A <IP list> --syslog-ident
> >> >  spamd/main
> >> >
>
This is probably irrelevant, but I notice that --max-children isn't
specified. 

Would it help to set it, adjusting --min-children to a somewhat lower
value?

Martin
Permalink | Reply | 1 comment |
headers
RW | 1 Feb 14:51

Re: Burst of large messages sometimes causes spamd to lock up

On Wed, 1 Feb 2012 00:07:18 +0000
RW wrote:

> What's happening is that spamd has a test to stop it deleting
> children at or below min-children, but it's missing the test to add
> them based on min-children. It initializes with max-spare children,
> the number montonically increases to satisfy min-spare until it
> reaches min-children, and thereafter behaves as expected.
> 

I've submitted a patch as bug 6754.

https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6754
Permalink | Reply | 0 comments |
headers
Kris Deugau | 1 Feb 16:46
Picon

Re: Burst of large messages sometimes causes spamd to lock up

RW wrote:
> As regards the original question I just remembered that something like
> this has come up before. IIRC there's a problem with timing-out a spamd
> child while it's inside a regex, so a single bad regex can cause serious
> problems. It's most likely  a rawbody, or possibly a body rule.

How would that affect the master spamd process though?  Once it's 
locked, new connection attempts just time out.  If it's a rogue regex, 
I'd expect to see a bunch of spamd children crunching away, but new 
connections going through just fine (at least up to max-children).

-kgd
Permalink | Reply | 5 comments |
headers
Bowie Bailey | 1 Feb 17:15
Favicon

Re: Burst of large messages sometimes causes spamd to lock up

On 2/1/2012 5:56 AM, Martin Gregorie wrote:
> On Tue, 2012-01-31 at 15:17 -0800, John Hardin wrote:
>> On Tue, 31 Jan 2012, Kris Deugau wrote:
>>
>>> John Hardin wrote:
>>>>  On Tue, 31 Jan 2012, John Hardin wrote:
>>>>>  You posted this command line:
>>>>>
>>>>>  /usr/local/bin/spamd -d -x -q -r /var/run/spamd.pid --min-children=59
>>>>>  --min-spare=1 --max-spare=1 --max-conn-per-child=100 -m 60 -s local1
>>>>>  -u spamd --timeout-child=60 -i 0.0.0.0 -A <IP list> --syslog-ident
>>>>>  spamd/main
>>>>>
> This is probably irrelevant, but I notice that --max-children isn't
> specified. 
>
> Would it help to set it, adjusting --min-children to a somewhat lower
> value?

-m is another name for --max-children.

--

-- 
Bowie
Permalink | Reply | 0 comments |
headers
Miguel Fernandes | 1 Feb 17:37
Favicon

Spamassassin rules from sql

Hi!

I'm just wondering, is there a limitation on the type of rules that can 
be added to the SpamassAssin rules table?
Because adding something like (subject scoring):

header TEST_SUBJECT Subject =~ /test/i (I've tried several combinations 
of fields: preference & value, with no success)
score TEST_SUBJECT 30 (this one is possible)

Does not seem straightforward...

My goal is to be able to score subjects through sql rules.

Thanks!

--

-- 
Cumprimentos,
Miguel Fernandes
Permalink | Reply | 7 comments |
headers
Kris Deugau | 1 Feb 17:52
Picon

Re: Spamassassin rules from sql

Miguel Fernandes wrote:
> Hi!
>
> I'm just wondering, is there a limitation on the type of rules that can
> be added to the SpamassAssin rules table?
> Because adding something like (subject scoring):
>
> header TEST_SUBJECT Subject =~ /test/i (I've tried several combinations
> of fields: preference & value, with no success)
> score TEST_SUBJECT 30 (this one is possible)
>
> Does not seem straightforward...
>
> My goal is to be able to score subjects through sql rules.

It should be possible with something like (untested):

INSERT INTO userpref (username, preference, value) VALUES 
('@GLOBAL','header TEST_SUBJECT','Subject =~ /test/i');

I don't know if @GLOBAL entries like that will be preloaded like the 
stock rules or any of the other file-based config, or if that's just a 
shortcut for something to be retrieved for all users on a userprefs 
lookup.  If it's the latter (I'm pretty sure it is), you'll have to 
enable allow_user_rules.

FWIW, this is probably the wrong way to solve whatever problem you're 
seeing.  If you can describe why you want to do this, someone can 
probably give you a better way to do it.
(Continue reading)

Permalink | Reply | 4 comments |
headers
Axb | 1 Feb 18:01
Picon

Re: Spamassassin rules from sql

On 02/01/2012 05:52 PM, Kris Deugau wrote:
> Miguel Fernandes wrote:
>> Hi!
>>
>> I'm just wondering, is there a limitation on the type of rules that can
>> be added to the SpamassAssin rules table?
>> Because adding something like (subject scoring):
>>
>> header TEST_SUBJECT Subject =~ /test/i (I've tried several combinations
>> of fields: preference & value, with no success)
>> score TEST_SUBJECT 30 (this one is possible)
>>
>> Does not seem straightforward...
>>
>> My goal is to be able to score subjects through sql rules.
>
> It should be possible with something like (untested):
>
> INSERT INTO userpref (username, preference, value) VALUES
> ('@GLOBAL','header TEST_SUBJECT','Subject =~ /test/i');
>
> I don't know if @GLOBAL entries like that will be preloaded like the
> stock rules or any of the other file-based config, or if that's just a
> shortcut for something to be retrieved for all users on a userprefs
> lookup. If it's the latter (I'm pretty sure it is), you'll have to
> enable allow_user_rules.
>
> FWIW, this is probably the wrong way to solve whatever problem you're
> seeing. If you can describe why you want to do this, someone can
> probably give you a better way to do it.
(Continue reading)

Permalink | Reply | 3 comments |
headers
Kris Deugau | 1 Feb 18:11
Picon

Re: Burst of large messages sometimes causes spamd to lock up

John Hardin wrote:
> That was just a stab in the dark. From your log excerpt SA seems to be
> getting wedged when bringing active the last idle child rather than when
> adding new children. It would be interesting to see if it wedges in the
> new config when idle child number 5 is needed...

The particulars of how many active children there are, and how many are 
busy, varies from incident to incident;  IIRC I've seen it get stuck 
with 6 idle children and 2 busy.  :/

>> OTOH... I'm having trouble seeing where this could cause the whole
>> spamd process tree to lock up completely for ~15 minutes. (It locks
>> hard enough that a monitoring process gets "connection timed out",
>> even though there are only 5 spamd children running - as per an
>> incident just after noon today.)
>
> So there were 5 busy children and one idle, and it wedged, and when it
> recovered it started adding children?

.. or 4 and 2, or 3 and 5, or 2 and 3, or....  :/

> --min-spare 5 would be something else to try.

I've bumped up min-spare and max-spare on one machine, we'll see how it 
goes.

>> On the third hand... if there *is* a subtle bug in spamd's process
>> scaling, is it worth jumping *way* out there and trying --round-robin?
>
> If this is a dedicated mail server, why not? round robin on (say) 16
(Continue reading)

Permalink | Reply | 0 comments |
headers
RW | 1 Feb 18:36

Re: Burst of large messages sometimes causes spamd to lock up

On Wed, 01 Feb 2012 10:46:57 -0500
Kris Deugau wrote:

> RW wrote:
> > As regards the original question I just remembered that something
> > like this has come up before. IIRC there's a problem with
> > timing-out a spamd child while it's inside a regex, so a single bad
> > regex can cause serious problems. It's most likely  a rawbody, or
> > possibly a body rule.
> 
> How would that affect the master spamd process though?  Once it's 
> locked, new connection attempts just time out.  If it's a rogue
> regex, I'd expect to see a bunch of spamd children crunching away,
> but new connections going through just fine (at least up to
> max-children).

spamd adjusts the number of children when it gets  a message from one
of them reporting that it's idle. If you have all the busy children
locked-up, and have N idle children and then exactly N messages that
trigger the buggy regex come in at the same time, you lock-up all the
children and wont get any new idle events. 

In your case (--min-spare=1 --max-spare=1) you have N=1 which turns a
rare scenario into a common one.
Permalink | Reply | 4 comments |

Gmane