headers
Julian Field | 1 Aug 2008 11:09
Picon
Favicon

Re: dying children?


Richard Siddall wrote:
> Julian Field wrote:
>> Someone else showed me a message that suffered the same problem a few 
>> weeks ago. Unfortunately I don't think there's anything I can do 
>> about it, sorry. It's to do with nesting in the HTML analysis code. 
>> Once it gets too nested up, Perl segfaults.
>>
>> Jules
>>
>
> Jules,
>
> Does that mean it's something like an out-of-memory error in one of 
> the CPAN modules?  Can we fix it by getting the module author to 
> handle excessive nesting?
The most likely culprit is HTML::Parser, but I have direct evidence to 
back that. I just know that it's in the HTML parsing where it falls 
over. Does HTML::Parser contain any non-Perl code?

Jules

--

-- 
Julian Field MEng CITP CEng
www.MailScanner.info
Buy the MailScanner book at www.MailScanner.info/store

Need help customising MailScanner?
Contact me!
Need help fixing or optimising your systems?
(Continue reading)

Permalink | Reply | 2 comments |
headers
Paul Houselander (SME | 1 Aug 2008 11:48
Picon

Spam from Free mail accounts

Hi

 

Just wondered if anyone else was experiencing a lot of spam getting through that has been sent from yahoo.com, hotmail.com accounts etc….

 

Have seen a big increase in the last couple of weeks, they do actually come from hotmails and yahoo’s servers so the network based checks don’t flag anything.

 

I added a plugin from http://sa.hege.li/FreeMail.pm which just checks if the message is from a freemail account, which is working but a lot of my users receive legitimate mail from hotmail etc… so I can’t score to highly (currently set to 1).

 

The messages aren’t really hitting any other rules (I use SA 3.2.5, sa-update daily, SARE, KAM, DCC, razor, pyzor) – my BAYES db has been running for some time so I’ve removed it and started again with the starter one from fsl.com

 

Subjects are pretty random

 

Beauty latin girl posing bill pain

Kuuimshot on boiiobs hole teeth

Inteirraciial pee threesome bat cook

Cindy gaping snatch and Office glrIs in stockings

 

Just wondered if anyone else was seeing the same?

 

Cheers

 

Paul

 

 

 

--

-- 
MailScanner mailing list
mailscanner <at> lists.mailscanner.info
http://lists.mailscanner.info/mailman/listinfo/mailscanner

Before posting, read http://wiki.mailscanner.info/posting

Support MailScanner development - buy the book off the website!
Permalink | Reply | 2 comments |
headers
Steve Freegard | 1 Aug 2008 12:35

Re: Spam from Free mail accounts

Hi Paul,

Paul Houselander (SME) wrote:
> Hi
> 
>  
> Just wondered if anyone else was experiencing a lot of spam getting 
> through that has been sent from yahoo.com, hotmail.com accounts etc….
> 
> Have seen a big increase in the last couple of weeks, they do actually 
> come from hotmails and yahoo’s servers so the network based checks don’t 
> flag anything.

I've been getting a lot of hits from these on our spam trap too.

You can get network tests to work on Yahoo and Hotmail as they supply 
the injection IP address in the headers (either through a Received or 
X-Originating-IP).

The CBL (e.g. Spamhaus XBL works pretty good on some of these injection 
addresses) however SpamAssassin isn't configured to do these tests.

These rules will enable XBL tests on all the received headers for 
messages from Yahoo and Hotmail and should not cause FPs:

# Freemailers
header __FSL_HOST_YAHOO Received =~ /\.yahoo\.com/
header __FSL_HOST_HOTMAIL Received =~ /\.hotmail\.com/

# Check for SBL/XBL listings for all received headers from Yahoo and Hotmail
header __FSL_DEEP_RCVD_IN_SBLXBL 
eval:check_rbl_sub('zen','127.0.0.[2345678]')
tflags __FSL_DEEP_RCVD_IN_SBLXBL net
meta FSL_FREEMAIL_SBLXBL __FSL_DEEP_RCVD_IN_SBLXBL && (__FSL_HOST_YAHOO 
|| __FSL_HOST_HOTMAIL)
score FSL_FREEMAIL_SBLXBL 4.0

I've also got another rule that nukes all the mail to the trap, but 
isn't really tested well for FPs:

header __FSL_RCVD_YAHOO_BOT Received =~ /from unknown \(HELO 
(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})\) \(\S+ <at> \1 with login\)/
meta FSL_YAHOO_BOT __FSL_HOST_YAHOO && __FSL_RCVD_YAHOO_BOT
score FSL_YAHOO_BOT 3.0

Feel free to score it low and see if it hits the junk you are getting 
and then increase the score if it does.

> I added a plugin from http://sa.hege.li/FreeMail.pm which just checks if 
> the message is from a freemail account, which is working but a lot of my 
> users receive legitimate mail from hotmail etc… so I can’t score to 
> highly (currently set to 1).

FreeMail.pm isn't really meant for scoring messages from freemail 
providers (although you can do this like you are); but it's more for 
catching 419 scams that typically come from one FreeMail address and ask 
you to send details to another different freemail address (which it 
works pretty well on).

> 
> Just wondered if anyone else was seeing the same?
> 

Yup - I'm scoring them just high enough to mark them as spam:

Jul 31 22:19:18 mail spamd[18417]: spamd: result: Y 6 -

BMX_GREY,FROM_FREEMAIL,FSL_YAHOO_BOT,RCVD_NUMERIC_HELO,FSL_FREEMAIL_SBLXBL
scantime=1.5,size=2311,user=(unknown),uid=99,required_score=5.0,rhost=localhost.localdomain,raddr=127.0.0.1,rport=35384,mid=<EMEW-k6UMJB75b3176a00335c50134e9307355ed6be-822954.14081.bm <at> omp203.mail.ukl.yahoo.com>,autolearn=disabled,shortcircuit=no 

Kind regards,
Steve.
--

-- 
MailScanner mailing list
mailscanner <at> lists.mailscanner.info
http://lists.mailscanner.info/mailman/listinfo/mailscanner

Before posting, read http://wiki.mailscanner.info/posting

Support MailScanner development - buy the book off the website!
Permalink | Reply | 1 comment |
headers
ram | 1 Aug 2008 13:42
Picon

Custom spam scanner score not getting added

I have MailScanner 4.64 on my servers where I see that sometimes the
score given by is not added to the total score while marking a mail
spam 

So if SpamAssassin returns 0 score and Custom Scanner returns 7 the mail
should be marked as spam. In most cases this works but some mails
strangely enough dont get marked spam 

It happens rarely , 3-4 times in a day ,  and is not replicated , so I
dont now how do I check this 

Any pointers ?? 

Thanks
Ram

--

-- 
MailScanner mailing list
mailscanner <at> lists.mailscanner.info
http://lists.mailscanner.info/mailman/listinfo/mailscanner

Before posting, read http://wiki.mailscanner.info/posting

Support MailScanner development - buy the book off the website!
Permalink | Reply | 0 comments |
headers
Dave Jenkins | 1 Aug 2008 13:58

Basic Postfix/MS question: avoid scanning mail for invalid recipients

Hi,

We have Postfix/MS/SA running on our main & backup mail servers. The
main server is both incoming MX and outgoing, auth required. We'd like
every mail to a valid recipient to be scanned, but don't want to waste
CPU scanning mail to invalid recipients.

Main MX: am I right in thinking Postfix will remove mail to invalid
recipients before it gets to MS & so that setting MS to scan
everything will achieve what we want?

Backup MX: I've written a script to turn /etc/postfix/virtusertable
from the main MX into a valid rules file to be rsync'd to the backup
MX's rules directory, is that a sensible approach? Does MS need to be
restarted when the rules files are updated?

Thanks and sorry for the basic nature of the question,

Dave

CentOS 5.2
postfix-2.3.3-2
mailscanner-4.69.8-1
spamassassin-3.1.9-1.el5
--

-- 
MailScanner mailing list
mailscanner <at> lists.mailscanner.info
http://lists.mailscanner.info/mailman/listinfo/mailscanner

Before posting, read http://wiki.mailscanner.info/posting

Support MailScanner development - buy the book off the website!
Permalink | Reply | 2 comments |
headers
ram | 1 Aug 2008 14:50
Picon

Re: Basic Postfix/MS question: avoid scanning mail for invalid recipients


On Fri, 2008-08-01 at 12:58 +0100, Dave Jenkins wrote:
> Hi,
> 
> We have Postfix/MS/SA running on our main & backup mail servers. The
> main server is both incoming MX and outgoing, auth required. We'd like
> every mail to a valid recipient to be scanned, but don't want to waste
> CPU scanning mail to invalid recipients.
> 

MailScanner is not involved here at all 

Deal with invalid recipients at postfix level. Do not accept mails for
invalid recipients at all by using recipient checks right at the entry
point. That way you save a lot of resources and avoid sending
misdirected bounces 

> Main MX: am I right in thinking Postfix will remove mail to invalid
> recipients before it gets to MS & so that setting MS to scan
> everything will achieve what we want?
> 
> Backup MX: I've written a script to turn /etc/postfix/virtusertable
> from the main MX into a valid rules file to be rsync'd to the backup
> MX's rules directory, is that a sensible approach? Does MS need to be
> restarted when the rules files are updated?
> 

Yes Using a virtusertable with all valid recipients is a good way for
this 
Use a hash/CDB database. You will not have restart anything. Just
postmap the file and you are done

Thanks
Ram

--

-- 
MailScanner mailing list
mailscanner <at> lists.mailscanner.info
http://lists.mailscanner.info/mailman/listinfo/mailscanner

Before posting, read http://wiki.mailscanner.info/posting

Support MailScanner development - buy the book off the website!
Permalink | Reply | 0 comments |
headers
Drew Marshall | 1 Aug 2008 15:30

Re: dying children?

On 1 Aug 2008, at 10:09, Julian Field wrote:
>
> Richard Siddall wrote:
>> Julian Field wrote:
>>> Someone else showed me a message that suffered the same problem a  
>>> few weeks ago. Unfortunately I don't think there's anything I can  
>>> do about it, sorry. It's to do with nesting in the HTML analysis  
>>> code. Once it gets too nested up, Perl segfaults.
>>>
>>> Jules
>>>
>>
>> Jules,
>>
>> Does that mean it's something like an out-of-memory error in one of  
>> the CPAN modules?  Can we fix it by getting the module author to  
>> handle excessive nesting?
> The most likely culprit is HTML::Parser, but I have direct evidence  
> to back that. I just know that it's in the HTML parsing where it  
> falls over. Does HTML::Parser contain any non-Perl code?

Jules

I seem to get a number of, what I think are, these types of mail that  
choke MS and hold the child process up until it times out. Is there  
any way that a mail that causes this sort of time out to be  
automatically quarantined? Perhaps by changing the scan time out from  
a batch time out to a message time out?

The problem that I see is that if a batch has 10 messages in it (Often  
mainly Spam) 1 of the messages chokes spam scanning, the whole batch  
times out and lets the all the other spam messages through for delivery.

The benefit of quarantining is that:
1. I can find the dodgy message and perhaps we can find a solution to  
a common problem
2. If it's spam the user won't care
3. Users will be notified as normal (E.g. through Mail Watch  
notification or warning message etc depending on set up)
4. All other users still get their mail scanned

While I can't code, I'm happy to test!

Kind regards

Drew

--
In line with our policy, this message has been scanned for viruses and dangerous
 content by Technology Tiger's Mail Launder system <www.mail-launder.com>
Our email policy can be found at www.technologytiger.net/policy

Technology Tiger Limited is registered in Scotland with registration number: 310997
Registered Office 55-57 West High Street Inverurie AB51 3QQ

--

-- 
MailScanner mailing list
mailscanner <at> lists.mailscanner.info
http://lists.mailscanner.info/mailman/listinfo/mailscanner

Before posting, read http://wiki.mailscanner.info/posting

Support MailScanner development - buy the book off the website!
Permalink | Reply | 0 comments |
headers
Julian Field | 1 Aug 2008 15:36
Picon
Favicon

Re: dying children?


Drew Marshall wrote:
> On 1 Aug 2008, at 10:09, Julian Field wrote:
>>
>> Richard Siddall wrote:
>>> Julian Field wrote:
>>>> Someone else showed me a message that suffered the same problem a 
>>>> few weeks ago. Unfortunately I don't think there's anything I can 
>>>> do about it, sorry. It's to do with nesting in the HTML analysis 
>>>> code. Once it gets too nested up, Perl segfaults.
>>>>
>>>> Jules
>>>>
>>>
>>> Jules,
>>>
>>> Does that mean it's something like an out-of-memory error in one of 
>>> the CPAN modules?  Can we fix it by getting the module author to 
>>> handle excessive nesting?
>> The most likely culprit is HTML::Parser, but I have direct evidence 
>> to back that. I just know that it's in the HTML parsing where it 
>> falls over. Does HTML::Parser contain any non-Perl code?
>
>
> Jules
>
> I seem to get a number of, what I think are, these types of mail that 
> choke MS and hold the child process up until it times out. Is there 
> any way that a mail that causes this sort of time out to be 
> automatically quarantined? Perhaps by changing the scan time out from 
> a batch time out to a message time out?
>
> The problem that I see is that if a batch has 10 messages in it (Often 
> mainly Spam) 1 of the messages chokes spam scanning, the whole batch 
> times out and lets the all the other spam messages through for delivery.
It's not a timeout issue. If it hits this, it brings the entire Perl 
system crashing down. Wrapping it in an eval and timeout may not help. 
It would certainly add more overhead.

I am open to all suggestions though!
>
> The benefit of quarantining is that:
> 1. I can find the dodgy message and perhaps we can find a solution to 
> a common problem
> 2. If it's spam the user won't care
> 3. Users will be notified as normal (E.g. through Mail Watch 
> notification or warning message etc depending on set up)
> 4. All other users still get their mail scanned
>
> While I can't code, I'm happy to test!
>
> Kind regards
>
> Drew
>
> -- 
> In line with our policy, this message has been scanned for viruses and 
> dangerous
> content by Technology Tiger's Mail Launder system <www.mail-launder.com>
> Our email policy can be found at www.technologytiger.net/policy
>
> Technology Tiger Limited is registered in Scotland with registration 
> number: 310997
> Registered Office 55-57 West High Street Inverurie AB51 3QQ
>
>

Jules

-- 
Julian Field MEng CITP CEng
www.MailScanner.info
Buy the MailScanner book at www.MailScanner.info/store

Need help customising MailScanner?
Contact me!
Need help fixing or optimising your systems?
Contact me!
Need help getting you started solving new requirements from your boss?
Contact me!

PGP footprint: EE81 D763 3DB0 0BFD E1DC 7222 11F6 5947 1415 B654

-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.

--

-- 
MailScanner mailing list
mailscanner <at> lists.mailscanner.info
http://lists.mailscanner.info/mailman/listinfo/mailscanner

Before posting, read http://wiki.mailscanner.info/posting

Support MailScanner development - buy the book off the website!
Permalink | Reply | 1 comment |
headers
Drew Marshall | 1 Aug 2008 15:50

Re: dying children?

On 1 Aug 2008, at 14:36, Julian Field wrote:

>
>
> Drew Marshall wrote:
>> On 1 Aug 2008, at 10:09, Julian Field wrote:
>>>
>>> Richard Siddall wrote:
>>>> Julian Field wrote:
>>>>> Someone else showed me a message that suffered the same problem  
>>>>> a few weeks ago. Unfortunately I don't think there's anything I  
>>>>> can do about it, sorry. It's to do with nesting in the HTML  
>>>>> analysis code. Once it gets too nested up, Perl segfaults.
>>>>>
>>>>> Jules
>>>>>
>>>>
>>>> Jules,
>>>>
>>>> Does that mean it's something like an out-of-memory error in one  
>>>> of the CPAN modules?  Can we fix it by getting the module author  
>>>> to handle excessive nesting?
>>> The most likely culprit is HTML::Parser, but I have direct  
>>> evidence to back that. I just know that it's in the HTML parsing  
>>> where it falls over. Does HTML::Parser contain any non-Perl code?
>>
>>
>> Jules
>>
>> I seem to get a number of, what I think are, these types of mail  
>> that choke MS and hold the child process up until it times out. Is  
>> there any way that a mail that causes this sort of time out to be  
>> automatically quarantined? Perhaps by changing the scan time out  
>> from a batch time out to a message time out?
>>
>> The problem that I see is that if a batch has 10 messages in it  
>> (Often mainly Spam) 1 of the messages chokes spam scanning, the  
>> whole batch times out and lets the all the other spam messages  
>> through for delivery.
> It's not a timeout issue. If it hits this, it brings the entire Perl  
> system crashing down. Wrapping it in an eval and timeout may not  
> help. It would certainly add more overhead.
>
> I am open to all suggestions though!

Ahh, I see. Can I request this anyway? I am seeing a number of time  
outs (Like upwards of 50 since midnight today!) from SpamAssassin  
which I just can't diagnose. The problem message always seems to stop  
at the point it starts running body checks (Or at least that's the  
line it displays last before hanging). I have done all the usual stuff  
but as I can't easily capture the problem messages as they are  
delivered after the time out limit is hit, I am struggling to put them  
somewhere for some expert help.

Drew

--
In line with our policy, this message has been scanned for viruses and dangerous
 content by Technology Tiger's Mail Launder system <www.mail-launder.com>
Our email policy can be found at www.technologytiger.net/policy

Technology Tiger Limited is registered in Scotland with registration number: 310997
Registered Office 55-57 West High Street Inverurie AB51 3QQ

--

-- 
MailScanner mailing list
mailscanner <at> lists.mailscanner.info
http://lists.mailscanner.info/mailman/listinfo/mailscanner

Before posting, read http://wiki.mailscanner.info/posting

Support MailScanner development - buy the book off the website!
Permalink | Reply | 0 comments |
headers
Julian Field | 1 Aug 2008 15:59
Picon
Favicon

Re: dying children?


Drew Marshall wrote:
> On 1 Aug 2008, at 14:36, Julian Field wrote:
>
>>
>>
>> Drew Marshall wrote:
>>> On 1 Aug 2008, at 10:09, Julian Field wrote:
>>>>
>>>> Richard Siddall wrote:
>>>>> Julian Field wrote:
>>>>>> Someone else showed me a message that suffered the same problem a 
>>>>>> few weeks ago. Unfortunately I don't think there's anything I can 
>>>>>> do about it, sorry. It's to do with nesting in the HTML analysis 
>>>>>> code. Once it gets too nested up, Perl segfaults.
>>>>>>
>>>>>> Jules
>>>>>>
>>>>>
>>>>> Jules,
>>>>>
>>>>> Does that mean it's something like an out-of-memory error in one 
>>>>> of the CPAN modules?  Can we fix it by getting the module author 
>>>>> to handle excessive nesting?
>>>> The most likely culprit is HTML::Parser, but I have direct evidence 
>>>> to back that. I just know that it's in the HTML parsing where it 
>>>> falls over. Does HTML::Parser contain any non-Perl code?
>>>
>>>
>>> Jules
>>>
>>> I seem to get a number of, what I think are, these types of mail 
>>> that choke MS and hold the child process up until it times out. Is 
>>> there any way that a mail that causes this sort of time out to be 
>>> automatically quarantined? Perhaps by changing the scan time out 
>>> from a batch time out to a message time out?
>>>
>>> The problem that I see is that if a batch has 10 messages in it 
>>> (Often mainly Spam) 1 of the messages chokes spam scanning, the 
>>> whole batch times out and lets the all the other spam messages 
>>> through for delivery.
>> It's not a timeout issue. If it hits this, it brings the entire Perl 
>> system crashing down. Wrapping it in an eval and timeout may not 
>> help. It would certainly add more overhead.
>>
>> I am open to all suggestions though!
>
> Ahh, I see. Can I request this anyway? I am seeing a number of time 
> outs (Like upwards of 50 since midnight today!) from SpamAssassin 
> which I just can't diagnose. The problem message always seems to stop 
> at the point it starts running body checks (Or at least that's the 
> line it displays last before hanging). I have done all the usual stuff 
> but as I can't easily capture the problem messages as they are 
> delivered after the time out limit is hit, I am struggling to put them 
> somewhere for some expert help.
I've actually got a couple of things to go to this weekend, so may well 
not have much time for coding.

It may be the start of next week....

Jules

-- 
Julian Field MEng CITP CEng
www.MailScanner.info
Buy the MailScanner book at www.MailScanner.info/store

Need help customising MailScanner?
Contact me!
Need help fixing or optimising your systems?
Contact me!
Need help getting you started solving new requirements from your boss?
Contact me!

PGP footprint: EE81 D763 3DB0 0BFD E1DC 7222 11F6 5947 1415 B654

-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.

--

-- 
MailScanner mailing list
mailscanner <at> lists.mailscanner.info
http://lists.mailscanner.info/mailman/listinfo/mailscanner

Before posting, read http://wiki.mailscanner.info/posting

Support MailScanner development - buy the book off the website!
Permalink | Reply | 1 comment |

Gmane