Miles Crawford | 1 Apr 06:31

Re: httpd-guardian errors not in error_log?


Thanks, I guess I knew I could append the output to any file I liked in this fashion.  I suppose my question was more of a "why?" than a "how?" ;)

Perhaps a note should be made here?
Or within the comments of httpd-guardian itself, which seem to indicate this isn't a necessary step.

Either way, it's not a big deal.  Thanks for mod_security, it's been working great for us!

-Miles

On 3/31/07, Ryan Barnett < Ryan.Barnett <at> breach.com> wrote:

Miles,

Just change the location of your open command to point to your Apache error_log like this –

 

open STDERR, ">>/path/to/apache/logs/error_log";

 

Now, once httpd-guardian takes action, you will see entries like this in your error_log file –

 

httpd-guardian: IP address HASH(0x92e08d0) reached the 1 min threshold (speed = 2.76712328767123 req/sec, threshold = 0.01 req/sec)

httpd-guardian: Executing: /usr/bin/logger DoS Attack Identified from 127.0.0.1

 

The 2nd line above is from my own "test" configuration where I am just using logger to create a Syslog alert and not actually blackholing the client on the firewall.

 

# If defined, execute this command when a threshold is reached

# block the IP address for one hour.

# $PROTECT_EXEC = "/sbin/blacklist block %s 3600";

# $PROTECT_EXEC = "/sbin/samtool -block -ip %s -dur 3600 snortsam.example.com";

my $PROTECT_EXEC;

 

# For testing only:

# $PROTECT_EXEC = "/sbin/blacklist-webclient %s 3600";

$PROTECT_EXEC = "/usr/bin/logger DoS Attack Identified from %s";

 

The point is that a message will be generated when httpd-guardian identifies a client that has gone over your defined threshold of requests/time interval and when it executes a responsive action.

 

Hope this helps.

 

--
Ryan C. Barnett
ModSecurity Community Manager

Breach Security: Director of Application Security Training
Web Application Security Consortium (WASC) Member
Author: Preventing Web Attacks with Apache

 

--------------

Web Security Threat Report Webinar on May 9, 2007 (12 pm EST)

Learn More About the Breach Webinar Series:

http://www.breach.com/webinars.asp

--------------

 

From: mod-security-users-bounces <at> lists.sourceforge.net [mailto:mod-security-users-bounces <at> lists.sourceforge.net] On Behalf Of Miles Crawford
Sent: Friday, March 30, 2007 5:52 PM
To: mod-security-users <at> lists.sourceforge.net
Subject: [mod-security-users] httpd-guardian errors not in error_log?

 


Hey folks, I'm playing with using mod_security and httpd-guardian to protect against scripting and automation.

httpd-guardian is configured like:

SecGuardianLog "|/usr/local/bin/httpd-guardian"

I think I'm off to a great start, the alerts are being triggered and so forth, but I get no httpd-guardian messages in the apache error_log.

Added the line:

open STDERR, ">>/tmp/hg_log";

at the top of the httpd-guardian script, and the messages all show up just fine in hg_log.  I was hoping to have them show alongside the rest of the Apache error_log, and that seems to be what the documentation suggests will happen.  Any advice?

I'm using:

ModSecurity for Apache 2.1.0
Apache/2.2.4 (Unix) mod_ssl/2.2.4 OpenSSL/0.9.8b DAV/2 mod_apreq2-20051231/2.6.0 mod_perl/2.0.3 Perl/v5.8.8
Rev 1.6 of httpd-guardian from the sourceforge CVS site

Thanks, and let me know if I can provide any other info that might help.
-Miles


-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
mod-security-users mailing list
mod-security-users <at> lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/mod-security-users
Jerry | 1 Apr 16:10
Picon
Picon

Re: Inktomi triggers "Host header is a numericIPaddress"

Ok, I get this now. The visitor is trying to access the site via the ip 
address rather than the domain name. In this case Inktomi is probably 
following a link somewhere where the IP was posted and not the url

(sidenote: this is a new dedicated server with a new IP. When ordering a new 
server it is always wise to ask the hoster what the IP address will be and 
to search for that IP to find out it's history! )

I guess it is a good thing to block these kind of request from search 
engines as it can prevent duplicate penalties. If a search engine has a site 
listed under url and IP then that's not good. I'll keep this rule as is.

Thanks. 

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
Ofer Shezaf | 2 Apr 00:42

Re: Netcraft triggering a false positive


Hi Jerry,

While such an exception can certainly be made to the Core Rule Set, some
questions come to mind:

- Why do you want to allow NetCraft in? By allowing others to learn
about your server without leaving any traces on the server itself
NetCraft can serve the bad guys. Is there a legitimate reason to allow
the NetCraft crawler in?

- Assuming we want to allow NetCraft in: it is probably not the only
one. Do you know about a publicly available white list of IP addresses
that can be used to exclude the rule?

~ Ofer Shezaf
ModSecurity Core Rule Set project leader

> -----Original Message-----
> From: mod-security-users-bounces <at> lists.sourceforge.net [mailto:mod-
> security-users-bounces <at> lists.sourceforge.net] On Behalf Of Jerry
> Sent: Saturday, March 31, 2007 1:26 PM
> To: mod-security-users <at> lists.sourceforge.net
> Subject: [mod-security-users] Netcraft triggering a false positive
> 
> --------------------------------------------------------
> [Fri Mar 30 09:30:17 2007] [error] [client 194.72.238.62] ModSecurity:
> Warning. Operator EQ match: 0. [id "960008"] [msg "Request Missing a
> Host
> Header"] [severity "WARNING"] [uri "/"] [unique_id
> "rmWrWsPy7C4AABykGm4AAAAM"]
> --------------------------------------------------------
> 
> That IP is netcraft which I believe is bona fide. Can this rule be
> modified
> or something put in to allow 194.72.238.62 to bypass the rule?
> 
> 
> 
> 
>
-----------------------------------------------------------------------
> --
> Take Surveys. Earn Cash. Influence the Future of IT
> Join SourceForge.net's Techsay panel and you'll get the chance to
share
> your
> opinions on IT & business topics through brief surveys-and earn cash
>
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVD
> EV
> _______________________________________________
> mod-security-users mailing list
> mod-security-users <at> lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/mod-security-users

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
Brian Rectanus | 2 Apr 18:07

Re: mod_security not doing what it's told to do

Ryan Barnett wrote:
>> Hi!
>>
>>
>> Am 31.03.2007 um 09:01 schrieb trol anon:
>>> Attempting to redirect a portion of a subnet to a particular
>>> website/URL:
>>>
>>>    SecFilterSelective REQUEST_URI "portal.html" log,allow
>>>    SecFilterSelective REMOTE_ADDR "^192\.168\.0\.200-210$"
>>> log,redirect: http://192.168.0.1/portal.html
>>>
> [Ryan Barnett] Your RegEx for REMOTE_ADDR is incorrect as it is not
> properly representing a digit range.  Use this -
> 
> SecFilterSelective REMOTE_ADDR "^192\.168\.0\.2([0][0-9]|10)$"
> log,redirect:http://192.168.0.1/portal.html
> 
>>> works except for the situation when the browser is first started,
>>> in which case I get a standard 404 (oh, at this point, I should
>>> probably let you know that I'm intercepting DNS queries from hosts
>>> in the 192.168.0.200-210 range and resolving *everything* to
>>> 192.168.0.1).  So, why is the web server is seeing URL with tail-
>>> end that look like this?:
>> The 404-error is a problem with mod_security/apache that I had
>> problems with, too. The thing is that apache processes 404-errors
>> slightly different/on its own,  probably circumventing some (all?) of
>> its modules which makes this being handled out of modsecurity's
>> scope. 
> [Ryan Barnett] This is correct.  Please take a look the Apache Request
> Cycle/ModSecurity Processing Phases Diagram in the Reference Manual -
> http://www.modsecurity.org/documentation/modsecurity-apache/2.1.0/modsec
> urity2-apache-reference.html#04-processing-phases.  ModSecurity rules
> that run in phase:1 (the Apache post-read-request hook) do not know if a
> request (for a file) is valid or not meaning that it doesn't know if a
> the file exists or will generate a 404 Not Found error.  The next
> request cycle hook that ModSecurity has is in phase:2 which is down in
> the "fixups" Request cycle phase.  ModSecurity 1.9's inbound hook is in
> the fixups phase which means that Apache is handling the 404s before it
> gets to Mod.  
> 
> Unfortunately, 404 are generated internally by Apache in one of the
> phases that come inbetween post-read-request and fixups.  Also, since
> Apache handles 404s internally, it doesn't go through the normal
> Response hook either where Mod could potentially intercept the response
> in either phase:3 or phase:4.  Keep in mind that this situation only
> exists when Mod is running in embedded-mode.  When Mod is installed on a
> reverse proxy server and the 404 is generated by a back-end system then
> it is possible for Mod to intercept the responses and take action.

I am working on a fix for this so that modsec can will see requests that
it normally would not have seen because they were handled internally by
Apache.  Hopefully in the 2.2.x timeframe.

>> To solve your problem, you could use apache's error-document
>> directive which will catch the above case.
>>
>> 	ErrorDocument 404 http://192.168.0.1/portal.html
>>
>> Perhaps Brian can bring in some more apache-related internal details
>> for the interested audience ;-)
>>
> [Ryan Barnett] Agreed, Brian is extremely skilled at bushwhacking
> through the Apache source code jungle :)

Heh.  Basically, Apache did not have a handler configured to handle the
request, so it used the default handler (file lookup) and the files was
not there, so it issued the 404 and terminated the request and did not
pass it on to the other modules (including mod_security).

The above ErrorDocument works fine if you want to hard-code the
servername in (I don't like that, personally).  If you don't then you
can do something like this:

  # Make a request for /_redirect/portal.html
  ErrorDocument 404 /_redirect/portal.html

  # Redirect your IP range to the real page
  RewriteCond %{REMOTE_ADDR} "^192\.168\.0\.2([0][0-9]|10)$"
  RewriteRule ^/_redirect(/.*) $1 [R,L]

Now, if you don't need to redirect the browser, you can just serve the
/portal.html page as any page that would have normally generated a 404:

  ErrorDocument 404 /portal.html

In this case, the page:

  http://your.server.name/foo/bar/does_not_exist.html

would show in the browser URL box, but the /portal.html page would
server the request.  This can be a bit confusing, though.

-B

--

-- 
Brian Rectanus
Breach Security

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
Vince Tingey | 2 Apr 22:49
Picon
Picon
Favicon

Remove 404 Errors from modsec_audit.log

Hi Everyone!

Another newb question here (probably going to see alot from me in the 
future as I learn to use this wonderful tool).

Is there anyway to remove apache 404 errors from the mod security log 
(modsec_audit.log) so I can see just mod security errors?  I have a 
separate apache error log for each website and am happy to see the 404 
errors there.  I'm trying to create custom rules and find it hard 
filtering through all the 404 errors (mostly generated by old links that 
web crawlers follow) to get to the ones that matter. 

--3801821a-A--
[01/Apr/2007:04:08:20 --0700] H1s85IlSYSIAABAbGSAAAAAH 72.30.215.27 
57048 137.82.97.34 80
--3801821a-B--
GET /robots.txt HTTP/1.0
Host: www.ellislab.biotech.ubc.ca
Accept: */*
User-Agent: Mozilla/5.0 (compatible; Yahoo! Slurp; 
http://help.yahoo.com/help/us/ysearch/slurp)
Accept-Encoding: gzip, x-gzip

--3801821a-F--
HTTP/1.1 404 Not Found
Content-Length: 331
Connection: close
Content-Type: text/html; charset=iso-8859-1

--3801821a-H--
Apache-Error: [file "core.c"] [line 3518] [level 3] File does not exist: 
/www/htdocs/www_ellislab/robots.txt
Stopwatch: 1175425700805860 2746 (585 2100 -)
Producer: ModSecurity v2.1.0 (Apache 2.x)
Server: Apache 2

--3801821a-Z--

Thanks,

--

-- 

Vince Tingey            |  Michael Smith Laboratories
IT Systems Coordinator  |  University of British Columbia
Tel:  604.822.8895      |  #301 - 2185 East Mall
www.msl.ubc.ca          |  Vancouver, BC, Canada, V6T 1Z4

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
Ryan Barnett | 2 Apr 23:14

Re: Remove 404 Errors from modsec_audit.log

What do you have SecAuditEngine set to?  You should probably go with
RelevantOnly for performance reasons.  If you have this directive set to
RelevantOnly, then check your SecAuditLogRelevantStatus setting.  The
default one in the Core Rules file will capture all 4xx and 5xx level
codes (which includes 404, which you don't want...).  Update the
directive to this -

SecAuditLogRelevantStatus "^(?:5|4\d[^4])"

Which will exclude 404s from the audit log.
-- 
Ryan C. Barnett
ModSecurity Community Manager
Breach Security: Director of Application Security Training
Web Application Security Consortium (WASC) Member
Author: Preventing Web Attacks with Apache

--------------
Web Security Threat Report Webinar on May 9, 2007 (12 pm EST)
Learn More About the Breach Webinar Series:
http://www.breach.com/webinars.asp
--------------

> -----Original Message-----
> From: mod-security-users-bounces <at> lists.sourceforge.net [mailto:mod-
> security-users-bounces <at> lists.sourceforge.net] On Behalf Of Vince
Tingey
> Sent: Monday, April 02, 2007 4:49 PM
> To: mod-security-users <at> lists.sourceforge.net
> Subject: [mod-security-users] Remove 404 Errors from modsec_audit.log
> 
> Hi Everyone!
> 
> Another newb question here (probably going to see alot from me in the
> future as I learn to use this wonderful tool).
> 
> Is there anyway to remove apache 404 errors from the mod security log
> (modsec_audit.log) so I can see just mod security errors?  I have a
> separate apache error log for each website and am happy to see the 404
> errors there.  I'm trying to create custom rules and find it hard
> filtering through all the 404 errors (mostly generated by old links
that
> web crawlers follow) to get to the ones that matter.
> 
> 
> --3801821a-A--
> [01/Apr/2007:04:08:20 --0700] H1s85IlSYSIAABAbGSAAAAAH 72.30.215.27
> 57048 137.82.97.34 80
> --3801821a-B--
> GET /robots.txt HTTP/1.0
> Host: www.ellislab.biotech.ubc.ca
> Accept: */*
> User-Agent: Mozilla/5.0 (compatible; Yahoo! Slurp;
> http://help.yahoo.com/help/us/ysearch/slurp)
> Accept-Encoding: gzip, x-gzip
> 
> --3801821a-F--
> HTTP/1.1 404 Not Found
> Content-Length: 331
> Connection: close
> Content-Type: text/html; charset=iso-8859-1
> 
> --3801821a-H--
> Apache-Error: [file "core.c"] [line 3518] [level 3] File does not
exist:
> /www/htdocs/www_ellislab/robots.txt
> Stopwatch: 1175425700805860 2746 (585 2100 -)
> Producer: ModSecurity v2.1.0 (Apache 2.x)
> Server: Apache 2
> 
> --3801821a-Z--
> 
> 
> Thanks,
> 
> --
> 
> Vince Tingey            |  Michael Smith Laboratories
> IT Systems Coordinator  |  University of British Columbia
> Tel:  604.822.8895      |  #301 - 2185 East Mall
> www.msl.ubc.ca          |  Vancouver, BC, Canada, V6T 1Z4
> 
> 
>
------------------------------------------------------------------------
-
> Take Surveys. Earn Cash. Influence the Future of IT
> Join SourceForge.net's Techsay panel and you'll get the chance to
share
> your
> opinions on IT & business topics through brief surveys-and earn cash
>
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDE
V
> _______________________________________________
> mod-security-users mailing list
> mod-security-users <at> lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/mod-security-users

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
gotroot | 3 Apr 09:48

Binary packages for debian

Hi !

I've noticed that there's no updates of Debian binary packages since
1.9.4 get out.

There's no implementation of the 2.x series.
And that's the Alberto Gonzalez Iniesta packages !

In official Debian repository you find only the 1.8.x.

Is there any reason about that ?

Sioban.

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
Favicon

Re: Binary packages for debian

On Tue, Apr 03, 2007 at 09:48:51AM +0200, gotroot <at> sioban.net wrote:
> Hi !
> 
> I've noticed that there's no updates of Debian binary packages since
> 1.9.4 get out.
> 
> There's no implementation of the 2.x series.
> And that's the Alberto Gonzalez Iniesta packages !
> 
> In official Debian repository you find only the 1.8.x.
> 
> Is there any reason about that ?

mod-security packages had to be removed from Debian due to licensing
problems. I still maintain them (updated) in my site:

http://etc.inittab.org/~agi/debian/

You may find there mod-sec 1.9.x and 2.x for both etch/sid and sarge.

Regards,

Alberto

--

-- 
Alberto Gonzalez Iniesta    | Formación, consultoría y soporte técnico
agi@(inittab.org|debian.org)| en GNU/Linux y software libre
Encrypted mail preferred    | http://inittab.com

Key fingerprint = 9782 04E7 2B75 405C F5E9  0C81 C514 AF8E 4BA4 01C3

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
gotroot | 3 Apr 15:32

Re: Binary packages for debian

Hi !

Is there any doc to help the migration between 1.9.x and 2.1.x ???

I'm somewhat lost

I would like a sample of modsecurity.conf with a little more than what
is in the .deb...

Thanks...

Sioban

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
Ryan Barnett | 3 Apr 15:37

Re: Binary packages for debian

I am putting the finishing touches on a 1.9 to 2.0 Migration Matrix that
will help users to translate directives/functionality between the Mod
versions.  It should be up on the Mod site soon (possibly later today).

-- 
Ryan C. Barnett
ModSecurity Community Manager
Breach Security: Director of Application Security Training
Web Application Security Consortium (WASC) Member
Author: Preventing Web Attacks with Apache

--------------
Web Security Threat Report Webinar on May 9, 2007 (12 pm EST)
Learn More About the Breach Webinar Series:
http://www.breach.com/webinars.asp
--------------

> -----Original Message-----
> From: mod-security-users-bounces <at> lists.sourceforge.net [mailto:mod-
> security-users-bounces <at> lists.sourceforge.net] On Behalf Of
> gotroot <at> sioban.net
> Sent: Tuesday, April 03, 2007 9:33 AM
> To: mod-security-users <at> lists.sourceforge.net
> Subject: Re: [mod-security-users] Binary packages for debian
> 
> Hi !
> 
> Is there any doc to help the migration between 1.9.x and 2.1.x ???
> 
> I'm somewhat lost
> 
> I would like a sample of modsecurity.conf with a little more than what
> is in the .deb...
> 
> Thanks...
> 
> Sioban
> 
>
------------------------------------------------------------------------
-
> Take Surveys. Earn Cash. Influence the Future of IT
> Join SourceForge.net's Techsay panel and you'll get the chance to
share
> your
> opinions on IT & business topics through brief surveys-and earn cash
>
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDE
V
> _______________________________________________
> mod-security-users mailing list
> mod-security-users <at> lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/mod-security-users

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

Gmane