Hi!
I got a simple single rule like this:
SecRule REQUEST_BODY "<wsse:Username>(.*)</wsse:Username>" phase:2,capture,t:none,setenv:SOAPUser=%{TX.1}
And no other rules configured at all!
When I send in the request with curl like this:
curl -k http://hostname/whatever/the/url --data-binary "<wsse:Username>mytest</wsse:Username>"
Then mod_security does match the regexp and set the environment variable.
But when I send the SAME request like this:
curl -k http://hostname/whatever/the/url --data-binary "<wsse:Username>mytest</wsse:Username>" -H "Content-Type: text/xml;charset=UTF-8"
it does not match and thus does not set the environment variable – even though it contains the SAME post.
In the matching case I get in the debug logs (set to 9) a line like this:
[05/Oct/2011:22:57:36 +0000] [hostname/sid#7f1582e28e28][rid#7f1582f0b8f8][/whatever/the/url][5] Adding request argument (BODY): name "<wsse:Username>mytest</wsse:Username>", value ""
This does not happen when sending the Content-Type header…
Is there an easy way to make it work in BOTH cases or at least to achieve the same in both cases
Thanks,
Martin
P.s: Here the full debug log:
The request without the content-type:
[05/Oct/2011:22:57:36 +0000] [hostname/sid#7f1582e28e28][rid#7f1582f0b8f8][/whatever/the/url][4] Initialising transaction (txid Tozg4AoKCvwAADp5kI4AAAAB).
[05/Oct/2011:22:57:36 +0000] [hostname/sid#7f1582e28e28][rid#7f1582f0b8f8][/whatever/the/url][4] Transaction context created (dcfg 7f1582da1e18).
[05/Oct/2011:22:57:36 +0000] [hostname/sid#7f1582e28e28][rid#7f1582f0b8f8][/whatever/the/url][4] First phase starting (dcfg 7f1582da1e18).
[05/Oct/2011:22:57:36 +0000] [hostname/sid#7f1582e28e28][rid#7f1582f0b8f8][/whatever/the/url][4] Starting phase REQUEST_HEADERS.
[05/Oct/2011:22:57:36 +0000] [hostname/sid#7f1582e28e28][rid#7f1582f0b8f8][/whatever/the/url][9] This phase consists of 0 rule(s).
[05/Oct/2011:22:57:36 +0000] [hostname/sid#7f1582e28e28][rid#7f1582f0b8f8][/whatever/the/url][4] Second phase starting (dcfg 7f1582da1e18).
[05/Oct/2011:22:57:36 +0000] [hostname/sid#7f1582e28e28][rid#7f1582f0b8f8][/whatever/the/url][4] Input filter: Reading request body.
[05/Oct/2011:22:57:36 +0000] [hostname/sid#7f1582e28e28][rid#7f1582f0b8f8][/whatever/the/url][9] Input filter: Bucket type HEAP contains 37 bytes.
[05/Oct/2011:22:57:36 +0000] [hostname/sid#7f1582e28e28][rid#7f1582f0b8f8][/whatever/the/url][9] Input filter: Bucket type EOS contains 0 bytes.
[05/Oct/2011:22:57:36 +0000] [hostname/sid#7f1582e28e28][rid#7f1582f0b8f8][/whatever/the/url][5] Adding request argument (BODY): name "<wsse:Username>mytest</wsse:Username>", value ""
[05/Oct/2011:22:57:36 +0000] [hostname/sid#7f1582e28e28][rid#7f1582f0b8f8][/whatever/the/url][4] Input filter: Completed receiving request body (length 37).
[05/Oct/2011:22:57:36 +0000] [hostname/sid#7f1582e28e28][rid#7f1582f0b8f8][/whatever/the/url][4] Starting phase REQUEST_BODY.
[05/Oct/2011:22:57:36 +0000] [hostname/sid#7f1582e28e28][rid#7f1582f0b8f8][/whatever/the/url][9] This phase consists of 1 rule(s).
[05/Oct/2011:22:57:36 +0000] [hostname/sid#7f1582e28e28][rid#7f1582f0b8f8][/whatever/the/url][4] Recipe: Invoking rule 7f1582e0d370; [file "/etc/httpd/modsecurity.d/ourrules.conf"] [line "4"].
[05/Oct/2011:22:57:36 +0000] [hostname/sid#7f1582e28e28][rid#7f1582f0b8f8][/whatever/the/url][5] Rule 7f1582e0d370: SecRule "REQUEST_BODY" " <at> rx <wsse:Username>(.*)</wsse:Username>" "phase:2,log,auditlog,pass,capture,t:none,setenv:SOAPUser=%{TX.1}"
[05/Oct/2011:22:57:36 +0000] [hostname/sid#7f1582e28e28][rid#7f1582f0b8f8][/whatever/the/url][4] Transformation completed in 7 usec.
[05/Oct/2011:22:57:36 +0000] [hostname/sid#7f1582e28e28][rid#7f1582f0b8f8][/whatever/the/url][4] Executing operator "rx" withparam "<wsse:Username>(.*)</wsse:Username>" against REQUEST_BODY.
[05/Oct/2011:22:57:36 +0000] [hostname/sid#7f1582e28e28][rid#7f1582f0b8f8][/whatever/the/url][9] Target value: "<wsse:Username>mytest</wsse:Username>"
[05/Oct/2011:22:57:36 +0000] [hostname/sid#7f1582e28e28][rid#7f1582f0b8f8][/whatever/the/url][9] Added regex subexpression toTX.0: <wsse:Username>mytest</wsse:Username>
[05/Oct/2011:22:57:36 +0000] [hostname/sid#7f1582e28e28][rid#7f1582f0b8f8][/whatever/the/url][9] Added regex subexpression toTX.1: mytest
[05/Oct/2011:22:57:36 +0000] [hostname/sid#7f1582e28e28][rid#7f1582f0b8f8][/whatever/the/url][4] Operator completed in 83 usec.
[05/Oct/2011:22:57:36 +0000] [hostname/sid#7f1582e28e28][rid#7f1582f0b8f8][/whatever/the/url][9] Setting env variable: SOAPUser=%{TX.1}
[05/Oct/2011:22:57:36 +0000] [hostname/sid#7f1582e28e28][rid#7f1582f0b8f8][/whatever/the/url][9] Resolved macro %{TX.1} to: mytest
[05/Oct/2011:22:57:36 +0000] [hostname/sid#7f1582e28e28][rid#7f1582f0b8f8][/whatever/the/url][9] Set env variable "SOAPUser" to: mytest
[05/Oct/2011:22:57:36 +0000] [hostname/sid#7f1582e28e28][rid#7f1582f0b8f8][/whatever/the/url][2] Warning. Pattern match "<wsse:Username>(.*)</wsse:Username>" at REQUEST_BODY. [file "/etc/httpd/modsecurity.d/ourrules.conf"] [line "4"]
[05/Oct/2011:22:57:36 +0000] [hostname/sid#7f1582e28e28][rid#7f1582f0b8f8][/whatever/the/url][4] Rule returned 1.
[05/Oct/2011:22:57:36 +0000] [hostname/sid#7f1582e28e28][rid#7f1582f0b8f8][/whatever/the/url][9] Match -> mode NEXT_RULE.
[05/Oct/2011:22:57:36 +0000] [hostname/sid#7f1582e28e28][rid#7f1582f0b8f8][/whatever/the/url][4] Hook insert_filter: Adding input forwarding filter (r 7f1582f0b8f8).
[05/Oct/2011:22:57:36 +0000] [hostname/sid#7f1582e28e28][rid#7f1582f0b8f8][/whatever/the/url][4] Hook insert_filter: Adding output filter (r 7f1582f0b8f8).
[05/Oct/2011:22:57:36 +0000] [hostname/sid#7f1582e28e28][rid#7f1582f0b8f8][/whatever/the/url][4] Input filter: Forwarding input: mode=0, block=0, nbytes=8192 (f 7f1582f14908, r 7f1582f0b8f8).
[05/Oct/2011:22:57:36 +0000] [hostname/sid#7f1582e28e28][rid#7f1582f0b8f8][/whatever/the/url][4] Input filter: Forwarded 37 bytes.
[05/Oct/2011:22:57:36 +0000] [hostname/sid#7f1582e28e28][rid#7f1582f0b8f8][/whatever/the/url][4] Input filter: Sent EOS.
[05/Oct/2011:22:57:36 +0000] [hostname/sid#7f1582e28e28][rid#7f1582f0b8f8][/whatever/the/url][4] Input filter: Input forwarding complete.
[05/Oct/2011:22:57:36 +0000] [hostname/sid#7f1582e28e28][rid#7f1582f0b8f8][/whatever/the/url][4] Hook insert_error_filter: Adding output filter (r 7f1582f0b8f8).
[05/Oct/2011:22:57:36 +0000] [hostname/sid#7f1582e28e28][rid#7f1582f0b8f8][/whatever/the/url][9] Output filter: Receiving output (f 7f1582f14bb0, r 7f1582f0b8f8).
[05/Oct/2011:22:57:36 +0000] [hostname/sid#7f1582e28e28][rid#7f1582f0b8f8][/whatever/the/url][4] Starting phase RESPONSE_HEADERS.
[05/Oct/2011:22:57:36 +0000] [hostname/sid#7f1582e28e28][rid#7f1582f0b8f8][/whatever/the/url][9] This phase consists of 0 rule(s).
[05/Oct/2011:22:57:36 +0000] [hostname/sid#7f1582e28e28][rid#7f1582f0b8f8][/whatever/the/url][9] Content Injection: Not enabled.
[05/Oct/2011:22:57:36 +0000] [hostname/sid#7f1582e28e28][rid#7f1582f0b8f8][/whatever/the/url][9] Output filter: Bucket type HEAP contains 296 bytes.
[05/Oct/2011:22:57:36 +0000] [hostname/sid#7f1582e28e28][rid#7f1582f0b8f8][/whatever/the/url][9] Output filter: Bucket type EOS contains 0 bytes.
[05/Oct/2011:22:57:36 +0000] [hostname/sid#7f1582e28e28][rid#7f1582f0b8f8][/whatever/the/url][4] Output filter: Completed receiving response body (buffered full - 296 bytes).
[05/Oct/2011:22:57:36 +0000] [hostname/sid#7f1582e28e28][rid#7f1582f0b8f8][/whatever/the/url][4] Starting phase RESPONSE_BODY.
[05/Oct/2011:22:57:36 +0000] [hostname/sid#7f1582e28e28][rid#7f1582f0b8f8][/whatever/the/url][9] This phase consists of 0 rule(s).
[05/Oct/2011:22:57:36 +0000] [hostname/sid#7f1582e28e28][rid#7f1582f0b8f8][/whatever/the/url][4] Output filter: Output forwarding complete.
[05/Oct/2011:22:57:36 +0000] [hostname/sid#7f1582e28e28][rid#7f1582f0b8f8][/whatever/the/url][4] Initialising logging.
[05/Oct/2011:22:57:36 +0000] [hostname/sid#7f1582e28e28][rid#7f1582f0b8f8][/whatever/the/url][4] Starting phase LOGGING.
[05/Oct/2011:22:57:36 +0000] [hostname/sid#7f1582e28e28][rid#7f1582f0b8f8][/whatever/the/url][9] This phase consists of 0 rule(s).
[05/Oct/2011:22:57:36 +0000] [hostname/sid#7f1582e28e28][rid#7f1582f0b8f8][/whatever/the/url][4] Recording persistent data took 0 microseconds.
[05/Oct/2011:22:57:36 +0000] [hostname/sid#7f1582e28e28][rid#7f1582f0b8f8][/whatever/the/url][4] Audit log: Not configured torun for this request.
The request with the mime-type header set:
[05/Oct/2011:22:57:46 +0000] [hostname/sid#7f1582e28e28][rid#7f1582f0b8f8][/whatever/the/url][4] Initialising transaction (txid Tozg6goKCvwAADp6kUAAAAAC).
[05/Oct/2011:22:57:46 +0000] [hostname/sid#7f1582e28e28][rid#7f1582f0b8f8][/whatever/the/url][4] Transaction context created (dcfg 7f1582da1e18).
[05/Oct/2011:22:57:46 +0000] [hostname/sid#7f1582e28e28][rid#7f1582f0b8f8][/whatever/the/url][4] First phase starting (dcfg 7f1582da1e18).
[05/Oct/2011:22:57:46 +0000] [hostname/sid#7f1582e28e28][rid#7f1582f0b8f8][/whatever/the/url][4] Starting phase REQUEST_HEADERS.
[05/Oct/2011:22:57:46 +0000] [hostname/sid#7f1582e28e28][rid#7f1582f0b8f8][/whatever/the/url][9] This phase consists of 0 rule(s).
[05/Oct/2011:22:57:46 +0000] [hostname/sid#7f1582e28e28][rid#7f1582f0b8f8][/whatever/the/url][4] Second phase starting (dcfg 7f1582da1e18).
[05/Oct/2011:22:57:46 +0000] [hostname/sid#7f1582e28e28][rid#7f1582f0b8f8][/whatever/the/url][4] Input filter: Reading request body.
[05/Oct/2011:22:57:46 +0000] [hostname/sid#7f1582e28e28][rid#7f1582f0b8f8][/whatever/the/url][9] Input filter: Bucket type HEAP contains 37 bytes.
[05/Oct/2011:22:57:46 +0000] [hostname/sid#7f1582e28e28][rid#7f1582f0b8f8][/whatever/the/url][9] Input filter: Bucket type EOS contains 0 bytes.
[05/Oct/2011:22:57:46 +0000] [hostname/sid#7f1582e28e28][rid#7f1582f0b8f8][/whatever/the/url][4] Request body no files length: 0
[05/Oct/2011:22:57:46 +0000] [hostname/sid#7f1582e28e28][rid#7f1582f0b8f8][/whatever/the/url][4] Input filter: Completed receiving request body (length 37).
[05/Oct/2011:22:57:46 +0000] [hostname/sid#7f1582e28e28][rid#7f1582f0b8f8][/whatever/the/url][4] Starting phase REQUEST_BODY.
[05/Oct/2011:22:57:46 +0000] [hostname/sid#7f1582e28e28][rid#7f1582f0b8f8][/whatever/the/url][9] This phase consists of 1 rule(s).
[05/Oct/2011:22:57:46 +0000] [hostname/sid#7f1582e28e28][rid#7f1582f0b8f8][/whatever/the/url][4] Recipe: Invoking rule 7f1582e0d370; [file "/etc/httpd/modsecurity.d/ourrules.conf"] [line "4"].
[05/Oct/2011:22:57:46 +0000] [hostname/sid#7f1582e28e28][rid#7f1582f0b8f8][/whatever/the/url][5] Rule 7f1582e0d370: SecRule "REQUEST_BODY" " <at> rx <wsse:Username>(.*)</wsse:Username>" "phase:2,log,auditlog,pass,capture,t:none,setenv:SOAPUser=%{TX.1}"
[05/Oct/2011:22:57:46 +0000] [hostname/sid#7f1582e28e28][rid#7f1582f0b8f8][/whatever/the/url][4] Rule returned 0.
[05/Oct/2011:22:57:46 +0000] [hostname/sid#7f1582e28e28][rid#7f1582f0b8f8][/whatever/the/url][9] No match, not chained -> mode NEXT_RULE.
[05/Oct/2011:22:57:46 +0000] [hostname/sid#7f1582e28e28][rid#7f1582f0b8f8][/whatever/the/url][4] Hook insert_filter: Adding input forwarding filter (r 7f1582f0b8f8).
[05/Oct/2011:22:57:46 +0000] [hostname/sid#7f1582e28e28][rid#7f1582f0b8f8][/whatever/the/url][4] Hook insert_filter: Adding output filter (r 7f1582f0b8f8).
[05/Oct/2011:22:57:46 +0000] [hostname/sid#7f1582e28e28][rid#7f1582f0b8f8][/whatever/the/url][4] Input filter: Forwarding input: mode=0, block=0, nbytes=8192 (f 7f1582f148f0, r 7f1582f0b8f8).
[05/Oct/2011:22:57:46 +0000] [hostname/sid#7f1582e28e28][rid#7f1582f0b8f8][/whatever/the/url][4] Input filter: Forwarded 37 bytes.
[05/Oct/2011:22:57:46 +0000] [hostname/sid#7f1582e28e28][rid#7f1582f0b8f8][/whatever/the/url][4] Input filter: Sent EOS.
[05/Oct/2011:22:57:46 +0000] [hostname/sid#7f1582e28e28][rid#7f1582f0b8f8][/whatever/the/url][4] Input filter: Input forwarding complete.
[05/Oct/2011:22:57:46 +0000] [hostname/sid#7f1582e28e28][rid#7f1582f0b8f8][/whatever/the/url][4] Hook insert_error_filter: Adding output filter (r 7f1582f0b8f8).
[05/Oct/2011:22:57:46 +0000] [hostname/sid#7f1582e28e28][rid#7f1582f0b8f8][/whatever/the/url][9] Output filter: Receiving output (f 7f1582f14b98, r 7f1582f0b8f8).
[05/Oct/2011:22:57:46 +0000] [hostname/sid#7f1582e28e28][rid#7f1582f0b8f8][/whatever/the/url][4] Starting phase RESPONSE_HEADERS.
[05/Oct/2011:22:57:46 +0000] [hostname/sid#7f1582e28e28][rid#7f1582f0b8f8][/whatever/the/url][9] This phase consists of 0 rule(s).
[05/Oct/2011:22:57:46 +0000] [hostname/sid#7f1582e28e28][rid#7f1582f0b8f8][/whatever/the/url][9] Content Injection: Not enabled.
[05/Oct/2011:22:57:46 +0000] [hostname/sid#7f1582e28e28][rid#7f1582f0b8f8][/whatever/the/url][9] Output filter: Bucket type HEAP contains 296 bytes.
[05/Oct/2011:22:57:46 +0000] [hostname/sid#7f1582e28e28][rid#7f1582f0b8f8][/whatever/the/url][9] Output filter: Bucket type EOS contains 0 bytes.
[05/Oct/2011:22:57:46 +0000] [hostname/sid#7f1582e28e28][rid#7f1582f0b8f8][/whatever/the/url][4] Output filter: Completed receiving response body (buffered full - 296 bytes).
[05/Oct/2011:22:57:46 +0000] [hostname/sid#7f1582e28e28][rid#7f1582f0b8f8][/whatever/the/url][4] Starting phase RESPONSE_BODY.
[05/Oct/2011:22:57:46 +0000] [hostname/sid#7f1582e28e28][rid#7f1582f0b8f8][/whatever/the/url][9] This phase consists of 0 rule(s).
[05/Oct/2011:22:57:46 +0000] [hostname/sid#7f1582e28e28][rid#7f1582f0b8f8][/whatever/the/url][4] Output filter: Output forwarding complete.
[05/Oct/2011:22:57:46 +0000] [hostname/sid#7f1582e28e28][rid#7f1582f0b8f8][/whatever/the/url][4] Initialising logging.
[05/Oct/2011:22:57:46 +0000] [hostname/sid#7f1582e28e28][rid#7f1582f0b8f8][/whatever/the/url][4] Starting phase LOGGING.
[05/Oct/2011:22:57:46 +0000] [hostname/sid#7f1582e28e28][rid#7f1582f0b8f8][/whatever/the/url][9] This phase consists of 0 rule(s).
[05/Oct/2011:22:57:46 +0000] [hostname/sid#7f1582e28e28][rid#7f1582f0b8f8][/whatever/the/url][4] Recording persistent data took 0 microseconds.
[05/Oct/2011:22:57:46 +0000] [hostname/sid#7f1582e28e28][rid#7f1582f0b8f8][/whatever/the/url][4] Audit log: Not configured torun for this request.