This is an updated section from my previous book Preventing Web Attacks with Apache and discusses a log analysis script for searching the ModSecurity audit log file.
The data provided by the ModSecurity audit_log file is invaluable when conducting incident response for web attacks. The data represents the entire client request including the client headers and data payloads. While the value of this data is undeniable, there are some practical issues to deal with if this data is to be used in an efficient manner. For those of you who have been using ModSecurity, you have almost certainly run into this issue that I am about to discuss.
Many ModSecurity users have thier SecAuditLogType directive set to Serial which will log all transactional data to one file rather than breaking them out into separate files with Concurrent. While this may be easier to manage, it does present some challenges with regards to quickly reviewing log data.
OK, let's imagine that you have received a ModSecurity alert in your Apache error_log that looks something like this:
[Thu Aug 11 04:30:20 2011] [error] [client 67.195.115.98] ModSecurity: Warning. Pattern match "(\\\\/\\\\*\\\\!?|\\\\*\\\\/|\\\\-\\\\-[\\\\s\\\\r\\\\n\\\\v\\\\f]|(?:--[^-]*-)|([^\\\\-&])#.*[\\\\s\\\\r\\\\n\\\\v\\\\f]|;?\\\\x00)" at ARGS:artist. [file "/etc/httpd/modsecurity.d/crs/base_rules/modsecurity_crs_41_sql_injection_attacks.conf"] [line "37"] [id "981231"] [rev "2.2.2"] [msg "SQL Comment Sequence Detected."] [data "n#foo*/*bar\\x0d\\x0aselect#foo\\x0d\\x0a"] [tag "WEB_ATTACK/SQL_INJECTION"] [tag "WASCTC/WASC-19"] [tag "OWASP_TOP_10/A1"] [tag "OWASP_AppSensor/CIE1"] [tag "PCI/6.5.2"] [hostname "www.modsecurity.org"] [uri "/testphp.vulnweb.com/artists.php"] [unique_id "chVQpcCo8AoAAGd3pV0AAAAM"]
This alert indicates that a remote client triggered an SQL Injection - Comment Sequence alert. With these types of terse messages, we often do not have enough information to accurately categorize this as a false positive or if it is a real attack. In this case, the logdata information seems to indicate that this is true positive however many times the data is inconclusive. In order to validate the alert, we must review the full transactional data within the audit_log file.
This leads us to our issue with the audit_log format. If I wanted to inspect the entire audit_log record for this alert, I would have a tough time extracting out the desired data due to the fact that the audit_log is a multi-line record. If I use a simple grep command, I only get a small amount of data:
# grep 67.195.115.98 audit_log[11/Aug/2011:04:30:21 --0500] chVQpcCo8AoAAGd3pV0AAAAM 67.195.115.98 46833 192.168.240.10 80
This only extracts out Section A (Audit Log Header) from the transaction. It is possible to use the "context" grep flags to try and extract out more lines around the match. As an example, this command uses the -A flag to also print "N" number of lines after the match:
# grep -A10 67.195.115.98 audit_log[11/Aug/2011:04:30:21 --0500] chVQpcCo8AoAAGd3pV0AAAAM 67.195.115.98 46833 192.168.240.10 80--8802d071-B--GET /testphp.vulnweb.com/artists.php?artist=0+div+1+union%23foo*%2F*bar%0D%0Aselect%23foo%0D%0A1%2C2%2Ccurrent_user HTTP/1.0Host: www.modsecurity.orgUser-Agent: Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp)Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5Accept-Language: en-us,en;q=0.5Accept-Encoding: gzipAccept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7--8802d071-F--
While this does show more data from the log entry, it does not perform well in bulk searches as there is never a consistent number of client headers being sent and will miss data if a request body exists, so the accuracy of the returned data is a bit off. What I need wis a script that functions similarly to grep; however, it is able to understand the ModSecurity audit_log format and be able to extract out the multi-line record.
This lead to the creation of a script called modgrep. Modgrep is a PERL script that accomplishes the tasks that we need to successfully parse and extract entire audit_log records that contain the desired search text string.
You can download the script here and it will soon be added to the ModSecurity archive however we are presenting the complete code here as well:
#!/usr/bin/perluse Getopt::Std;use vars qw/ %opt /;my $opt_string = 'hf:s:v:';getopts( "$opt_string", \%opt ) or usage();usage() if $opt{h};if ($opt{f}) { open(FH,"<$opt{f}") || die "cant find $opt{f} file $!\n";}if ($opt{s}) { $sstr = $opt{s};}$/ = "-Z--\n";while(my $line = ) { chomp $line; if ($line =~ m/$sstr/g) { print "========================================\nMatched Transaction for Search String ($sstr)\n========================================\n"; print $line,"-Z--\n"; print "\n"; }}close(FH);exit;sub usage(){ print STDERR << "EOF";This program does...usage: $0 [-hf:s:v:] if a file is compressed then it will be uncompressed on the fly default : display usage -f file : file to search through -s string : string to match on - enclosed in quotes if it contains spaces -v level : verbose output -h : this (help) messageexample: $0 $0 -f $0 -s $0 -v level Verbose/Debug messages, where level = 0..9EOF exit;}
They key line in the script is highlighted, which specifies the Section Z - Audit log footer separator. Here is an example session of using modgrep to search the audit_log file for the same search string as was used with standardgrep:
# modgrep -f audit_log -s "67.195.115.98"========================================Matched Transaction for Search String (67.195.115.98)========================================--8802d071-A--[11/Aug/2011:04:30:21 --0500] chVQpcCo8AoAAGd3pV0AAAAM 67.195.115.98 46833 192.168.240.10 80--8802d071-B--GET /testphp.vulnweb.com/artists.php?artist=0+div+1+union%23foo*%2F*bar%0D%0Aselect%23foo%0D%0A1%2C2%2Ccurrent_user HTTP/1.0Host: www.modsecurity.orgUser-Agent: Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp)Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5Accept-Language: en-us,en;q=0.5Accept-Encoding: gzipAccept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7--8802d071-F--HTTP/1.1 200 OKX-Powered-By: PHP/5.1.2Content-Length: 1177Content-Type: text/html; charset=UTF-8Connection: close--8802d071-E--<br /><strong>Warning</strong>: mysql_connect() [<a href="function.mysql-connect">function.mysql-connect</a>]: Too many connections in <strong>/var/www/vhosts/default/htdocs/database_connect.php</strong> on line <strong>2</strong><br />Website is out of order. Please visit back later. Thank you for understanding.--8802d071-H--Message: Warning. Pattern match "(\\/\\*\\!?|\\*\\/|\\-\\-[\\s\\r\\n\\v\\f]|(?:--[^-]*-)|([^\\-&])#.*[\\s\\r\\n\\v\\f]|;?\\x00)" at ARGS:artist. [file "/etc/httpd/modsecurity.d/crs/base_rules/modsecurity_crs_41_sql_injection_attacks.conf"] [line "37"] [id "981231"] [rev "2.2.2"] [msg "SQL Comment Sequence Detected."] [data "n#foo*/*bar\x0d\x0aselect#foo\x0d\x0a"] [tag "WEB_ATTACK/SQL_INJECTION"] [tag "WASCTC/WASC-19"] [tag "OWASP_TOP_10/A1"] [tag "OWASP_AppSensor/CIE1"] [tag "PCI/6.5.2"]Message: Warning. Pattern match "(?i:\\bunion\\b.{1,100}?\\bselect\\b)" at ARGS:artist. [file "/etc/httpd/modsecurity.d/crs/base_rules/modsecurity_crs_41_sql_injection_attacks.conf"] [line "318"] [id "959047"] [rev "2.2.2"] [msg "SQL Injection Attack"] [data "union#foo*/*bar\x0d\x0aselect"] [severity "CRITICAL"] [tag "WEB_ATTACK/SQL_INJECTION"] [tag "WASCTC/WASC-19"] [tag "OWASP_TOP_10/A1"] [tag "OWASP_AppSensor/CIE1"] [tag "PCI/6.5.2"]Message: Warning. Pattern match "(?i:(?:(?:s(?:t(?:d(?:dev(_pop|_samp)?)?|r(?:_to_date|cmp))|u(?:b(?:str(?:ing(_index)?)?|(?:dat|tim)e)|m)|e(?:c(?:_to_time|ond)|ssion_user)|ys(?:tem_user|date)|ha(1|2)?|oundex|chema|ig?n|pace|qrt)|i(?:s(null|_(free_lock|ipv4_compat|ipv4_mapped|ipv4|ipv ..." at ARGS:artist. [file "/etc/httpd/modsecurity.d/crs/base_rules/modsecurity_crs_41_sql_injection_attacks.conf"] [line "496"] [id "959073"] [rev "2.2.2"] [msg "SQL Injection Attack"] [data "union#foo*/*bar\x0d\x0aselect"] [severity "CRITICAL"] [tag "WEB_ATTACK/SQL_INJECTION"] [tag "WASCTC/WASC-19"] [tag "OWASP_TOP_10/A1"] [tag "OWASP_AppSensor/CIE1"] [tag "PCI/6.5.2"]Message: Warning. Pattern match "([\\~\\!\\@\\#\\$\\%\\^\\&\\*\\(\\)\\-\\+\\=\\{\\}\\[\\]\\|\\:\\;\"\\'\\\xc2\xb4\\\xe2\x80\x99\\\xe2\x80\x98\\`\\<\\>].*){4,}" at ARGS:artist. [file "/etc/httpd/modsecurity.d/crs/base_rules/modsecurity_crs_41_sql_injection_attacks.conf"] [line "523"] [id "981173"] [rev "2.2.1"] [msg "Restricted SQL Character Anomaly Detection Alert - Total # of special characters exceeded"] [data "#foo\x0d\x0a1,2,current_user"]Message: Warning. Pattern match "(?i:(?:\\d(\"|'|`|\xc2\xb4|\xe2\x80\x99|\xe2\x80\x98)\\s+(\"|'|`|\xc2\xb4|\xe2\x80\x99|\xe2\x80\x98)\\s+\\d)|(?:^admin\\s*(\"|'|`|\xc2\xb4|\xe2\x80\x99|\xe2\x80\x98)|(\\/\\*)+(\"|'|`|\xc2\xb4|\xe2\x80\x99|\xe2\x80\x98)+\\s?(?:--|#|\\/\\*|{)?)|(?:(\"|'| ..." at ARGS:artist. [file "/etc/httpd/modsecurity.d/crs/base_rules/modsecurity_crs_41_sql_injection_attacks.conf"] [line "533"] [id "981244"] [msg "Detects basic SQL authentication bypass attempts 1/3"] [data "div"] [severity "CRITICAL"] [tag "WEB_ATTACK/SQLI"] [tag "WEB_ATTACK/ID"] [tag "WEB_ATTACK/LFI"]Message: Warning. Pattern match "(?i:(?:\\d(\"|'|`|\xc2\xb4|\xe2\x80\x99|\xe2\x80\x98)\\s+(\"|'|`|\xc2\xb4|\xe2\x80\x99|\xe2\x80\x98)\\s+\\d)|(?:^admin\\s*(\"|'|`|\xc2\xb4|\xe2\x80\x99|\xe2\x80\x98)|(\\/\\*)+(\"|'|`|\xc2\xb4|\xe2\x80\x99|\xe2\x80\x98)+\\s?(?:--|#|\\/\\*|{)?)|(?:(\"|'| ..." at ARGS:artist. [file "/etc/httpd/modsecurity.d/crs/base_rules/modsecurity_crs_41_sql_injection_attacks.conf"] [line "533"] [id "981244"] [msg "Detects basic SQL authentication bypass attempts 1/3"] [data "div"] [severity "CRITICAL"] [tag "WEB_ATTACK/SQLI"] [tag "WEB_ATTACK/ID"] [tag "WEB_ATTACK/LFI"]Message: Warning. Pattern match "(?i:(?:@.+=\\s*\\(\\s*select)|(?:\\d+\\s*x?or|div|like|between|and\\s*\\d+\\s*[\\-+])|(?:\\/\\w+;?\\s+(?:having|and|x?or|div|like|between|and|select)\\W)|(?:\\d\\s+group\\s+by.+\\()|(?:(?:;|#|--)\\s*(?:drop|alter))|(?:(?:;|#|--)\\s*(?:update|insert)\\s ..." at ARGS:artist. [file "/etc/httpd/modsecurity.d/crs/base_rules/modsecurity_crs_41_sql_injection_attacks.conf"] [line "539"] [id "981248"] [msg "Detects chained SQL injection attempts 1/2"] [data "div"] [severity "CRITICAL"] [tag "WEB_ATTACK/SQLI"] [tag "WEB_ATTACK/ID"]Message: Warning. Pattern match "(?i:(?:@.+=\\s*\\(\\s*select)|(?:\\d+\\s*x?or|div|like|between|and\\s*\\d+\\s*[\\-+])|(?:\\/\\w+;?\\s+(?:having|and|x?or|div|like|between|and|select)\\W)|(?:\\d\\s+group\\s+by.+\\()|(?:(?:;|#|--)\\s*(?:drop|alter))|(?:(?:;|#|--)\\s*(?:update|insert)\\s ..." at ARGS:artist. [file "/etc/httpd/modsecurity.d/crs/base_rules/modsecurity_crs_41_sql_injection_attacks.conf"] [line "539"] [id "981248"] [msg "Detects chained SQL injection attempts 1/2"] [data "div"] [severity "CRITICAL"] [tag "WEB_ATTACK/SQLI"] [tag "WEB_ATTACK/ID"]Message: Warning. Pattern match "(?i:(?:(\"|'|`|\xc2\xb4|\xe2\x80\x99|\xe2\x80\x98)\\s*x?or|div|like|between|and\\s*(\"|'|`|\xc2\xb4|\xe2\x80\x99|\xe2\x80\x98)?\\d)|(?:\\\\x(?:23|27|3d))|(?:^.?(\"|'|`|\xc2\xb4|\xe2\x80\x99|\xe2\x80\x98)$)|(?:(?:^[(\"|'|`|\xc2\xb4|\xe2\x80\x99|\xe2\x80 ..." at ARGS:artist. [file "/etc/httpd/modsecurity.d/crs/base_rules/modsecurity_crs_41_sql_injection_attacks.conf"] [line "565"] [id "981242"] [msg "Detects classic SQL injection probings 1/2"] [data "0 div 1"] [severity "CRITICAL"] [tag "WEB_ATTACK/SQLI"] [tag "WEB_ATTACK/ID"] [tag "WEB_ATTACK/LFI"]Message: Warning. Pattern match "(?i:(?:(\"|'|`|\xc2\xb4|\xe2\x80\x99|\xe2\x80\x98)\\s*x?or|div|like|between|and\\s*(\"|'|`|\xc2\xb4|\xe2\x80\x99|\xe2\x80\x98)?\\d)|(?:\\\\x(?:23|27|3d))|(?:^.?(\"|'|`|\xc2\xb4|\xe2\x80\x99|\xe2\x80\x98)$)|(?:(?:^[(\"|'|`|\xc2\xb4|\xe2\x80\x99|\xe2\x80 ..." at ARGS:artist. [file "/etc/httpd/modsecurity.d/crs/base_rules/modsecurity_crs_41_sql_injection_attacks.conf"] [line "565"] [id "981242"] [msg "Detects classic SQL injection probings 1/2"] [data "0 div 1"] [severity "CRITICAL"] [tag "WEB_ATTACK/SQLI"] [tag "WEB_ATTACK/ID"] [tag "WEB_ATTACK/LFI"]Message: Warning. Pattern match "(?i:(?:(\"|'|`|\xc2\xb4|\xe2\x80\x99|\xe2\x80\x98)\\s*\\*.+(?:x?or|div|like|between|and|id)\\W*(\"|'|`|\xc2\xb4|\xe2\x80\x99|\xe2\x80\x98)\\d)|(?:\\^(\"|'|`|\xc2\xb4|\xe2\x80\x99|\xe2\x80\x98))|(?:^[\\w\\s(\"|'|`|\xc2\xb4|\xe2\x80\x99|\xe2\x80\x98)-]+( ..." at ARGS:artist. [file "/etc/httpd/modsecurity.d/crs/base_rules/modsecurity_crs_41_sql_injection_attacks.conf"] [line "573"] [id "981243"] [msg "Detects classic SQL injection probings 2/2"] [data "div"] [severity "CRITICAL"] [tag "WEB_ATTACK/SQLI"] [tag "WEB_ATTACK/ID"] [tag "WEB_ATTACK/LFI"]Message: Warning. Pattern match "(?i:(?:(\"|'|`|\xc2\xb4|\xe2\x80\x99|\xe2\x80\x98)\\s*\\*.+(?:x?or|div|like|between|and|id)\\W*(\"|'|`|\xc2\xb4|\xe2\x80\x99|\xe2\x80\x98)\\d)|(?:\\^(\"|'|`|\xc2\xb4|\xe2\x80\x99|\xe2\x80\x98))|(?:^[\\w\\s(\"|'|`|\xc2\xb4|\xe2\x80\x99|\xe2\x80\x98)-]+( ..." at ARGS:artist. [file "/etc/httpd/modsecurity.d/crs/base_rules/modsecurity_crs_41_sql_injection_attacks.conf"] [line "573"] [id "981243"] [msg "Detects classic SQL injection probings 2/2"] [data "div"] [severity "CRITICAL"] [tag "WEB_ATTACK/SQLI"] [tag "WEB_ATTACK/ID"] [tag "WEB_ATTACK/LFI"]Message: Warning. Pattern match ".*" at TX:981231-WEB_ATTACK/SQL_INJECTION-ARGS:artist. [file "/etc/httpd/modsecurity.d/crs/base_rules/modsecurity_crs_48_bayes_analysis.conf"] [line "1"] [data "Completed Bayesian Training on SQLi Payload: n#foo*/*bar\x0d\x0aselect#foo\x0d\x0a."]Message: Warning. Pattern match ".*" at TX:959047-WEB_ATTACK/SQL_INJECTION-ARGS:artist. [file "/etc/httpd/modsecurity.d/crs/base_rules/modsecurity_crs_48_bayes_analysis.conf"] [line "1"] [data "Completed Bayesian Training on SQLi Payload: union#foo*/*bar\x0d\x0aselect."]Message: Warning. Pattern match ".*" at TX:959073-WEB_ATTACK/SQL_INJECTION-ARGS:artist. [file "/etc/httpd/modsecurity.d/crs/base_rules/modsecurity_crs_48_bayes_analysis.conf"] [line "1"] [data "Completed Bayesian Training on SQLi Payload: union#foo*/*bar\x0d\x0aselect."]Message: Warning. Pattern match ".*" at TX:981173-WEB_ATTACK/RESTRICTED_SQLI_CHARS-ARGS:artist. [file "/etc/httpd/modsecurity.d/crs/base_rules/modsecurity_crs_48_bayes_analysis.conf"] [line "1"] [data "Completed Bayesian Training on SQLi Payload: #foo*/*bar\x0d\x0aselect#foo\x0d\x0a1,2,current_user."]Message: Warning. Pattern match ".*" at TX:981244-Detects basic SQL authentication bypass attempts 1/3-WEB_ATTACK/SQLI-ARGS:artist. [file "/etc/httpd/modsecurity.d/crs/base_rules/modsecurity_crs_48_bayes_analysis.conf"] [line "1"] [data "Completed Bayesian Training on SQLi Payload: div."]Message: Warning. Pattern match ".*" at TX:981244-Detects basic SQL authentication bypass attempts 1/3-WEB_ATTACK/ID-ARGS:artist. [file "/etc/httpd/modsecurity.d/crs/base_rules/modsecurity_crs_48_bayes_analysis.conf"] [line "1"] [data "Completed Bayesian Training on SQLi Payload: div."]Message: Warning. Pattern match ".*" at TX:981244-Detects basic SQL authentication bypass attempts 1/3-WEB_ATTACK/LFI-ARGS:artist. [file "/etc/httpd/modsecurity.d/crs/base_rules/modsecurity_crs_48_bayes_analysis.conf"] [line "1"] [data "Completed Bayesian Training on SQLi Payload: div."]Message: Warning. Pattern match ".*" at TX:981248-Detects chained SQL injection attempts 1/2-WEB_ATTACK/SQLI-ARGS:artist. [file "/etc/httpd/modsecurity.d/crs/base_rules/modsecurity_crs_48_bayes_analysis.conf"] [line "1"] [data "Completed Bayesian Training on SQLi Payload: div."]Message: Warning. Pattern match ".*" at TX:981248-Detects chained SQL injection attempts 1/2-WEB_ATTACK/ID-ARGS:artist. [file "/etc/httpd/modsecurity.d/crs/base_rules/modsecurity_crs_48_bayes_analysis.conf"] [line "1"] [data "Completed Bayesian Training on SQLi Payload: div."]Message: Warning. Pattern match ".*" at TX:981242-Detects classic SQL injection probings 1/2-WEB_ATTACK/SQLI-ARGS:artist. [file "/etc/httpd/modsecurity.d/crs/base_rules/modsecurity_crs_48_bayes_analysis.conf"] [line "1"] [data "Completed Bayesian Training on SQLi Payload: 0 div 1."]Message: Warning. Pattern match ".*" at TX:981242-Detects classic SQL injection probings 1/2-WEB_ATTACK/ID-ARGS:artist. [file "/etc/httpd/modsecurity.d/crs/base_rules/modsecurity_crs_48_bayes_analysis.conf"] [line "1"] [data "Completed Bayesian Training on SQLi Payload: 0 div 1."]Message: Warning. Pattern match ".*" at TX:981242-Detects classic SQL injection probings 1/2-WEB_ATTACK/LFI-ARGS:artist. [file "/etc/httpd/modsecurity.d/crs/base_rules/modsecurity_crs_48_bayes_analysis.conf"] [line "1"] [data "Completed Bayesian Training on SQLi Payload: 0 div 1."]Message: Warning. Pattern match ".*" at TX:981243-Detects classic SQL injection probings 2/2-WEB_ATTACK/SQLI-ARGS:artist. [file "/etc/httpd/modsecurity.d/crs/base_rules/modsecurity_crs_48_bayes_analysis.conf"] [line "1"] [data "Completed Bayesian Training on SQLi Payload: div."]Message: Warning. Pattern match ".*" at TX:981243-Detects classic SQL injection probings 2/2-WEB_ATTACK/ID-ARGS:artist. [file "/etc/httpd/modsecurity.d/crs/base_rules/modsecurity_crs_48_bayes_analysis.conf"] [line "1"] [data "Completed Bayesian Training on SQLi Payload: div."]Message: Warning. Pattern match ".*" at TX:981243-Detects classic SQL injection probings 2/2-WEB_ATTACK/LFI-ARGS:artist. [file "/etc/httpd/modsecurity.d/crs/base_rules/modsecurity_crs_48_bayes_analysis.conf"] [line "1"] [data "Completed Bayesian Training on SQLi Payload: div."]Message: Warning. Pattern match "Warning<\\/b>.{0,100}?:.{0,1000}?\\bon line\\b" at RESPONSE_BODY. [file "/etc/httpd/modsecurity.d/crs/base_rules/modsecurity_crs_50_outbound.conf"] [line "28"] [id "970009"] [rev "2.2.1"] [msg "PHP Information Leakage"] [severity "ERROR"] [tag "LEAKAGE/ERRORS_PHP"] [tag "WASCTC/WASC-13"] [tag "OWASP_TOP_10/A6"] [tag "PCI/6.5.6"]Message: Warning. Operator GE matched 1 at TX. [file "/etc/httpd/modsecurity.d/crs/base_rules/modsecurity_crs_60_correlation.conf"] [line "22"] [id "981201"] [msg "Correlated Successful Attack Identified: (Total Score: 70, SQLi=14, XSS=) Inbound Attack (981243-Detects classic SQL injection probings 2/2 - Inbound Anomaly Score: 66) + Outbound Data Leakage (PHP Information Leakage - Outbound Anomaly Score: 4)"] [severity "EMERGENCY"]Message: Warning. Operator GE matched 1 at TX. [file "/etc/httpd/modsecurity.d/modsecurity_demo_end.conf"] [line "66"] [msg "Correlated Successful Attack Identified: Inbound Attack (981243-Detects classic SQL injection probings 2/2) + Outbound Data Leakage (PHP Information Leakage) - (Transactional Anomaly Score: 70)"] [severity "EMERGENCY"]Apache-Handler: proxy-serverStopwatch: 1313055020765349 576490 (- - -)Stopwatch2: 1313055020765349 576490; combined=339694, p1=321, p2=275977, p3=34, p4=61395, p5=1871, sr=46, sw=96, l=0, gc=0Response-Body-Transformed: DechunkedProducer: ModSecurity for Apache/2.6.1-rc1 (http://www.modsecurity.org/); core ruleset/2.2.1.Server: Apache/2.2.3 (CentOS) DAV/2 mod_ssl/2.2.3 OpenSSL/0.9.8e-fips-rhel5--8802d071-Z--
As you can see, modgrep was able to extract out the entire record for this sql injection attempt. Another nice feature of this script is that you are also able to pipe multiple searches together. This is possible due to the -f flag's ability to take standard input instead of a file. This capability is tremendously useful when you need to narrow down a search based on multiple connection characteristics.
For instance, let's say that we wanted to search the logs for all connections from the client 67.195.115.98 and where they accessed the "artists.php" page with the "artist" parameter. In order to narrow down our results, we need to run two piped modgrep commands that take the output of the first sgrep search and feed it into the second search.
# modgrep -f audit_log -s "67.195.115.98" | modgrep -f - -s "artists.php\?artist=" | less========================================Matched Transaction for Search String (artists.php\?artist=)================================================================================Matched Transaction for Search String (67.195.115.98)========================================--8802d071-A--[11/Aug/2011:04:30:21 --0500] chVQpcCo8AoAAGd3pV0AAAAM 67.195.115.98 46833 192.168.240.10 80--8802d071-B--GET /testphp.vulnweb.com/artists.php?artist=0+div+1+union%23foo*%2F*bar%0D%0Aselect%23foo%0D%0A1%2C2%2Ccurrent_user HTTP/1.0Host: www.modsecurity.orgUser-Agent: Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp)Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5Accept-Language: en-us,en;q=0.5Accept-Encoding: gzipAccept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7--8802d071-F--HTTP/1.1 200 OKX-Powered-By: PHP/5.1.2Content-Length: 1177Content-Type: text/html; charset=UTF-8Connection: close--CUT----8802d071-Z--========================================Matched Transaction for Search String (artists.php\?artist=)================================================================================Matched Transaction for Search String (67.195.115.98)========================================--0921bb7c-A--[11/Aug/2011:03:20:56 --0500] edx6ycCo8AoAAGSIXVQAAAAf 67.195.115.98 44599 192.168.240.10 80--0921bb7c-B--GET /testphp.vulnweb.com/artists.php?artist=%40%40new%20union%23sqlmapsqlmapsqlmapsqlmapsqlmapsqlmapsqlmapsqlmapsqlmapsqlmapsqlmapsqlmapsqlmapsqlmapsqlmapsqlmapsql%0Aselect%201,2,database%23sqlmap%0A%28%29 HTTP/1.0Host: www.modsecurity.orgUser-Agent: Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp)Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5Accept-Language: en-us,en;q=0.5Accept-Encoding: gzipAccept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7--0921bb7c-F--HTTP/1.1 200 OKX-Powered-By: PHP/5.1.2Content-Length: 1217Content-Type: text/html; charset=UTF-8Connection: close--CUT--
Perfect! We were able to extract out the entire audit_log entries for transaction that matched our search criteria. As you might expect, this script makes searching the audit_log file much easier and shortens the time needed for log analysis during incident response. Please test out the script and let us know if you find any issues or have any recommendations for improvements.