Handling False Positives and Creating Custom Rules

It is inevitable; you will run into some False Positive hits when using web application firewalls. This is not something that is unique to ModSecurity. All web application firewalls will generate false positives from time to time. The following information will help to guide you through the process of identifying, fixing, implementing and testing new custom rules to address false positives.

Every rule set can have false positive in new environments
False Positives happen with ModSecurity + the Core Rules mainly as a by product of the fact that the rules are "generic" in nature. There is no way to know exactly what web application is going to be run behind it. That is why the Core Rules are geared towards blocking the known bad stuff and forcing some HTTP compliance. This catches the vast majority of attacks.

Use DetectionOnly mode
Any new installation should initially use the log only Rule Set version or if no such version is available, set ModSecurity to Detection only using the SecRuleEngine DetectionOnly command. After running ModSecurity in a detection only mode for a while review the events generated and decide if any modification to the rule set should be made before moving to protection mode.

Don't be too hasty to remove a rule
Just because a particular rule is generating a false positive on your site does not mean that you should remove the rule entirely. Remember, these rules were created for a reason. They are intended to block a known attack. By removing this rule completely, you might expose your website to the very attack that the rule was created for. This would be the dreaded False Negative.

ModSecurity rules are open source
Thankfully, since ModSecurity's rules are open source, this allows you the capability to see exactly what the rule is matching on and also allows you to create your own rules. With closed-source rules, you can not verify what it is looking for so you really have no other option but to remove the offending rule.

The logs are your friend
In order to verify if you indeed have a false positive, you need to review your logs. This means that you need to look in the audit_log file first to see what the ModSecurity message states. It will provide information as to which rule triggered. This same information is also available within the error_log file. The last place to look, and actually the best source of information, is the modsec_debug.log file. This file can show everything that ModSecurity is doing, especially if you turn up the SecDebugLogLevel to 9. Keep in mind, however, that increasing the verboseness of the debug log does impact performance. While increasing the verboseness for all traffic is usually not feasible, what you can do is to create a new rule that uses the "ctl" action to turn up the debugloglevel selectively. For instance, if you identify a False Positive from only one specific user, you could add in a rule such as this:

SecRule REMOTE_ADDR "^192\.168\.10\.69$" phase:1,log,pass,ctl:debugLogLevel=9

This will set the debugLogLevel to 9 only for requests coming from that specific source IP address. Perhaps that still generates a bit too much traffic. You could tighten this down a bit to increase the logging only for the specific file or argument that is causing the false positive:

SecRule REQUEST_URI "^/path/to/script.pl$" phase:1,log,pass,ctl:debugLogLevel=9  or  SecRule ARGS:variablename "something" phase:1,pass,ctl:debugLogLevel=9

Now that you have verbose information in the debug log file, you can review it to ensure that you understand what portion of the request was being inspected when the specific rule trigger and you can also view the payload after all of the transformation functions have been applied.

Try to avoid altering the Core Rules
In general, it is recommended that you try to limit your alteration of the Core Rules as much as possible. The more you alter the rule files, the less likely it will be that you will want to upgrade to the newer releases since you would have to recreate your customizations. What we recommend is that you try to contain your changes to your own custom rules file(s) that are particular to your site. This is where you would want to add new signatures and to also create rules to exclude False Positives from the normal Core Rules files. There are two main ways to integrate your custom rules so that they work with the Core Rules.

1. Adding new white-listing rules

If you need to add new white-listing rules so that you can, for instance, allow a specific client IP address to pass through all of the ModSecurity rules you should place this type of rule after the modsecurity_crs_10_config.conf file but BEFORE the other Core Rules. This is accomplished by creating a new rule file called – modsecurity_crs_15_customrules.conf and place it in the same directory as the other Core Rules. This is assuming you are using the Apache Include directive to call up the Core Rules like this –

<IfModule security2_module>  Include conf/rules/*.conf  </IfModule>

By naming your file with the "_15_" string in it, it will be called up just after the config file. This will ensure that your new white-list rule will be executed early and you can then use such actions as allow and ctl:ruleEngine=Off to allow the request through the remainder of the rules.

2. Adding new negative policy rules

If you need to add new negative policy rules, such as when you need to update a Core Rule that is causing a false positive, you should add these rules to a new rule file that come AFTER all of the other Core Rules. Call this new file something like – modsecurity_crs_60_customrules.conf. Just make sure that number in the filename is higher than any other rules file so it is read last. The rationale for placing these types of rules after the other rules is that you can then match up these new replacement rules with corresponding SecRuleRemoveByID directives that will then disable the specific Core Rule(s) that are causing False Positives. It is important to note that you need to use SecRuleRemoveById AFTER ModSecurity has knowledge of the Rule ID you are actually removing. If you were to place this directive in the modsecurity_crs_15_customrules.conf file, it would not work correctly as the rule ID you are specifying does not exist yet. That is why this directive should be called up in your custom rules file that comes at the end. Using this method allows you to turn off rules without having to actually go into the Core Rules files and comment out or update specific rules.

Fixing the false positive
OK, so now you have identified the specific Core Rule that is causing the false positive. Let's say that the rule that is causing a false positive is the following one in the modsecurity_crs_40_generic_attacks.conf file –

# XSS  SecRule REQUEST_FILENAME|ARGS|ARGS_NAMES|REQUEST_HEADERS   "(?:\b(?:on(?:(?:mo(?:use(?:o(?:ver|ut)|down|move|up)|ve)|ke  y(?:press|down|up)|c(?:hange|lick)|s(?:elec|ubmi)t|(?:un)?load|dragdrop|resize|focus|blur)\b\W*?=|abort\b)|(?:l(?:ows  rc\b\W*?\b(?:(?:java|vb)script|shell)|ivescript)|(?:href|url)\b\W*?\b(?:(?:java|vb)script|shell)|mocha):|type\b\W*?\b  (?:text\b(?:\W*?\b(?:j(?:ava)?|ecma)script\b| [vbscript])|application\b\W*?\bx-(?:java|vb)script\b)|s(?:(?:tyle\b\W*=  .*\bexpression\b\W*|ettimeout\b\W*?)\(|rc\b\W*?\b(?:(?:java|vb)script|shell|http):)|(?:c(?:opyparentfolder|reatetextr  ange)|get(?:special|parent)folder|background-image:|@import)\b|a(?:ctivexobject\b|lert\b\W*?\())|<(?:(?:body\b.*?\b(?  :backgroun|onloa)d|input\b.*?\\btype\b\W*?\bimage)\b|!\[CDATA\[|script|meta)|.(?:(?:execscrip|addimpor)t|(?:fromcharc  od|cooki)e|innerhtml)\b)" \  "log,id:950004,severity:2,msg:'Cross-site Scripting (XSS) Attack'"

Your next step is to just copy and paste it into the new modsecurity_crs_60_customrules.conf file. Let's assume that the false positive hit with this rule is when it is inspecting a specific portion of your Cookie header called Foo. The Cookie data is included within the REQUEST_HEADERS variable. You now need to make a few edits to the rule to update it to remove the false hit. The bolded sections of code are the relevant updates -

# XSS  SecRule REQUEST_FILENAME|ARGS|ARGS_NAMES|REQUEST_HEADERS|  !REQUEST_HEADERS:Cookie|REQUEST_COOKIES|REQUEST_COOKIES_NAMES|!REQUEST_COOKIES:/^Foo$/   "(?:\b(?:on(?:(?:mo(?:use(?:o(?:ver|ut)|down|move|up)|ve)|ke  y(?:press|down|up)|c(?:hange|lick)|s(?:elec|ubmi)t|(?:un)?load|dragdrop|resize|focus|blur)\b\W*?=|abort\b)|(?:l(?:ows  rc\b\W*?\b(?:(?:java|vb)script|shell)|ivescript)|(?:href|url)\b\W*?\b(?:(?:java|vb)script|shell)|mocha):|type\b\W*?\b  (?:text\b(?:\W*?\b(?:j(?:ava)?|ecma)script\b| [vbscript])|application\b\W*?\bx-(?:java|vb)script\b)|s(?:(?:tyle\b\W*=  .*\bexpression\b\W*|ettimeout\b\W*?)\(|rc\b\W*?\b(?:(?:java|vb)script|shell|http):)|(?:c(?:opyparentfolder|reatetextr  ange)|get(?:special|parent)folder|background-image:|@import)\b|a(?:ctivexobject\b|lert\b\W*?\())|<(?:(?:body\b.*?\b(?  :backgroun|onloa)d|input\b.*?\\btype\b\W*?\bimage)\b|!\[CDATA\[|script|meta)|.(?:(?:execscrip|addimpor)t|(?:fromcharc  od|cooki)e|innerhtml)\b)" \  "log,id:1,severity:2,msg:'Cross-site Scripting (XSS) Attack'"

This updated rule is doing three things –

1. We are using the exclamation point character to create an inverted rule meaning do NOT inspect the REQUEST_HEADERS variable whose name is Cookie. The problem here is that this variable location is too generic/broad and we are only interested is excluding one specific Cookie location from this check and not the entire Cookie value. We don't want to allow other possible XSS attack vectors within the Cookie value.

2. Since we still want to inspect the Cookie values, we have now opted to include additional Cookie variables that were not present before. We can now include both REQUEST_COOKIES and REQUEST_COOKIES_NAMES variables to the check. We are then finally using another inverted rule to exclude checking this rule against any Cookie whose name is exactly "Foo." This is accomplished by using a regular expression argument to the REQUEST_COOKIES variable.

3. Finally, we are also updating the "id" meta-data action by changing to a new number that represents a custom rule range. The range: 1 -99999 is reserved for your internal use.

The last thing to do is to use SecRuleRemoveById to disable the Core Rule that was causing the problem –

SecRuleRemoveById 950004

Testing the new rules
The final step is to actually test out your new configs and verify that the old rule is not executing and the new rule is not triggering a false positive hit. The easiest method to use is to just resend the previously offending request to the web server and then monitor the audit_log file to see if the request becomes blocked or if the ModSecurity message is generated.

Easy Implementation of new Core Rules
With this type of methodology, you can create custom exclusions and fix false positives and it also allows for easy updating of the Core Rules themselves. What we don't want to have happen is that current Mod users have altered the Core Rules files extensively for their environment that they do not want to upgrade when new Core Rule releases are available for fear of having to re-implement all of their custom configs. With this scenario, you can download new Core Rules versions as they are released and then just copy over your new ModSecurity custom rule files and you are ready to go!