(Updated) Advanced Topic of the Week: Handling Authorized Scanning Traffic

• To identify all vulnerabilities within a target web application, or

• To identify all vulnerabilities within a target web application that are remotely exploitable by an external attacker.

Identifying the underlying vulnerabilities is a critical goal to have when scanning. If you can identify all of the existing vulnerabilities, or at least those which can be identified by scanning, you can then formulate a remediation plan to address the issue at the root cause. Remediation should include both source code fixes and custom rules within a web application firewall such as ModSecurity for protection in the interim.

In order to identify all of the actual vulnerabilities, you will need to allow the vulnerability scanning host to interact completely with the back-end web application. The advantages of this approach are twofold:

The downside of this approach is that it reduces the true exploitability accuracy of the scans as it does not simulate what a "normal" attacker would be able to access since the ModSecurity is not blocking inbound attacks and outbound information leakages. For whitelisting examples, please refer to the Implementing Exceptions - Whitelisting based on Client Location recipe.

From a ModSecurity administration perspective, it is important to have proper configuration so as to allow for the vulnerability scanning goal while simultaneously not flooding the alert monitoring processes with thousands of events from authorized vulnerability scanning traffic. This data may impact both the overall performance of ModSecurity and it may skew reporting outputs that include raw attack data.

If you do not implement any specify policy changes to handle authorized scanning traffic, and ModSecurity is running with SecRuleEngine On to allow for disruptive blocking actions, then it will give a truer picture of what an attacker would or wouldn't be able to exploit since ModSecurity is providing its normal protections. In this configuration, however, the scans will possibly generate huge volumes of alerts especially if the scan frequency is high (daily). Additionally, vulnerabilities within the web application may not be identified and therefore are not being remediated through other means. It is for these reasons that you should consider the following scanning configuration policies.

Another scanning method to use is to coordinate with the vulnerability scanning team to conduct two different scans – one when the scanner is whitelisted and one when it is not. After the scans have been completed, the reduction of the identified vulnerabilities can almost assuredly be attributed to the ModSecurity blocking rule sets. The benefits of this approach are that it allows the scanning team to identify actually vulnerabilities, as well as, provides a tangible return on investment (ROI) of ModSecurity's protections. The drawback of this approach are that you have to run two scans in order to meet your desired scanning goals.

One option you might want to consider and is a "block but don't log" configuration which is a hybrid approach that is suitable for most day to day usage. With this setup, you allow ModSecurity to block requests/responses when it sees attacks however in the phase:5 logging phase it will then inspect the IP address range and if it is coming from an authorized scanning host it will not generate alerts.

SecRule REMOTE_ADDR "@streq 192.168.1.101" \
    "phase:5,t:none,nolog,pass,ctl:auditEngine=Off"

While this rule is functionally correct, it could be improved from a performance perspective and moved from phase:5 to phase:1. This is a functionally equivalent rule, however the overall performance is improved since the audit engine is not burdened with creating internal audit events that will later be discarded.

# Recommended "Block but Don't Log" rule for scanning traffic
SecRule REMOTE_ADDR "@streq 192.168.1.101" \
    "phase:1,t:none,nolog,pass,ctl:auditEngine=Off"

Ideally, there should be a balance between both the vulnerability scanner and ModSecurity's viewpoints. This configuration certainly removed an audit log flooding scenario based on identified inbound attack traffic, however it is at the expense of ModSecurity's visibility of outbound monitoring. We don't really care about inbound attack attempts, however successful attack transactions are another story. If the vulnerability scanner was able to either send an attack payload that evaded the ModSecurity rulesets or if the protected web application generated some sort of error or information leakage, then there would be no audit log of the transaction. It is for this scenario that this final rule set is recommended:

# Recommended "Block but Don't Log" rule for scanning traffic
SecRule REMOTE_ADDR "@streq 192.168.1.101" \
    "phase:1,t:none,nolog,pass,ctl:auditEngine=Off"

# Recommended phase 3 rule that will re-enable the audit engine if the request
# was not blocked by one of the normal rules.
SecRule REMOTE_ADDR "@streq 192.168.1.101" \
    "phase:3,t:none,nolog,pass,ctl:auditEngine=On"

This rule set toggles off the the audit engine in phase:1 if it is coming from the authorized scanning host IP address. If ModSecurity blocks an inbound attack, it will immediately proceed to phase:5 and then exit. If, on the otherhand, the scanner traffic does not trigger any inbound rules, then when it reaches phase:3 response headers, this rule will toggle the audit engine back on. This will allow ModSecurity to generate proper alerts for successful attacks and information leakage issues. In this final configuration, any audit events that are created based on the vulnerability scanning traffic should most certainly be investigated for remediation.