On July 30th, we announced our public ModSecurity XSS Evasion Challenge. This blog post will provide an overview of the challenge and results.
First of all, I would like to thank all those people that participated in the challenge. All told, we had > 730 participants (based on unique IP addresses) which is a tremendous turn out. This type of community testing has helped to both validate the strengths and expose the weaknesses of the XSS blacklist filter protections of the OWASP ModSecurity Core Rule Set Project. The end result of this challenge is that the XSS Injection rules within the CRS have been updated within the Trunk release in GitHub.
The form on this page is vulnerable to reflected XSS. Data passed within the test parameter (either GET or POST) will be reflected back to this same page without any output encoding/escaping.
We activated updated XSS filters from the OWASP ModSecurity Core Rule Set (CRS). When clients send attack payloads, they are evaluated by the CRS rules and then the detection scores are popagated to the HTML form as such:
CRS XSS Anomaly Score Exceeded (score 10): NoScript XSS InjectionChecker: HTML Injection
We added XSS filter regular expressions from the following 3rd party sources:
It is important to understand what a JS sandbox is and how it works. You may be able to execute JS code however it is in a sandboxed environment. For example - preventing a JS alert popup box is not the goal here but rather protecting DOM elements from being accessed.
In some cases, MentalJS prevents the injected code from executing at all. In other cases, the code will execute, however it does so from within a JS sandbox. The result is that the code is not able to access DOM elements outside of the sandbox. For instance, if an injection attempts to access document.cookie, it returns data as "undefined" -
The challenge was twofold and to win the participants needed to develop a single payload that achieved both of these goals:
You must execute a reflected XSS attack accessing one of the DOM elements listed below WITHOUT triggering an XSS filter alert. Example -
No CRS XSS Filter Alerts Triggered (score 0)
Depending on the payload, you may need to review the raw HTML to verify the CRS score.
You must bypass the MentalJS JS Sandbox protections and successfully execute a reflected XSS attack that executes JS code in your browser. A successful attack will be able to access one of the following DOM elements:
Example -
You may toggle On/Off the defenses by checking the box in the form below. This includes disable the MentalJS Sandbox injection and also will add the X-XSS-Protection: 0 response header to temporarily disable any browser side XSS filtering. This will help to facilitate testing of working XSS payloads.
While various community submissions did include bypasses for each of the two protection layers individually, there were no official winners who were able to bypass BOTH the CRS filters and break out of the MentalJS sandbox all within a single payload. That being said, we do want to highlight the individual component bypasses as they still have value for the community.
Security Researcher Rafay Baloch actually was a bit ahead of the curve on this challenge as he notified us of a bypass using Nul Bytes within the JS script tag name just prior to the XSS Evasion Challenge being publicly announced.
He was testing our public Smoke Test page here.
As you can see from the output, rule ID 960901 triggered and was specifically designed to flag the presence of Nul Bytes. We took this approach rather than attempting to deal with Nul Bytes within every remaining rule. That being said, there is still a valid point here - if a Nul Byte can be used to obfuscate payloads and evade detections then this should be addressed specifically.
If we research Gareth Heyes'Shazzer online fuzzer info, we find that indeed IE v9 allows the use of Nul Bytes within the script tag name -
The JSON export API shows the example vector payloads -
So, by adding %00 within the script tag name, many of the XSS regular expression checks would not match and IE9 would still parse the tag correct and execute javascript... Perfect.
To combat this tactic, we added the "t:removeNulls" transformation function action to our XSS filters:
This Nul Byte issue demonstrates just one different is browser parsing quirks. There are many, many others... It is for this reason that we decided for regression testing of XSS filters we needed to leverage the Shazzer dataset to identify payloads that trigger JS across multiple browser types and versions. I created a number of ruby scripts that interact with the Shazzer API that can extract successful fuzz payloads and test them against the live XSS evasion challenge page running the latest OWASP ModSecurity CRS rules.
Again Rafay Baloch sent in a submission that used this technique -
http://www.modsecurity.org/demo/demo-deny-noescape.html?test=%3Cscript%3Edocument.body.innerHTML=%22%3Ca%20onmouseover%0B=location=%27\x6A\x61\x76\x61\x53\x43\x52\x49\x50\x54\x26\x63\x6F\x6C\x6F\x6E\x3B\x63\x6F\x6E\x66\x69\x72\x6D\x26\x6C\x70\x61\x72\x3B\x64\x6F\x63\x75\x6D\x65\x6E\x74\x2E\x63\x6F\x6F\x6B\x69\x65\x26\x72\x70\x61\x72\x3B%27%3E%3Cinput%20name=attributes%3E%22;%3C/script%3E&disable_xss_defense=on&disable_browser_xss_defense=on
We also received a similar submission from ONsec_lab:
There are a number of characters that browsers will treat as "spaces" characters. Here is a table of these characters for each browser type:
IExplorer = [0x09,0x0B,0x0C,0x20,0x3B]Chrome = [0x09,0x20,0x28,0x2C,0x3B]Safari = [0x2C,0x3B]FireFox = [0x09,0x20,0x28,0x2C,0x3B]Opera = [0x09,0x20,0x2C,0x3B]Android = [0x09,0x20,0x28,0x2C,0x3B]
The evasion method used by both of these submissions was to place one of the characters in between the "onevent" name attribute (such as onmouseover and onfocus) and the equal sign (=) character -
As you can see from the table above, Internet Explorer (IE) would treat the %0B character as a space and parse/execute the payload however it would bypass blacklist regular expression filters such as this -
(?i)([\s\"'`;\/0-9\=]+on\w+\s*=)
The problem with this regex is the use of the "\s" meta-character where %0B is not within the class.
Similar to the Evasion #1 - we can utilize Shazzer Fuzz DB data to obtain payloads with these non-space chars separating on-event handlers.
We modified our regex rules to catch these non-space characters:
Here is what the updatd regex visual looks like:
We then ran our ruby scripts to extract and test this Shazzer Fuzz data against our rules to verify that they were caught:
The second half of the challenge was to try and escape from the MentalJS sandbox and access the DOM elements. There were a few people who were successful. Remember, however, that while these payloads were able to break out of the MentalJS sandbox they were all caught by the regular expression filters.
The most interesting MentalJS bypass submission was from Roman Shafigullin which included a method of breaking the parser. My analyzing the JS, he found that he could break the parsing by setting an emply script tag:
With this as the payload, it then brok this section of the MentalJS parsing:
He was then able to set the "src" attribute to anything he wanted and chose to pull in a JS file hosted from his own server:
When resulting full payload returned the following:
Guiseppe Trotta sent in the first succesful submission.
Here were the payloads he used -
There were others who submitted similar payloads that used document.body.innerHTML to override the MentalJS sandboxing-
There are a few DOM events that are not covered within the current version of MentalJS including:
These were also reported by Vladimir Vorontsov in the sla.ckers.org forum .
Ok, so in this conclusion section, I am going to hit you with some Earth-shattering revelations! No, not really. What I will cover here, however is information that re-enforces security themes you have heard before with relevant attack testing data.
Trust, but Verify - Ronald Reagan.
Our rules are of a high quality because they have been field tested by the community. Only when top-tier web app pentesters focus some QA effort on the rules will they become better. It is through these types of public, community hacking challenges that we are able to strengthen our rules. The question is - what are other WAF vendors doing to test their rules???
Negative security models used on their own is not enough to prevent a determined attacker. It is only a matter of time before they are able to identify an evasion method. While this is true, negative security models do still have value:
There are many other methods of detecting malicious payloads besides regular expressions. Some other examples are:
The combination of server-side blacklist filtering and client-side javascript challenge proved to be extremely effective. While some submissions were successful in evading the filters, they were not able to break out of the MentalJS sandbox. On the flip side, those submissions that were able to break out of the MentalJS sandbox, were not able to evade the filters. Use multple different methods of identifying attacks to increase coverage and to compensate for weaknesses in other layers.
