ModSecurity Python Bindings: Parsing ModSecurity rules from Python
One of the good things about the next generation of ModSecurity, libModSecurity (AKA ModSecurity version 3), is the fact that it portable to almost any platform. This extensibility makes the use of bindings for other languages, beyond C/C++, very useful. For those that are unfamiliar with the concept of a 'binding', the term describes an interface between two different languages. In the case of ModSecurity this binding will provide an interface that allows you to use libModSecurity functionally inside your scripting language of choice, with negligible loss of performance.
The libModSecurity Python bindings may serve many purposes; for instance, it could be leveraged to rapidly (and easily) create a custom web interface for ModSecurity. However, we are not limited to simply displaying information, the new bindings give you nearly all the capabilities that you would have interacting with libModSecurity from native C, allowing you to, for example, display ModSecurity rules in any fashion that you desire, with almost zero effort. A simple usage is demonstrated on the Figure 1.
Figure 1. ModSecurity being used inside a Python shell.
Bindings for languages others than Python will be also created. In fact, the utility that was used in the creation of the Python bindings, SWIG, can also generates interfaces to other languages such as: Ruby. You are welcome to volunteer to expand your favorite language to add ModSecurity capabilities.
In this blog post I will dive into the creation of a simple console utility to load and list a set of rules in a elegant way. For this I will start with the compilation of the ModSecurity-Python-bindings.
Notice: The ModSecurity Python bindings depend on the libModSecurity which is publicly available on ModSecurity GitHub, but it is not considered stable release at this point yet.
Installing ModSecurity Python Bindings
Before starting the installation of the ModSecurity Python bindings, make sure you have libModSecurity installed and operational on your machine. For further information about libModSecurity and how to install it, please see the README at ModSecurity's GitHub repository, here: https://github.com/SpiderLabs/ModSecurity/tree/libmodsecurity . Of course, you will also want to make sure you have Python installed and all the developer utilities you may need on your Linux .
Once libModSecurity is installed, it is time to proceed with the installation of the ModSecurity Python bindings. Simply download the code from our GitHub repository, and proceed with the compilation, as demonstrated below:
$ git clone http://www.github.com/SpiderLabls/ModSecurity-Python-bindings
$ cd ModSecurity-Python-bindings
$ make
$ sudo make install
Once this process is finished, it is a good idea to test your new installation. You can do this by running the test script provided. This can be accomplished using the following command:
./test/t.py
If everything is alright, you should not get any error messages from Python.
The hello world script
With the ModSecurity Python bindings installed and tested it is time to create your first 'hello world' application using the ModSecurity library. Let's start with something very simply like checking the ModSecurity version that we are playing with.
Inside libModSecurity we expose the whoAmI() method which describes which version of ModSecurity you are bound with. Further information about this method, can be found here: https://github.com/SpiderLabs/ModSecurity/blob/libmodsecurity/src/modsecurity.cc#L67-L78
Let's use the whoAmI() method, to print the libModSecurity version. We demonstrate proper usage in Script 1 below.
#!/usr/bin/python
from modsecurity import *
modsec = ModSecurity()
s = "Hello World, I am: " + str(modsec.whoAmI())
print s
Script 1. Hello world using libModSecurity.
As output, you should be able to see something similar to the Figure 2.
Figure 2. The output of the hello world script.
Loading the rules
The next step is to add a few more pieces to our script allowing it to actually load a given set of rules. Make sure you have a workable set of ModSecurity rules before you start to code. A good set for this example might be the OWASP CRS 3.0 ruleset, available here: https://github.com/SpiderLabs/owasp-modsecurity-crs/tree/v3.0.0-dev
Notice: Make sure you download the version 3.0.0-dev. Our parser is not 100% compatible with ModSecurity v2.9.x yet. It may not work with other versions of CRS.
Using our first example, start by commenting out the "Hello World..." string and the associated print statement. These two pieces won't be necessary for this step. Your script should look similar to example provided in Script 2.
#!/usr/bin/python
from modsecurity import *
modsec = ModSecurity()
#s = "Hello World, I am: " + str(modsec.whoAmI())
#print s
Script 2. Hello World script with the unnecessary strings commented.
The rules in ModSecurity are loaded through a Rules object. While the Rules object may be merged with other objects of the same type, in this script let's keep it simple. For this example we just need to load a set of rules from a file and print them to the console. Again, a very simple utilization for libModSecurity.
We can load the rules, using the loadFromUri() method, which takes one argument as follows: "loadFromUri('/path/to/the/rules/file.txt')". The loadFromUri() method allow us to load a set of rules into memory. Notice that this method will return "-1" if there are any problems while loading the rules. If this occurs you can call the getParserError() method to get more information about any possible errors. The loadFromUri() method must be called with the path to a rules file as a target, as demonstrated in Script 3.
#!/usr/bin/python
from modsecurity import *
modsec = ModSecurity()
#s = "Hello World, I am: " + str(modsec.whoAmI())
#print s
rules = Rules.loadFromFile("/path/to/your/v3.0.0-dev/rules/REQUEST-10-IP-REPUTATION.conf")
Script 3. Loading the rules from a given file.
The Rules object has provides the getRulesForPhase() method which is called as follows: "rules.getRulesForPhase(phase_number)". Calling this method will return a vector of Rule objects. The Rule object contains all the properties associated with a given rule. To list all the rules from the target file that will load during a given phase you may simply iterate over the returned value from getRulesForPhase(). This is demonstrated in Script 4.
#!/usr/bin/python
from modsecurity import *
modsec = ModSecurity()
#s = "Hello World, I am: " + str(modsec.whoAmI())
#print s
rules = Rules()
r = rules.loadFromUri("/path/to/your/v3.0.0-dev/rules/REQUEST-10-IP-REPUTATION.conf")
if r == -1:
print rules.getParserError()
sys.exit()
i = 0
while i < modsec.NUMBER_OF_PHASES:
r = rules.getRulesForPhase(i)
print "-- Phase " + str(i)
for x in r:
if x.rule_id == 0:
continue
print " Rule Id: " + str(x.rule_id)
print " From: " + str(x.m_fileName) + " at " + str(x.m_lineNumber)
i = i + 1
Script 4. Print all rules ID from a ModSecurity configuration file.
The output of Script 4 should appear similar to the output illustrated below in Figure 3.
Figure 3. The output of the script which prints the rules ID in the console.
Of course this script can be extended to print this information in any manner desired. A slightly more refined example is illustrated in Figure 4. Remember that as this is Python it is easy to display this information as part of GUI or web application.
Figure 4. ModSecurtity rules information extracted using the libModSecurity parser, inside a Python script.
Conclusions
The new ModSecurity Python bindings should make it easy and fast to utilize libModSecurity as demonstrated in the examples above. These Python bindings are just one of a host of new features we will be presenting in upcoming blog posts, so be on the lookout.
The advantage of fast prototyping provided by the script language utilization, plus, the performance of a core in C++ opens a wide range of possibilities. Like the construction of a Rule editor in Python, and/or a Django web application to navigate inside the rules. In fact, why not implement those as open source projects?
Notice that those are just simple examples, but it is just as easy to use these bindings to actually process a transaction, but this is left as exercise for the reader, enjoy :)