At precisely 155 commits ahead of the latest version, ModSecurity version 3.0.3 contains a number of improvements and features to enhance the ModSecurity experience. In this blog post, we'll explain some of the new capabilities in the latest release.
Better user experience while reading the logs
ModSecurity has three different logging outputs: AuditLogs, ErrorLogs, and DebugLogs. Each of the logs has a specific purpose. There is one factor that can be used to make a connection between those logs which is the request unique identificator, also known as request id.
Internally in ModSecurity, the request id is a randomly generated number that is initiated at the beginning of a request. This number persists until the request is finished.
Depending on how you are using ModSecurity, there will be another log entry of interest. This log entry isn't generated by ModSec, but by ModSec's consumer. This use case was well illustrated by contributor @defanator on the issue #1627. In this issue, the user expressed the desire to clearly and easily match nginx logging with ModSec`s log. This is an interesting feature to have, allowing to match the auditlogs giving an entry on nignx error log, and vice versa.
Adding this functionality to ModSecurity first required ModSecurity to be aware of the existence of an external Unique ID, and then be able to use it for its own transaction IDs. This functionality was implemented in the API by extending the method msc_new_transaction which counts with a new version: msc_new_transaction_with_id. The existing msc_new_transaction method was also kept and continues to work as it did before.
Code Snippet 1. ModSecurity`s methods signature to create a new transaction. On GitHub:
https://github.com/SpiderLabs/ModSecurity/blob/4e6e4243a899ff46dccd189693b2f5d9421dcd98/src/transaction.cc#L1767-L1770
Once the API was capable of handling such functionality the connectors were also extended. Details on that are a subject for another blog post :)
More capable customization in different folders and vhosts
One of the important features of ModSecurity is the ability to handle customized rules per directory or virtual host. This feature includes the capability of dynamic changes the rules behavior in runtime.
For the runtime changes, the ctl:* actions are most important as they are responsible to change, for example, the kind on inspection that is going to be performed on a request body, based on its content: JSON, XML, etc. In this new release, you can also count with the ctl:requestBodyProcessor=URLENCODED ctl action. It is not yet on the modsecurity.conf-recommended but it meant to be in further releases as under discussion as the issue #1797.
As for the static changes, the SecRuleUpdateActionById action has an important role. It's meant to be used to change a possible disruptive action in the consequence of rule match, as discussed at #1800.
Improving the integration and/or communication with other peers
ModSecurity is well known to be the swiss army knife of WAFs. One of the reasons for that description is, as an open source project, it integrates with different tools and utilities. As part of this communication channel, we now have the ability to set and read environment variables. Once set, all other software in execution on the same context can read and write to it. Further details are discussed at #1044.
Yes, the code readability is better
As part of good coding practices, we are always working to improve the code quality in ModSecurity. The version 3 project was initiated as an agile project. The architecture was designed with great care and all test utilities and security tools were built to be robust, but with space for improvements on the implementation. That was the case for the variables which before were declared/expressed by an entire class, but now with just a simple macro expansion.
These improvements are very important. By avoiding code repetition we not only make the readability better but allow a single change to reflect in different parts of code. Making code easier to read also means more chances for less experienced [or lazy :)] developers to be able to contribute to the project with patches. Take your shot :)
Couple of issues related to code readability: 0xb7c36, 0x5ac20, #1697, 0xb2840, 0xca270
An uptake on performance
As a consequence of a well-designed architecture, every best practice improvement also leads to better performance. f An example is the Rules class. In its first version, it was made to support very few features. As time passed, new features were being added and the class methods were no longer making sense. There was no point to refactor as other new features were on the pipe to be developed, mainly the "changes" actions such as: SecRemoveRuleById.
Now that almost all needed features were in place, it seemed like a good time to do a punctual refactoring on the Rule class. The consequence of applying the best practices in this specific class turned out to have a very beneficial side consequence: overall better performance.
What next?
Hard to tell precisely what will be released next. New features that are on the pipe are the distributed collections and other awesome stuff that can be seen on GitHub:
https://github.com/SpiderLabs/ModSecurity/milestone/13
