Configuring Firefox For Web App Pen Testing

You know the routine: you get a gig doing a web app pen test. You break out Burp (or whatever lesser proxy you prefer), and get ready to ruin some developer’s day. And then, just as you get ready to load the target URL and start, you see a ton of update requests hit the proxy.

It’s annoying. Your logs are polluted, and if you have to turn them over to the client, the extra noise strips some of the professionalism from your image (as a sidenote: Burp’s “only save in-scope items” feature helps quite a lot with this).

Here then, is a quick guide on how to¬†tweak Firefox so that it doesn’t spew stupid crap in your web app pen test log files. I may come back and explain the “why” behind some of these later, but for now, just the “how” will have to do. (Note: some of these settings reduce the security of the browser. My presumption here is that Firefox will only be used for testing, not for general purpose browsing. The settings below reflect that.)

1) Open about:config
2) Disable Safe Browsing
3) Disable Pipelining
4) Disable Pre-fetching
5) Remove all bookmarks
6) Set homepage to about:blank for startup
7) Make sure history is enabled, but disable search suggestions
8) Disable checking for updates
9) Just say no to helping developers
10) Disable updates for sync

That’s it. Now you can go forth, and break all the things, knowing that your log files will be nice and tidy afterwards.

Kivlad – Initial Thoughts

Recently, the folks over at Matasano Security released a tool to decompile Android Dalvik binaries. The tool is named Kivlad and it can be found on their site.

Having a need to dissassemble Android binaries on a fairly frequent basis, I’m always looking for a new tool to help out, so I took this one for a spin.

The first thing to point out is the disclaimer on the Matasano site, which reads:

This is very much an alpha release and while it will be production-quality in the near future, we wanted to give the community a taste of what’s to come.

Experimenting with Kivlad shows that disclaimer to be very warranted. Kivlad is a cool concept, and is somewhat unique in the field because the output format is HTML. At the core, Kivlad offers a tree view of the Android elements contained within the Dex. I like this idea, and it will be very cool to see it get developed further.

That said, the tool is pretty near non-functional (for me at least) – I’m unable to get it to parse any APK other than the included HelloWorld.apk that comes with the download.

Further, because the background of the HTML output is an image loaded via CSS, if the output file is not in the source code directory, it is completely unreadable (because the font color is set to white, and the default browser background is also white.) This is easily fixed, either open the file in the source code directory, or change the CSS to use whatever color scheme you wish.

I should note that I find both of these shortcoming to be perfectly acceptable, (Matasano did say the tool was pre-alpha afterall), and as I said, I like the concept and where this seems headed quite a lot.

The process to getting Kivlad working is pretty straightforward IMO, but I’ve included my notes here in case someone finds them useful. (These are for the Ubuntu Linux distribution, tested on versions 10.10 and 11.04)

  1. Install the required software and libraries:
    sudo apt-get install ruby1.9.1 libzip-ruby1.9.1 graphviz
    sudo ln -s /usr/bin/ruby1.9.1 /usr/bin/ruby
    sudo ln -s /usr/bin/gem1.9.1 usr/bin/gem
    sudo gem install metasm
  2. Download Kivlad and extract it:
    wget http://www.matasano.com/research/kivlad/kivlad-0.1.tar.gz
    tar zxf kivlad-0.1.tar.gz
    cd kivlad-0.1
  3. Run the tool:
    ruby ./reflect.rb HelloWorld.apk HelloWorld.html

This will spit out a bunch of stuff, and you’ll end up with a HelloWorld.html file in the directory:

Lcom/daeken/helloworld/HelloWorld;/
Succeed

Lcom/daeken/helloworld/HelloWorld;/bar
Succeed

...

Lcom/daeken/helloworld/R$string;/
Succeed

Lcom/daeken/helloworld/R;/
Succeed
[]

As I mentioned, the only APK I’ve been able to successfully use Kivlad on is the HelloWorld.apk. Every other APK I’ve tried has resulted in the following error (example below using OI_Safe_1.2.4 from freewarelovers.com):

ruby ./reflect.rb OI_Safe_1.2.4.apk OI_Safe_1.2.4.html

Lestreamj/ciphers/trivium/Trivium$Maker;/
Succeed

Lestreamj/ciphers/trivium/Trivium$Maker;/create
/home/rossja/Desktop/kivlad-0.1/instruction_form.rb:11:in `inspect': wrong number of arguments(1 for 0) (ArgumentError)
 from /home/rossja/Desktop/kivlad-0.1/instruction_form.rb:11:in `convert_insn'
 from /home/rossja/Desktop/kivlad-0.1/decompiler.rb:61:in `disassemble_all'
 from /home/rossja/Desktop/kivlad-0.1/decompiler.rb:85:in `disassemble_blocks'
 from /home/rossja/Desktop/kivlad-0.1/decompiler.rb:713:in `decompile'
 from /home/rossja/Desktop/kivlad-0.1/decompiler.rb:774:in `decompile'
 from /home/rossja/Desktop/kivlad-0.1/dex.rb:70:in `method'
 from /home/rossja/Desktop/kivlad-0.1/dex.rb:46:in `block (2 levels) in initialize'
 from /home/rossja/Desktop/kivlad-0.1/dex.rb:45:in `map'
 from /home/rossja/Desktop/kivlad-0.1/dex.rb:45:in `block in initialize'
 from /home/rossja/Desktop/kivlad-0.1/dex.rb:30:in `map'
 from /home/rossja/Desktop/kivlad-0.1/dex.rb:30:in `initialize'
 from ./reflect.rb:9:in `new'
 from ./reflect.rb:9:in `initialize'
 from ./reflect.rb:21:in `new'
 from ./reflect.rb:21:in `'

At some point I’ll try to chase down where the issue is and see if I can figure out whom to send a patch to =)

Detecting URL Rewriting (part 2)

This post is a continuation of my documenting the process I go through to come up with some way a client of a web site can first: determine if URL rewriting is occurring on a given web server, and second: in cases where it is used, determine what the rewrite rules are.

I left off with Apache configured, and a simple rule established for mod_rewrite. I now need to decide whether to use mod_rewrite to handle the rewrite using a redirect (via an HTTP 302 response), or to process it internally. As I mentioned, the difference between these two methods is quite large.

For example, if I choose to send a redirect, (eg. by amending our rule to include an [R] flag), like so …

RewriteRule    /litterbox/(.*)  /sandbox/$1 [R]

… the rewrite rule will cause an incoming request to http://bar.com/litterbox/bar1.php to be redirected to the location http://bar.com/sandbox/bar1.php instead by using HTTP server headers.

Examining the relevant portion of the HTTP request and response headers associated with this process, the conversation looks like this:

Initial request:

GET /litterbox/bar1.php HTTP/1.1
Host: bar.com

Initial response:

HTTP/1.1 302 Found
Date: Wed, 06 Oct 2010 04:50:18 GMT
Server: Apache/2.2.9 (Debian) PHP/5.2.6-1+lenny9 with Suhosin-Patch
Location: http://bar.com/sandbox/bar1.php

In the response above, notice that the server has returned an HTTP 302 status response, and included a Location: header which contains the URL to the content. The browser receives this, and sends a new request to that location:

Redirected request:

GET /sandbox/bar1.php HTTP/1.1
Host: bar.com

This request is met with the final response, which includes the content at /sandbox/bar1.php:

HTTP/1.1 200 OK
Date: Wed, 06 Oct 2010 04:50:29 GMT

This is how I’ve used mod_rewrite in the past. The rules I’ve set to enforce SSL have been very similar to the one given in the example. At first glance, it seems that it will be easy to tell when rewriting is occurring… all that’s required is to look for the 302 response!

Not so fast

There are a couple problems with this theory. The first is: there are other mechanisms which can be used to provide this same HTTP response code. For example, the following PHP code will cause an HTTP 302 response to be sent by the server:

<?php
header("Location: http://bar.com/sandbox/bar1.php");
?>

When I put that code into a file located at http://bar.com/redir.php, the response to a GET request for that file looks pretty much exactly like the one generated natively by Apache above:

HTTP/1.1 302 Found
Date: Wed, 06 Oct 2010 05:38:09 GMT
Server: Apache/2.2.9 (Debian) PHP/5.2.6-1+lenny9 with Suhosin-Patch
X-Powered-By: PHP/5.2.6-1+lenny9
Location: http://bar.com/sandbox/bar1.php

From this, it would seem that there is no way to distinguish between a redirect coming from mod_rewrite, and one stemming from some other mechanism.

More importantly though, and a bigger blow to my high hopes for an easy answer, is that the [R] flag is optional. By default, a redirect header isn’t returned by Apache at all when mod_rewrite is used. Looking up how Apache handles rewriting, there’s a fair amount of documentation on the process specific to the 2.2 version of Apache I’m using:

The nutshell version is this: Requests which are rewritten and not using a 302 response to the client are processed completely within the Apache Kernel only. There’s no indication given to the client that a redirect has occurred.

In fact, it appears that the only way an application hosted on the server can know that it has been reached via a rewritten request is by checking for the presence of one or both of two server headers which only appear when Apache has processed a rewrite … they do not appear on a redirect, despite their name =)

(Recall that I can see these because the PHP script I wrote includes a printout of every server header. It seems that doing this was a good idea indeed!):

REDIRECT_STATUS = 200
REDIRECT_URL = /litterbox/bar1.php

Note that these headers are different than the ones the Apache documentation says it adds. I’m not sure why that is, but since these headers are only available as server variables, they are completely outside the reach of a client accessing a given URL on the host.

That sucks.

At this point, I give up on the 302 response and Location: header theory: it’s both misleading (in that a 302 response may not be the result of a URL rewrite), and inconsistent in that rewritten URLs may not provide a 302 response at all.

I start thinking of other mechanisms I could use. One that comes immediately to mind is the Referer header. This is an HTTP header which is provided to a web server when, for example, a user clicks a link. The destination host the link resolves to receives the request for a URL, along with where the user came from. An example of this can be seen here:

Initial Request:

GET /litterbox/bar1.php HTTP/1.1
Host: bar.com

Initial Response:

HTTP/1.1 200 OK
Date: Fri, 08 Oct 2010 05:51:54 GMT
[content]
  <div><a href="bar2.php">bar2</div>
[more-content]

The content served in the response contains a link to bar2.php. When I click that link, the fact that I’m coming from the bar1.php page is sent in the request, as shown below:
Request to bar2.php:

GET /sandbox/bar2.php HTTP/1.1
Host: bar.com
Referer: http://bar.com/litterbox/bar1.php

That’s all well and good, but as you can see, the Referer still shows /litterbox as the URL I was coming from. That’s because the referer is specified by the user agent (a browser in this case). Since the browser didn’t receive any indication that the content it is being served has come from a different location than it requested, it thinks it’s still at /litterbox and so sends that location in the headers.

So much for using that as a detection of rewriting. What’s next…
So far, I’ve tried a couple of different ideas to try to determine if a client can tell whether URL rewriting is in use or not. I’ve ruled out using a 302 response and accompanying Location: header as being unfit for this purpose. I’ve also briefly played with the idea of using Referer, and quickly ruled that out as an option as well. I need to come up with some more creative way to try to tell.

How about timing?

Thinking about this problem a bit, it occurs to me that, since the Apache kernel has to map rewritten URLs internally to come up with a computed URL to serve content from, that I may be able to use how long a request takes to load as an indicator.

To test this theory out, I’m going to use ruby, because I’m familiar with it, and it allows me to quickly throw together some proof-of-concept code.

Since I have the advantage in this case of knowing for sure what is being rewritten and what is not, I can use the benchmark module in ruby to measure the time it takes to get a file where rewriting is occurring, and where it is not. I can then compare the two to see if this theory bears further investigation.

For the intital test, I decide to use the bmbm method of the benchmark module for two reasons: 1) it automatically gives me two iterations to compare. But more importantly it 2) initializes the environment and tries to minimize skewed results by going through a rehearsal process before benchmarking “for reals”. Once I decided that, I came up with the following script:

#!/usr/bin/env ruby
require 'net/http'
require 'uri'
require 'benchmark'
include Benchmark

bmbm do |test|
  test.report("rewrite:") do
    Net::HTTP.get_response URI.parse('http://bar.com/litterbox/bar1.php')
  end
  test.report("non-rewrite:") do
    Net::HTTP.get_response URI.parse('http://bar.com/sandbox/bar1.php')
  end
end

I’ve created two labels in this benchmark: one for the known rewritten URL, and one for the known non-rewritten URL. When I run this script, I get the following results:

Rehearsal ------------------------------------------------
rewrite:       0.010000   0.000000   0.010000 (  0.001429)
non-rewrite:   0.000000   0.000000   0.000000 (  0.000876)
--------------------------------------- total: 0.010000sec

                   user     system      total        real
rewrite:       0.000000   0.000000   0.000000 (  0.001105)
non-rewrite:   0.000000   0.000000   0.000000 (  0.000907)

That’s pretty interesting! When I run this on the same host the web server is located at, I can definitely tell a difference between rewritten and non-rewritten content!

I need to look into this further. The first thing that needs to happen is, I need to perform these requests many more times and look at the timing. A single request is useful for a quick “is there merit to this”, but the fact that it appears this may work could just be a fluke in the given requests at that particular time. I need to increase the number of times I perform this test and prove whether, statistically, there is a difference in the time it takes to serve a rewritten URL vs a non-rewritten one.

I also need to look at what factors may affect the results. Some immediate considerations that come to mind are:

  1. is the Apache server cacheing content, causing it to be served faster the second time?
  2. Am I able to prevent that if so?
  3. On a local machine, this may work, but what happens across a LAN?
  4. What happens to the timing when requests go across the Internet?
  5. How much does “heavy” content (video, images, etc.) affect the timing?
  6. Can I time just getting the HTTP headers, to avoid loading content?

I need to answer some of these before testing, and some of these will be answered as the testing progresses.

[to be continued]