Monday, January 11, 2010

Volatility's Output Rendering Functions

Lately I've been playing around writing plugins for Volatility. A few of these will will be released at the end of this blog post, some others are still in the works to be released later. During the writing of some of the more complicated plugins, I decided that I needed to have some temporary storage while doing complex processing. Sure I could dump to a file and process that later, but why not do it within Volatility itself?

SQLite is good for this. There's an option to use an in-memory database (":memory:") that will remain in memory until the process dies. I also started thinking that some people might like to have a SQLite database of all the information they could get from a memory image for various reasons. Hence this is what the first release of plugins is all about.

Luckily Volatility has an option for plugins to have more than one output option. If you look at the code in forensics/commands.py you'll see the following (line numbers not included):


82 function_name = "render_%s" % self.opts.output
83 if not self.opts.out_file:
84 outfd = sys.stdout
85 else:
86 outfd = open(self.opts.out_file,'w')



This allows plugins to have more than one output function. For example a plugin might have a render_text function that would print to stdout as usual, a render_html function that prints out in html style, a render_sql function that does some SQL actions etc etc... The framework allows the user to pick which output option s/he wants and the output file on the command line as defined in vutils.py:


45 def get_standard_parser(cmdname):
...
...
59 op.add_option('-H','--output',default = 'text',
60 help='(optional, default="text") Output format (xml, html, sql)')
61 op.add_option('-O', '--out_file', default=None,
62 help='(output filename to write results onto - default stdout)')



Therefore, if there is a plugin that has an html output option, it can be invoked from the command line like so:


./volatility <plugin> -H html -O <out_file> -f mem.dd


Quite cool :-)

Plugin Structure

If you are interested in writing plugins for Volatility, you really should read Andreas Schuster's slides. They go into nice detail on how to write plugins for the framework. Here I will simply give you the gist :-)

The "skeleton" for the plugins is defined in forensics/commands.py. Items of interest include the help() function which is the plugin description you see when you run Volatility with the help option:


./volatility -h


Also of interest is the parser() function, which allows the plugin to modify its command line options. There is also the calculate() function, which is where the real work is done. The last item of our interest is the execute() function which allows us to calculate and collect the desired data from the memory image and then output it using the plugin's chosen render_* function.

The plugins I'm releasing now consist of core commands (defined in vmodules.py) that have been converted to this code structure so I could have more than one type of output for each of these commands. The plugins in this package are:

memory_plugins/connections_2.py
memory_plugins/dlllist_2.py
memory_plugins/files_2.py
memory_plugins/modules_2.py
memory_plugins/pslist_2.py
memory_plugins/sockets_2.py


Schema

The schema for these plugins is quite simple and not much different than the original output for these core commands. There is an extra field for the name of the memory image that was analyzed in case someone would like to place information for more than one memory image into a SQLite database. This may change at some point and of course you are free to change it as you like. It's enough for what I needed, however.

connections_2.py

Table Name: connections

pid Process ID
local Local connection information
remote Remote connection information
memimage Memory image information was extracted from


memory_plugins/dlllist_2.py [Edit: 2/16/10 new schema]

Table Name: dlls

image_file_name Process name
pid Process ID
cmdline Command Line text
base Base Address
size Size
path Path of DLL
memimage Memory image information was extracted from


memory_plugins/files_2.py

Table Name: files

pid Process ID
file Open file
num Number of times file is open by pid
memimage Memory image information was extracted from


memory_plugins/modules_2.py

Table Name: modules

file Module Path
base Base Address
size Size
name Module Name
memimage Memory image information was extracted from


memory_plugins/pslist_2.py

Table Name: process

pname Process Name
pid Process ID
ppid Parent Process ID
thrds Threads
hndl Handle Count
ctime Creation Time
memimage Memory image information was extracted from


memory_plugins/sockets_2.py

Table Name: sockets

pid Process ID
port Port
proto Protocol
ctime Creation Time
memimage Memory image information was extracted from


Installation

First, make sure you have SQLite3 installed along with support for Python. Now download the Volatility code from the SVN. Download the plugins from here. A listing of the plugins is as follows:


$ tar -tzf vol_sql.tgz
vutils.py
forensics/commands.py
memory_plugins/connections_2.py
memory_plugins/dlllist_2.py
memory_plugins/files_2.py
memory_plugins/modules_2.py
memory_plugins/pslist_2.py
memory_plugins/sockets_2.py



Make a backup of your vutils.py and forensics/commands.py files if you like. I had to make a small modification to both of these files to get the plugins working properly. Then place the tar file into your Volatility directory and type:


$ tar -xvzf vol_sql.tgz


Each of the redefined core commands end with "_2" so pslist becomes pslist_2 and connections becomes connections_2 and so on. So if I wanted to dump the output of the connections_2 plugin to a SQLite file I would type the following:


./volatility connections_2 -H sql -O test.db -f mem.dd


After running all of the new commands to the same SQLite3 file, I can then look at what I have stored:


$ sqlite3 test.db
SQLite version 3.5.9
Enter ".help" for instructions
sqlite> .table
connections dlls files modules process sockets
sqlite> .schema
CREATE TABLE connections (pid integer, local text, remote text, memimage text);
CREATE TABLE dlls (image_file_name text, pid integer, cmdline text, base text, size text, path text, memimage text);
CREATE TABLE files (pid, file, num, memimage);
CREATE TABLE modules (file text, base text, size text, name text, memimage text);
CREATE TABLE process (pname text, pid integer, ppid integer, thrds text, hndl text, ctime text, memimage text);
CREATE TABLE sockets (pid integer, port integer, proto text, ctime text, memimage text);
sqlite> select * from files where pid = 4 and file like '%SEC%';
4|\WINDOWS\system32\config\SECURITY|1|/home/levy/forensic/evidence/10.vmem
4|\WINDOWS\system32\config\SECURITY.LOG|1|/home/levy/forensic/evidence/10.vmem
sqlite> .quit



You can make as many complex queries as you like now :-)

4 comments:

Scudette said...

Gleeda,
Thanks for a great post. Maybe new plugins should be developed with the 1.4 branch - i think you will find the new API much easier to work with and examples for using it will really help get more people interested.

Scudette

JL said...

Hey Scudette,

Thanks! I haven't looked at the 1.4 branch in a little while, but I will soon. I appreciate the work you've done getting the rendering capability built in - it's really quite useful :-)

-Gleeda

Scudette said...

Gleeda,
Check out the same code implemented for the 1.4beta branch:

http://code.google.com/p/volatility/source/browse/branches/experimental/plugins/sqlite.py

Love to hear what you think...

Scudette.

Bruce said...

Thanks you for Vol2html. I was lost trying to use Volatility output by hand.

--Bruce D. Meyer