Thursday, January 28, 2010

Cybercrime Studies: File Carving for Forensics Recovery

There is an upcoming talk at John Jay College that should be interesting:

File Carving for Forensics Recovery

Nasir Memon

Professor of Computer Science
Director of the Information Systems and Internet Security (ISIS) Lab
Polytechnic Institute of New York University


As the number of digital devices in use continues to increase, there has also been an increase in the seizure and analysis of digital data for forensic purposes. One of the areas of high forensic interest is in the recovery of digital data from devices. In cases where the file system information for a digital device is missing or corrupt, newer data recovery techniques involving a process known as file carving are used to recover the data. This talk describes the need for and evolution of file carving, and presents the various technologies that have been used to improve file carving recovery, including our own Smart Carving techniques.

Date: Tuesday, February 9, 2010
Time: Reception – 1:45pm, Lecture – 2:00 pm
Location: Room 630T, Haaren Hall
899 Tenth Avenue, New York City 10019


RSVP: Nicole Daniels at 212-237-8920 or email ndaniels@jjay.cuny.edu.
For additional information please contact Professor Doug Salane, Director of the Center for Cybercrime Studies, at 212-237-8836 or email dsalane@jjay.cuny.edu.

Monday, January 11, 2010

Volatility's Output Rendering Functions

Lately I've been playing around writing plugins for Volatility. A few of these will will be released at the end of this blog post, some others are still in the works to be released later. During the writing of some of the more complicated plugins, I decided that I needed to have some temporary storage while doing complex processing. Sure I could dump to a file and process that later, but why not do it within Volatility itself?

SQLite is good for this. There's an option to use an in-memory database (":memory:") that will remain in memory until the process dies. I also started thinking that some people might like to have a SQLite database of all the information they could get from a memory image for various reasons. Hence this is what the first release of plugins is all about.

Luckily Volatility has an option for plugins to have more than one output option. If you look at the code in forensics/commands.py you'll see the following (line numbers not included):


82 function_name = "render_%s" % self.opts.output
83 if not self.opts.out_file:
84 outfd = sys.stdout
85 else:
86 outfd = open(self.opts.out_file,'w')



This allows plugins to have more than one output function. For example a plugin might have a render_text function that would print to stdout as usual, a render_html function that prints out in html style, a render_sql function that does some SQL actions etc etc... The framework allows the user to pick which output option s/he wants and the output file on the command line as defined in vutils.py:


45 def get_standard_parser(cmdname):
...
...
59 op.add_option('-H','--output',default = 'text',
60 help='(optional, default="text") Output format (xml, html, sql)')
61 op.add_option('-O', '--out_file', default=None,
62 help='(output filename to write results onto - default stdout)')



Therefore, if there is a plugin that has an html output option, it can be invoked from the command line like so:


./volatility <plugin> -H html -O <out_file> -f mem.dd


Quite cool :-)

Plugin Structure

If you are interested in writing plugins for Volatility, you really should read Andreas Schuster's slides. They go into nice detail on how to write plugins for the framework. Here I will simply give you the gist :-)

The "skeleton" for the plugins is defined in forensics/commands.py. Items of interest include the help() function which is the plugin description you see when you run Volatility with the help option:


./volatility -h


Also of interest is the parser() function, which allows the plugin to modify its command line options. There is also the calculate() function, which is where the real work is done. The last item of our interest is the execute() function which allows us to calculate and collect the desired data from the memory image and then output it using the plugin's chosen render_* function.

The plugins I'm releasing now consist of core commands (defined in vmodules.py) that have been converted to this code structure so I could have more than one type of output for each of these commands. The plugins in this package are:

memory_plugins/connections_2.py
memory_plugins/dlllist_2.py
memory_plugins/files_2.py
memory_plugins/modules_2.py
memory_plugins/pslist_2.py
memory_plugins/sockets_2.py


Schema

The schema for these plugins is quite simple and not much different than the original output for these core commands. There is an extra field for the name of the memory image that was analyzed in case someone would like to place information for more than one memory image into a SQLite database. This may change at some point and of course you are free to change it as you like. It's enough for what I needed, however.

connections_2.py

Table Name: connections

pid Process ID
local Local connection information
remote Remote connection information
memimage Memory image information was extracted from


memory_plugins/dlllist_2.py [Edit: 2/16/10 new schema]

Table Name: dlls

image_file_name Process name
pid Process ID
cmdline Command Line text
base Base Address
size Size
path Path of DLL
memimage Memory image information was extracted from


memory_plugins/files_2.py

Table Name: files

pid Process ID
file Open file
num Number of times file is open by pid
memimage Memory image information was extracted from


memory_plugins/modules_2.py

Table Name: modules

file Module Path
base Base Address
size Size
name Module Name
memimage Memory image information was extracted from


memory_plugins/pslist_2.py

Table Name: process

pname Process Name
pid Process ID
ppid Parent Process ID
thrds Threads
hndl Handle Count
ctime Creation Time
memimage Memory image information was extracted from


memory_plugins/sockets_2.py

Table Name: sockets

pid Process ID
port Port
proto Protocol
ctime Creation Time
memimage Memory image information was extracted from


Installation

First, make sure you have SQLite3 installed along with support for Python. Now download the Volatility code from the SVN. Download the plugins from here. A listing of the plugins is as follows:


$ tar -tzf vol_sql.tgz
vutils.py
forensics/commands.py
memory_plugins/connections_2.py
memory_plugins/dlllist_2.py
memory_plugins/files_2.py
memory_plugins/modules_2.py
memory_plugins/pslist_2.py
memory_plugins/sockets_2.py



Make a backup of your vutils.py and forensics/commands.py files if you like. I had to make a small modification to both of these files to get the plugins working properly. Then place the tar file into your Volatility directory and type:


$ tar -xvzf vol_sql.tgz


Each of the redefined core commands end with "_2" so pslist becomes pslist_2 and connections becomes connections_2 and so on. So if I wanted to dump the output of the connections_2 plugin to a SQLite file I would type the following:


./volatility connections_2 -H sql -O test.db -f mem.dd


After running all of the new commands to the same SQLite3 file, I can then look at what I have stored:


$ sqlite3 test.db
SQLite version 3.5.9
Enter ".help" for instructions
sqlite> .table
connections dlls files modules process sockets
sqlite> .schema
CREATE TABLE connections (pid integer, local text, remote text, memimage text);
CREATE TABLE dlls (image_file_name text, pid integer, cmdline text, base text, size text, path text, memimage text);
CREATE TABLE files (pid, file, num, memimage);
CREATE TABLE modules (file text, base text, size text, name text, memimage text);
CREATE TABLE process (pname text, pid integer, ppid integer, thrds text, hndl text, ctime text, memimage text);
CREATE TABLE sockets (pid integer, port integer, proto text, ctime text, memimage text);
sqlite> select * from files where pid = 4 and file like '%SEC%';
4|\WINDOWS\system32\config\SECURITY|1|/home/levy/forensic/evidence/10.vmem
4|\WINDOWS\system32\config\SECURITY.LOG|1|/home/levy/forensic/evidence/10.vmem
sqlite> .quit



You can make as many complex queries as you like now :-)

Saturday, January 02, 2010

Briefly: Misc News

Into the Boxes Issue 0x0 is now out. It contains a small article on Linux memory forensics I wrote as well as a cool article on Windows 7 UserAssist Registry Keys by Didier Stevens, a hardware quick tip and FTK imager quick tip by Don C. Weber and a PCI interview with Harlan Carvey. For those who are interested in contributing to future publications, check out the Collaboration Box. Congratulations to Don and Harlan on the first release!

There is a new meetup group that I am helping to organize: NYC4Sec. The group consists of computer security, forensics and compliance professionals based in the Tri-State area and we will be meeting to discuss the latest trends in threats and responses as well as what to do when attacked. We are aiming to meet at the end of the month. Feel free to sign up and come to the meeting. If anyone is interested in presenting at the meeting, please contact myself or Morton Swimmer.

I have also been asked to give a talk at CEIC 2010. So if you are planning to attend, I'll see you there :-)