Examining the capabilities of malicious software allows your IT team to better assess the nature of a security incident, and may help prevent further infections. Here's how to set up a controlled malware-analysis lab—for free.
Page 4 of 5
Your 5-Step Malware-Analysis Toolkit
A LARGE NUMBER of computer intrusions
involve some form of malicious software (malware),
which finds its way to the victim's workstation or to
a server. When investigating the incident, the IT
responder typically seeks to answer questions
such as: What actions can the malware specimen
perform on the system? How does it spread? How,
if at all, does it maintain contact with the attacker?
These questions can all be answered by analyzing
the offending malware in a controlled environment.
A simple analysis toolkit, built from free and readily
available software, can help you and your IT
team develop the skills critical to responding to
today's security incidents. The steps below will
help get you started. We'll focus on malware analysis
in a Windows environment, since that platform
is particularly popular among malware authors.
Step 1: Allocate physical or virtual
systems for the analysis lab
A common approach to examining malicious software
involves infecting a system with the malware
specimen and then using the appropriate
monitoring tools to observe how it behaves. This requires
a laboratory system you can infect without affecting your
The most popular and flexible way to set up such a lab
system involves virtualization software, which allows you
to use a single physical computer for hosting multiple virtual
systems, each running a potentially different operating
system. Free virtualization software options include:
Running multiple virtual systems simultaneously on a single
physical computer is useful for analyzing malware that
seeks to interact with other systems, perhaps for leaking
data, obtaining instructions from the attacker, or upgrading
itself. Virtualization makes it easy to set up and use such
systems without procuring numerous physical boxes.
Another useful feature of many virtualization tools is the
ability to take instantaneous snapshots of the laboratory
system. This way, you can record the state of the system
before you infect it, and revert to the pristine environment
with a click of a button at the end of your analysis.
If using virtualization software, install as much RAM into
the physical system as you can, as the availability of memory
is arguably the most important performance factor for virtualization tools. In addition, having a large hard drive will
allow you to host many virtual machines, whose virtual file
systems typically are stored as files on the physical system's
Take precautions to isolate the malware-analysis lab from the production network, to mitigate the risk that a malicious program will escape.
Because malware may detect that it's running in a virtualized
environment, some analysts prefer to rely on physical,
rather than virtual, machines for implementing laboratory
systems. Your old and unused PCs or servers can
make excellent systems for your malware-analysis lab,
which usually doesn't need high-performing CPUs or highly
redundant hardware components.
To allow malware to reach its full potential in the lab, laboratory
systems typically are networked with each other.
This helps you observe the malicious program's network
interactions. If using physical systems, you can connect
them with each other using an inexpensive hub or a switch.
Step 2: Isolate laboratory systems from the
You must take precautions to isolate the malware-analysis
lab from the production network, to mitigate the risk that a
malicious program will escape. You can separate the laboratory
network from production using a firewall. Better yet,
don't connect laboratory and production networks at all, to
avoid firewall configuration issues that might allow malware
to bypass filtering restrictions.
If your laboratory network is strongly isolated, you can
use removable media to bring tools and malware into the
lab. It's best to use write-once media, such as CDs, to prevent
malicious software from escaping the lab's confines by
writing itself to a USB key. If using a USB key, which is
more convenient than a CD, get a model that includes a
physical write-protect switch.
Some malware-analysis scenarios benefit from the lab
being connected to the internet. Avoid using the production
network for such connectivity. If possible, provision a
separate, and usually inexpensive, internet connection, perhaps
by dedicating a DSL line to this purpose. Avoid keeping
the lab connected to the internet all the time to minimize
the chance of malware in your lab attacking someone else's
system on the internet.
If virtualizing your lab, be sure to keep up with security
patches released by the virtualization-software vendor. Such
software may have vulnerabilities that could allow malware
to escape from the virtual system you infected and onto the
physical host. Furthermore, don't use the physical machine
that's hosting your virtualized lab for any other purpose.
Step 3: Install behavioral analysis tools
Before you're ready to infect your laboratory system with
the malware specimen, you need to install and activate the
appropriate monitoring tools. Free utilities that will let you
observe how Windows malware interacts with its environment
- File system and registry monitoring: Process Monitor and Capture BAT offer a powerful way to observe in
real time how local processes read, write, or delete
registry entries and files. These tools can help you
understand how malware attempts to embed into the
system upon infection.
- Process monitoring: Process Explorer and Process Hacker replace the built-in Windows Task Manager, helping
you observe malicious processes, including local network
ports they may attempt to open.
- Network monitoring:Wireshark and SmartSniff are
network sniffers, which can observe laboratory network
traffic for malicious communication attempts, such as
DNS resolution requests, bot traffic, or downloads.
- Change detection: Regshot is a lightweight tool for comparing the system's
state before and after the infection, to highlight
the key changes malware made to the file system and
Behavioral monitoring tools can give you a sense for the
key capabilities of malicious software. For further details
about its characteristics, you may need to roll up your
sleeves and perform some code analysis.
Step 4: Install code-analysis tools
Examining the code that comprises the specimen helps
uncover characteristics that may be difficult to obtain through
behavioral analysis. In the case of a malicious executable, you
rarely will have the luxury of access to the source code from
which it was created. Fortunately, the following free tools can
help you reverse compiled Windows executables:
- Disassembler and debugger: OllyDbg and IDA Pro Freeware can parse compiled Windows
executables and, acting as disassemblers, display their
code as Intel x86 assembly instructions. These tools
also have debugging capabilities, which allow you to
execute the most interesting parts of the malicious program
slowly and under highly controlled conditions, so
you can better understand the purpose of the code.
- Memory dumper: LordPE and OllyDump help obtain protected code located in the
lab system's memory and dump it to a file. This technique
is particularly useful when analyzing packed executables,
which are difficult to disassemble because
they encode or encrypt their instructions, extracting
them into RAM only during run-time.
Step 5: Utilize online analysis tools
To round off your malware-analysis toolkit, add to it some
freely available online tools that may assist with the reverse engineering
process. One category of such tools performs
automated behavioral analysis of the executables you supply.
These applications look similar at first glance, but use
different technologies on the back end. Consider submitting
your malware specimen to several of these sites;
depending on the specimen, some sites will be more effective
than others. Such tools include:
Another set of potentially useful online tools provides
details about websites that are suspected of hosting malicious
code. Some of these tools examine the sites you
specify in real time; others provide historical information.
Consider submitting a suspicious URL to several of these
sites, because each may offer a slightly different perspective
on the website in question:
With your initial toolkit assembled,
start experimenting in the lab with
malware you come across on the
web, in your e-mail box, on your systems,
and so on. You may find this
one-page cheat sheet convenient.
Begin analysis with the tools and
approaches most familiar to you.
Then, as you become more familiar
with the inner workings of the malware
specimen, venture out of your
comfort zone to try other tools and
techniques. The tools I've listed within
each step operate virtually identically.
Since they're all free, you should
feel free to try them all. You'll find that
one tool will work better than another,
depending on the situation. And with
time, patience, and practice, you will
learn to turn malware inside out.