fusil@...

Fusil on windows

Fusil project moved to bitbucket

Hi,

Since I'm usable to fix the *.hachoir.org website issues (well, I'm too 
lazy to fix it), I choosed to move all my personal projects to 
Bitbucket. I already moved python-ptrace, and now it's Fusil.

So the Fusil project moved to bitbucket.org server, and the source code 
is now using Mercurial (instead of Subversion).

  http://bitbucket.org/haypo/fusil/wiki/Home

The Mercurial repository contains the whole svn history, but only the 
trunk branch.

Note : My web server (serving all *.hachoir.org websites) will be down 
between the 1st August and around the 20 August, because I'm moving to a 
new flat...

Victor

python-ptrace project moved to bitbucket

Hi,

Since I'm usable to fix the *.hachoir.org website issues (well, I'm too 
lazy to fix it), I choosed to move all my personal projects to 
Bitbucket. The first one is python-ptrace (one of the smallest).

So the python-ptrace project moved to bitbucket.org server, and the 
source code is now using Mercurial (instead of Subversion).

   http://bitbucket.org/haypo/python-ptrace/wiki/Home

The Mercurial repository contains the whole svn history.

Note : My web server (serving all *.hachoir.org websites) will be down 
between the 1st August and around the 20 August, because I'm moving to a 
new flat...

Victor

[PATCH] python-ptrace: following a byte string across a program's execution

Hello all,

attached you will find a small patch that allows one to "follow" a 
sequence of bytes (a regular string or a byte string) across the
execution of a program.

The gdb.py "follow" command takes one argument, the term that the user
wishes to follow. The user can add multiple terms to the list of terms
to be "followed", as shown below:

(gdb) follow "dimitris"
(gdb) follow "\x01\x02\x03\x04"

The "showfollow" command shows the current list of "followed" terms

(gdb) showfollow
['dimitris', '\x01\x02\x03\x04']

At any point during the debugging of the program, one can use the
"xray" command to inspect the memory of all debugged processes
for instances of the "followed" terms:

(gdb) xray
term['dimitris'] pid[2686] 0x093a0000-0x09c29000 => [heap] (rw-p) 
0x09552aa0 pointers: 0x09552b88
term['dimitris'] pid[2686] 0x093a0000-0x09c29000 => [heap] (rw-p) 
0x09552b40 pointers: 0x09552b20
term['dimitris'] pid[2686] 0x093a0000-0x09c29000 => [heap] (rw-p) 
0x095532a0 pointers: 0x09553290
term['dimitris'] pid[2686] 0x093a0000-0x09c29000 => [heap] (rw-p) 
0x095e9ce0 pointers: 0x095e82d8
term['dimitris'] pid[2686] 0x093a0000-0x09c29000 => [heap] (rw-p) 
0x09a272a0 pointers: 0x09a2730c
term['dimitris'] pid[2686] 0x093a0000-0x09c29000 => [heap] (rw-p) 
0x09bc3f9e pointers:
...

The "pointers: " clause, shows the addresses of possible pointers 
pointing to a "followed" term.

The command "resetfollow" resets the list of "followed" terms.
(gdb) resetfollow
(gdb) showfollow
[]

It seems that python-ptrace has some serious potential in the field
of taint analysis 

Hope you will find this useful!

cheers,

dimitris

--- gdb.py	2009-04-14 00:53:03.000000000 +0300
+++ /usr/local/bin/gdb.py	2009-04-15 01:20:29.000000000 +0300
@@ -12,7 +12,7 @@
 from ptrace.binding import HAS_PTRACE_SINGLESTEP
 from ptrace.disasm import HAS_DISASSEMBLER
 from ptrace.ctypes_tools import (truncateWord,
-    formatWordHex, formatAddress, formatAddressRange)
+    formatWordHex, formatAddress, formatAddressRange, word2bytes)
 from ptrace.process_tools import dumpProcessInfo
 from ptrace.tools import inverseDict
 from ptrace.func_call import FunctionCallOptions
@@ -22,6 +22,8 @@
 from errno import ESRCH
 from ptrace.cpu_info import CPU_POWERPC
 from ptrace.debugger import ChildError
+from ptrace.debugger.memory_mapping import readProcessMappings
+
 import re
 try:
     # Use readline for better raw_input()
@@ -66,12 +68,27 @@
     ("proclist", "list of traced processes"),
     ("switch", "switch active process (switch or switch <pid>)"),
 
+    ("follow", 'follow a term (eg. "follow \'\\0x12\\0x14\\0x27\\0x13\'")'),
+    ("showfollow", 'show all "followed" terms'),
+    ("resetfollow", 'reset all "followed" terms'),
+    ("xray", 'show addresses of (and pointers to) "followed" terms'),
+
     # other
     ("dbginfo", "informations about the debugger"),
     ("quit", "quit debugger"),
     ("help", "display this help"),
 )
 
+# finds possible pointer values in process memory space, 
+# pointing to address
+def getPointers(process, address):
+    retlist = []
+    procmaps = readProcessMappings(process)
+    for pm in procmaps:
+        for found in pm.search(word2bytes(address)):
+            retlist.append(found)
+    return retlist
+
 class Gdb(Application):
     def __init__(self):
         Application.__init__(self)
@@ -94,6 +111,8 @@
         # FIXME: Remove self.breaks!
         self.breaks = dict()
 
+        self.followterms = []
+
     def setupLog(self):
         self._setupLog(stdout)
 
@@ -205,6 +224,33 @@
             values.append(value)
         return values
 
+    def addfollowterm(self, term):
+        # Allow terms of the form 'string', "string", '\x04', "\x01\x14"
+        #
+        # fixme: this is not really safe, since the user can always
+        # input a string like 'bla\'
+        if ((term.startswith("'") and term.endswith("'")) or
+            (term.startswith('"') and term.endswith('"'))):
+            eval("self.followterms.append(%s)" % term)
+        else:
+            return 'Follow term must be enclosed in quotes!'
+
+    def showfollowterms(self):
+        print self.followterms
+
+    # displays the offsets of all terms found in the process memory mappings
+    # along with possible addresses of pointers pointing to these terms
+    def xray(self):
+        for term in self.followterms:
+            for process in self.debugger:
+                for procmap in readProcessMappings(process):
+                    for found in procmap.search(term):
+                        print "term[%s] pid[%i] %s %s pointers: %s" % (
+                            repr(term), process.pid, procmap, 
+                            formatAddress(found),
+                            " ".join([formatAddress(x) for x in 
+                                      getPointers(process, found)]))
+
     def execute(self, command):
         errmsg = None
         if command == "cont":
@@ -257,6 +303,14 @@
             errmsg = self.signal(command[7:])
         elif command.startswith("print "):
             errmsg = self.print_(command[6:])
+        elif command.startswith("follow "):
+            errmsg = self.addfollowterm(command[7:])
+        elif command == "showfollow":
+            self.showfollowterms()
+        elif command == "resetfollow":
+            self.followterms = []
+        elif command == "xray":
+            self.xray()
         else:
             errmsg = "Unknown command: %r" % command
         if errmsg:
--- ptrace/debugger/memory_mapping.py	2009-04-13 19:11:35.000000000 +0300
+++ /usr/local/lib/python2.5/site-packages/ptrace/debugger/memory_mapping.py	2009-04-15
01:08:04.000000000 +0300
@@ -31,12 +31,16 @@
      - major_device / minor_device (int): major / minor device number
      - inode (int)
      - pathname (str)
+     - pid (int)
 
     Operations:
      - "address in mapping" checks the address is in the mapping.
+     - "search(somestring)" returns the offsets of "somestring" in the mapping
      - "str(mapping)" create one string describing the mapping
+     - "repr(mapping)" create a string represantation of the mapping,
+       useful in list contexts
     """
-    def __init__(self, start, end, permissions, offset, major_device, minor_device, inode, pathname):
+    def __init__(self, start, end, permissions, offset, major_device, minor_device, inode, pathname, pid):
         self.start = start
         self.end = end
         self.permissions = permissions
@@ -45,6 +49,7 @@
         self.minor_device = minor_device
         self.inode = inode
         self.pathname = pathname
+        self.pid = pid
 
     def __contains__(self, address):
         return self.start <= address < self.end
@@ -55,6 +60,29 @@
             text += " => %s" % self.pathname
         text += " (%s)" % self.permissions
         return text
+    
+    def search(self, bytestr):
+        retlist = []
+        bytestr_len = len(bytestr)
+        proc_mem = open("/proc/%i/mem" % self.pid, "r")
+        proc_mem.seek(self.start)
+        covered = self.start
+        data = proc_mem.read(self.end - self.start)
+        while (data != ""):
+            offset = data.find(bytestr)
+            if (offset == -1):
+                proc_mem.close()
+                return retlist
+            else:
+                retlist.append(offset + covered)
+                covered += offset + bytestr_len
+                proc_mem.seek(covered)
+                data = proc_mem.read(self.end - covered)
+        proc_mem.close()
+        return retlist
+
+    def __repr__(self):
+        return self.__str__()
 
 def readProcessMappings(process):
     """
@@ -86,7 +114,7 @@
                 int(match.group(5), 16),
                 int(match.group(6), 16),
                 int(match.group(7)),
-                match.group(8))
+                match.group(8), process.pid)
             maps.append(map)
     finally:
         mapsfile.close()

Release of Fusil 1.2.1

Changes from Fusil 1.1 to 1.2.1:

Fusil 1.2.1 (2009-02-06)
------------------------

 * Fix mangle agent of the Image Magick fuzzer
 * Fix AttachProcessPID() probe: stop the probe at process exit

Fusil 1.2 (2009-02-04)
----------------------

User visible changes:

 * Fusil now requires Python 2.5
 * Documentation: write an index (index.rst) and an user guide (usage.rst)
 * Replay script: copy HOME environment for GDB and catch setuid() error
 * fusil-firefox: support more file formats (bmp, gif, ico, png, svg), 
create
   --test command line option, write the HTML page into index.html file
 * fusil-python: write errors to stderr (instead of stdout) to avoid unicode
   error (especially with Python3)
 * FileWatch: rename the session with "long_output" if the program wrote 
more
   than max_nbline lines
 * fusil-python: blacklist posix.fork() to avoid false positive
 * If the process is killed by a signal, rename the session using the
   signal name (already worked if the debugger was disabled)

Developer changes:

 * MangleAgent supports multiple input files
 * Create DummyMangle: agent with MangleFile API but don't touch file 
content
   to test the fuzzer
 * Network: close() method of NetworkClient and ServerClient use
   shutdown(SHUT_RDWR)
 * NetworkServer uses a backlog of 5 clients for socket.listen() 
(instead of 1)

Bugfixes:

 * Fix Directory.rmtree() and replay script for Python 3.0
 * Fix ServerClient.sendBytes(): use socket.send() result to get the next
   data offset

Victor
http://fusil.hachoir.org/

[PATCH] Add -i option for displaying the instruction pointer

Here's a patch to add the -i option that the regular strace supports.

A minor problem is that it doesn't work on the exit_group() syscall.

diff --git a/strace.py b/strace.py
index 9de9213..eec013e 100755
--- a/strace.py
+++ b/strace.py
@@ -62,6 +62,8 @@ class SyscallTracer(Application):
             action="store_true", default=False)
         parser.add_option("--list-syscalls", help="Display system calls and exit",
             action="store_true", default=False)
+        parser.add_option("-i", help="print instruction pointer at time of syscall",
+            action="store_true", default=False, dest="show_ip")

         self.createLogOptions(parser)

@@ -136,8 +138,10 @@ class SyscallTracer(Application):
         text = syscall.format()
         if syscall.result is not None:
             text = "%-40s = %s" % (text, syscall.result_text)
+        if self.options.show_ip:
+            text = "[%x] %s" % (syscall.process.getInstrPointer(), text)
         if self.options.show_pid:
-            text = "[%s] %s" % (syscall.process.pid, text)
+            text = "[pid %s] %s" % (syscall.process.pid, text)
         error(text)

     def syscallTrace(self, process):

[announce] Release of Fusil version 1.1

User visible changes:
 * replay.py: ask confirmation if the fuzzer will not be running under a
   different user or as root
 * Even with --force-unsafe, show safety warning if the fuzzer is
   running as the root user
 * Close files for child processes (close_fds=True)

Developer changes:
 * Create IntegerRangeGenerator in fusil.unicode_generator
 * Create EnvVarIntegerRange in fusil.process.env
 * Create fusil-wizzard fuzzer
 * Write timestamp in session.log
 * Add session() method to ProjectAgent
 * Add NAME attribute to a fuzzer, reused to choose the project directory name

Bugfixes:
 * Fix Debugger.processSignal(): use the process agent to send the message
   (session_rename) since the debugger agent may be disabled
 * Fix replay.py: quote gdb arguments escape quote and antislash characters
   (eg. "text=\"Hello\\n\".")
 * replay.py uses /dev/null for stdin as Fusil does
 * FileWatch: open file in binary mode to use bytes in Python3

Victor

---

python-ptrace 0.5 and Fusil 1.0 final released

\o/ Fusil 1.0 \o/

Fusil 1.0 final
---------------

Visiable changes:

 * Create fusil-zzuf fuzzer (use the zzuf library)
 * Create fusil-vlc fuzzer (VLC media player)
 * For each session, generate a Python script (replay.py) to replay the
   session. The script can run the target in gdb, valgrind or gdb.py
   (python-ptrace debugger), with many options (--user, --limit, etc.)
 * Create --force-unsafe option, like --unsafe with without the confirmation
 * CreateProcess is now a probe (witch a score): if the debugger catchs a
   fatal signal, the session stops
 * Always use a null device as stdin for child processes to avoid blocking the
   fuzzer if the process reads stdin (eg. call getchar())
 * Write the created process identifier in the logs

Developer:

 * Create EnvVarIntegerRange: environment variable with an integer value
   in a fixed range
 * Changes to get a minimal Windows support: disable "change user/group"
   feature on Windows; remove log file before removing the project directory;
   use ":NUL" instead of /dev/null for null input/output
 * On setupProject() error, make sure that the project is cleaned
 * Close stdout files (input and output) at process exit (fix needed
   by Windows)
 * Rename long2raw() to uint2bytes(), and bytes2long() to bytes2uint()
 * Normalize score that make sure that a probe score is in range [-1; +1]
   and so that score*weight is in range[-weight; +weight]
 * CodeC: remove method lines(), writeCode() is renamed writeIntoFile(),
   use unicode strings (instead of byte strings)
 * Remove StdoutFile class, code merged in CreateProcess

python-ptrace 0.5 (2008-09-13)
------------------------------

Visible changes:

 * Write an example (the most simple debugger) and begin to document the code
 * gdb.py: create "dbginfo" command
 * Parse socket syscalls on FreeBSD
 * On invalid memory access (SIGSEGV), eval the dereference expression to get
   the fault address on OS without siginfo (eg. FreeBSD)
 * Fixes to get minimal Windows support: fix imports, fix locateProgram()

Other changes:

 * Break the API:
   - Rename PtraceDebugger.traceSysgood() to PtraceDebugger.enableSysgood()
   - Rename PtraceDebugger.trace_sysgood to PtraceDebugger.use_sysgood
   - Remove PtraceProcess.readCode()
 * Create createChild() function which close all files except stdin,
   stdout and stderr
 * On FreeBSD, on process exit recalls waitpid(pid) to avoid zombi process

--

-- 
Victor Stinner aka haypo
http://www.haypocalc.com/blog/

---

Release of python-ptrace 0.4.1 and Fusil 1.0beta1

Fusil 1.0beta1
--------------

This is the first beta of the final Fusil 1.0 release. Please try all fuzzers 
as much as possible: on different OS, with different CPU, different Python 
version, etc.  Changes:

 * Convert projects to programs so it's possible to execute directly a fuzzer
   and a fuzzer has its own command line options
 * Remove all generated files: use --keep-generated-files to keep them
 * Use ptrace debugger in CreateProcess to watch process signals
 * Fix "Too many files open" bug: CreateProcess waits until process death to
   avoid creation of process zombi
 * Create a shell script (replay.sh) to replay a session, and gdb.sh to
   replay it in gdb
 * Create a configuration file, fusil.conf, to choose some global options like
   using the CPU probe or a debugger
 * Replace usage of FileWatch.patterns by FileWatch.addRegex()
 * Create AttachProcessPID() to watch a running process identified by its
   identifier (instead of its name)
 * Remove many debug messages: from the Multi Agent System, from the network
   client and server (don't log data exchange by default), don't show
   file/process informations
 * NetworkClient: close socket on session stop
 * NetworkClient: support non-blocking receive (timeout=0)
 * Create MangleProcess to simplify the fuzzers using an MangleAgent
 * A fuzzer is now a Python executable program and a Python module:
   move all projects from projects/ to fuzzers/, and remove 
   run_fusil.sh script
 * AutoMangle: use increment operation for aggressivity >= 0.25
 * Create FileWatch.fromFilename() static method
 * Improve Python 3.0 support
 * Run Fusil as an different user and group to avoid arbitrary file remove
   or process kill
 * Rename a session using strings like "abort", "timeout", 
   "invalid_write", ...

python-ptrace 0.4.1
-------------------

This version just adds some minor features needed by Fusil 1.0beta1:

 * The project has a new dedicated website: http://python-ptrace.hachoir.org/
 * Create cptrace: optional Python binding of ptrace written in C (faster
   than ptrace, the Python binding written in Python with ctypes)
 * Add name attribute to SignalInfo classes
 * Fixes to help Python 3.0 compatibility: don't use sys.exc_clear()
   (was useless) in writeBacktrace()
 * ProcessState: create utime, stime, starttime attributes

--

-- 
Victor Stinner aka haypo
http://www.haypocalc.com/blog/

---