RAND_poll() and CreateToolhelp32Snapshot() stability
Tanguy Fautré <tfautre <at> telenet.be>
2009-03-02 16:56:39 GMT
We've been observing in our application several crashes on Windows related to RAND_poll(). We've been
working on this issue for 3 days now, and came up with a possible explanation and fix. Bare with me on this
rather lengthy email, as I'll try to document as best I can everything we've done and come up with.
The crash (an access violation, usually -but not always- a null pointer) always happens in Heap32Next().
Because RAND_poll uses Heap32Next to traverse through the heaps, our first assessment was that this
function is not thread safe and would leave RAND_poll traversing through garbage data, causing an access
violation. After some googling, several old posts on openssl mailing lists appeared to confirm our
initial fear that RAND_poll implementation is not thread-safe.
However, this does not seem to be a valid explanation. RAND_poll uses CreateToolhelp32Snapshot to create
a snapshot of the heap list; snapshot that is supposedly safe.
<Quote from MSDN>
Snapshots are at the core of the tool help functions. A snapshot is a read-only copy of the current state of
one or more of the following lists that reside in system memory: processes, threads, modules, and heaps.
Processes that use tool help functions access these lists from snapshots instead of directly from the
operating system. The lists in system memory change when processes are started and ended, threads are
created and destroyed, executable modules are loaded and unloaded from system memory, and heaps are
created and destroyed. The use of information from a snapshot prevents inconsistencies. Otherwise,
changes to a list could possibly cause a thread to incorrectly traverse the list or cause an access
violation (a GP fault). For example, if an application traverses the thread list while other threads are
created or terminated, information that the application is using to traverse the thread list might
become outdated and could cause an error for the application traversing the list.
</Quote from MSDN>