jmk | 3 May 18:33 2010
Picon

Re: NTLMv2 Challenge/Response Cracking

On Sun, 2010-04-25 at 06:16 +0400, Solar Designer wrote:
> As to your "--config" patch, I don't understand the rationale behind
> your introduction of FLG_CONFIG_DEFAULT, but I kept it in the patch.
> I think we may drop it later to free up that bit.

Unfortunately, neither do I. It's been a few years since I made that
initial patch. It certainly doesn't appear to be necessary. 

Joe

Solar Designer | 8 May 16:56 2010

OpenMP

Hi,

Some of you might recall my past comments on OpenMP:

http://www.openwall.com/lists/john-users/2006/05/23/1
http://openwall.info/wiki/internal/gcc-local-build#Application-to-John-the-Ripper

While this is not the most efficient way to parallelize JtR, it does
have its advantages - most notably, simplicity of code changes (when we
talk about a single hash type) and ease of use.  So I have sort of
temporarily "given up" (procrastinating a "proper" parallel processing
implementation for JtR) and implemented OpenMP support for one of the
hash types.

Attached is a patch against JtR 1.7.5 to crack OpenBSD bcrypt hashes
fast. ;-)  I've also uploaded the patch to the wiki:

http://openwall.info/wiki/john/patches

It uses OpenMP - tested with gcc 4.5.0 (on Linux) and Sun Studio 12 (on
Solaris).  The only new requirement is an OpenMP-capable C compiler,
such as recent gcc.  Overall, this is easier and more reliable to use
than "MPI-patched" builds of JtR are, but drawbacks do exist as well
(per-hash-type code and a performance hit - see below).

I've measured the efficiency (vs. multiple separate-process instances of
JtR) to be as high as 98.5% for a build with gcc on otherwise idle Linux
systems with Intel CPUs.  With Sun Studio, Solaris, and Opterons, it was
down to between 78% and 91% (I did not investigate why).  Multi-threaded
code involves more complicated addressing modes and synchronization
(Continue reading)

Erik Winkler | 9 May 13:18 2010
Picon

Re: OpenMP

Patch compiles and works flawlessly on MacOS 10.6 as OpenMPI is part of the OS.  CPU is a Intel "Core 2 Duo"
"Penryn" (P8600).

./john -test -format:bf
Benchmarking: OpenBSD Blowfish (x32) [32/64 X2]... DONE
Raw:	1211 c/s real, 627 c/s virtual

Running with more threads on this CPU degrades performance:

OMP_NUM_THREADS=4 ./john --test=1 --format=bf
Benchmarking: OpenBSD Blowfish (x32) [32/64 X2]... DONE
Raw:	1165 c/s real, 628 c/s virtual

This would be very useful for other hash types as well.  Nice work Alexander.

Erik

On May 8, 2010, at 10:56 AM, Solar Designer wrote:

> 
> Have fun.  As usual, any feedback is welcome.
> 
> Alexander
> <john-1.7.5-omp-1.diff>

Solar Designer | 9 May 14:21 2010

Re: OpenMP

On Sun, May 09, 2010 at 07:18:50AM -0400, Erik Winkler wrote:
> Patch compiles and works flawlessly on MacOS 10.6 as OpenMPI is part of the OS.  CPU is a Intel "Core 2 Duo"
"Penryn" (P8600).

The patch uses OpenMP, which is part of recent versions of gcc.  It does
not use Open MPI.

Can you check what libraries the resulting "john" binary is linked
against and what directories those are in?  On Linux, I am getting:

$ ldd john
        libgomp.so.1 => /home/solar/gcc-4.5.0/lib64/libgomp.so.1 (0x00002b64f231c000)
        libpthread.so.0 => /lib64/libpthread.so.0 (0x00002b64f242c000)
        libc.so.6 => /lib64/libc.so.6 (0x00002b64f2541000)
        librt.so.1 => /lib64/librt.so.1 (0x00002b64f2766000)
        /lib64/ld-linux-x86-64.so.2 (0x00002b64f2207000)

The dependency on libgomp.so.1 in my gcc install tree means that this
binary won't work on a system without a similar gcc install.  Of course,
I can simply add -static to LDFLAGS, which removes all of those runtime
dependencies on external libraries - I actually did just that to test
performance on other Linux machines (with different CPUs).

I wonder if your "john" binary on Mac OS X is dependent on having Xcode
installed or not.  And if it is, does adding -static work (produce a
working fully-static binary)?  This is going to be important for those
using binary builds of JtR for Mac OS X, many of whom do not have Xcode
installed (I guess).

It might be better to attempt a "half-static" build, though - link
(Continue reading)

Erik Winkler | 9 May 18:49 2010
Picon

Re: OpenMP

Alexander,

On May 9, 2010, at 8:21 AM, Solar Designer wrote:
> 
> Can you check what libraries the resulting "john" binary is linked
> against and what directories those are in?  On Linux, I am getting:
> 

For MacOSX, the command is Otool -L,

$ Otool -L john
john:
	/usr/local/lib/libgomp.1.dylib (compatibility version 2.0.0, current version 2.0.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 125.0.1)

> To see how efficient this build is, you need to compare the 1211 c/s
> figure vs. combined c/s rate for two separate "john" processes built
> without OpenMP (JtR 1.7.5 without any patch).  You'd need to run those
> two simultaneously, with a script like:
> 
> ./john -te -fo=bf &
> ./john -te -fo=bf &
> 
> 
Here is what I get when running the non-OpenMP john on both cores:

Benchmarking: OpenBSD Blowfish (x32) [32/64 X2]... 
Benchmarking: OpenBSD Blowfish (x32) [32/64 X2]... 

DONE
(Continue reading)

Solar Designer | 9 May 23:21 2010

Re: OpenMP

On Sun, May 09, 2010 at 12:49:01PM -0400, Erik Winkler wrote:
> For MacOSX, the command is Otool -L,
> 
> $ Otool -L john
> john:
> 	/usr/local/lib/libgomp.1.dylib (compatibility version 2.0.0, current version 2.0.0)
> 	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 125.0.1)

So I guess that /usr/local/lib/libgomp.1.dylib came from Xcode (and is
not available on systems without Xcode installed).

> Here is what I get when running the non-OpenMP john on both cores:
> 
> Benchmarking: OpenBSD Blowfish (x32) [32/64 X2]... 
> Benchmarking: OpenBSD Blowfish (x32) [32/64 X2]... 
> 
> DONE
> Raw:	599 c/s real, 615 c/s virtual
> 
> DONE
> Raw:	606 c/s real, 617 c/s virtual

This is very close to your 1211 c/s with OpenMP, so the efficiency is
close to 100% - but we can't easily tell just how close it is.  There
appears to be some load on the system (perhaps from the GUI), which
appears to be in excess of any efficiency loss from the OpenMP build.

> I did note the following compile error when compiling SSE2 version:
> 
> gcc -c -Wall -O2 -fomit-frame-pointer -fopenmp -m32 -funroll-loops BF_std.c
(Continue reading)

Erik Winkler | 10 May 04:37 2010
Picon

Re: OpenMP

Alexander,

> So I guess that /usr/local/lib/libgomp.1.dylib came from Xcode (and is
> not available on systems without Xcode installed).

Yes.  It is Xcode 3.2.2 for 10.6 that I am using, so even 10.5 might be unsupported with OpenMP.  Another issue
is the libgomp.dylib is x86_64 only.  I guess Apple didn't see this as an issue due to the fact that almost all
of Apple's Intel CPUs are 64-bit capable (except for some early core solos).

> Indeed.  OK, you got me to make a 32-bit build of gcc 4.5.0 as well and
> to complete that part of the patch.  The above should be fixed in
> 1.7.5-omp-2, which I've just uploaded to:
> 

Thanks, but it won't link due to the libgomp.dylib being x86_64 only.  I may build my own version of gcc 4.5.0
to test out as well.

Erik
jmk | 14 May 22:11 2010
Picon

NetLMv2 / NetNTLMv2 Bug Fix Patch

Hi,

I've uploaded a small patch against John 1.7.5 + Jumbo 3 for my
NetLMv2/NetNTLMv2 formats:

http://openwall.info/wiki/john/patches
http://www.foofus.net/jmk/tools/jtr/john-1.7.5-jumbo-3-netv2-fix.diff

The patch makes the following changes:

* Fixes a bug which surfaces when cracking NTLMv2 authentication
exchanges with very long client challenges. These seem to occur when
both ends of the exchange are Windows 7 hosts. 

* Fixes domain parsing for LMv2. I was previously upper-casing the
domain name and should not have been.

* Makes username format for NTLMv2/LMv2 more flexible. The file can now
contain either "DOMAIN\USER:::" or "USER::DOMAIN:".

Please let me know if there are any questions or issues.

Thanks,
Joe

Dan Tentler | 16 May 08:32 2010

MediaWiki password hashes

Out of curiosity, is anyone aware of a patch or a method in which JtR 
may be used to crack mediawiki passwords?
The hash style is defined here:

http://www.mediawiki.org/wiki/Manual_talk:User_table#.22B.22_type_password_.28current_default.29

The "b style" password hashes.

-Dan

Solar Designer | 16 May 18:03 2010

OpenMP benchmarks on UltraSPARC T2

Hi,

For the curious, here are benchmarks of 1.7.5-omp-2 on UltraSPARC T2
(quad-core, 8 threads per core).  The system:

$ uname -a
SunOS host 5.10 Generic_142900-10 sun4v sparc SUNW,SPARC-Enterprise-T5120

"/usr/sbin/psrinfo -v" reports 32 "virtual processors", all of which are
"online".  "/usr/platform/sun4v/sbin/prtdiag -v" also reports all 32,
but somehow only 28 of them are reported as "on-line".  I did not look
into this discrepancy.  Both report the clock rate as 1165 MHz.

The compiler:

$ cc -V
cc: Sun C 5.9 SunOS_sparc Patch 124867-14 2010/03/30

The default BF_mt of 24 (in BF_std.h) obviously would not use more than
24 threads, so I edited it to be 32.  Then I built with:

gmake solaris-sparc64-cc -j32

One thread (but OpenMP-capable build):

$ ../run/john -te -fo=bf
Benchmarking: OpenBSD Blowfish (x32) [32/64]... DONE
Raw:    96.7 c/s real, 96.6 c/s virtual

4, 8, 16, and 32 threads:
(Continue reading)


Gmane