Jeffrey Walton | 2 Oct 2011 20:34
Picon

Re: AES-CTR Counter Incrementing


On Sep 27, 5:02 am, Niknak <nick.pate... <at> certivox.com> wrote:
> Hi there,
>
> I'm currently having some issues with the AES-CTR implementation, no
> doubt I'm missing some crucial step, but it seems that the counter in
> my IV is not being incremented with every block.
>
> I'm calling ProcessData to perform encryption on the blocks of data as
> I'm currently integrating it into an existing system.
>
> I know something isn't quite right as when I attempt to decrypt using
> WinZip/7Zip etc. it only manages to do the first block of any file,
> everything after that fails with a password mismatch.  I assume this
> is IV related.
If you are using ProcessData directly, I believe you need to manually
increment the IV for each block processed. See
http://www.mail-archive.com/cryptopp-users <at> googlegroups.com/msg05066.html
for inspiration.

Jeff

--

-- 
You received this message because you are subscribed to the "Crypto++ Users" Google Group.
To unsubscribe, send an email to cryptopp-users-unsubscribe <at> googlegroups.com.
More information about Crypto++ and this group is available at http://www.cryptopp.com.

Jeffrey Walton | 2 Oct 2011 20:39
Picon

Re: gcc 4.6 AutoSeededX917RNG segfault


On Sep 27, 4:41 pm, David Irvine <david.irv... <at> maidsafe.net> wrote:
> Hi
>
> I am testing gcc 4.6 and found what appears to be a bug. If I use
> CryptoPP::AutoSeededX917RNG< CryptoPP::AES >
> g_srandom_number_generator;
> (any cypher fails)
>
> Then it's a segfault (see below)
>
> However switching to (which I do not want to do, or more likely
> cannot)
> CryptoPP::AutoSeededRandomPool g_srandom_number_generator;
>
> Is fine, if not as secure.
> ########################################################################
> System is Linux
> Linux di-ubuntu-64 3.0.0-11-generic #18-Ubuntu SMP Tue Sep 13 23:38:01
> UTC 2011 x86_64 x86_64 x86_64 GNU/Linux
> Ubuntu oneiric (development branch)
>
> I am not wishing to make too much of this as this is a beta release
> OS, but it seems more of a cryptopp issue I think. Is anyone else
> using ?
> gcc (Ubuntu/Linaro 4.6.1-9ubuntu3) 4.6.1
> Copyright (C) 2011 Free Software Foundation, Inc.
> This is free software; see the source for copying conditions.  There
> is NO
> warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR
(Continue reading)

David Irvine | 2 Oct 2011 21:30
Favicon
Gravatar

Re: gcc 4.6 AutoSeededX917RNG segfault

On Sun, Oct 2, 2011 at 7:39 PM, Jeffrey Walton <noloader <at> gmail.com> wrote:


On Sep 27, 4:41 pm, David Irvine <david.irv... <at> maidsafe.net> wrote:
> Hi
>
> I am testing gcc 4.6 and found what appears to be a bug. If I use
> CryptoPP::AutoSeededX917RNG< CryptoPP::AES >
> g_srandom_number_generator;
> (any cypher fails)
>
> Then it's a segfault (see below)
>
> However switching to (which I do not want to do, or more likely
> cannot)
> CryptoPP::AutoSeededRandomPool g_srandom_number_generator;
>
> Is fine, if not as secure.
> ########################################################################
> System is Linux
> Linux di-ubuntu-64 3.0.0-11-generic #18-Ubuntu SMP Tue Sep 13 23:38:01
> UTC 2011 x86_64 x86_64 x86_64 GNU/Linux
> Ubuntu oneiric (development branch)
>
> I am not wishing to make too much of this as this is a beta release
> OS, but it seems more of a cryptopp issue I think. Is anyone else
> using ?
> gcc (Ubuntu/Linaro 4.6.1-9ubuntu3) 4.6.1
> Copyright (C) 2011 Free Software Foundation, Inc.
> This is free software; see the source for copying conditions.  There
> is NO
> warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR
> PURPOSE
>
> Segfault below
>
> #0  0x000000000071c3ad in
> CryptoPP::BufferedTransformation::ChannelPut2(std::string const&,
> unsigned char const*, unsigned long, int, bool)
>     ()
> #1  0x00000000007b543e in
> CryptoPP::X917RNG::GenerateIntoBufferedTransformation(CryptoPP::BufferedTransformation&,
> std::string const&, unsigned long long) ()
> #2  0x000000000071c88e in
> CryptoPP::RandomNumberGenerator::GenerateBlock(unsigned char*,
> unsigned long) ()
> #3  0x00000000007b576b in
> CryptoPP::X917RNG::X917RNG(CryptoPP::BlockTransformation*, unsigned
> char const*, unsigned char const*) ()
> #4  0x0000000000743cc5 in
> CryptoPP::AutoSeededX917RNG<CryptoPP::Rijndael>::Reseed(unsigned char
> const*, unsigned long, unsigned char const*, unsigned char const*) ()
> #5  0x0000000000743ed8 in
> CryptoPP::AutoSeededX917RNG<CryptoPP::Rijndael>::Reseed(bool, unsigned
> char const*, unsigned long) ()
> #6  0x0000000000744000 in
> CryptoPP::AutoSeededX917RNG<CryptoPP::Rijndael>::AutoSeededX917RNG(bool,
> bool) ()
> #7  0x00000000006eddc0 in __static_initialization_and_destruction_0
> (__initialize_p=1, __priority=65535)
>     at /home/dirvine/Devel/MaidSafe-Common/maidsafe_common_lib/src/
> maidsafe/common/crypto.cc:61
> #8  0x00000000006ede24 in _GLOBAL__sub_I_crypto.cc(void) ()
>     at /home/dirvine/Devel/MaidSafe-Common/maidsafe_common_lib/src/
> maidsafe/common/crypto.cc:299AES/CTR IV custom increment
> #9  0x00000000007e9ded in __libc_csu_init ()
> #10 0x00007ffff70a02a0 in __libc_start_main (main=0x5f3114 <main(int,
> char**)>, argc=1, ubp_av=0x7fffffffe118,
>     init=0x7e9d90 <__libc_csu_init>, fini=<optimized out>,
> rtld_fini=<optimized out>, stack_end=0x7fffffffe108) at libc-start.c:
> 185
> #11 0x00000000005f305d in _start ()
> (gdb) quit
I would guess that g_srandom_number_generator is in one compilation
unit, and the code using it is in another. Ie,
g_srandom_number_generator is in my_rng.cpp, and the code using it is
in my_key_gen.cpp. In this case, you would be violating C++'s static
non-local rules.

The fix is relatively easy. Offer a static/global function to serve up
a static local:

RandomNumberGenerator& GetRandomNumberGenerator()
{
 static AutoSeededX917RNG< CryptoPP::AES > s_prng;
 return s_prng;
}

Jeff
Thanks Jeff. It's possibly the case,, although this was initialised in the same file it was used, however what I noticed was that merely initialising the rng and never calling it anywhere would cause this issue. 

To be a bit more exact this was in a my_rng.cc which was compiled into a lib that was then linked to another file.cc (or object, but you know what I mean). The file.oo did have some cryptopp code in there (AES, HASH etc. but no explicit RNG calls). This is why it took a wee while to figure it out. It looks like there may have been a static somewhere I had not seen (inside cryptopp) or something very similar. 

I have tried your suggestion and looks a good, although it does still segfault in some circumstances (I am tracking that down just now). I think it may be a threading issue though, but will post again if it turns out to be more (which I doubt). 

Thanks again.

All suggestions/critique welcomed though. 

--
You received this message because you are subscribed to the "Crypto++ Users" Google Group.
To unsubscribe, send an email to cryptopp-users-unsubscribe <at> googlegroups.com.
More information about Crypto++ and this group is available at http://www.cryptopp.com.
Jeffrey Walton | 2 Oct 2011 21:42
Picon

Re: gcc 4.6 AutoSeededX917RNG segfault


On Oct 2, 3:30 pm, David Irvine <david.irv... <at> maidsafe.net> wrote:
> On Sun, Oct 2, 2011 at 7:39 PM, Jeffrey Walton <noloa... <at> gmail.com> wrote:
>
> > On Sep 27, 4:41 pm, David Irvine <david.irv... <at> maidsafe.net> wrote:
> > > Hi
>
> > > I am testing gcc 4.6 and found what appears to be a bug. If I use
> > > CryptoPP::AutoSeededX917RNG< CryptoPP::AES >
> > > g_srandom_number_generator;
> > > (any cypher fails)
>
> > > Then it's a segfault (see below)
>
> > > However switching to (which I do not want to do, or more likely
> > > cannot)
> > > CryptoPP::AutoSeededRandomPool g_srandom_number_generator;
>
> > > Is fine, if not as secure.
> > > ########################################################################
> > > System is Linux
> > > Linux di-ubuntu-64 3.0.0-11-generic #18-Ubuntu SMP Tue Sep 13 23:38:01
> > > UTC 2011 x86_64 x86_64 x86_64 GNU/Linux
> > > Ubuntu oneiric (development branch)
>
> > > I am not wishing to make too much of this as this is a beta release
> > > OS, but it seems more of a cryptopp issue I think. Is anyone else
> > > using ?
> > > gcc (Ubuntu/Linaro 4.6.1-9ubuntu3) 4.6.1
> > > Copyright (C) 2011 Free Software Foundation, Inc.
> > > This is free software; see the source for copying conditions.  There
> > > is NO
> > > warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR
> > > PURPOSE
>
> > > Segfault below
>
> > > #0  0x000000000071c3ad in
> > > CryptoPP::BufferedTransformation::ChannelPut2(std::string const&,
> > > unsigned char const*, unsigned long, int, bool)
> > >     ()
> > > #1  0x00000000007b543e in
>
> > CryptoPP::X917RNG::GenerateIntoBufferedTransformation(CryptoPP::BufferedTransformation&,
> > > std::string const&, unsigned long long) ()
> > > #2  0x000000000071c88e in
> > > CryptoPP::RandomNumberGenerator::GenerateBlock(unsigned char*,
> > > unsigned long) ()
> > > #3  0x00000000007b576b in
> > > CryptoPP::X917RNG::X917RNG(CryptoPP::BlockTransformation*, unsigned
> > > char const*, unsigned char const*) ()
> > > #4  0x0000000000743cc5 in
> > > CryptoPP::AutoSeededX917RNG<CryptoPP::Rijndael>::Reseed(unsigned char
> > > const*, unsigned long, unsigned char const*, unsigned char const*) ()
> > > #5  0x0000000000743ed8 in
> > > CryptoPP::AutoSeededX917RNG<CryptoPP::Rijndael>::Reseed(bool, unsigned
> > > char const*, unsigned long) ()
> > > #6  0x0000000000744000 in
> > > CryptoPP::AutoSeededX917RNG<CryptoPP::Rijndael>::AutoSeededX917RNG(bool,
> > > bool) ()
> > > #7  0x00000000006eddc0 in __static_initialization_and_destruction_0
> > > (__initialize_p=1, __priority=65535)
> > >     at /home/dirvine/Devel/MaidSafe-Common/maidsafe_common_lib/src/
> > > maidsafe/common/crypto.cc:61
> > > #8  0x00000000006ede24 in _GLOBAL__sub_I_crypto.cc(void) ()
> > >     at /home/dirvine/Devel/MaidSafe-Common/maidsafe_common_lib/src/
> > > maidsafe/common/crypto.cc:299AES/CTR IV custom increment
> > > #9  0x00000000007e9ded in __libc_csu_init ()
> > > #10 0x00007ffff70a02a0 in __libc_start_main (main=0x5f3114 <main(int,
> > > char**)>, argc=1, ubp_av=0x7fffffffe118,
> > >     init=0x7e9d90 <__libc_csu_init>, fini=<optimized out>,
> > > rtld_fini=<optimized out>, stack_end=0x7fffffffe108) at libc-start.c:
> > > 185
> > > #11 0x00000000005f305d in _start ()
> > > (gdb) quit
> > I would guess that g_srandom_number_generator is in one compilation
> > unit, and the code using it is in another. Ie,
> > g_srandom_number_generator is in my_rng.cpp, and the code using it is
> > in my_key_gen.cpp. In this case, you would be violating C++'s static
> > non-local rules.
>
> > The fix is relatively easy. Offer a static/global function to serve up
> > a static local:
>
> > RandomNumberGenerator& GetRandomNumberGenerator()
> > {
> >  static AutoSeededX917RNG< CryptoPP::AES > s_prng;
> >  return s_prng;
> > }
>
> > Jeff
>
> Thanks Jeff. It's possibly the case,, although this was initialised in the
> same file it was used, however what I noticed was that
> merely initialising the rng and never calling it anywhere would cause this
> issue.
>
> To be a bit more exact this was in a my_rng.cc which was compiled into a lib
> that was then linked to another file.cc (or object, but you know what I
> mean). The file.oo did have some cryptopp code in there (AES, HASH etc. but
> no explicit RNG calls). This is why it took a wee while to figure it out. It
> looks like there may have been a static somewhere I had not seen (inside
> cryptopp) or something very similar.
I believe Crypto++ is fairly safe in its use of globals. They are
used, and they are easy to audit: searh for "g_*" in the source files.

> I have tried your suggestion and looks a good, although it does still
> segfault in some circumstances (I am tracking that down just now). I think
> it may be a threading issue though, but will post again if it turns out to
> be more (which I doubt).
Crypto++ objects are thread safe, meaning they do not depend on global
data which might become corrupted across calls in different threads.
You still have to lock on concurrent access to an object.

For what it worth, Wei uses a different technique for initializing a
singleton (his method results in a memory leak under 'worst case', if
I recall correctly).

If you suspect you might have threading issues, Helgrind would
probably be very useful. But you need a good test program to get the
most out of it.
    valgrind --tool=helgrind my_test_program

Jeff

--

-- 
You received this message because you are subscribed to the "Crypto++ Users" Google Group.
To unsubscribe, send an email to cryptopp-users-unsubscribe <at> googlegroups.com.
More information about Crypto++ and this group is available at http://www.cryptopp.com.

David Irvine | 2 Oct 2011 21:47
Favicon
Gravatar

Re: gcc 4.6 AutoSeededX917RNG segfault

On Sun, Oct 2, 2011 at 8:42 PM, Jeffrey Walton <noloader <at> gmail.com> wrote:


On Oct 2, 3:30 pm, David Irvine <david.irv... <at> maidsafe.net> wrote:
> On Sun, Oct 2, 2011 at 7:39 PM, Jeffrey Walton <noloa... <at> gmail.com> wrote:
>
> > On Sep 27, 4:41 pm, David Irvine <david.irv... <at> maidsafe.net> wrote:
> > > Hi
>
> > > I am testing gcc 4.6 and found what appears to be a bug. If I use
> > > CryptoPP::AutoSeededX917RNG< CryptoPP::AES >
> > > g_srandom_number_generator;
> > > (any cypher fails)
>
> > > Then it's a segfault (see below)
>
> > > However switching to (which I do not want to do, or more likely
> > > cannot)
> > > CryptoPP::AutoSeededRandomPool g_srandom_number_generator;
>
> > > Is fine, if not as secure.
> > > ########################################################################
> > > System is Linux
> > > Linux di-ubuntu-64 3.0.0-11-generic #18-Ubuntu SMP Tue Sep 13 23:38:01
> > > UTC 2011 x86_64 x86_64 x86_64 GNU/Linux
> > > Ubuntu oneiric (development branch)
>
> > > I am not wishing to make too much of this as this is a beta release
> > > OS, but it seems more of a cryptopp issue I think. Is anyone else
> > > using ?
> > > gcc (Ubuntu/Linaro 4.6.1-9ubuntu3) 4.6.1
> > > Copyright (C) 2011 Free Software Foundation, Inc.
> > > This is free software; see the source for copying conditions.  There
> > > is NO
> > > warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR
> > > PURPOSE
>
> > > Segfault below
>
> > > #0  0x000000000071c3ad in
> > > CryptoPP::BufferedTransformation::ChannelPut2(std::string const&,
> > > unsigned char const*, unsigned long, int, bool)
> > >     ()
> > > #1  0x00000000007b543e in
>
> > CryptoPP::X917RNG::GenerateIntoBufferedTransformation(CryptoPP::BufferedTransformation&,
> > > std::string const&, unsigned long long) ()
> > > #2  0x000000000071c88e in
> > > CryptoPP::RandomNumberGenerator::GenerateBlock(unsigned char*,
> > > unsigned long) ()
> > > #3  0x00000000007b576b in
> > > CryptoPP::X917RNG::X917RNG(CryptoPP::BlockTransformation*, unsigned
> > > char const*, unsigned char const*) ()
> > > #4  0x0000000000743cc5 in
> > > CryptoPP::AutoSeededX917RNG<CryptoPP::Rijndael>::Reseed(unsigned char
> > > const*, unsigned long, unsigned char const*, unsigned char const*) ()
> > > #5  0x0000000000743ed8 in
> > > CryptoPP::AutoSeededX917RNG<CryptoPP::Rijndael>::Reseed(bool, unsigned
> > > char const*, unsigned long) ()
> > > #6  0x0000000000744000 in
> > > CryptoPP::AutoSeededX917RNG<CryptoPP::Rijndael>::AutoSeededX917RNG(bool,
> > > bool) ()
> > > #7  0x00000000006eddc0 in __static_initialization_and_destruction_0
> > > (__initialize_p=1, __priority=65535)
> > >     at /home/dirvine/Devel/MaidSafe-Common/maidsafe_common_lib/src/
> > > maidsafe/common/crypto.cc:61
> > > #8  0x00000000006ede24 in _GLOBAL__sub_I_crypto.cc(void) ()
> > >     at /home/dirvine/Devel/MaidSafe-Common/maidsafe_common_lib/src/
> > > maidsafe/common/crypto.cc:299AES/CTR IV custom increment
> > > #9  0x00000000007e9ded in __libc_csu_init ()
> > > #10 0x00007ffff70a02a0 in __libc_start_main (main=0x5f3114 <main(int,
> > > char**)>, argc=1, ubp_av=0x7fffffffe118,
> > >     init=0x7e9d90 <__libc_csu_init>, fini=<optimized out>,
> > > rtld_fini=<optimized out>, stack_end=0x7fffffffe108) at libc-start.c:
> > > 185
> > > #11 0x00000000005f305d in _start ()
> > > (gdb) quit
> > I would guess that g_srandom_number_generator is in one compilation
> > unit, and the code using it is in another. Ie,
> > g_srandom_number_generator is in my_rng.cpp, and the code using it is
> > in my_key_gen.cpp. In this case, you would be violating C++'s static
> > non-local rules.
>
> > The fix is relatively easy. Offer a static/global function to serve up
> > a static local:
>
> > RandomNumberGenerator& GetRandomNumberGenerator()
> > {
> >  static AutoSeededX917RNG< CryptoPP::AES > s_prng;
> >  return s_prng;
> > }
>
> > Jeff
>
> Thanks Jeff. It's possibly the case,, although this was initialised in the
> same file it was used, however what I noticed was that
> merely initialising the rng and never calling it anywhere would cause this
> issue.
>
> To be a bit more exact this was in a my_rng.cc which was compiled into a lib
> that was then linked to another file.cc (or object, but you know what I
> mean). The file.oo did have some cryptopp code in there (AES, HASH etc. but
> no explicit RNG calls). This is why it took a wee while to figure it out. It
> looks like there may have been a static somewhere I had not seen (inside
> cryptopp) or something very similar.
I believe Crypto++ is fairly safe in its use of globals. They are
used, and they are easy to audit: searh for "g_*" in the source files.

> I have tried your suggestion and looks a good, although it does still
> segfault in some circumstances (I am tracking that down just now). I think
> it may be a threading issue though, but will post again if it turns out to
> be more (which I doubt).
Crypto++ objects are thread safe, meaning they do not depend on global
data which might become corrupted across calls in different threads.
You still have to lock on concurrent access to an object.

For what it worth, Wei uses a different technique for initializing a
singleton (his method results in a memory leak under 'worst case', if
I recall correctly).

If you suspect you might have threading issues, Helgrind would
probably be very useful. But you need a good test program to get the
most out of it.
   valgrind --tool=helgrind my_test_program

Jeff

Doing that right now Jeff, thanks 

For anyone who is interested the (offending) code is located here   https://github.com/maidsafe/MaidSafe-Common

David

--
You received this message because you are subscribed to the "Crypto++ Users" Google Group.
To unsubscribe, send an email to cryptopp-users-unsubscribe <at> googlegroups.com.
More information about Crypto++ and this group is available at http://www.cryptopp.com.
Jeffrey Walton | 2 Oct 2011 22:22
Picon

Re: gcc 4.6 AutoSeededX917RNG segfault


On Oct 2, 3:47 pm, David Irvine <david.irv... <at> maidsafe.net> wrote:
> On Sun, Oct 2, 2011 at 8:42 PM, Jeffrey Walton <noloa... <at> gmail.com> wrote:
>
> > On Oct 2, 3:30 pm, David Irvine <david.irv... <at> maidsafe.net> wrote:
> > > On Sun, Oct 2, 2011 at 7:39 PM, Jeffrey Walton <noloa... <at> gmail.com>
> > wrote:
>
> > > > On Sep 27, 4:41 pm, David Irvine <david.irv... <at> maidsafe.net> wrote:
> > > > > Hi
>
> > > > > I am testing gcc 4.6 and found what appears to be a bug. If I use
> > > > > CryptoPP::AutoSeededX917RNG< CryptoPP::AES >
> > > > > g_srandom_number_generator;
> > > > > (any cypher fails)
>
> > > > > Then it's a segfault (see below)
>
> > > > > However switching to (which I do not want to do, or more likely
> > > > > cannot)
> > > > > CryptoPP::AutoSeededRandomPool g_srandom_number_generator;
>
> > > > > Is fine, if not as secure.
>
> > ########################################################################
> > > > > System is Linux
> > > > > Linux di-ubuntu-64 3.0.0-11-generic #18-Ubuntu SMP Tue Sep 13
> > 23:38:01
> > > > > UTC 2011 x86_64 x86_64 x86_64 GNU/Linux
> > > > > Ubuntu oneiric (development branch)
>
> > > > > I am not wishing to make too much of this as this is a beta release
> > > > > OS, but it seems more of a cryptopp issue I think. Is anyone else
> > > > > using ?
> > > > > gcc (Ubuntu/Linaro 4.6.1-9ubuntu3) 4.6.1
> > > > > Copyright (C) 2011 Free Software Foundation, Inc.
> > > > > This is free software; see the source for copying conditions.  There
> > > > > is NO
> > > > > warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR
> > > > > PURPOSE
>
> > > > > Segfault below
>
> > > > > #0  0x000000000071c3ad in
> > > > > CryptoPP::BufferedTransformation::ChannelPut2(std::string const&,
> > > > > unsigned char const*, unsigned long, int, bool)
> > > > >     ()
> > > > > #1  0x00000000007b543e in
>
> > CryptoPP::X917RNG::GenerateIntoBufferedTransformation(CryptoPP::BufferedTransformation&,
> > > > > std::string const&, unsigned long long) ()
> > > > > #2  0x000000000071c88e in
> > > > > CryptoPP::RandomNumberGenerator::GenerateBlock(unsigned char*,
> > > > > unsigned long) ()
> > > > > #3  0x00000000007b576b in
> > > > > CryptoPP::X917RNG::X917RNG(CryptoPP::BlockTransformation*, unsigned
> > > > > char const*, unsigned char const*) ()
> > > > > #4  0x0000000000743cc5 in
> > > > > CryptoPP::AutoSeededX917RNG<CryptoPP::Rijndael>::Reseed(unsigned char
> > > > > const*, unsigned long, unsigned char const*, unsigned char const*) ()
> > > > > #5  0x0000000000743ed8 in
> > > > > CryptoPP::AutoSeededX917RNG<CryptoPP::Rijndael>::Reseed(bool,
> > unsigned
> > > > > char const*, unsigned long) ()
> > > > > #6  0x0000000000744000 in
>
> > CryptoPP::AutoSeededX917RNG<CryptoPP::Rijndael>::AutoSeededX917RNG(bool,
> > > > > bool) ()
> > > > > #7  0x00000000006eddc0 in __static_initialization_and_destruction_0
> > > > > (__initialize_p=1, __priority=65535)
> > > > >     at /home/dirvine/Devel/MaidSafe-Common/maidsafe_common_lib/src/
> > > > > maidsafe/common/crypto.cc:61
> > > > > #8  0x00000000006ede24 in _GLOBAL__sub_I_crypto.cc(void) ()
> > > > >     at /home/dirvine/Devel/MaidSafe-Common/maidsafe_common_lib/src/
> > > > > maidsafe/common/crypto.cc:299AES/CTR IV custom increment
> > > > > #9  0x00000000007e9ded in __libc_csu_init ()
> > > > > #10 0x00007ffff70a02a0 in __libc_start_main (main=0x5f3114 <main(int,
> > > > > char**)>, argc=1, ubp_av=0x7fffffffe118,
> > > > >     init=0x7e9d90 <__libc_csu_init>, fini=<optimized out>,
> > > > > rtld_fini=<optimized out>, stack_end=0x7fffffffe108) at libc-start.c:
> > > > > 185
> > > > > #11 0x00000000005f305d in _start ()
> > > > > (gdb) quit
> > > > I would guess that g_srandom_number_generator is in one compilation
> > > > unit, and the code using it is in another. Ie,
> > > > g_srandom_number_generator is in my_rng.cpp, and the code using it is
> > > > in my_key_gen.cpp. In this case, you would be violating C++'s static
> > > > non-local rules.
>
> > > > The fix is relatively easy. Offer a static/global function to serve up
> > > > a static local:
>
> > > > RandomNumberGenerator& GetRandomNumberGenerator()
> > > > {
> > > >  static AutoSeededX917RNG< CryptoPP::AES > s_prng;
> > > >  return s_prng;
> > > > }
>
> > > > Jeff
>
> > > Thanks Jeff. It's possibly the case,, although this was initialised in
> > the
> > > same file it was used, however what I noticed was that
> > > merely initialising the rng and never calling it anywhere would cause
> > this
> > > issue.
>
> > > To be a bit more exact this was in a my_rng.cc which was compiled into a
> > lib
> > > that was then linked to another file.cc (or object, but you know what I
> > > mean). The file.oo did have some cryptopp code in there (AES, HASH etc.
> > but
> > > no explicit RNG calls). This is why it took a wee while to figure it out.
> > It
> > > looks like there may have been a static somewhere I had not seen (inside
> > > cryptopp) or something very similar.
> > I believe Crypto++ is fairly safe in its use of globals. They are
> > used, and they are easy to audit: searh for "g_*" in the source files.
>
> > > I have tried your suggestion and looks a good, although it does still
> > > segfault in some circumstances (I am tracking that down just now). I
> > think
> > > it may be a threading issue though, but will post again if it turns out
> > to
> > > be more (which I doubt).
> > Crypto++ objects are thread safe, meaning they do not depend on global
> > data which might become corrupted across calls in different threads.
> > You still have to lock on concurrent access to an object.
>
> > For what it worth, Wei uses a different technique for initializing a
> > singleton (his method results in a memory leak under 'worst case', if
> > I recall correctly).
>
> > If you suspect you might have threading issues, Helgrind would
> > probably be very useful. But you need a good test program to get the
> > most out of it.
> >    valgrind --tool=helgrind my_test_program
>
> > Jeff
>
> > Doing that right now Jeff, thanks
>
> For anyone who is interested the (offending) code is located here https://github.com/maidsafe/MaidSafe-Common
At the risk of a flame war from the Boost fan boi club..... Don't use
Boost::Threads or Boost::Threading. From a recent audit, I believe the
subproject is not production quality. For example:
* Ignoring return values from locking functions
      (ie, pthread_mutex_lock and WaitForSingleObject)
* Race conditions
* Resource leaks on negative code paths

Most frustrating is the 20 or so bugs filed against Boost are still
sitting in their bug tracker unacknowledged (I believe two were
acknowledged and closed to date).

Jeff

--

-- 
You received this message because you are subscribed to the "Crypto++ Users" Google Group.
To unsubscribe, send an email to cryptopp-users-unsubscribe <at> googlegroups.com.
More information about Crypto++ and this group is available at http://www.cryptopp.com.

David Irvine | 2 Oct 2011 22:32
Favicon
Gravatar

Re: gcc 4.6 AutoSeededX917RNG segfault

Argh crossed emails there. 


Yes we noticed this in the office, the amount of help you give in cryptopp and then the total lack of feedback from boost on some pretty decent bug reports. We have found some issues as well but tamed a lot of them using condition variables to get over some race conditions and the likes. Often some weird errors though. I would hope some of the boost guys take a look at your detailed explanations here and maybe up the game a wee bit in replying. 
 
I am not sure if you have ever tried OpenMP, but for the self encrypting file stuff I did use this and really enjoyed the threading experience (yes you are reading correctly). Whilst admittedly it may not be the most efficient, when you can remove dependencies from loops then it's almost a simple one line pre processor statement and immediate responses, you know sometimes what should go faster slows down, but other times the difference is amazing. 

What is cool is the speed at which you can try a thread model with it. 

It's not multi purpose though and not as efficient as had crafting a thread model, butt it's very efficient in it's own right and does back off a thread to let the computer still appear responsive whilst the rest of the cores are hammering away. Really recommend a play around with it.  
 
Best Regards
David Irvine
---------------------------------------------------------------------
Be independent of the good opinions of other people and have a healthy disregard for the impossible!
-----------------------------------------------------------------
Einstein was right when he said "Great spirits have always encountered violent opposition from mediocre minds."
------------------------------------------------------------
maidsafe.net Ltd is a limited liability company incorporated in Scotland with number SC297540. Registered Office; 72 Templehill, Troon; KA10 6BE. VAT Registered 889 0608 77.
Telephone Scotland:  +44 1292 750020




On Sun, Oct 2, 2011 at 9:22 PM, Jeffrey Walton <noloader <at> gmail.com> wrote:


On Oct 2, 3:47 pm, David Irvine <david.irv... <at> maidsafe.net> wrote:
> On Sun, Oct 2, 2011 at 8:42 PM, Jeffrey Walton <noloa... <at> gmail.com> wrote:
>
> > On Oct 2, 3:30 pm, David Irvine <david.irv... <at> maidsafe.net> wrote:
> > > On Sun, Oct 2, 2011 at 7:39 PM, Jeffrey Walton <noloa... <at> gmail.com>
> > wrote:
>
> > > > On Sep 27, 4:41 pm, David Irvine <david.irv... <at> maidsafe.net> wrote:
> > > > > Hi
>
> > > > > I am testing gcc 4.6 and found what appears to be a bug. If I use
> > > > > CryptoPP::AutoSeededX917RNG< CryptoPP::AES >
> > > > > g_srandom_number_generator;
> > > > > (any cypher fails)
>
> > > > > Then it's a segfault (see below)
>
> > > > > However switching to (which I do not want to do, or more likely
> > > > > cannot)
> > > > > CryptoPP::AutoSeededRandomPool g_srandom_number_generator;
>
> > > > > Is fine, if not as secure.
>
> > ########################################################################
> > > > > System is Linux
> > > > > Linux di-ubuntu-64 3.0.0-11-generic #18-Ubuntu SMP Tue Sep 13
> > 23:38:01
> > > > > UTC 2011 x86_64 x86_64 x86_64 GNU/Linux
> > > > > Ubuntu oneiric (development branch)
>
> > > > > I am not wishing to make too much of this as this is a beta release
> > > > > OS, but it seems more of a cryptopp issue I think. Is anyone else
> > > > > using ?
> > > > > gcc (Ubuntu/Linaro 4.6.1-9ubuntu3) 4.6.1
> > > > > Copyright (C) 2011 Free Software Foundation, Inc.
> > > > > This is free software; see the source for copying conditions.  There
> > > > > is NO
> > > > > warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR
> > > > > PURPOSE
>
> > > > > Segfault below
>
> > > > > #0  0x000000000071c3ad in
> > > > > CryptoPP::BufferedTransformation::ChannelPut2(std::string const&,
> > > > > unsigned char const*, unsigned long, int, bool)
> > > > >     ()
> > > > > #1  0x00000000007b543e in
>
> > CryptoPP::X917RNG::GenerateIntoBufferedTransformation(CryptoPP::BufferedTransformation&,
> > > > > std::string const&, unsigned long long) ()
> > > > > #2  0x000000000071c88e in
> > > > > CryptoPP::RandomNumberGenerator::GenerateBlock(unsigned char*,
> > > > > unsigned long) ()
> > > > > #3  0x00000000007b576b in
> > > > > CryptoPP::X917RNG::X917RNG(CryptoPP::BlockTransformation*, unsigned
> > > > > char const*, unsigned char const*) ()
> > > > > #4  0x0000000000743cc5 in
> > > > > CryptoPP::AutoSeededX917RNG<CryptoPP::Rijndael>::Reseed(unsigned char
> > > > > const*, unsigned long, unsigned char const*, unsigned char const*) ()
> > > > > #5  0x0000000000743ed8 in
> > > > > CryptoPP::AutoSeededX917RNG<CryptoPP::Rijndael>::Reseed(bool,
> > unsigned
> > > > > char const*, unsigned long) ()
> > > > > #6  0x0000000000744000 in
>
> > CryptoPP::AutoSeededX917RNG<CryptoPP::Rijndael>::AutoSeededX917RNG(bool,
> > > > > bool) ()
> > > > > #7  0x00000000006eddc0 in __static_initialization_and_destruction_0
> > > > > (__initialize_p=1, __priority=65535)
> > > > >     at /home/dirvine/Devel/MaidSafe-Common/maidsafe_common_lib/src/
> > > > > maidsafe/common/crypto.cc:61
> > > > > #8  0x00000000006ede24 in _GLOBAL__sub_I_crypto.cc(void) ()
> > > > >     at /home/dirvine/Devel/MaidSafe-Common/maidsafe_common_lib/src/
> > > > > maidsafe/common/crypto.cc:299AES/CTR IV custom increment
> > > > > #9  0x00000000007e9ded in __libc_csu_init ()
> > > > > #10 0x00007ffff70a02a0 in __libc_start_main (main=0x5f3114 <main(int,
> > > > > char**)>, argc=1, ubp_av=0x7fffffffe118,
> > > > >     init=0x7e9d90 <__libc_csu_init>, fini=<optimized out>,
> > > > > rtld_fini=<optimized out>, stack_end=0x7fffffffe108) at libc-start.c:
> > > > > 185
> > > > > #11 0x00000000005f305d in _start ()
> > > > > (gdb) quit
> > > > I would guess that g_srandom_number_generator is in one compilation
> > > > unit, and the code using it is in another. Ie,
> > > > g_srandom_number_generator is in my_rng.cpp, and the code using it is
> > > > in my_key_gen.cpp. In this case, you would be violating C++'s static
> > > > non-local rules.
>
> > > > The fix is relatively easy. Offer a static/global function to serve up
> > > > a static local:
>
> > > > RandomNumberGenerator& GetRandomNumberGenerator()
> > > > {
> > > >  static AutoSeededX917RNG< CryptoPP::AES > s_prng;
> > > >  return s_prng;
> > > > }
>
> > > > Jeff
>
> > > Thanks Jeff. It's possibly the case,, although this was initialised in
> > the
> > > same file it was used, however what I noticed was that
> > > merely initialising the rng and never calling it anywhere would cause
> > this
> > > issue.
>
> > > To be a bit more exact this was in a my_rng.cc which was compiled into a
> > lib
> > > that was then linked to another file.cc (or object, but you know what I
> > > mean). The file.oo did have some cryptopp code in there (AES, HASH etc.
> > but
> > > no explicit RNG calls). This is why it took a wee while to figure it out.
> > It
> > > looks like there may have been a static somewhere I had not seen (inside
> > > cryptopp) or something very similar.
> > I believe Crypto++ is fairly safe in its use of globals. They are
> > used, and they are easy to audit: searh for "g_*" in the source files.
>
> > > I have tried your suggestion and looks a good, although it does still
> > > segfault in some circumstances (I am tracking that down just now). I
> > think
> > > it may be a threading issue though, but will post again if it turns out
> > to
> > > be more (which I doubt).
> > Crypto++ objects are thread safe, meaning they do not depend on global
> > data which might become corrupted across calls in different threads.
> > You still have to lock on concurrent access to an object.
>
> > For what it worth, Wei uses a different technique for initializing a
> > singleton (his method results in a memory leak under 'worst case', if
> > I recall correctly).
>
> > If you suspect you might have threading issues, Helgrind would
> > probably be very useful. But you need a good test program to get the
> > most out of it.
> >    valgrind --tool=helgrind my_test_program
>
> > Jeff
>
> > Doing that right now Jeff, thanks
>
> For anyone who is interested the (offending) code is located here https://github.com/maidsafe/MaidSafe-Common
At the risk of a flame war from the Boost fan boi club..... Don't use
Boost::Threads or Boost::Threading. From a recent audit, I believe the
subproject is not production quality. For example:
* Ignoring return values from locking functions
     (ie, pthread_mutex_lock and WaitForSingleObject)
* Race conditions
* Resource leaks on negative code paths

Most frustrating is the 20 or so bugs filed against Boost are still
sitting in their bug tracker unacknowledged (I believe two were
acknowledged and closed to date).

Jeff

--
You received this message because you are subscribed to the "Crypto++ Users" Google Group.
To unsubscribe, send an email to cryptopp-users-unsubscribe <at> googlegroups.com.
More information about Crypto++ and this group is available at http://www.cryptopp.com.

--
You received this message because you are subscribed to the "Crypto++ Users" Google Group.
To unsubscribe, send an email to cryptopp-users-unsubscribe <at> googlegroups.com.
More information about Crypto++ and this group is available at http://www.cryptopp.com.
Pedja | 5 Oct 2011 14:16
Picon
Gravatar

Re: Assertion Failure when using RSA

Hello!
I need to build cryptopp 5.6.1 for OS/X using Xcode 4.0+
When I set the compiler to GCC 4.2 I am able to build the static lib.
However, when trying to build the provided cryptest application
against it, I get linking errors. (I am using Snow Leopard 10.6.7 and
I used xcode 3.2.5 project as a reference form the bottom of this
<http://www.cryptopp.com/wiki/IOS> page.)

Ld /Users/svp/Library/Developer/Xcode/DerivedData/cryptopp-test-
fhbrajkkbingzucdhvpvqnfsjfbf/Build/Products/Debug/cryptest normal
x86_64
    cd /Volumes/data/Repos/hookflash-libs/cryptopp/projects/xcode/
cryptopp
    setenv MACOSX_DEPLOYMENT_TARGET 10.6
    /Developer/usr/bin/g++-4.2 -arch x86_64 -isysroot /Developer/SDKs/
MacOSX10.6.sdk -L/Users/svp/Library/Developer/Xcode/DerivedData/
cryptopp-test-fhbrajkkbingzucdhvpvqnfsjfbf/Build/Products/Debug -F/
Users/svp/Library/Developer/Xcode/DerivedData/cryptopp-test-
fhbrajkkbingzucdhvpvqnfsjfbf/Build/Products/Debug -filelist /Users/svp/
Library/Developer/Xcode/DerivedData/cryptopp-test-
fhbrajkkbingzucdhvpvqnfsjfbf/Build/Intermediates/cryptopp-test.build/
Debug/cryptest.build/Objects-normal/x86_64/cryptest.LinkFileList -
mmacosx-version-min=10.6 -lcryptopp -o /Users/svp/Library/Developer/
Xcode/DerivedData/cryptopp-test-fhbrajkkbingzucdhvpvqnfsjfbf/Build/
Products/Debug/cryptest

Undefined symbols for architecture x86_64:
  "CryptoPP::AlignedAllocate(unsigned long)", referenced from:
      CryptoPP::AllocatorWithCleanup<unsigned char,
true>::allocate(unsigned long, void const*)in bench.o
      CryptoPP::AllocatorWithCleanup<unsigned char,
true>::allocate(unsigned long, void const*)in bench2.o
      CryptoPP::AllocatorWithCleanup<unsigned char,
true>::allocate(unsigned long, void const*)in dlltest.o
      CryptoPP::AllocatorWithCleanup<unsigned char,
true>::allocate(unsigned long, void const*)in fipstest.o
      CryptoPP::AllocatorWithCleanup<unsigned char,
true>::allocate(unsigned long, void const*)in regtest.o
      CryptoPP::AllocatorWithCleanup<unsigned char,
true>::allocate(unsigned long, void const*)in test.o
      CryptoPP::AllocatorWithCleanup<unsigned char,
true>::allocate(unsigned long, void const*)in validat1.o
      ...
    "CryptoPP::AlignedDeallocate(void*)", referenced from:
      CryptoPP::AllocatorWithCleanup<unsigned char,
true>::deallocate(void*, unsigned long)in bench.o
      CryptoPP::AllocatorWithCleanup<unsigned char,
true>::deallocate(void*, unsigned long)in bench2.o
      CryptoPP::AllocatorWithCleanup<unsigned char,
true>::deallocate(void*, unsigned long)in dlltest.o
      CryptoPP::AllocatorWithCleanup<unsigned char,
true>::deallocate(void*, unsigned long)in fipstest.o
      CryptoPP::AllocatorWithCleanup<unsigned char,
true>::deallocate(void*, unsigned long)in regtest.o
      CryptoPP::AllocatorWithCleanup<unsigned char,
true>::deallocate(void*, unsigned long)in test.o
      CryptoPP::AllocatorWithCleanup<unsigned char,
true>::deallocate(void*, unsigned long)in validat1.o
      ...
    "vtable for CryptoPP::SHA224", referenced from:
      CryptoPP::SHA224::~SHA224()in fipstest.o
      CryptoPP::SHA224::SHA224(CryptoPP::SHA224 const&)in fipstest.o
      CryptoPP::SHA224::SHA224()in fipstest.o
      CryptoPP::SHA224::~SHA224()in regtest.o
      CryptoPP::SHA224::SHA224(CryptoPP::SHA224 const&)in regtest.o
      CryptoPP::SHA224::SHA224()in regtest.o
ld: symbol(s) not found for architecture x86_64
collect2: ld returned 1 exit status

On Aug 22, 1:37 am, Jeffrey Walton <noloa... <at> gmail.com> wrote:
> On Aug 21, 6:58 pm, Rob <ro... <at> zingsphere.com> wrote:
>
>
>
>
>
>
>
> > I'm on a mac compiling with xcode 4 on Snow Leapord for OS/X build
> > 64bit using cryptopp 5.6.1
>
> > Assertion failed: (memcmp(P, A, 4*WORD_SIZE)==0), function
> > AtomicDivide, file cryptopp/integer.cpp, line 2530.
>
> > Related cryptopp code:
> >                 assert(!T[2] && !T[3] && (T[1] < B[1] || (T[1]==B[1] &&
> > T[0]<B[0])));
> >                 word P[4];
> >                 s_pMul[0](P, Q, B);
> >                 Add(P, P, T, 4);
> >                 assert(memcmp(P, A, 4*WORD_SIZE)==0);
> >  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ fails here
>
> > Seems that P has a value of
> > [0] 0x0
> > [1] 0x4000000000000001
> > [2] 0x4000000000000000
> > [0] 0x0
>
> > Whereas A has a value of
> > [0] 0x0
> > [1] 0x4000000000000000
> > [2] 0x4000000000000000
> > [0] 0x0
>
> > Thus the assertion fails since it's off by 1 in the [1] word array
> > position.
>
> > FYI WORD_SIZE is set to "8" in that code.
>
> > My test code:
>
> >           AutoSeededRandomPool rnd;
>
> >           RSA::PrivateKey rsaPrivate;
> >           rsaPrivate.GenerateRandomWithKeySize(rnd, 2048);
>
> > Also I had a compile error that I had to fix:
>
> >         pointer allocate(size_type n, const void * = NULL)
> >         {
> > -               CheckSize(n);
> > +    AllocatorBase<T>::CheckSize(n);
>
> > ^ I had to put the "AllocatorBase<T>::" in front of "CheckSize" or it
> > wouldn't see "CheckSize" routine.
>
> > Any ideas?
>
> What compiler are you using?
>
> For what its worth, I have not had problems on the command line or in
> Xcode 3.2.x running Snow Leopard or Lion from a MacBook when using
> GCC.
>
> Jeff

--

-- 
You received this message because you are subscribed to the "Crypto++ Users" Google Group.
To unsubscribe, send an email to cryptopp-users-unsubscribe <at> googlegroups.com.
More information about Crypto++ and this group is available at http://www.cryptopp.com.

Rawler | 9 Oct 2011 16:01
Picon
Gravatar

Base32Encoder fails with test-vectors.

Hi,

I was just trying to use the base32-decoder, noticing it produced
unexpected output. Some further investigation, and I've now concluded
it also produces incorrect output for the test-vectors in
http://www.faqs.org/rfcs/rfc4648.html.

CODE:

#include <crypto++/filters.h>
#include <crypto++/base32.h>

#include <iostream>

int main() {
  std::string testdata("foobar");
  std::string base32str;
  CryptoPP::StringSource(testdata, true,
                new CryptoPP::Base32Encoder(
                    new CryptoPP::StringSink(base32str)));
  std::cout << base32str << std::endl;
  return 0;
}

RUN:
g++ -o test test.cpp -lcrypto++ && ./test

EXPECTED:
MZXW6YTBOI

GOT:
N3ZY82VBQI

--

-- 
You received this message because you are subscribed to the "Crypto++ Users" Google Group.
To unsubscribe, send an email to cryptopp-users-unsubscribe <at> googlegroups.com.
More information about Crypto++ and this group is available at http://www.cryptopp.com.

Mouse | 10 Oct 2011 00:30
Picon

Re: Base32Encoder fails with test-vectors.

I concur.

It appears that Base32 does not encode correctly. Also, there seems to
be no support for padding (that RFC 4648 seems to require).

RFC 4648 says: BASE32("foobar") = "MZXW6YTBOI======"  (but it's actually
Base32Hex - not strict Base32)

My own test-code output matched Rawler's, producing N3ZY82VBQI

On 10/9/11 10:01 AM, Rawler wrote:
> Hi,
>
> I was just trying to use the base32-decoder, noticing it produced
> unexpected output. Some further investigation, and I've now concluded
> it also produces incorrect output for the test-vectors in
> http://www.faqs.org/rfcs/rfc4648.html.
>
> CODE:
>
> #include <crypto++/filters.h>
> #include <crypto++/base32.h>
>
> #include <iostream>
>
> int main() {
>   std::string testdata("foobar");
>   std::string base32str;
>   CryptoPP::StringSource(testdata, true,
>                 new CryptoPP::Base32Encoder(
>                     new CryptoPP::StringSink(base32str)));
>   std::cout << base32str << std::endl;
>   return 0;
> }
>
> RUN:
> g++ -o test test.cpp -lcrypto++ && ./test
>
> EXPECTED:
> MZXW6YTBOI
>
> GOT:
> N3ZY82VBQI
>

-- 
Regards,
Mouse

--

-- 
You received this message because you are subscribed to the "Crypto++ Users" Google Group.
To unsubscribe, send an email to cryptopp-users-unsubscribe <at> googlegroups.com.
More information about Crypto++ and this group is available at http://www.cryptopp.com.


Gmane