dave | 1 Dec 2008 09:06

Wesnoth memory usage investigation

I've done some investigation into Wesnoth's memory usage which I've  
documented in my blog here: http://dave.wesnoth.org/?p=7

As a result of this investigation, I've implemented a custom memory  
allocator for Wesnoth, which saves about 10% memory on my system. I  
appreciate others testing it and letting me know their results.
Ignacio Morelle | 1 Dec 2008 16:36
Picon
Gravatar

Re: Translation process

Hello

On 11/24/08, allefant <allefant <at> gmail.com> wrote:
> On Mon, Nov 24, 2008 at 5:13 PM, Fabian Mueller <fabianmueller5 <at> gmx.de>
> wrote:
>
> What would be nice would be something so poedit itself could show you
> what changed - back when I was translating I also found it annoying
> when a string was marked "fuzzy" but I had no idea what did change in
> the English text. (And going through huge diffs to find the change for
> each single string is rather time consuming no matter which diff
> program).
>
> But not sure there exists a solution.

There seems to be a tool labeled to able to show word-by-word
differences in PO files ... or it was supposed to do so at some time.
I don't know; it was written by ott and I haven't tested it myself.

# wxdiff: highlight word differences in gettext po file translations
# use when the original and translation are close but not identical
# usage: wxdiff en_GB.po > tmp.po; less -R tmp.po
#   Copyright (C) 2005 by ott

This is /trunk/utils/wxdiff in the mainline SVN repository.

Regards
-- Ignacio R. Morelle <shadowmaster>
Ilor | 1 Dec 2008 20:51
Picon

Re: Wesnoth memory usage investigation

2008/12/1  <dave <at> whitevine.net>:
> I've done some investigation into Wesnoth's memory usage which I've
> documented in my blog here: http://dave.wesnoth.org/?p=7
> As a result of this investigation, I've implemented a custom memory
> allocator for Wesnoth, which saves about 10% memory on my system. I
> appreciate others testing it and letting me know their results.

Where can we get it? How portable is it? A good read in general, as
for the hashmaps, I did a quick test by replacing the two big std::map
typedefs in config.hpp to boost::unordered_map. Memory usage increased
by around 10%, measured at the start of the test scenario. Perhaps
combining the hash maps with a custom allocator would yield better
results. Also note thet boost::unordered_map needs boost >= 1.36.

--
Ilor
dave | 1 Dec 2008 21:17

Re: Wesnoth memory usage investigation

Quoting Ilor <kailoran <at> gmail.com>:

> 2008/12/1  <dave <at> whitevine.net>:
>> I've done some investigation into Wesnoth's memory usage which I've
>> documented in my blog here: http://dave.wesnoth.org/?p=7
>> As a result of this investigation, I've implemented a custom memory
>> allocator for Wesnoth, which saves about 10% memory on my system. I
>> appreciate others testing it and letting me know their results.
>
> Where can we get it? How portable is it?

It's checked in under src/poolalloc.c

If you use scons to build, it should already be setup to be built and  
used right now.

The only non-standard thing about it is that it calls mmap(), which  
I'm not sure is available on Windows. This should be fairly easy to  
hook in to an equivalent call on any platform though.

Other than that it should be completely portable to any C99 platform.

> A good read in general, as
> for the hashmaps, I did a quick test by replacing the two big std::map
> typedefs in config.hpp to boost::unordered_map. Memory usage increased
> by around 10%, measured at the start of the test scenario. Perhaps
> combining the hash maps with a custom allocator would yield better
> results. Also note thet boost::unordered_map needs boost >= 1.36.

I doubt using a hashmap would save much, though we could try Google's  
(Continue reading)

Ignacio Morelle | 2 Dec 2008 00:06
Picon
Gravatar

Re: Wesnoth memory usage investigation

Hello

On 12/1/08, dave <at> whitevine.net <dave <at> whitevine.net> wrote:
> The only non-standard thing about it is that it calls mmap(), which
> I'm not sure is available on Windows. This should be fairly easy to
> hook in to an equivalent call on any platform though.
>
> Other than that it should be completely portable to any C99 platform.

I have read that Boost [1] also has a memory pool library
(boost::pool) that even provides a std::allocator interface, as far as
1.33.0 goes. I haven't used it, and the documentation does not give me
a clue about its portability, but I guess that it wouldn't be in Boost
in the first place, if it wasn't broadly portable.

I wonder if it could be another possible solution.

[1]: http://www.boost.org/doc/libs/1_37_0/libs/pool/doc/index.html

Regards
-- Ignacio R. Morelle <shadowmaster>
dave | 2 Dec 2008 01:55

Re: Wesnoth memory usage investigation

Quoting Ignacio Morelle <shadowm2006 <at> gmail.com>:

> Hello
>
> On 12/1/08, dave <at> whitevine.net <dave <at> whitevine.net> wrote:
>> The only non-standard thing about it is that it calls mmap(), which
>> I'm not sure is available on Windows. This should be fairly easy to
>> hook in to an equivalent call on any platform though.
>>
>> Other than that it should be completely portable to any C99 platform.
>
> I have read that Boost [1] also has a memory pool library
> (boost::pool) that even provides a std::allocator interface, as far as
> 1.33.0 goes. I haven't used it, and the documentation does not give me
> a clue about its portability, but I guess that it wouldn't be in Boost
> in the first place, if it wasn't broadly portable.

I've used Boost pools before. They're quite nice and is memory efficient.

However, it would be quite difficult for us to use. We can't really  
use the allocator interface easily because, (1) it only works for STL  
containers that use allocators, not for other sources of memory use;  
(2) we'd have to go through and modify all of our STL containers to  
use this allocator instead of the default one. This is quite an  
intrusive and annoying change to make.

Calling the pools from malloc() itself isn't really possible, because  
when free() is called, we wouldn't be able to efficiently tell where  
the memory came from.

(Continue reading)

Nils Kneuper | 3 Dec 2008 19:37
Picon

The end of 1.4.x


Hi everybody!
Today I just wanted to announce that I plan to get the last release of the 1.4.x
series out by the end of next week. That is I would like to get 1.4.7 out then
and it shall be the last one of this branch (that is unless some really big and
bad bug occurs that also gets fixed in this branch).
The reason for the 1.4.7 release is that YogiHH just added a patch to make
(newly created) replays actually work. The reason to abandon 1.4.x is that we
are getting close to the new stable series and shall enter the beta time for it
soon...
Cheers,
Nils Kneuper aka Ivanovic
David Philippi | 3 Dec 2008 23:25
Picon

Re: [Wesnoth-commits] r31239 - /trunk/SConstruct

Am Mittwoch 03 Dezember 2008 schrieb loonycyborg <at> gmail.com:
> URL: http://svn.gna.org/viewcvs/wesnoth?rev=31239&view=rev
> Log:
> Change CXXFLAGS to CCFLAGS so .c sources will be affected too.

I wouldn't consider this a good idea. It's perfectly valid to put options into 
CXXFLAGS which are not available for C sources at all. People wanting to 
optimize .c sources will have to set CFLAGS accordingly.

Bye,
David
Sergey Popov | 4 Dec 2008 11:11
Picon

Re: [Wesnoth-commits] r31239 - /trunk/SConstruct

На Wed, 3 Dec 2008 23:25:10 +0100
David Philippi <david <at> torangan.de> записано:

> Am Mittwoch 03 Dezember 2008 schrieb loonycyborg <at> gmail.com:
> > URL: http://svn.gna.org/viewcvs/wesnoth?rev=31239&view=rev
> > Log:
> > Change CXXFLAGS to CCFLAGS so .c sources will be affected too.
> 
> I wouldn't consider this a good idea. It's perfectly valid to put options into 
> CXXFLAGS which are not available for C sources at all. People wanting to 
> optimize .c sources will have to set CFLAGS accordingly.
Sorry for misleading commit message! It should be:
"Append release and debug flags to CCFLAGS scons contruction variable instead
of CXXFLAGS since they work for both C and C++ and should be used for both."

_______________________________________________
Wesnoth-dev mailing list
Wesnoth-dev <at> gna.org
https://mail.gna.org/listinfo/wesnoth-dev
ben | 4 Dec 2008 19:03
Favicon

Re: The end of 1.4.x

Sounds good. My computer is now fixed and ready to be picked up, so I should have it by then and should be able to
make a Mac release. :)

Ben / crimson_penguin

Nils Kneuper <crazy-ivanovic <at> gmx.net> wrote :

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> Hi everybody!
> Today I just wanted to announce that I plan to get the last release of the
> 1.4.x
> series out by the end of next week. That is I would like to get 1.4.7 out
> then
> and it shall be the last one of this branch (that is unless some really big
> and
> bad bug occurs that also gets fixed in this branch).
> The reason for the 1.4.7 release is that YogiHH just added a patch to make
> (newly created) replays actually work. The reason to abandon 1.4.x is that
> we
> are getting close to the new stable series and shall enter the beta time for
> it
> soon...
> Cheers,
> Nils Kneuper aka Ivanovic
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v2.0.9 (GNU/Linux)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
> 
(Continue reading)


Gmane