Installation problems (solved) & feature requests

Hey there!

I have been searching the web today for a good decentralized bug tracker
and stumbled upon bugs-everywhere. After I removed other projects from
my selection as they seemed abandoned, not mature etc., be was the only
project that seemed worth trying.

However, it really took me a while to get going:

I'm currently using Ubuntu Natty Narwhal, the 11.04 release, and the
package is still very old. I know that you guys are working on it,
however, I decided to compile it myself, but I ran into many errors.
After some searching I found out that I had to use "make DOC= install"
to prevent the missing dependencies. The installation process afterwards
seemed ok, but calling "be" just told me that the class
"libbe.ui.command_line.py" could not be imported. I had to investigate
that there's a PYTHONPATH variable that I could set to the appropriate
path, in my case

export PYTHONPATH=/home/heinrich/lib/python2.7/site-packages/

Afterwards, it worked! Now, please, could you put that on your website?
It really took me ages to find that out and I suppose there are many
people who run into these problems.

--------

I tried "be" a little bit, and there are some suggestions I would like
to make.

Re: Installation problems (solved) & feature requests

On Sun, May 01, 2011 at 02:52:13AM +0200, Christian Heinrich wrote:
> However, it really took me a while to get going:

Sorry.  Hopefully the packaging gets better soon!

> After some searching I found out that I had to use "make DOC= install"

This is mentioned in doc/install.txt.

> calling "be" just told me that the class "libbe.ui.command_line.py"
> could not be imported. I had to investigate that there's a
> PYTHONPATH variable that I could set to the appropriate path, in my
> case
> 
> export PYTHONPATH=/home/heinrich/lib/python2.7/site-packages/

This should be fixed in my branch since

    Author:     W. Trevor King <wking <at> drexel.edu>
    AuthorDate: Sat Apr 16 21:26:02 2011 -0400

    Fix /64c by installing with `--user` by default (vs. --prefix=${HOME}).

    This way users don't need to mess with PYTHONPATH.  They'll still need
    to tweak PATH so they can find the be entry script.

It should probably be merged into the trunk (<bumps Chris>).

> 1.) 4a4:os Is a good point. The reason for including something like
> "medium" or "normal" is that I want to differentiate between e.g. a

Re: Installation problems (solved) & feature requests

On Sun, May 01, 2011 at 10:17:31PM -0400, W. Trevor King wrote:
> On Sun, May 01, 2011 at 02:52:13AM +0200, Christian Heinrich wrote:
> > 3.) Using wrong syntax should receive better help messages.
> > ...
> 
> How about:
>   "be new" -> ERROR:
>   Missing required argument SUMMARY
>   Run `be help new` for usage details
> 
> Most of the times I mistype commands it's silly, and I only want a
> short reminder.  For newer users, the full help details may be
> interesting, so I'm fine with pointing towards them, but they are too
> long to print every time.

I've just added something like this to my tree.  It's hard for me to
decide what errors should be UsageErrors (which get the usage string
and a pointer towards the command's help) and which should be
UserErrors (which only get the error message), but that's easy enough
to fine tune as we move forward.

--

-- 
This email may be signed or encrypted with GPG (http://www.gnupg.org).
The GPG signature (if present) will be attached as 'signature.asc'.
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy

My public key is at http://www.physics.drexel.edu/~wking/pubkey.txt

_______________________________________________

Re: Installation problems (solved) & feature requests

On Sun, May 01, 2011 at 10:17:31PM -0400, W. Trevor King wrote:
> On Sun, May 01, 2011 at 02:52:13AM +0200, Christian Heinrich wrote:
> > 4.) "be show" should also list the "target" just as "tags"
> 
> I don't understand what you mean here.  Can you give an explicit
> example?  I don't think `be show` shows either targets, tags, or any
> extra_string values.  It should though, and as a stop-gap solution we
> could just print the strings.

Just pushed printing of extra strings to my branch.

--

-- 
This email may be signed or encrypted with GPG (http://www.gnupg.org).
The GPG signature (if present) will be attached as 'signature.asc'.
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy

My public key is at http://www.physics.drexel.edu/~wking/pubkey.txt

_______________________________________________
Be-devel mailing list
Be-devel <at> bugseverywhere.org
http://void.printf.net/cgi-bin/mailman/listinfo/be-devel

Performance issue

Hi all,

I just recently discovered BE, and I must congratulate you guys, the
feature set and the documentation are very interesting. There are quite
a few distributed BTS out there now, but some are really minimalist
and/or very poorly documented. BE's documentation is a really nice
surprise.

A year back, I had considered using ditz, and the first thing that hit
me when I saw how it works was that scanning its bug directory would be
a nasty performance killer once filled with a huge number of entries, if
not properly optimized, and that the proper optimization would be far
from trivial because of the distributeness (I suppose no present DB
format is amenable to merging). A quick showed that performance indeed
dropped with a big number of bugs.

I have seen that in 2010, someone here ran a few performance tests on
BE, that showed this performance drop at most at a few thousands bugs.
My own tests was to create empty bugs with just a message and to list
them (discarding the output to /dev/null) and it showed the following
results:

number of commits       time
0                       baseline
128                     1.2x slower
512                       4x slower
1024                     17x slower

On my system, the baseline was actually 400ms, which is barely
noticeable, but at 512 bugs, it already becomes 1.6s, which easily seems

Re: Performance issue

On Tue, May 10, 2011 at 02:29:27PM -0400, W. Trevor King wrote:
> After the 4096 bugs, the best fit was
> 
>   <`be list` time> = 470 ms + 7.6 ms * N + 3.0 us * N**2

After 4096 bugs and commits, the best fit was

  <`be list` time> = 470 ms + 8.7 ms * N + 2.3 us * N**2

So post-bug commits have no impact (as expected).

--

-- 
This email may be signed or encrypted with GPG (http://www.gnupg.org).
The GPG signature (if present) will be attached as 'signature.asc'.
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy

My public key is at http://www.physics.drexel.edu/~wking/pubkey.txt

1	0.47
2	0.49
4	0.49
8	0.53
16	0.58
32	0.70
64	0.96
128	1.52
256	2.73
512	5.43

Re: Performance issue

Some of my earlier responses to Pierre's message seem to have been
lost in the ether.  Here they are again:

On Tue, May 10, 2011 at 01:51:22PM -0400, W. Trevor King wrote:
> On Tue, May 10, 2011 at 12:45:14AM +0200, Pierre THIERRY wrote:
> > I have seen that in 2010, someone here ran a few performance tests on
> > BE, that showed this performance drop at most at a few thousands bugs.
> > My own tests was to create empty bugs with just a message and to list
> > them (discarding the output to /dev/null) and it showed the following
> > results:
> > 
> > number of commits       time
> > 0                       baseline
> > 128                     1.2x slower
> > 512                       4x slower
> > 1024                     17x slower
> > 
> > On my system, the baseline was actually 400ms, which is barely
> > noticeable, but at 512 bugs, it already becomes 1.6s, which easily seems
> > like a small pause to the user.
> 
> By "number of commits" I expect you mean "number of empty bugs".  I
> expect BE should scale well vs number of commits.  I just ran some
> tests (benchmark-be-list.sh attached) on my own system (ext4
> filesystem), and got something like
>   <`be list` time> = 470 ms + 8.3 ms * N + 2.6 us * N**2
> where N is the number of bugs.
> 
> Which is a bit slower than your times, but qualitatively similar.
> I'll run the tests on my btrfs system when I get home tonight.

Re: Performance issue

On Tue, May 10, 2011 at 01:51:22PM -0400, W. Trevor King wrote:
> 3) Use `be serve` to cache in memory.  As long as you avoid other
>   direct repository manipulation, this should work well until you run
>   out of memory ;).

Actually, it is roughly equivalent to direct access since `be serve`
doesn't cache (it handles raw Storage requests).  A hypothetical
command-based server that loaded the BugDir while handling requests
would cache in memory as I suggested above.

-- 
This email may be signed or encrypted with GPG (http://www.gnupg.org).
The GPG signature (if present) will be attached as 'signature.asc'.
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy

My public key is at http://www.physics.drexel.edu/~wking/pubkey.txt

1	0.29
2	0.30
4	0.34
8	0.39
16	0.49
32	0.72
64	1.19
128	2.17
256	4.22
512	8.61
1024	17.64

Re: Installation problems (solved) & feature requests

On Sun, May 01, 2011 at 10:17:31PM -0400, W. Trevor King wrote:
> > 5.) Interesting: The ID (the long hash) displayed with "be show" is
> > *NOT* the BUG-ID, which actually is the "short name". Maybe change the
> > name? Confused me a lot!
> 
> Ah.  That is a good point.  The long hash *is* the bug ID, but when
> commands ask for bug IDs, they actually want a compound
> BUGDIR-ID/BUG-ID[/COMMENT-ID].  There should be links from the
> relevant sections of the tutorial to libbe.util.id:
>   http://www.physics.drexel.edu/~wking/code/be/doc/libbe/libbe.util.id.html
> But I seem to have borked the Sphinx cross referencing.  I'll try
> and make this clearer.

Sphinx cross referencing fixed for tutoial.txt -> libbe.util.id.

--

-- 
This email may be signed or encrypted with GPG (http://www.gnupg.org).
The GPG signature (if present) will be attached as 'signature.asc'.
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy

My public key is at http://www.physics.drexel.edu/~wking/pubkey.txt

_______________________________________________
Be-devel mailing list
Be-devel <at> bugseverywhere.org
http://void.printf.net/cgi-bin/mailman/listinfo/be-devel

Re: Performance issue

On Tue, May 10, 2011 at 01:51:22PM -0400, W. Trevor King wrote:
> 1) Use a better file system.  There's no fundamental difference
>   between a file system and a database like MySQL.  With a properly
>   tuned filesystem geared towards many small files, BE's current
>   file-based backends should perform acceptably.

I ran the tests on a tmpfs partition on the same computer instead.  I
didn't deactivat swapping, but there wasn't much else on the computer
during the test run, so I doubt it swapped much.  The results were
suprisingly similar to the other direct-access runs.  I'll attach all
the results here (as well as the tmpfs script) for those interested in
the raw numbers.

--

-- 
This email may be signed or encrypted with GPG (http://www.gnupg.org).
The GPG signature (if present) will be attached as 'signature.asc'.
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy

My public key is at http://www.physics.drexel.edu/~wking/pubkey.txt

1	0.47
2	0.49
4	0.49
8	0.52
16	0.58
32	0.70
64	0.97
128	1.56
256	2.79