headers
Olly Betts | 1 Jul 2005 02:06
Favicon
Gravatar

Re: Xapian Search

On 2005-06-26, Lars Magne Ingebrigtsen <larsi <at> gnus.org> wrote:
> Just let me know how you'd like Xapian to be called, and I can whip
> something up.

Previously I thought it simplest to just "nc sea 8002" myself, but now
I've thought about this more, I think it's better if the "log-datespool"
script tells Xapian when it has added an article to the date sorted
spoool.  That way I don't need to worry about whether the article that
sea has just announced has been processed by log-datespool and is
actually readable there or not.

So I guess if it can dump paths like this:

2005/07/01/00/11/21

to a file, then after a certain number (or a certain time) start a new
file and rename the existing one which the xapian indexer will then see
and pick up.

The obvious alternative seems to be for log-datespool to open the
xapian indexer as a pipe, which means that log-datespool needs to be
restarted whenever the indexer is changed.

It's probably simplest if I just adjust the scripts to show what I have
in mind.

Cheers,
    Olly
Permalink | Reply | 2 comments |
headers
Steinar Bang | 1 Jul 2005 07:00
Picon
Picon
Favicon

Re: Xapian Search

>>>>> Olly Betts <olly <at> survex.com>:

> Yeah - we:search ignores the "#" in "C#" and the "++" in "C++", and
> interprets both as "C",

Sounds like Karmic Justice to me.
Permalink | Reply | 0 comments |
headers
Lars Magne Ingebrigtsen | 2 Jul 2005 11:59
Face
Picon
Favicon
Gravatar

Re: Xapian Search

Olly Betts <olly <at> survex.com> writes:

> This is actually pretty efficient - we record a list of the articles
> indexed, in date order, so we just sweep the spool in date order
> performing a set disjunction operation with this list from the previous
> run, which is easy because it's between two sorted lists.  Much nicer
> than having to use a hash table as we-search did before the date-sorted
> spool.

Yup.  

--

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi <at> gnus.org * Lars Magne Ingebrigtsen
Permalink | Reply | 0 comments |
headers
Lars Magne Ingebrigtsen | 2 Jul 2005 12:01
Face
Picon
Favicon
Gravatar

Re: Xapian Search

Olly Betts <olly <at> survex.com> writes:

> Previously I thought it simplest to just "nc sea 8002" myself, but now
> I've thought about this more, I think it's better if the "log-datespool"
> script tells Xapian when it has added an article to the date sorted
> spoool.  That way I don't need to worry about whether the article that
> sea has just announced has been processed by log-datespool and is
> actually readable there or not.

That's true.  On the other hand, you can just read via nfs from
/var/spool/news -- there's only a few tens of messages per minute, so
the nfs traffic that would result from that is negligible.

--

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi <at> gnus.org * Lars Magne Ingebrigtsen
Permalink | Reply | 1 comment |
headers
Lars Magne Ingebrigtsen | 2 Jul 2005 12:01
Face
Picon
Favicon
Gravatar

Re: gmane.network.openldap.general: posting possible?

Adrian Gschwend <ktk <at> netlabs.org> writes:

> I posted a message to the list in %subj% and it did not appear till now 
> (about 3 hours ago).

[...]

> am I too impatient or is there a problem with the list?

Impossible to say.  Mail is mail.

--

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi <at> gnus.org * Lars Magne Ingebrigtsen
Permalink | Reply | 0 comments |
headers
Lars Magne Ingebrigtsen | 3 Jul 2005 14:31
Face
Picon
Favicon
Gravatar

Towards better statistics

I've been looking at the stats scripts again.  It would be cool to
have more "live stats" -- "what's the most popular mailing list right
now", "what project is gaining most momentum right now", and so on.

But to do that, I really need to be running a database, and I don't
really want to be doing that, so I've just been looking at the
rates-per-day stuff.

Here's an example:

http://gmane.org/details.php?group=gmane.games.fps.bf1942

The normal, easy-to-generate stats are

http://dir.gmane.org/gmane.games.fps.bf1942

which, you can see, are pretty much unusable, because all the messages
in the group are there after an import, and the normal stat stuff uses
the arrival date, not the posting date.

The details.php thing is based on a script that parses the group .nov
headers and gathers the required information.  It's a slow Perl
script.  To figure out what took so long (after a tip from a small
dog), I cat-ed all the .nov files to /dev/null, and timed that.  It
took an hour.  The .nov files were quite fragmented, which shouldn't
have come as a surprise.  I mean, they're just appended to, and
haven't been regenerated in a couple of years.

I whipped up a script that did "cat file > file.tmp; mv file.tmp file" 
for all the .nov files, and catting the 9GB .nov files now takes five
(Continue reading)

Permalink | Reply | 9 comments |
headers
Alan Post | 4 Jul 2005 06:26

Re: Towards better statistics

In article <m3u0jc6pgo.fsf <at> quimbies.gnus.org>, Lars Magne Ingebrigtsen wrote:
> 
> However, the Perl stats script is still very slow.  It takes 2½
> hours.  Which is much too slow to be running every day.
> 
> So I (or somebody else) may be writing a stats gathering thing in a
> different language...  

If you post the current script and an example input/output pair, it
might happen.  :)

  Alan
Permalink | Reply | 8 comments |
headers
Rakotomandimby Mihamina | 6 Jul 2005 17:50
Picon

Re: dowmload gmane

Olly Betts wrote:
> On 2005-06-02, Rakotomandimby (R12y) Mihamina <mihamina.rakotomandimby <at> etu.univ-orleans.fr> wrote:
>>But... tell me... is gmane open source/specifications?
> Yes.  But it's not anything approaching a turn-key solution.

Well. If someone is planning to write some documentation on how to run
an INN that would mirror gmane (in order to share the BW needs or some
other things) just tell me, I'm volunteer.

We dont have cash money, we just have been sponsorised the rent of 5
dedicated servers that are under-used, so... If they could help...

--

-- 
Miroir de logiciels libres        http://www.etud-orleans.fr
Permalink | Reply | 1 comment |
headers
Felix Wiemann | 10 Jul 2005 16:51
Picon

Re: Followups to another group

Lars Magne Ingebrigtsen wrote:

> Felix Wiemann writes:
>
>> Mmmh...  Any chance of getting this fixed?  We really have a lot of
>> postings which are posted to a different group than the group of the
>> referenced article.  (Followups from a check-in mailing list to a
>> developer mailing list.)  They all don't appear on the web interface.
>
> I haven't had time to look into it, but it does appear to be a pretty
> reproducible problem.

*nudging-lars-gently* ;-)

--

-- 
For private mail please ensure that the header contains 'Felix Wiemann'.

"the number of contributors [...] is strongly and inversely correlated with the
number of hoops each project makes a contributing user go through."      -- ESR
Permalink | Reply | 2 comments |
headers
Luca P. Stevens | 11 Jul 2005 12:05
Picon
Favicon

Mail delivery software won't authorize me.


Hi.

I repeatedly tried to send private emails in
gmane.recreation.pescasub.italian, I got many "Please confirm your message"
message, one for each mail, I followed the instruction every time, but I'm not
authorized yet, my emails never reached recipients (I tried posting to myself
either).

I used the address which I'm subscribed to that group with: 
p.stevens at libero.it. 

This is the last confirmation request I got:

From: Gmane Remailer
<public-p.stevens-VGgt2q2+T+FeoWH0uzbU5w <at> ciao.gmane.org>
Reply-To:
public-p.stevens-VGgt2q2+T+FeoWH0uzbU5w-confirm-1120998358.5491.8f08df <at> ciao.gmane.org

Newsgroup posting is ok, instead.

Thanx for any help.

--

-- 
Luca - e-mail: p.stevens at libero.it
Permalink | Reply | 1 comment |

Gmane