headers
Jeffrey M. Vinocur | 1 Nov 2009 01:02

Re: INN Stat command can not get article number

On Thu, 29 Oct 2009, [iso-8859-1] Julien ÉLIE wrote:

> > Is there any command get article number from specific message id?
> 
> No.  INN does not give the article number.

Well, you can retrieve the Xref header.

Of course, you will still have confusion if the article is crossposted 
into more than one group.  

--

-- 
Jeffrey M. Vinocur
jeff <at> litech.org
_______________________________________________
inn-workers mailing list
inn-workers <at> lists.isc.org
https://lists.isc.org/mailman/listinfo/inn-workers
Permalink | Reply | 0 comments |
headers
Julien ÉLIE | 1 Nov 2009 17:52
Favicon

Re: HDR/OVER/XPAT and keyword generation

Hi Nix,

> I thought the point of keyword generation was that it kicked in only
> for those posts that *don't* already have a keywords header?
>
> ... but, hm, it looks like it appends generated keywords to the
> existing keyword header value. That's really not ideal, is it :/

It appears that keyword generation is done for every article, even
though it already has a Keywords: header.
I would tend to think like you that it should not regenerate it for
such articles.

We can change the behaviour of keyword generation.
Does someone think we should go on regenerating the Keywords: header
of articles which already have one?

>> "HDR keywords" would give the contents of the real Keywords: header
>> and "HDR :keywords" would give the contents of the computed Keywords:
>> header...
>> But I doubt clients know how to deal with that...
>
> I was assuming that the point of the keyword generation was that it
> made those articles which didn't have keywords look as if they did,
> leaving those which *did* have human-written keywords alone. But it
> looks like that's not quite what it does...

We do not modify the article, but only its overview data, as it is said
in INN 2.5.1 documentation:  "In order to use this feature, the regex
library should be available and INN configured with the --enable-keywords
(Continue reading)

Permalink | Reply | 2 comments |
headers
Julien ÉLIE | 1 Nov 2009 18:06
Favicon

Configuration parser and integers

Hi,

The configuration parser defines numbers:
    integer = ["-"] 1*DIGIT
but a lot of them must be positive.

Should we do something about that at the level of the parser?
(defining a positive integer?)
or at the level of innconf_validate? (exiting if something
is bad)

or do nothing because it is the fault of the news admin?
INN will then probably crash (SERVER cant malloc 4294967292 bytes,
or something like that).

--

-- 
Julien ÉLIE

« -- Tu dois avoir un messager zélé autant qu'ailé
  pour faire rapidement le trajet.
  -- Oui ! et c'est une fine mouche ! » (Astérix) 

_______________________________________________
inn-workers mailing list
inn-workers <at> lists.isc.org
https://lists.isc.org/mailman/listinfo/inn-workers
Permalink | Reply | 5 comments |
headers
Julien ÉLIE | 1 Nov 2009 21:11
Favicon

Use of cvtbatch

Hi,

I wonder what kind of use is cvtbatch for.  Is someone here using it?

It takes storage API tokens in entry and can give for each of them
the size of the article headers (yes, only the headers) and the message-ID.
(The arrival time is new in INN 2.5.2.)

--

-- 
Julien ÉLIE

« Persica, pira, poma, requirent uina bona. »

_______________________________________________
inn-workers mailing list
inn-workers <at> lists.isc.org
https://lists.isc.org/mailman/listinfo/inn-workers
Permalink | Reply | 2 comments |
headers
Julien ÉLIE | 1 Nov 2009 21:04
Favicon

Re: Batcher and it's man page.

Hi Kamil,

> Is it planned restoration of handling lines with filenames? (Or I have
> to write it by myself? :) )

Did you finally find something else to use batcher?

In case it would help, I see that there is some code in innxmit that
could be of help if you want to add support for filenames to batcher:

/*
**  Open an article.  If the argument is a token, retrieve the article via
**  the storage API.  Otherwise, open the file and fake up an ARTHANDLE for
**  it.  Only fill in those fields that we'll need.  Articles not retrieved
**  via the storage API will have a type of TOKEN_EMPTY.
*/
static ARTHANDLE *
article_open(const char *path, const char *id)
{
    TOKEN token;
    ARTHANDLE *article;
    int fd;
    size_t length;
    struct stat st;
    char *p;

    if (IsToken(path)) {
        token = TextToToken(path);
        article = SMretrieve(token, RETR_ALL);
        if (article == NULL) {
(Continue reading)

Permalink | Reply | 1 comment |
headers
Nix | 2 Nov 2009 21:22
Picon

Re: HDR/OVER/XPAT and keyword generation

On 1 Nov 2009, julien <at> trigofacile.com said:

> We can change the behaviour of keyword generation.
> Does someone think we should go on regenerating the Keywords: header
> of articles which already have one?

(not me, but you already know that)

>> I was assuming that the point of the keyword generation was that it
>> made those articles which didn't have keywords look as if they did,
>> leaving those which *did* have human-written keywords alone. But it
>> looks like that's not quite what it does...
>
> We do not modify the article, but only its overview data, as it is said
> in INN 2.5.1 documentation:  "In order to use this feature, the regex

Yeah, sorry, I misspoke through rephrasing the response one too many
times (the 'look as if' lost an 'in the overview').

> Incidentally, I have just caught another bug in keyword generation,
> depending on the value of the keylimit: inn.conf parameter.
>
> Just set it to "-5" and innd will die with:

That's just missing parameter validation, presumably.

>    SERVER cant malloc 4294967292 bytes at keywords.c line 138
> Because of a xmalloc().
>
> And also, just set it to "5" and wait for an article containing
(Continue reading)

Permalink | Reply | 1 comment |
headers
Nix | 2 Nov 2009 22:07
Picon

Re: Configuration parser and integers

On 1 Nov 2009, julien <at> trigofacile.com stated:

> INN will then probably crash (SERVER cant malloc 4294967292 bytes,
> or something like that).

In my case (big server, multi-tens-of-gigabytes of physical RAM and all
64-bit) it probably *won't* crash: it'll just eat an insane amount of
storage. That's probably even worse.
_______________________________________________
inn-workers mailing list
inn-workers <at> lists.isc.org
https://lists.isc.org/mailman/listinfo/inn-workers
Permalink | Reply | 0 comments |
headers
Kamil Jońca | 2 Nov 2009 21:59
Picon
Favicon

Re: Batcher and it's man page.

Julien ÉLIE <julien <at> trigofacile.com> writes:

> Hi Kamil,
>
>> Is it planned restoration of handling lines with filenames? (Or I have
>> to write it by myself? :) )
>
> Did you finally find something else to use batcher?
>

Finally I wrote simple perl script, which collects articles accordingo
to given list (of files).

I thought about it, and I think, that this is not necessary in batcher
if it will cause extra coding. 
KJ

_______________________________________________
inn-workers mailing list
inn-workers <at> lists.isc.org
https://lists.isc.org/mailman/listinfo/inn-workers
Permalink | Reply | 0 comments |
headers
Julien ÉLIE | 2 Nov 2009 22:09
Favicon

gdb and memory caching?

Hi,

Trying to debug keyword generation code because of a
    *** glibc detected *** free(): invalid next size (fast): 0x083338b8 ***
I had last night when freeing hc->Value, I do not understand the output given
by gdb:

(gdb) step
246         strlcpy(hc->Value + l, ",\377", innconf->keylimit + 1 - l);
(gdb)
strlcpy (dst=0x8312a15 "", src=0x80a2af2 ",ÿ", size=1) at strlcpy.c:31

hc->Value is a pointer to the value of the Keywords: header.
l is the length of that value.
We see that "hc->Value + l" is 0x8312a15, which is '\0'.
That's fine.  We are at the end of the header.

innconf->keylimit = 5
And the header is "12345" at 0x8312a10.

Then, what I do not understand is that gdb mentions that l is 7
whereas it was changed to 5.
Is it a gdb bug?  (Very strange.)
Isn't gdb returning the current value of l?

247         for (chase = hc->Value + l + 2, word_index = 0;
(gdb) print l
$7 = 7
(gdb) print  hc->Value + l + 2
$9 = 0x8312a19 ""
(Continue reading)

Permalink | Reply | 1 comment |
headers
Russ Allbery | 3 Nov 2009 01:42
Picon
Favicon
Gravatar

Re: gdb and memory caching?

Julien ÉLIE <julien <at> trigofacile.com> writes:

> Then, what I do not understand is that gdb mentions that l is 7
> whereas it was changed to 5.
> Is it a gdb bug?  (Very strange.)
> Isn't gdb returning the current value of l?

I suspect the compiler may be aggressively optimizing because the code
here is invalid.

>    /* Scribble onto end of Keywords: after a magic separator. */
>    strlcpy(hc->Value + l, ",\377", innconf->keylimit + 1 - l);
>    for (chase = hc->Value + l + 2, word_index = 0;
>        word_index < distinct_words;
>        word_index++) {

>        /* Add to list. */
>        *chase++ = ',';
>        strlcpy(chase, word[word_vec[word_index].index],
>                innconf->keylimit + 1 - (chase - hc->Value));
>        chase += word_vec[word_index].length;

>        if (chase - hc->Value > (innconf->keylimit - (MAX_WORD_LENGTH + 4)))
>            break;
>    }

> chase is first set to "hc->Value + l + 2" even though it is beyond
> the end of hc->Value; then a "," is added even though it is beyond
> the end of hc->Value; and the comparison might be unsigned (size_t)
> and therefore be wrong if the second member is negative.
(Continue reading)

Permalink | Reply | 0 comments |

Gmane