perl_access hash value length limit?

Hi gang,

I think I've found a bug in INN 2.5.2 with the readers.conf perl_access
access() returned hash value length.  I added a bunch of newsgroups
today that pushed the character count in the 'read' key's value from
6456 characters to 8238 characters.  With a 'read' string that long, INN
didn't work anymore for incoming connections.  Syslog reported the error
"syntax error in (null)(2), Expected value." and terminated the
connection.

If I reverted my newsgroup adds from the flatfile that my perl_access
script uses to build the hash, back down to its this-morning length of
6456 characters, it worked again.  Add one or two newsgroups more and it
still worked.  So somewhere between those two value's character lengths,
INN stopped liking it.

I don't think perl cares, or that we've hit a limit in perl, since a
small test perl script that calls the same perl_access library and
simply prints out the resulting hash showed the thing in entirity
(that's how I was able to see the character counts).  I'm hoping it's
something simple like a buffer limit being hit in INN.

Backstory: We've been using INN for years to support internal newsgroups
for CS course discussions.  18 months ago we started naming the groups
per-semester (for more granular access control and versioning), but that
means each term we add 100 or so newsgroups.  Either wildmats didn't
work in the perl_access hash at one time, or we decided not to use them
but each newsgroup is enumerated individually in the read and post hash
values.  FWIW, I was able to squeeze all of this semester's groups in by
just putting a comma inbetween groups (I had comma-space before) and my

Re: perl_access hash value length limit?

Hi David,

On Tue, Jan 11, 2011 at 05:28:42PM -0600, David E Mussulman wrote:
> I think I've found a bug in INN 2.5.2 with the readers.conf perl_access
> access() returned hash value length.  I added a bunch of newsgroups
> today that pushed the character count in the 'read' key's value from
> 6456 characters to 8238 characters.  With a 'read' string that long, INN
> didn't work anymore for incoming connections.  Syslog reported the error
> "syntax error in (null)(2), Expected value." and terminated the
> connection.

The reason is that CONFgetword() in lib/conffile.c limits the size of
any single configuration line to BIG_BUFFER, which by default is 8192.
So, I believe a simple work-around would be to recompile INN with
BIG_BUFFER set to a larger value, like 16384, in include/inn/options.h

The rejection of longer configuration lines looks quite intentional in
the parser code. I'm not entirely sure about the new parser
(lib/confparse.c); it uses a buffer size of 8192 as well, but I think it
will resize the buffer if necessary, and it's also possible to continue
a string over multiple lines. So I think a proper fix would be to
convert readers.conf to the new parser, but I'm not sure if anybody
intends to go forward with that in the forseeable future...

Florian

Hi David,

Re: perl_access hash value length limit?

Florian Schlichting <fschlich <at> CIS.FU-Berlin.DE> writes:

> The rejection of longer configuration lines looks quite intentional in
> the parser code. I'm not entirely sure about the new parser
> (lib/confparse.c); it uses a buffer size of 8192 as well, but I think it
> will resize the buffer if necessary, and it's also possible to continue
> a string over multiple lines. So I think a proper fix would be to
> convert readers.conf to the new parser, but I'm not sure if anybody
> intends to go forward with that in the forseeable future...

Yup, the new parser has no arbitrary limits; the buffer there will be
resized as needed to get all of a token.

Switching to the new parser is unfortunately a fair bit of work, although
it would clean up a lot of nnrpd code.

--

-- 
Russ Allbery (rra <at> stanford.edu)             <http://www.eyrie.org/~eagle/>

    Please send questions to the list rather than mailing me directly.
     <http://www.eyrie.org/~eagle/faqs/questions.html> explains why.

[PATCH] improve wording of newgroup log message

Hi,

today, innreport found a log line with a slightly peculiar sounding
reason:

controlchan[29384]: skipping newgroup free.de.etc.selbsthilfe.angst y dk3uz <at> arrl.net (would no
change): Empty description

The below patch tries to address the matter.

Florian

--- a/control/modules/newgroup.pl
+++ b/control/modules/newgroup.pl
 <at>  <at>  -133,13 +133,13  <at>  <at>  sub control_newgroup {
             $status = 'be made moderated';
         } else {
             if ($ngdesc eq $olddesc) {
-                $status = 'no change';
+                $status = 'not change';
             } else {
                 $status = 'have a new description';
             }
         }
     } elsif (not $approved) {
-        $status = 'unapproved';
+        $status = 'be unapproved';
     } else {
         $status = 'be created';
     }

Some problems with himarks in active file

Good day, All!

What is the maximum possible number of himark/lowmark in active file?

I have a problem with numbers more than 2147483647 (31 bits, signed 
long, or something?)

inn-STABLE-20110119

Petr Novopashenniy

Re: [PATCH] improve wording of newgroup log message

Hi Florian,

> today, innreport found a log line with a slightly peculiar sounding 
> reason:
> 
> controlchan[29384]: skipping newgroup free.de.etc.selbsthilfe.angst y dk3uz <at> arrl.net (would no
change): Empty description
>
> The below patch tries to address the matter.

Yes, it does the trick.
Thanks for the patch.

I also patched rmgroup.pl because the same issue exists with "no change".
This way, the two files are now homogeneous.

--- rmgroup.pl	(r?vision 9112)
+++ rmgroup.pl	(copie de travail)
 <at>  <at>  -37,18 +37,18  <at>  <at> 
     close ACTIVE;
     my $status;
     if (not  <at> oldgroup) {
-        $status = 'no change';
+        $status = 'not change';
     } elsif (not $approved) {
-        $status = 'unapproved';
+        $status = 'be unapproved';
     } else {
-        $status = 'removed';
+        $status = 'be removed';

Re: Some problems with himarks in active file

Hi Petr,

> What is the maximum possible number of himark/lowmark in active file?
>
> I have a problem with numbers more than 2147483647 (31 bits, signed
> long, or something?)
>
> inn-STABLE-20110119

I'm afraid it is the maximum possible number…

See Section 6 of RFC 3977:

    Article numbers MUST lie between 1 and 2,147,483,647, inclusive.  The
    client and server MAY use leading zeroes in specifying article
    numbers but MUST NOT use more than 16 digits.  In some situations,
    the value zero replaces an article number to show some special
    situation.

    Note that it is likely that the article number limit of 2,147,483,647
    will be increased by a future revision or extension to this
    specification.  While servers MUST NOT send article numbers greater
    than this current limit, client and server developers are advised to
    use internal structures and datatypes capable of handling larger
    values in anticipation of such a change.

The protocol does not define (yet) a way to use higher numbers for 
numbering articles.
INN does not handle that, and I do not think it will be implemented 
soon.  We first need to specify how the extension to the protocol should 

Counting posts

Older versions of inn counts articles like this ("cumulative" - i don't
know good word for this):
--8<---------------cut here---------------start------------->8---
[...]

alfa 127.0.0.1 innd: localhost:54 checkpoint seconds 922 accepted 28000 refused 1011 rejected 0
duplicate 0 accepted size 46665780 duplicate size 0 rejected size 0
alfa 127.0.0.1 innd: localhost:54 checkpoint seconds 949 accepted 30000 refused 1070 rejected 0
duplicate 0 accepted size 49817392 duplicate size 0 rejected size 0
alfa 127.0.0.1 innd: localhost:54 checkpoint seconds 965 accepted 32000 refused 1116 rejected 0
duplicate 0 accepted size 53231320 duplicate size 0 rejected size 0
[..]
--8<---------------cut here---------------end--------------->8---

newer counts like this (counters are reset between checkpoints):
--8<---------------cut here---------------start------------->8---
[...]
alfa innd: localhost:54 checkpoint seconds 26 accepted 2000 refused 73 rejected 0 duplicate 0 accepted
size 3669584 duplicate size 0 rejected size 0
alfa innd: localhost:54 checkpoint seconds 28 accepted 2000 refused 66 rejected 0 duplicate 0 accepted
size 3292148 duplicate size 0 rejected size 0
alfa innd: localhost:54 checkpoint seconds 30 accepted 2000 refused 71 rejected 0 duplicate 0 accepted
size 3468856 duplicate size 0 rejected size 0
alfa innd: localhost:54 checkpoint seconds 23 accepted 2000 refused 46 rejected 0 duplicate 0 accepted
size 3360176 duplicate size 0 rejected size 0
[...]
--8<---------------cut here---------------end--------------->8---

Whether there any possiblity to diplay this info in older way?

Re: Counting posts

> Jan 20 02:34:26 news innd: news.ecp.fr status seconds 114 accepted 0 refused 1 rejected 0 duplicate 0
accepted size 0 duplicate size 0 rejected size 0
> Jan 20 02:44:27 news innd: news.ecp.fr status seconds 711 accepted 1 refused 3 rejected 0 duplicate 0
accepted size 7212 duplicate size 0 rejected size 0
> Jan 20 02:54:28 news innd: news.ecp.fr status seconds 1307 accepted 11 refused 5 rejected 0 duplicate 0
accepted size 33577 duplicate size 0 rejected size 0
> Jan 20 03:04:29 news innd: news.ecp.fr status seconds 1902 accepted 16 refused 14 rejected 0 duplicate 0
accepted size 51052 duplicate size 0 rejected size 0
> Jan 20 03:14:30 news innd: news.ecp.fr status seconds 2498 accepted 18 refused 14 rejected 0 duplicate 0
accepted size 65720 duplicate size 0 rejected size 0
> Jan 20 03:19:33 news innd: news.ecp.fr:757 checkpoint seconds 2828 accepted 18 refused 14 rejected 0
duplicate 0 accepted size 65720 duplicate size 0 rejected size 0
> Jan 20 03:24:31 news innd: news.ecp.fr status seconds 3094 accepted 19 refused 14 rejected 0 duplicate 0
accepted size 73420 duplicate size 0 rejected size 0
> 
> I think it is what you are asking for (just look at "status" lines
> instead of "checkpoint" lines).

And, to be complete, also look at "closed" lines, because you would
otherwise miss the final count!

Jan 20 03:25:31 news innd: news.ecp.fr:757 closed seconds 3154 accepted 19 refused 14 rejected 0
duplicate 0 accepted size 73420 duplicate size 0 rejected size 0

--

-- 
Julien ÉLIE

« J'aurais pu faire forêt comble. » (Astérix)

Re: Counting posts

Hi Kamil,

> Older versions of inn counts articles like this ("cumulative" - i don't
> know good word for this):
> --8<---------------cut here---------------start------------->8---
> [...]
> 
> alfa 127.0.0.1 innd: localhost:54 checkpoint seconds 922 accepted 28000 refused 1011 rejected 0
duplicate 0 accepted size 46665780 duplicate size 0 rejected size 0
> alfa 127.0.0.1 innd: localhost:54 checkpoint seconds 949 accepted 30000 refused 1070 rejected 0
duplicate 0 accepted size 49817392 duplicate size 0 rejected size 0
> alfa 127.0.0.1 innd: localhost:54 checkpoint seconds 965 accepted 32000 refused 1116 rejected 0
duplicate 0 accepted size 53231320 duplicate size 0 rejected size 0
> [..]
> --8<---------------cut here---------------end--------------->8---
> 
> 
> newer counts like this (counters are reset between checkpoints):
> --8<---------------cut here---------------start------------->8---
> [...]
> alfa innd: localhost:54 checkpoint seconds 26 accepted 2000 refused 73 rejected 0 duplicate 0 accepted
size 3669584 duplicate size 0 rejected size 0
> alfa innd: localhost:54 checkpoint seconds 28 accepted 2000 refused 66 rejected 0 duplicate 0 accepted
size 3292148 duplicate size 0 rejected size 0
> alfa innd: localhost:54 checkpoint seconds 30 accepted 2000 refused 71 rejected 0 duplicate 0 accepted
size 3468856 duplicate size 0 rejected size 0
> alfa innd: localhost:54 checkpoint seconds 23 accepted 2000 refused 46 rejected 0 duplicate 0 accepted
size 3360176 duplicate size 0 rejected size 0
> [...]
> --8<---------------cut here---------------end--------------->8---