headers
Andrew Rodland | 1 Aug 2003 02:15

Re: Re: Manifest Not Found

FillaMent@... wrote:

>Manifest Not Found: This would be informative for advanced mode Fproxy - having some way of indicating
what it is we're DNF'ing on, being the the base redirect, the manifest, or the default doc.\n\n-Fill
>  
>
Strongly agree. Can we see this?

BTW, Filla, \n in anonymail does not do what you think it does. :)

--hobbs
Permalink | Reply | 501 comments |
headers
Niklas Bergh | 1 Aug 2003 10:01

null Stream from buffer

With build 6120

I get these about once an hour.. Note that the call stack always is exactly
the same (the node is always handling the state
'freenet.node.states.request.TransferInsertPending')

/N

Jul 30, 2003 5:21:53 AM (freenet.node.Node, QThread-2615, ERROR): Error
while receiving message freenet.Message: Accepted
 <at> freenet.ConnectionHandler <at> 11d402d for tcp/connection:
12.225.117.108:1419,freenet.transport.tcpConnection <at> 98b89e,freenet.session.F
npLink <at> 1606078  <at>  40d3af22eea706a2 in state
freenet.node.states.request.TransferInsertPending:
key=12a5a4ac8df28fac4a591d1a106b9b56d2fd23ae0f0203, hopsToLive=5,
id=40d3af22eea706a2,ft=freenet.node.states.FNP.FNPFeedbackToken <at> 98f6d4 <at> 10595
35313781, routedTime=1059535310765: java.lang.IllegalStateException: null
stream from buffer
java.lang.IllegalStateException: null stream from buffer
 at
freenet.node.ds.FSDataStoreElement$KeyInputStreamImpl.<init>(FSDataStoreElem
ent.java:237)
 at
freenet.node.ds.FSDataStoreElement.getKeyInputStream(FSDataStoreElement.java
:46)
 at
freenet.node.ds.FSDataStoreElement$KeyOutputStreamImpl.getKeyInputStream(FSD
ataStoreElement.java:137)
 at
freenet.node.states.data.ReceiveData.getKeyInputStream(ReceiveData.java:69)
(Continue reading)

Permalink | Reply | 0 comments |
headers
Niklas Bergh | 1 Aug 2003 10:07

message sending failed while more messages was queued

With build 6120

I am seeing huge amounts of these. Could we say that it is a normal
behaviour instead of an error?

ug 1, 2003 7:17:32 AM (freenet.ConnectionHandler,  write interface thread,
ERROR): sending a message failed, and the queue had 3 messages enqueueued.
Too bad!
Aug 1, 2003 7:17:40 AM (freenet.ConnectionHandler,  write interface thread,
ERROR): sending a message failed, and the queue had 58 messages enqueueued.
Too bad!
Aug 1, 2003 7:23:39 AM (freenet.ConnectionHandler,  write interface thread,
ERROR): sending a message failed, and the queue had 2 messages enqueueued.
Too bad!
Aug 1, 2003 7:24:19 AM (freenet.ConnectionHandler,  write interface thread,
ERROR): sending a message failed, and the queue had 199 messages enqueueued.
Too bad!
Aug 1, 2003 7:26:47 AM (freenet.ConnectionHandler,  write interface thread,
ERROR): sending a message failed, and the queue had 0 messages enqueueued.
Too bad!
Aug 1, 2003 7:30:11 AM (freenet.ConnectionHandler,  write interface thread,
ERROR): sending a message failed, and the queue had 17 messages enqueueued.
Too bad!
Aug 1, 2003 7:30:19 AM (freenet.ConnectionHandler,  write interface thread,
ERROR): sending a message failed, and the queue had 86 messages enqueueued.
Too bad!
Aug 1, 2003 7:30:54 AM (freenet.ConnectionHandler,  write interface thread,
ERROR): sending a message failed, and the queue had 4 messages enqueueued.
Too bad!
Aug 1, 2003 7:31:58 AM (freenet.ConnectionHandler,  write interface thread,
(Continue reading)

Permalink | Reply | 0 comments |
headers
pineapple | 1 Aug 2003 10:56
Picon
Favicon

possible node DOS?

Houston, we HAVE a problem :)  Looks like someone may
have found an exploit to render a node useless.  Right
now my node isn't doing much of anything because there
are too many active pooled jobs.  Looks like most of
them are for IP 80.108.88.116.  Here is the thread
hierarchy from my node (oh, and don't ask for a
logfile, I had to turn logging off because my node
keeps spamming messages for bug id 4729342; thank you
mr. node, but I don't need to be told 543619 times
about this!)

__________________________________
Do you Yahoo!?
Yahoo! SiteBuilder - Free, easy-to-use web site design software
http://sitebuilder.yahoo.com
Attachment (hierarchy.zip): application/x-zip-compressed, 3717 bytes
Permalink | Reply | 1 comment |
headers
Gordan | 1 Aug 2003 11:14

Re: To zip or not to zip

On Thursday 31 Jul 2003 11:57, panamerica334@... wrote:
> >I understand where you are coming from, and at the moment I am fairly
> > evenly split between considering zip vs bzip2.
>
> please try to stick to plain stupid zip if you have to.
> not only because it's been around with the jre for some time, so we can
> assume it's mostly bugfree, but mainly because i worry for 3rd party
> freenet nodes.

It would not lead to 3rd party nodes at all. Because the compression is the 
matter of a client-side protocol, in order to extract the data from the 
network, all clients would have to support the (de)compression algorithm. 
Therefore either everybody uses it or nobody uses it. There is little scope 
for compromise inbetween because all compressed files will not work for the 
people who don't have the node with the codec. So, IMHO, it comes down to the 
choice between better compression, and the convenience of a built-in library.

Gordan
Permalink | Reply | 0 comments |
headers
Niklas Bergh | 1 Aug 2003 11:37

Re: possible node DOS?

> Houston, we HAVE a problem :)  Looks like someone may
> have found an exploit to render a node useless.  Right
> now my node isn't doing much of anything because there
> are too many active pooled jobs.  Looks like most of
> them are for IP 80.108.88.116.  Here is the thread
> hierarchy from my node (oh, and don't ask for a
> logfile, I had to turn logging off because my node
> keeps spamming messages for bug id 4729342; thank you
> mr. node, but I don't need to be told 543619 times
> about this!)

543619 times you say. To me this indicates that your node is non-functional.
Could you give me an exact example of one log entries? It *might* be so that
your node is unable to send any data at all (due to that particular bug),
thereby causing all those threads to stand there waiting for jsut that to
happen...

/N
Permalink | Reply | 0 comments |
headers
Gordan | 1 Aug 2003 11:34

Re: To zip or not to zip

On Thursday 31 Jul 2003 18:55, Tom Kaitchuck wrote:

> The entire directory:		2617995
> The tar of the directory:	2703360
> The zip of the directory:	583382	Ratio: 4.48 to 1
> Gzip of the tar (default):	497157	Ratio: 5.26 to 1 (compaired to the dir
> size) Gzip of the tar (--best):	492510	Ratio: 5.31 to 1 (compaired to the
> dir size) Bzip2 of the tar:		437351	Ratio: 5.98 to 1 (compaired to the dir
> size)

> This means, that if we assume this compression ratio, on a hypothetical
> size index, Bzip will result in enough improvement to move to the next
> power of 2 size 3/8ths of the time. Bottom line: On Freesites that are
> using HTML containers that have between 4KB and 4MB of uncompressed
> content,  Bzips will only use 80% of bandwidth than zips. Please bare in
> mind that Zips are already reducing this bandwidth to about 23% of what it
> once was, and that space wise, this is a small part of Freenet's content.

Moving down to the next power of two yields 50% reduction in space. There is 
no inbetween. So, if bzip2 is 20% smaller than zip on average, given the 
powers of 2 distribution of file sizes, how often will that yield that 50% 
improvement?

> Is is taking this number down to 19% worth all the extra effort it would
> take?

It depends on how often that would result in reducing the space usage by a 
notch. And I am not all that concerned that the effort required to use one 
compression library instead of another is that great.
(Continue reading)

Permalink | Reply | 12 comments |
headers
Niklas Bergh | 1 Aug 2003 11:56

Re: Re: [freenet-support] Routing table

> > > Primarily the fact that the graphical representation of it (under
> "Routing
> > > table" or on the "Node Reference Status" page), instead of looking
nice
> > and
> > > healthy with plenty of routing 'knowledge' (resembling a barcode),
> starts
> > off
> > > almost completely white with just a handful of thin stripes. Over a
day
> or
> > > so, coverage increases.
> >
> > Same over here.... Cannot tell for sure that it happens every time I
> restart
> > but I definitely has seen the 'barcode' cleared when the node is freshly
> > started.
>
> I can now say that it definitely not happens every time...We'll see if I
can
> provoke it to happen again..

One thing that happens though is that is seems to appear a lot of new (old
really) nodes in the routing table during a restart. After a restart after
10 hrs of uptime the version histogram looks something like:
Fred,0.4,1.46,522  |=
Fred,0.5,1.46,5007 |=
Fred,0.5,1.46,5009 |=
Fred,0.5,1.46,5010 |==
Fred,0.5,1.46,5012 |==
(Continue reading)

Permalink | Reply | 4 comments |
headers
Gordan | 1 Aug 2003 12:04

Re: To zip or not to zip

On Thursday 31 Jul 2003 21:39, Menno Jonkers wrote:

> I don't think it'll be worth it. Stick to the compression algorithm
> that's readily available in Java. Focus the scarce development resources
> on things that can really make a difference, e.g. routing. Once the core
> functionality is okay, you can start looking at these kind of
> optimizations.

The probem with "looking at these kind of optimizations" is that every time 
the compression algorithm is changed, the network will break for all the 
people who have nodes that don't support it. This means that there would have 
to be a potentially rather long transition period where both methods would 
have to be supported, at least on the decompression side. But even if the new 
nodes are restricted to zip instead of bzip2, the old nodes that haven't been 
updated will find the content compressed with the new algorithm unreadable.

This effectively means it would require all the nodes to be updated and all 
the data to be re-inserted, effectively tearing down the network and starting 
from scratch.

Granted, with a long transition period, it would probably work out without too 
much disruption, but at the same time, I must say that I am in favour of 
getting it right the first time, rather than having to either fix things 
later at an increased cost or just putting up with the sub-optimal 
performance.

Gordan
Permalink | Reply | 2 comments |
headers
panamerica334-hX/r/F0l+Dw@public.gmane.org | 1 Aug 2003 12:10
Picon
Favicon

Re: To zip or not to zip

>> >I understand where you are coming from, and at the moment I am fairly
>> > evenly split between considering zip vs bzip2.
>>
>> please try to stick to plain stupid zip if you have to.
>> not only because it's been around with the jre for some time, so we can
>> assume it's mostly bugfree, but mainly because i worry for 3rd party
>> freenet nodes.
>
>It would not lead to 3rd party nodes at all. Because the compression is the 
>matter of a client-side protocol, in order to extract the data from the 
>network, all clients would have to support the (de)compression algorithm. 
>Therefore either everybody uses it or nobody uses it. There is little scope 
>for compromise inbetween because all compressed files will not work for the 
>people who don't have the node with the codec. So, IMHO, it comes down to the 
>choice between better compression, and the convenience of a built-in library.
>
>Gordan

>It's not in the node. It's in the client code. Which can work over FCP.
>So you can reimplement the node without reimplementing the client code.
>So it's no big deal.
[Toad]

then perhaps i misunderstood the approach.

i believed the container compression has been cleared by introducing the code from fish; supporting jar
and standard zip files?

by the way, you are using jar-specific classes to access the container ingredients:
+	    JarInputStream myJis=new JarInputStream(myIs);
(Continue reading)

Permalink | Reply | 26 comments |

Gmane