Re: Opennet connection replacement policies

On Wed, Mar 31, 2010 at 6:22 PM, Matthew Toseland
<toad@...> wrote:
> On Wednesday 17 March 2010 18:33:01 Evan Daniel wrote:
>> Currently, when we need to drop an opennet connection, we use the LRU
>> (least recently used) policy: we drop the connection that has least
>> recently served a successful request.  Unfortunately, I can't find a
>> link to the paper that describes why LRU was chosen, though the basic
>> idea is obvious: if the node has too many peers around a certain
>> location, then those peers will get fewer requests, and therefore one
>> of them is more likely to be the peer dropped.
>>
>> I'm curious whether other algorithms were explored, and if so how they
>> performed in comparison.  The obvious candidates are LFU (least
>> frequently used), LFU-Window (LFU within a finite time window), and
>> LFU-aging (LFU but with periodic reduction in the frequency counts, so
>> that formerly useful nodes that are no longer useful eventually age
>> out).
>>
>> LFU-aging is normally described as having discrete aging steps at
>> (somewhat) large intervals, because it is normally discussed in the
>> context of a large cache where the O(n) time to perform the aging
>> computation is problematic.  However, we could do continuous
>> exponential aging at every request without any problems, because the
>> number of connections is small.  That is, every time a request
>> succeeds, we first multiply the score for each connection by some
>> constant alpha (0 < alpha <= 1), and then increment the score for the
>> connection that had the success.  (For 0 < alpha <= 0.5 this is
>> precisely equivalent to LRU; for alpha = 1, it is precisely equivalent
>> to (non-window) LFU.)
>>

Re: FNP changes and transport plugins

On Wednesday 24 March 2010 17:42:12 Evan Daniel wrote:
> The current FNP protocol leaves a bit to be desired.  It has more
> overhead than we'd like, it can only pad packets with random data (not
> useful bulk transfers), it doesn't handle small MTUs, and its
> congestion avoidance isn't as TCP-compatible as we'd like.
> 
> The most immediate problem is probably that it would make a transport
> plugin infrastructure overly complicated.  For example, a lot of
> interesting steganographic transports (VoIP, chat, and video game
> imitators, for example) have a low MTU.  That means the plugins would
> need to support packet fragmenting and reassembly, or add another
> layer between FNP and the transport plugins.  Neither solution is
> particularly desirable.
> 
> Toad and I have talked about this some on IRC.  I've written up a
> partial draft proposal that draws heavily on the old "New protocol"
> proposal, but makes some improvements (I hope ;) ).  You can find it
> at http://new-wiki.freenetproject.org/User:Evanbd/New_Packet_Format_Proposal
> .  Comments are very welcome, especially from anyone well versed in
> network theory.

The MAC may need to be lengthened, or to scale depending on packet size. There is some doubt about
hash-then-encrypt, I was told after much searching that an outer hash is better, but in any case, it needs
to not be computable by a snooper. If a short MAC is not sufficient to guarantee that the message has not been
tampered with then we need to consider alternatives for small packets such as MACing at a higher level. We
need to research this before deploying small MACs.

We discussed this extensively on IRC, evanbd will be writing up a new proposal soon. Denial of service
attacks are potentially a problem because fragmentation is initiated by the sender, and could use a
significant amount of memory if the sender is malicious. We can avoid this by setting (and publishing to

Re: Getting rid of load limiting

On Monday 29 March 2010 20:40:04 Robert Hailey wrote:
> 
> On Mar 24, 2010, at 2:24 PM, Matthew Toseland wrote:
> 
> > Currently our load management system works like this:
> > - At the level of any node, a request will be preferentially routed  
> > to the best node by location (or FOAF location), but if many nodes  
> > are backed off, we can route to the worst node by location. In other  
> > words there is no limit whatever on misrouting.
> > - At the level of the node originating requests, we attempt to  
> > estimate the capacity of the network in a manner similar to TCP  
> > (although more vague as we operate on rates rather than actual  
> > windows).
> >
> > Hence we do not limit misrouting, but we do limit load. And limiting  
> > load on the sender side is very expensive and inflexible, costing us  
> > a great deal of performance.
> 
> True, yet from a pragmatic perspective... is there really a way around  
> this? Network theory: you can control what you send but not what you  
> receive.
> 
> > Recently the CCN paper, and since long ago various people on Frost,  
> > have pointed out that we do not in fact need to limit load. We don't  
> > need to limit it at the sender side and arguably we don't need to do  
> > more at the any-node level than basic limiting of the number of  
> > requests in flight. This is because every time data is transferred  
> > it is the result of a request, so congestion in the sense that TCP/ 
> > IP deals with cannot really occur.
> 

Re: French wiki rss feed down

should be fixed; let me know if it isn't.

X

On 03/25/2010 03:37 PM, cvollet@... wrote:
> Hi,
> 
> It seems that since some weeks now, the rss feed of the french wiki doesn't
> work.
> I have admin rights, but I don't think I can do anything about it (plus, I'm
> in internship, and they've blocked the freenetproject.org domain).
> 
> The error is below : (translated from french, so it can sound weird)
> 
> 
> http://doc-fr.freenetproject.org/Sp%C3%A9cial:Modifications_r%C3%A9centes
> 
> Database error
> A syntax error in the request to the database happened. This can suggest a
> bug in the software. The last request treated by the database was :
> 
> ...
> 
> from the function ? SpecialRecentChanges::
> doMainQuery ?. MySQL sent the error 1146 : Table 'freenet_doc.tag_summary'
> doesn't exist (localhost) ?.
> 
> Can someone please fix this ?
> 
> I'm used to use the rss feed to spot spam and revert it (so I hope there

Re: GSoC 2010 --- developing a firefox plugin

On Tuesday 30 March 2010 19:35:28 Kyle Messner wrote:
> No need to apologize, thanks for the feedback! 

Evan is essentially right - while it would be possible to run Freenet as a browser plugin, it would be a very
heavyweight plugin, it would still need Java, and if it only ran when the browser was running it would have
very low uptime, which would make life rather difficult for the network. I comment on other things below...

> I took a look at the idea 
> list, and I'm very interested in implementing at least one (if not several,
> time permitting) of the client improvement ideas. My top priority would be
> improving the web interface generally and helping to improve the content
> filters. 

You are allowed to make multiple applications to the same project (Freenet) for different tasks (web
interface cleanup, content filters, etc).

There is plenty of work to do in both areas!

> That is, I'd make improving the web interface my main goal, and if 
> I have time I'd like to help contribute to at least one or two content
> filters. I feel that a clean, streamlined, bug-free interface is one of the
> most important things to have when trying to get users interested in a piece
> of software and keep them interested. It's also something that I believe
> would be well within my ability to help implement while still presenting a
> challenge and helping to further my programming abilities. I also feel like
> it would be a great way to get my feet wet in the Freenet project, and
> perhaps after GSoC it would allow me to move on try to contribute to other
> various parts of the Freenet code.

I suggest you install Freenet to get an idea of the current web interface. You might like to try out the major

Re: [question:] A microblogging and/or real-time chat system of GSOC

On Friday 26 March 2010 19:49:56 Evan Daniel wrote:
> On Fri, Mar 26, 2010 at 2:28 PM, 陈天一 <chentianyi87 <at> gmail.com> wrote:
> > Hi,
> >
> > I am interested in A microblogging and/or real-time chat system of GSOC and
> > the description said that a fair amount of work on how to efficiently
> > implement microblogging over Freenet have been done. I am wordering that if
> > I want to join this project, I should build a independent use any technology
> > and framework I want such as Struts + Spring + Hibernate or I should
> > embedded the microblogging in to the Freenet code as the whole system. And I
> > think Freenet have had strong ability in file distribute so that add
> > real-time chat system is really easy for you because you only need to
> > connect others by knowing their IP address. Could give me a short
> > description about what is the objective of this project and what kind of
> > technology you really want? I am now builiding by own SNS website by using
> > Struts + Spring + Hibernate and Ajax and have some experience about file
> > download software with multithreading control. Could you tell me what is the
> > most different between this project and my provious projects?
> >
> > By the way, do I need to design the entity, relationship and attribute for
> > database by meself? What database you use? (I have MySQL and Oracle 10g
> > experience)?
> 
> I'll let someone else speak to the gsoc issues; here's a brief summary
> of the technical problems.
> 
> The first thing to realize about any sort of microblogging or chat
> application on Freenet is that it will be *very* different from a
> normal web app on the back end.  (The UI end can and should be as
> similar as possible, though.)  First, there is no central server to

Re: git tree weirdness in WoT\

On Tuesday 30 March 2010 23:38:00 David ‘Bombe’ Roden wrote:
> On Tuesday 30 March 2010 12:35:00 Ximin Luo wrote:
> 
> > > No, it will not. If work on the branch continues (which is unlikely
> > > because it has been merged so further work would happen on the master
> > > branch) I would just merge master again. Simple and beautiful.
> > 
> > What I meant by weird was it'll look like this:
> > 
> > -o---o       o---o <- master
> >       \     /
> >        \   /
> >         \ /
> > -o---o---o---o---o <- identicon
> > 
> > which looks like master was pulled into identicon then branched off again.
> > Which is basically what has happened here without --no-ff.
> 
> No, it doesn’t. The situation looked kind of like this:
> 
> -o-o-o-------o <- master
>       \       \
>        o-o-o---o <- bombe/identicon
> 
> After merging the identicon branch into master it looks like:
> 
> -o-o-o-------o
>       \       \
>        o-o-o---o <- master / bombe/identicon
> 

Re: [GSoC 2010] - Transport plug-ins

On Tuesday 30 March 2010 12:07:21 Pratibha Sundar wrote:
> Hi,
> 
> I'm interested in the Transport plug-ins project. Please tell me whom to
> contact for further information.

There is a fair amount of information on the 2 wiki's (wiki.freenetproject.org and
new-wiki.freenetproject.org) and the bug tracker. Apart from that, you'll have to look at the source
code. Note that this particular project is likely to be quite a challenging project, and you should aim to
submit at least 2 proposals to Freenet for Summer of Code.

_______________________________________________
Devl mailing list
Devl@...
http://osprey.vm.bytemark.co.uk/cgi-bin/mailman/listinfo/devl

Re: responses to WoT/search questions

On Wednesday 31 March 2010 14:16:11 Evan Daniel wrote:
> On Wed, Mar 31, 2010 at 5:42 AM, xor <xor@...> wrote:
> > On Wednesday 31 March 2010 06:32:58 am Evan Daniel wrote:
> >> On Wed, Mar 31, 2010 at 12:22 AM, Ximin Luo <xl269@...> wrote:
> >> > have you joined the freenet-dev mailing list? in future i'd like to have
> >> > these discussions there so that other people can see it too.
> >> >
> >> > (03:53:26) lusha: hi, can i ask a question about WOT?
> >> > (03:55:33) evanbd: No need to ask permission :)
> >> > (03:56:12) lusha: is there any document for this?
> >> > (03:56:32) lusha: i dont quite understand how they evaluate trust
> >> >
> >> > (i think) WoT uses a flow-based metric similar to advogato
> >> > (www.advogato.org) - see the source code (plugin-WoT-staging), or ask p0s
> >> > on IRC (xor on the mailing list) for specific details. atm the
> >> > implementation requires retrieving trust scores for everyone on the
> >> > network, which won't scale in the long run.
> >>
> >> No.  The current WoT code is neither flow-based nor particularly
> >> related to the Advogato algorithm.  It's purely alchemical, having
> >> neither a proper specification as to the problem being solved nor any
> >> sort of theoretical basis to believe it solves that unspecified
> >> problem.
> >
> > That is true. I should finally do this and answer to your mail w.r.t. to your
> > prosed alternative algorithm. I'm sorry,        I'm just trying to make everyone
> > happy. People want a release of FT/WoT soon so as long as I didn't have much
> > time/day I was trying to spend it on writing code.
> >
> > BUT we should also state that the algorithm itself fortunately is only a small

Re: French wiki rss feed down

Le jeudi 01 avril 2010 20:05:11, Ximin Luo a écrit :
> should be fixed; let me know if it isn't.
> 
> X
> 

It works. Thanks :)