Re: Bad performance. Something wrong about backoff? darknet.
Matthew Toseland <
toad@...>
2013-05-26 23:18:11 GMT
liberty-test-of-jFniki <at> fUGgvPvBMLTdR2C7rk8G-Om9xfMEcLgaQyKimk2pXvU wrote:
> FuckSárközi <at> lJpyCPT1hcDMA9~Lfa6LoGDqfH7ymInRuaqwH6CZwAE wrote :
>> oo <at> lkXpu0~CDV6dh0Idyw4MBwkUSgn~h~Bs3qqVXYOXSaY wrote :
>>> toad-notrust <at> h2RzPS4fEzP0zU43GAfEgxqK2Y55~kEUNR01cWvYApI wrote:
>>>
>>>>> We are getting downloads stale for days at 0 blocks. There is some bug
>>>>> in the wakeup mechanism.
>>>
>>> Yes, there is such a bug.
>>> <toad-mode on>Most likely<toad-mode off> once stalled all your downloads
>>> will proceed again if you change priority of an arbitrary download for a
>>> few seconds.
>>> I am anxious to read how toad will integrate this into his paranoid
>>> imagination that my patches are responsible for everything evil wrt
>>> freenet and the universe.
Okay how about this:
There are reports of downloads taking a lot longer than people had hoped.
Some have suggested this is "stalling".
"Stalling" in the most literal sense, as I have previously debugged, would mean that particular requests
are not getting selected AT ALL.
The patch by oo's sole function is to increase the frequency of *trying to send requests*. It does not affect
the client layer, where such stall supposedly would be. It does not affect the cooldown or RecentlyFailed
logic. All it does is run the same code more frequently; this still has to get past e.g. local load limiting,
and it still won't send requests for keys that are in cooldown.
Yet this patch is universally praised as solving the problem.
Conclusion? Requests are not stalling. Requests are being sent "too slowly" compared to some arbitrary
expectation - maybe even compared to past performance. But they are still being sent.
Having said that, there does seem to be an issue with bogus RecentlyFailed's, which needs looking into.
>>>
>>>> Perhaps there is a problem with the AIMD's. Your solution is to ignore
>>>> the AIMD's. Most likely the problem in fact is your patch: Nodes
>>>> running your patch are DoS'ing the network and the polite part of the
>>>> network throttles itself as a result.
>>>>
>>>> This is what comes from not fixing the incentives. Unfortunately, just
>>>> like on the broader internet, building systems that are perfect
>>>> incentives-wise is hard.
>>>
>>> This is what comes from not verifying your assumptions.
>>> You should date your therapist to talk about your paranoia.
>>> Did you ever notice that almost all of your reasonings are based upon
>>> "..probably..., most likely..., I assume,..." just to stay compatible
>>> with your biased view of the world?
>>> Does it make life easier for you?
I'm not the one that's calling for us to throw out load management principles based directly on TCP and
replace them with ... what? The hope that it won't cause backoff everywhere? I don't see what your
manifesto is. oo's patch, by effectively turning off sender-side load management, would increase load,
and there is no reason to think this wouldn't be catastrophic. It's the equivalent of a tweaked TCP stack
saying "damn congestion control, I'm just gonna send packets as fast as I can". There's a reason that real
TCP stack don't do that.
>>>
>>> I don't think so as you fear to verify your assumptions like hell. Even
>>> if they have been proven to be wrong (after release of months of coding
>>> that unfortunatly didn't work as expected) all you need to commence
>>> something completely different is "Oh well, there are too many unknown
>>> parameters...", always leaving broken garbage behind.
Months of coding? There really haven't been that many significant load-related changes over the last 18
months AFAIK. NLM was revoked. The period over which Freenet's performance is supposed to have collapsed
coincides with a period in which my productivity has been substantially reduced (see below). So it's not
due to my tinkering. Which means it either doesn't exist, or is due to something else.
"There are too many unknown parameters": I sometimes use this phrase to explain why we can't get an accurate
analysis of the impact of one build just by measuring insert/request performance: There is so much
background noise that only a really big change will have a clear effect. That's not just my view, it's
digger3's view too (and he maintains the stats). However, we can add specific stats to measure whether
specific changes do what they are supposed to do.
>>>
>>> It is beyond my imagination that *you* are going to study mathematics,
>>> that will be really interesting to follow.
FYI I'm not going to be doing maths at uni (although the course will include some). I have a conditional offer
at Cambridge for CS. I need to get an A in my Further Mathematics A-level in order to meet that offer.
>>
>> Thats the fun part, the funny story about going back to school and trying
>> to learn new stuff. Lot of places have popped up that try to reason about
>> this new academics endeavor and people strongly agree and come to the
>> same conclusions as you. Basic gist is: you cant teach an old dog new
>> tricks. Toad has for whatever reasons not opted in to acquiring skills
>> and a good mental health in early and young years but now as an elderly
>> person as it seems from pictures at least, tries to make up for it. Maybe
>> you cant blame him or whatever, maybe his religious attitude kept him
>> away from science many years back. Whatever it was, it is not healthy
>> that the Freenet userbase is this strongly relying on a person with such
>> behavioral aspects and attitude as he is expressing it all over the time.
ROFLMAO. There are no recent pictures of me anywhere on the internet AFAIK, including Freenet. And I'm not
so proud as to think that I know everything, unlike certain people here. Nor am I so foolish to think that I'm
incapable of learning anything new at the grand old age of 32. That's simply not true, for 3 reasons:
1. I have been successful so far in my recent education.
2. This week's New Scientist has a detailed article debunking the common fallacy that older people can't
learn.
3. There are lots of mature students at Cambridge (and other prestigious universities too), and many of
them do very well. Granted most "mature students" are early 20s, but many are much older, and still go on to
get good degrees. They offered me a place, so I am confident that I can do well: I trust their judgement on that.
>
> I do not agree with your negative attitude.
>
> Stop stupid illogical personal attacks.
>
> We should be thankful for his dedication and work, even if it was not
> always best.
Thank you. (And yes, I've made many mistakes. So has everyone else).
>
> I hope things are turning around, and I think this is anyway very big
> progress for such an epic project with basically 1 person.
I am hoping to make substantial progress after my exams are over, that is from June 22nd to the beginning of
October. It will be easier the more volunteers help.
>
>> He is maybe also a candidate for a psyciatrist or something, although
>> those kind of professions fuck you up even more if you dont bring in a
>> solid personality to the game.
>>
>> Freenet is taking literally decades to progress anywhere. And "where" its
>> progressing to thats still not clear.
I have documented most of my medium term goals for Freenet in some detail in various forums - bug trackers,
IRC logs, mailing lists. There is a reasonably clear vision. Granted, none of these things is really an
acceptable form of documentation. However the near term priorities should be IMHO:
- Client layer: Get rid of db4o, and replace it with something more robust and faster.
- Darknet: Make it easy, fast, and secure.
- Tunneling: Darknet-based tunnels could provide us with some fairly impressive security for inserts.
- Load management. We've discussed this. The current TCP-based model is wrong, but removing one section of
it without looking at the rest doesn't work. Changing the other areas as well, so that "too many requests"
doesn't result in "backed off hell", may work well.
>
> That is correct.
>
> So we are here to help develop Freenet and give more freedom to people.
>
> Instead just bitching you could give us your time, or money, or something.
>
> I would guess that you or your friends are at creating some patches, this
> is very good and I'm sure we will get around to use them. Same as maxpeers
> where finally applied :)
>
> Please help us with constructive work to improve freenet ok?
Yes, the main reason Freenet doesn't make rapid progress is because we have limited person-power. We
simply need more developers. Money is one way to do that, but the best way is to get more volunteers.
_______________________________________________
Devl mailing list
Devl@...
https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl
RSS Feed