Vaibhav Gupta | 30 Sep 13:13 2014
Steve Dougherty | 27 Sep 14:33 2014

Coding and commit message standards in effect

The coding standards documented in the wiki [0] apply to incoming code
and commit messages. Pull requests that do not meet them are likely to
be rejected.


Devl mailing list
Steve Dougherty | 25 Sep 03:25 2014

Your release manager is overwhelmed

Hi everyone,

I have too many tasks to do to be able complete them in the way I've
been approaching things thus far. The sheer magnitude of code I feel
obligated to review on top of other improvements I'd like to write for
1466 makes this feel more like endless unpaid work than fun volunteering.

I will try making a list of tasks to complete and going through them in
a more focused way instead of jumping around, and hopefully that will
help, but ultimately I think I need to have less fall to me for this to
be sustainable. One thing I would really appreciate is other people
reviewing and signing off on pull requests so that I can focus on other
aspects of releasing, at least for now. The purge-db4o pull request
alone (and there are others) changes ~100k LOC. This is more code than
I've ever reviewed at once, and I don't think I can do a good job on
reviewing all of it by myself in a timely manner.

Is anyone willing to volunteer to help review code? When reviewers are
happy with a pull request (which I hope will include following the
coding standards) I will merge it.

- Steve

Devl mailing list
Matthew Toseland | 19 Sep 15:39 2014

Purge-db4o release candidate (snapshot 19)

The client layer rewrite aka purge-db4o is finished. That is, it is
feature complete, and has had a reasonable amount of testing (but needs

Major changes:
- No longer uses db4o to store anything, but will migrate from old
node.db4o*'s (downloads and uploads will be restarted)
- Should be much more reliable, and cause much less disk I/O
- Uses the multi-container code even for persistent inserts
- Improved disk crypto (I may improve this further)
- Trivial changes to FCP
- New metadata format (fixes some minor bugs, but we use the old one for
now by default)
- Handles low disk space much better. Migration may use significant disk
space; if we have less than 512MB left, transient requests will fail to
avoid breaking the system, if we have less than 1GB left, persistent
requests will fail / not start.
- Various bugfixes
- Rename RandomAccessThing to RandomAccessBuffer, etc.

The latest version is here:
Branch "purge-db4o-crypt"
Signed tag "purge-db4o-snapshot-19"
Commit hash 9b7c9204fa5461ddbfa3236f81e257e379dc86eb

Uploaded jar and signature:
CHK <at> WiSJI8wGPKMZ4KJhxO7EHzd3k4OtJBmay95IQ2dUbS0,4ChtKmFXtb5EOegUgQQZ85EWpijCdUyKubelVP9AOU0,AAMC--8/freenet.purge-db4o-snapshot-19.jar
CHK <at> FwVHzsL8zuRv6cJix2B14xm~8fH2EZsO68nyggIwoqQ,P0etmFePAHcKmgBKJTtnFckxQ7ZX97qM4Wg43bokPus,AAMC--8/freenet.purge-db4o-snapshot-19.jar.sig
(Continue reading)

vmonmoonshine | 17 Sep 20:22 2014

Job opportunity: looking for a freenet dev and more

Hello Freenet folks,

We're starting a project called CeNo, based on the idea I shared with a
few of you at last year's CTS in Berlin where we had the freenet meet
up. Its about using freenet to cache censored web pages

The company where I work secured funding for a year and is
currently looking for a dev to work (alongside me!) on the project. If
the potential applicant has  extensive software dev experience with
other languages as well and is interested in other projects, there is a
good chance to get a full-time position here. If you cannot apply but
are still interested in contributing to CeNo! by sharing your thoughts
and ideas on it, please join the mailing

And here's the job:

I think is a pretty good team to work with. We concentrate on
 digital security for civil society and develop free software to defend
 online freedom  speech and other aspect of human rights. So it is the
 prefect place for those of you who have the activism itch.

Looking forward to work with you, 
Matthew Toseland | 16 Sep 19:42 2014

Should we remove disk crypto?

On 05/09/14 19:43,
localghost <at> eOC4Zm8KjRpMFhNBp6DmI8K4URaq8bQZH45y0dLHEnI wrote:>
creamsoda <at> 0vpcRHZV1ftyj4mJpZnuYaG8wpkNIvf3qa3b-LUcsZs wrote :
>> TAILS is meant to be for short-lived sessions of minutes to hours
right? That doesn't lend itself well to freenet which works better over
a longer time.
> Ya, I kinda figured that.. just curious if anybody had done it. I was
asking for a friend, as stated. I prefer to let Freenet run constantly,
to help the network- and have not used TAILS, personally..
This is exactly why "leave disk crypto to the operating system" isn't so
obviously the right policy for Freenet.

The arguments for not doing disk crypto:
- We're likely to get it wrong.
- If they download video files etc there will probably be leaks. (But it
IS possible to limit this)
- Memorising a good password is hard, and the users who are willing to
do so may be the same group as the users who will install a secure linux
distro just to run Freenet, or at least do full disk encryption on
Windows (presumably using BitLocker?)
- If we do disk crypto we need to turn on swap encryption. This is
trivial on recent Windows but arguably not a good idea.

The arguments for doing disk crypto:
- We want people to run Freenet long-term. They usually won't install a
new OS just to run Freenet, and they can't run it long-term from a
livecd. This is one of the reasons we support Windows!
(Continue reading)

Steve Dougherty | 16 Sep 05:42 2014

[RFC] Update channels

Update channels are a way to provide easier testing of development
versions, make it easier for unofficial builds to be distributed, and
can also enhance build security by allowing for multiple signatures with
offline keys.

## Channel definition

A channel definition describes an update channel: both the user-facing
information and where to fetch it. It also contains security measures
like a revocation key, a list of trusted key IDs, and the number of
valid signatures required to deploy. This number can be zero if trusting
the insert key is deemed sufficient.

It is a key-value list of:

* name
* user-facing description
* trusted key IDs
* number of valid signatures from trusted keys required to deploy
* channel revocation SSK

Depending on whether the definition is distributed with a build, the
name and description could either be literal or localization keys. Those
distributed with builds are copied to an `update_channels` directory on
the filesystem and used from there to allow for clearer operation and
have it make more sense to drop additional channel definitions in the

Trusted keys are exciting because although the update insert key must
(Continue reading)

Matthew Toseland | 11 Sep 23:42 2014

New symmetric crypto API issues

In order to implement disk encryption in purge-db4o, I'm reviewing one
of unixninja's crypto refactoring commits. This includes disk crypto but
also some important crypto infrastructure providing a single hopefully
clean API for our various different ciphers etc. I haven't looked at his
other commits, so I don't know whether something else was intended ...

Lots of the code abuses ByteBuffer.array(), making very bad assumptions
about the ByteBuffer's passed in etc (not direct, not a slice,
arrayOffset() == 0, etc etc); this is fixed. (No criticism intended, NIO
is odd)

CryptByteBuffer implements symmetric encryption:

The main methods are:
public ByteBuffer encrypt(byte[] input, int offset, int len)
public ByteBuffer decrypt(byte[] input, int offset, int len)

There are various wrappers e.g. ByteBuffer encrypt(ByteBuffer). This is
highly ambiguous: Lots of NIO code takes a ByteBuffer and returns the
same ByteBuffer. Here we are returning a different ByteBuffer. The
original code wasn't clear whether it was always a new buffer or
sometimes shared the array, anyway we have 3 requirements:
1. The API must be unambiguous. Ambiguity -> very bad bugs, e.g. stuff
not getting encrypted. Hence e.g. we should not return a value if we are
encrypting in place; this is convenient but ambiguous.
2. It needs to efficiently support encrypting in-place for big stuff:
Whole packets, whole CHKs.
3. It needs to conveniently support small encryptions. These happen in
e.g. setting up an EncryptedRandomAccessThing (new class used for
encrypted random access tempfiles).
(Continue reading)

Matthew Toseland | 10 Sep 16:46 2014

Please test purge-db4o pre6

This is nearly done now. The only thing that isn't working is disk
encryption (physical security level of NORMAL or higher). Please test,
especially if on Windows!

Please test! You need to:
- Switch to LOW physical security, or turn off "encrypt temporary
buckets" and "encrypt persistent temporary buckets" in advanced core config.
- Shutdown the node
- Backup master.keys, node.db4o/node.db4o.crypt and persistent-temp*/
- Download the jar.
- Check the signature if possible.
- Replace freenet.jar with the downloaded jar.
- Change wrapper.conf to point to freenet.jar if it's currently
referring to
- Start up Freenet.

It should automatically migrate your downloads and uploads.

It is not compatible with the previous test build due to some refactoring.

Please test, and report bugs! Thanks!


CHK <at> Q8lin1CHHh-xB5u4BDWASGvx72xscrpvNTl7fnWBsnc,Vl7u9WkjoCXFhhl3i7b3NJUZdvWvX2ZUWQzHE5x53oU,AAMC--8/freenet.purge-db4o-snapshot-6.jar

CHK <at> XQpzloFhnVLPHTPDMiDZL8yVK4vkji~XGe0bBx~uezQ,WeOeLqiMx0mt77kxo3T4vANhFoR3Xt-tuHrBYBgAhjo,AAMC--8/freenet.purge-db4o-snapshot-6.jar.sig

(Continue reading)

Matthew Toseland | 5 Sep 19:34 2014

Please don't deploy the Java 7 stuff yet

It looks like somebody messed up and removed a close() which was before
a rename, probably in BaseFileBucket. Such a bug could break Windows
nodes very badly, so please be very careful reviewing the change before

[18:11:18] <TheSeeker> toad_: looks like the plugin data saving bit is
not finishing the temp file rename (trying to rename while open?)
[18:11:28] <toad_> TheSeeker: ouch
[18:11:32] <toad_> TheSeeker: not a regression mind you
[18:11:34] <TheSeeker> I have files like:
plugins.floghelper.FlogHelper.data1029365099476005573.freenet-tmp  in my
plugin-data dir.
[18:11:36] <toad_> TheSeeker: always happens on Windows?
[18:11:44] <TheSeeker> that didn't used to happen :P
[18:11:53] <toad_> TheSeeker: hmmm ... it could have broken sometime?
[18:12:18] <TheSeeker> sometime during purge-db4o yes.
[18:12:33] <toad_> TheSeeker: I haven't done anything to that code :|
[18:12:37] <toad_> TheSeeker: is this Windows?
[18:15:20] <TheSeeker> toad_: yes, windows.  likely something to do with
a change I saw somewhere assuming that java will always close things
properly?  (so maybe on next, not purge-db4o?)
[18:15:38] <toad_> TheSeeker: yeah that might be it ... are you running
java 7?
[18:15:42] <TheSeeker> 8
[18:15:48] <toad_> so it should work then? :|
[18:16:04] <toad_> if you can show it doesn't work then something needs
to be done about it urgently as it may break windows nodes badly
[18:20:08] <TheSeeker> gah!  I don't understand why files are being
created with e.g. 1029365099476005573.freenet-tmp at the end... that
doesn't look like what ClientPersister is supposed to be doing...
(Continue reading)

Matthew Toseland | 4 Sep 19:20 2014

Please test purge-db4o

See my flog for details.

Devl mailing list