Matthew Toseland | 19 Sep 15:39 2014

Purge-db4o release candidate (snapshot 19)

The client layer rewrite aka purge-db4o is finished. That is, it is
feature complete, and has had a reasonable amount of testing (but needs

Major changes:
- No longer uses db4o to store anything, but will migrate from old
node.db4o*'s (downloads and uploads will be restarted)
- Should be much more reliable, and cause much less disk I/O
- Uses the multi-container code even for persistent inserts
- Improved disk crypto (I may improve this further)
- Trivial changes to FCP
- New metadata format (fixes some minor bugs, but we use the old one for
now by default)
- Handles low disk space much better. Migration may use significant disk
space; if we have less than 512MB left, transient requests will fail to
avoid breaking the system, if we have less than 1GB left, persistent
requests will fail / not start.
- Various bugfixes
- Rename RandomAccessThing to RandomAccessBuffer, etc.

The latest version is here:
Branch "purge-db4o-crypt"
Signed tag "purge-db4o-snapshot-19"
Commit hash 9b7c9204fa5461ddbfa3236f81e257e379dc86eb

Uploaded jar and signature:
CHK <at> WiSJI8wGPKMZ4KJhxO7EHzd3k4OtJBmay95IQ2dUbS0,4ChtKmFXtb5EOegUgQQZ85EWpijCdUyKubelVP9AOU0,AAMC--8/freenet.purge-db4o-snapshot-19.jar
CHK <at> FwVHzsL8zuRv6cJix2B14xm~8fH2EZsO68nyggIwoqQ,P0etmFePAHcKmgBKJTtnFckxQ7ZX97qM4Wg43bokPus,AAMC--8/freenet.purge-db4o-snapshot-19.jar.sig
(Continue reading)

vmonmoonshine | 17 Sep 20:22 2014

Job opportunity: looking for a freenet dev and more

Hello Freenet folks,

We're starting a project called CeNo, based on the idea I shared with a
few of you at last year's CTS in Berlin where we had the freenet meet
up. Its about using freenet to cache censored web pages

The company where I work secured funding for a year and is
currently looking for a dev to work (alongside me!) on the project. If
the potential applicant has  extensive software dev experience with
other languages as well and is interested in other projects, there is a
good chance to get a full-time position here. If you cannot apply but
are still interested in contributing to CeNo! by sharing your thoughts
and ideas on it, please join the mailing

And here's the job:

I think is a pretty good team to work with. We concentrate on
 digital security for civil society and develop free software to defend
 online freedom  speech and other aspect of human rights. So it is the
 prefect place for those of you who have the activism itch.

Looking forward to work with you, 
Matthew Toseland | 16 Sep 19:42 2014

Should we remove disk crypto?

On 05/09/14 19:43,
localghost <at> eOC4Zm8KjRpMFhNBp6DmI8K4URaq8bQZH45y0dLHEnI wrote:>
creamsoda <at> 0vpcRHZV1ftyj4mJpZnuYaG8wpkNIvf3qa3b-LUcsZs wrote :
>> TAILS is meant to be for short-lived sessions of minutes to hours
right? That doesn't lend itself well to freenet which works better over
a longer time.
> Ya, I kinda figured that.. just curious if anybody had done it. I was
asking for a friend, as stated. I prefer to let Freenet run constantly,
to help the network- and have not used TAILS, personally..
This is exactly why "leave disk crypto to the operating system" isn't so
obviously the right policy for Freenet.

The arguments for not doing disk crypto:
- We're likely to get it wrong.
- If they download video files etc there will probably be leaks. (But it
IS possible to limit this)
- Memorising a good password is hard, and the users who are willing to
do so may be the same group as the users who will install a secure linux
distro just to run Freenet, or at least do full disk encryption on
Windows (presumably using BitLocker?)
- If we do disk crypto we need to turn on swap encryption. This is
trivial on recent Windows but arguably not a good idea.

The arguments for doing disk crypto:
- We want people to run Freenet long-term. They usually won't install a
new OS just to run Freenet, and they can't run it long-term from a
livecd. This is one of the reasons we support Windows!
(Continue reading)

Steve Dougherty | 16 Sep 05:42 2014

[RFC] Update channels

Update channels are a way to provide easier testing of development
versions, make it easier for unofficial builds to be distributed, and
can also enhance build security by allowing for multiple signatures with
offline keys.

## Channel definition

A channel definition describes an update channel: both the user-facing
information and where to fetch it. It also contains security measures
like a revocation key, a list of trusted key IDs, and the number of
valid signatures required to deploy. This number can be zero if trusting
the insert key is deemed sufficient.

It is a key-value list of:

* name
* user-facing description
* trusted key IDs
* number of valid signatures from trusted keys required to deploy
* channel revocation SSK

Depending on whether the definition is distributed with a build, the
name and description could either be literal or localization keys. Those
distributed with builds are copied to an `update_channels` directory on
the filesystem and used from there to allow for clearer operation and
have it make more sense to drop additional channel definitions in the

Trusted keys are exciting because although the update insert key must
(Continue reading)

Matthew Toseland | 11 Sep 23:42 2014

New symmetric crypto API issues

In order to implement disk encryption in purge-db4o, I'm reviewing one
of unixninja's crypto refactoring commits. This includes disk crypto but
also some important crypto infrastructure providing a single hopefully
clean API for our various different ciphers etc. I haven't looked at his
other commits, so I don't know whether something else was intended ...

Lots of the code abuses ByteBuffer.array(), making very bad assumptions
about the ByteBuffer's passed in etc (not direct, not a slice,
arrayOffset() == 0, etc etc); this is fixed. (No criticism intended, NIO
is odd)

CryptByteBuffer implements symmetric encryption:

The main methods are:
public ByteBuffer encrypt(byte[] input, int offset, int len)
public ByteBuffer decrypt(byte[] input, int offset, int len)

There are various wrappers e.g. ByteBuffer encrypt(ByteBuffer). This is
highly ambiguous: Lots of NIO code takes a ByteBuffer and returns the
same ByteBuffer. Here we are returning a different ByteBuffer. The
original code wasn't clear whether it was always a new buffer or
sometimes shared the array, anyway we have 3 requirements:
1. The API must be unambiguous. Ambiguity -> very bad bugs, e.g. stuff
not getting encrypted. Hence e.g. we should not return a value if we are
encrypting in place; this is convenient but ambiguous.
2. It needs to efficiently support encrypting in-place for big stuff:
Whole packets, whole CHKs.
3. It needs to conveniently support small encryptions. These happen in
e.g. setting up an EncryptedRandomAccessThing (new class used for
encrypted random access tempfiles).
(Continue reading)

Matthew Toseland | 10 Sep 16:46 2014

Please test purge-db4o pre6

This is nearly done now. The only thing that isn't working is disk
encryption (physical security level of NORMAL or higher). Please test,
especially if on Windows!

Please test! You need to:
- Switch to LOW physical security, or turn off "encrypt temporary
buckets" and "encrypt persistent temporary buckets" in advanced core config.
- Shutdown the node
- Backup master.keys, node.db4o/node.db4o.crypt and persistent-temp*/
- Download the jar.
- Check the signature if possible.
- Replace freenet.jar with the downloaded jar.
- Change wrapper.conf to point to freenet.jar if it's currently
referring to
- Start up Freenet.

It should automatically migrate your downloads and uploads.

It is not compatible with the previous test build due to some refactoring.

Please test, and report bugs! Thanks!


CHK <at> Q8lin1CHHh-xB5u4BDWASGvx72xscrpvNTl7fnWBsnc,Vl7u9WkjoCXFhhl3i7b3NJUZdvWvX2ZUWQzHE5x53oU,AAMC--8/freenet.purge-db4o-snapshot-6.jar

CHK <at> XQpzloFhnVLPHTPDMiDZL8yVK4vkji~XGe0bBx~uezQ,WeOeLqiMx0mt77kxo3T4vANhFoR3Xt-tuHrBYBgAhjo,AAMC--8/freenet.purge-db4o-snapshot-6.jar.sig

(Continue reading)

Matthew Toseland | 5 Sep 19:34 2014

Please don't deploy the Java 7 stuff yet

It looks like somebody messed up and removed a close() which was before
a rename, probably in BaseFileBucket. Such a bug could break Windows
nodes very badly, so please be very careful reviewing the change before

[18:11:18] <TheSeeker> toad_: looks like the plugin data saving bit is
not finishing the temp file rename (trying to rename while open?)
[18:11:28] <toad_> TheSeeker: ouch
[18:11:32] <toad_> TheSeeker: not a regression mind you
[18:11:34] <TheSeeker> I have files like:
plugins.floghelper.FlogHelper.data1029365099476005573.freenet-tmp  in my
plugin-data dir.
[18:11:36] <toad_> TheSeeker: always happens on Windows?
[18:11:44] <TheSeeker> that didn't used to happen :P
[18:11:53] <toad_> TheSeeker: hmmm ... it could have broken sometime?
[18:12:18] <TheSeeker> sometime during purge-db4o yes.
[18:12:33] <toad_> TheSeeker: I haven't done anything to that code :|
[18:12:37] <toad_> TheSeeker: is this Windows?
[18:15:20] <TheSeeker> toad_: yes, windows.  likely something to do with
a change I saw somewhere assuming that java will always close things
properly?  (so maybe on next, not purge-db4o?)
[18:15:38] <toad_> TheSeeker: yeah that might be it ... are you running
java 7?
[18:15:42] <TheSeeker> 8
[18:15:48] <toad_> so it should work then? :|
[18:16:04] <toad_> if you can show it doesn't work then something needs
to be done about it urgently as it may break windows nodes badly
[18:20:08] <TheSeeker> gah!  I don't understand why files are being
created with e.g. 1029365099476005573.freenet-tmp at the end... that
doesn't look like what ClientPersister is supposed to be doing...
(Continue reading)

Matthew Toseland | 4 Sep 19:20 2014

Please test purge-db4o

See my flog for details.

Devl mailing list
unixninja92 | 17 Aug 19:24 2014

[GSoC 2014 Crypto API] Status Update 8

Hi all,

I've gotten a lot done since my last status update. The deadline for
GSoC is tomorrow, and I'm just about done. All I need to do is finish
up writing unit tests for ERAT (like 3 more to go) and CryptBucket. I
also need to write docs for ERAT and CryptSignature, but that should
not take very long.

This means that all the other classes I've written for the API are
finished and ready for review/merging. This includes: CryptByteBuffer
(used to be called CryptBitSet), Hash, KeyExchange, KeyGenUtils,
MessageAuthCode, and MasterSecret. I've submitted a pull request (#277
[1]) for EncryptedRandomAccessThing and it's dependencies so that toad
can start working with it as soon as possible. This includes
CryptByteBuffer, KeyGenUtils, MasterSecret, and MessageAuthCode.
CryptByteBuffer and MessageAuthCode have not had any review yet, so
please review!

Hash has been submitted in PR #258 [2]. Once #277 has been merged I
will submit a new pull request that will include KeyExchange,
CryptBucket and CryptSignature.

Unfortunately I don't have time to get JFKExchange classes working as
part of GSoC, but I will hopefully have time to work on them in the

So please review #277 and #271(upgrade to BC 1.51)[3] :)

(Continue reading)

xor | 11 Aug 21:04 2014

Good news: I have permission to write a Freenet filesharing app as my bachelor thesis

Hey folks!

The bachelor thesis is the last thing remaining to do in my computer science 
Today I had a meeting with my mentor, and I proposed to design and implement a 
Freenet client application: Filesharing. He is okay with that! :)

Easy-to-use, sharefolder-based filesharing with auto-insert has been requested 
for ages, and yet we didn't bother to finally implement it.
In fact it is the #1 request at uservoice:

I want to finally deal with this :)
It fits my current job very well: It will need spam filtering, so it  should be 
a WOT client application. 
Luckily, I was also able to convince my mentor that it is absolutely critical 
for me to fix WOT performance first! :) I told him that this shall take 4-6 
months, so I would start writing the thesis between December and February.

This should be manageable - there are 5 possible major WOT optimizations, 
sorted descending by efficiency:
1)  Optimization of computeAllScoresWithoutCommit(). Very pessimistically 
estimated should take a month.
2) Queuing of downloaded trust lists to prevent thread congestion. Not much 
work, should take 1-2 weeks. 
3) Event-notifications, which is essentially finished besides some fred-side 
refactoring and will be deployed very soon (2-3 weeks). 
4) Not subscribing to the USKs of ALL identities but only to direct trustees. 
Should be 2-4 weeks of work.
5) De-synchronizing the web-interface. While that own its own is only a week 
(Continue reading)

Steve Dougherty | 9 Aug 19:12 2014

Freenet 0.7.5 build 1465 released

The major network structure change in this release is preferential
opennet peer acceptance based on link length. The Roos, Schiller,
Hacker, and Strufe paper reports far too many long links. [0] The
statistics we've been collecting observe this as well. This release
accepts comparatively few long links, which should allow much better
navigation of the local keyspace. This will result in rejecting more
connections offered over announcement, so in a future version
announcements may indicate link length preference to lessen the load.

Matthew speculates that this will not interact well with the existing
behavior, so it will be mandatory 2014-08-16.

Java 6 has been EOL since February 2013. [1] This release adds an alert
when running with Java earlier than 1.7. [2] Freenet will require Java
1.7 or later in a future release.

Also in this release:

* Add Russian Windows installer translation. Thanks zabuldon! If you
  want to give a translation for another language please do so; the
  English source file is here: [2] In addition to Russian there are
  currently translations for Spanish, French, and Dutch.
* Update German, Finnish, French, Japanese, Dutch, Brazilian
  Portuguese, and Simplified Chinese translations thanks to volunteers
  on Transifex.
* Add permissions attribute to main jar manifest.
* Allow building with Bouncy Castle 1.50 and higher. 1.49 had a draft
  implementation of OCBBlockCipher, and an updated draft in 1.50
  limited the nonce size below what Freenet was using. Bouncy Castle
  1.51 will be deployed in a future release.
(Continue reading)