[freenet-dev] Freenet website - content management

Re: Freenet website - content management

If you know PHP, you could write a simple one with MySQL. 

++ 26/12/02 13:10 -0000 - Simon Porter:
>Hi everyone
> 
>I hope you've all had a good Christmas!
> 
>I am looking for suggestions on how content management should be
>implemented on the new site. Most content management systems I have
>looked at seem to use tables here there and everywhere. Seeing as I've
>only just learnt how to banish tables and such I don't really want to go
>back to using them again. Anyone know a decent content management system
>that works with CSS at all? Many thanks.
> 
>Simon Porter

--

-- 
--------------------------------
Freedom is slavery.
Ignorance is strength.
War is peace.
                -- George Orwell

Re: Re: [arodland <at> noln.com: Anonymity filter breakage]

On Thu, Dec 26, 2002 at 04:20:46AM +0100, Marco Gruß wrote:
> hi,
> 
> > [possible attack on anon filter]
> I think a usable workaround against this is to specify 
> a "non-existant" http proxy server (i.e. 127.0.0.1:3117   
> or whatever) and to enable direct access to localhost -
> this way, using fproxy will work perfectly while not
> allowing any content to be loaded from other hosts.
And we expect our users to do this to use freenet? That is unreasonable.
It is a reasonable workaround maybe, but not a long term solution.
> 
> so long,
> Marco
> 
> -- 
> +++ GMX - Mail, Messaging & more  http://www.gmx.net +++
> NEU: Mit GMX ins Internet. Rund um die Uhr für 1 ct/ Min. surfen!
> 

--

-- 
Matthew Toseland
toad@...
amphibian@...
Freenet/Coldstore open source hacker.
Employed full time by Freenet Project Inc. from 11/9/02 to 11/1/03
http://freenetproject.org/

Re: Freenet website - content management

The idea of creating our own CMS smells of "not invented here".

A simple approach might be to put the whole thing in a CVS module, and 
have it automatically checked out to the website whenever a change is 
made.

Not sure how easy the latter part would be though.

Ian.

On Thu, Dec 26, 2002 at 01:10:35PM -0000, Simon Porter wrote:
> 
>    Hi everyone
> 
> 
>    I hope you've all had a good Christmas!
> 
> 
>    I am looking for suggestions on how content management should be
>    implemented on the new site. Most content management systems I have
>    looked at seem to use tables here there and everywhere. Seeing as I've
>    only just learnt how to banish tables and such I don't really want to
>    go back to using them again. Anyone know a decent content management
>    system that works with CSS at all? Many thanks.
> 
> 
>    Simon Porter

--

-- 
Ian Clarke                ian <at> [freenetproject.org|locut.us|cematics.com]
Latest Project                                 http://cematics.com/kanzi
Personal Homepage				        http://locut.us/

Large Datastores, sponsorship

Hi. There is an interesting issue involving very large datastores on
machines with less than a hell of a lot of RAM. Basically, it is that we
keep a structure (NativeBuffer) for each file in the datastore in RAM.
This is necessary for two reasons:
a) To keep the LRU list up to date.
b) To optimize the common case datastore lookup, which is where we are
asked for a file and we probably don't have it. This would result in a
syscall (and maybe even some disk seeking) if we don't keep the above
datastructure.

Now, we can solve the above problems and greatly reduce the amount of
memory used by:
* Implementing a hack, storeSize=0, which does not attempt to limit
  store space usage, and therefore does not need the LRU list.
  Suggested by Pascal.
* Using a bit-vector hash to optimize lookups. A largish datastore might
  have 200,000 files; with a scarcity factor of 50, this is only
  ~10,000,000 bits, i.e. just over a meg. This would be a config
  parameter - the power of two to use for the number of bits.

I don't consider this to be a priority at the moment... however, it is a
long term problem, and a potential sponsor has emerged (Pascal). I suggest
that it will take up to approximately three days including maintenance and
debugging (wild guess :))... I am still working for freenet project, and
there's a good chance it will be able to renew my contract, so it seems
reasonable to do it through the project. I suggest that we should charge
a commission of say 50% to cover unexpected events, and this item not
being on the current perceived critical path... so 
$41*3*1.5 ~= $180 donation (3 days may be pessimistic... but experience 
has taught me to overestimate). The sponsor has agreed to this.

Comments? I'm going to start working on this on Saturday or Monday
(depending on various factors including whether I decide to work on
Saturday [I'm somewhat behind on hours due to recent illness, I
regard myself as a contractor and I need the flexibility to take time
off and delay the next payment when necessary]) if nobody objects.
--

-- 
Matthew Toseland
toad@...
amphibian@...
Freenet/Coldstore open source hacker.
Employed full time by Freenet Project Inc. from 11/9/02 to 11/1/03
http://freenetproject.org/

Re: Large Datastores, sponsorship

On Fri, Dec 27, 2002 at 03:39:10AM +0100, Frank v Waveren wrote:
> On Fri, Dec 27, 2002 at 02:08:11AM +0000, Matthew Toseland wrote:
> > Now, we can solve the above problems and greatly reduce the amount of
> > memory used by:
> > * Implementing a hack, storeSize=0, which does not attempt to limit
> >   store space usage, and therefore does not need the LRU list.
> Sounds good, if the sponsor (thanks btw!) is prepared to add disks
> indefinately. However, just purging the datastore (partially) once in
> a while seems far from optimal.
Yeah, the intention is to add disks indefinitely, or set the DS size to
something nonzero after a while.
> 
> > * Using a bit-vector hash to optimize lookups. A largish datastore might
> >   have 200,000 files; with a scarcity factor of 50, this is only
> >   ~10,000,000 bits, i.e. just over a meg. This would be a config
> >   parameter - the power of two to use for the number of bits.
> I'm not sure what you mean by a 'bit-vector hash' but I'd suggest
> using a bloom filter (if that wasn't already what you meant).
Well, I'm not familiar with all the CS terminology, but the idea is you
run the keys through a nice hash function, then take the first n bits of
the hash, and create a vector of 2^n bits, and set each bit to true if
there is a key in the store whose hash starts with that bitstring.
Bithashes are really cheap - 1,000,000 keys, 50:1 scarcity (total
overkill probably), makes 50,000,000 bits ~= 6MB (round it up to 8MB).
Compared to the reported hundreds of megabytes of RAM used by the
current structures, this is a bargain.
> A problem would however be having to regenerate the cache every time
> you remove something, which would mean having to either do store
> purges very coarsely, or not at all as you suggested.
Yeah. Eventually, we will have to deal with this properly. It will
involve messy stuff like on-disk LRU list structures.
> 
> -- 
> Frank v Waveren                                      Fingerprint: 21A7 C7F3
> fvw <at> [var.cx|stack.nl|chello.nl] ICQ#10074100            1FF3 47FF 545C CB53
> Public key: hkp://wwwkeys.pgp.net/fvw@...            7BD9 09C0 3AC1 6DF2
> 

--

-- 
Matthew Toseland
toad@...
amphibian@...
Freenet/Coldstore open source hacker.
Employed full time by Freenet Project Inc. from 11/9/02 to 11/1/03
http://freenetproject.org/

Re: Large Datastores, sponsorship

> * Using a bit-vector hash to optimize lookups. A largish datastore might
>   have 200,000 files; with a scarcity factor of 50, this is only
>   ~10,000,000 bits, i.e. just over a meg. This would be a config
>   parameter - the power of two to use for the number of bits.

I someone (possibly me) had an idea a long time ago about a novel way to 
implement the datastore - namely to implement it as a miniature Freenet 
in itself.  Split it up into bite-sized chunks which can reference 
each-other just as Freenet nodes reference other Freenet nodes, and 
implement each as a miniature LRU in itself.

Ian.

--

-- 
Ian Clarke                ian <at> [freenetproject.org|locut.us|cematics.com]
Latest Project                                 http://cematics.com/kanzi
Personal Homepage				        http://locut.us/

Freenet Website - Help needed

Hey everyone

I've been looking round for a decent content management system for the
website as I've mentioned before. I've come across Tiki
http://tikiwiki.sourceforge.net/ 

This seems to be a more advanced version of Wiki, the content management
system we are currently using. The best thing about it is it works using
CSS, XHTML and PHP.

I'd really like some help with implementing this into the website as I
have no experience with php. If you would like to help out with the site
then please e-mail me at hailstormxp@... 

Thanks.

Simon Porter

Re: Freenet Website - Help needed

On Fri, Dec 27, 2002 at 02:39:25PM -0000, Simon Porter wrote:
> I've been looking round for a decent content management system for the
> website as I've mentioned before. I've come across Tiki
> http://tikiwiki.sourceforge.net/ 

After some blood, sweat, and tears - Tiki is now installed (for the 
moment) at http://freenetproject.org/tiki/.

If anyone needs access, or any admin stuff modified - let me know.

Ian.

--

-- 
Ian Clarke                ian <at> [freenetproject.org|locut.us|cematics.com]
Latest Project                                 http://cematics.com/kanzi
Personal Homepage				        http://locut.us/

Tiki access