john skaller | 11 Jul 23:49 2014
Picon
Picon

Re: [felix] Web site down again


On 12/07/2014, at 2:46 AM, Ryan Gonzalez wrote:

> If you check out the source from GitHub, you can run the web server for your own computer.
> 
> I'm THIS close to writing an fdoc-to-html converter just because of this. THIS CLOSE!!!!

Actually fdoc IS already HTML PLUS some  <at> commands. 
 <at> h1,  <at> h2 etc should be easy to translate. In Python or any language.
 <at> felix, etc should be easy to translate too if you don't bother with 
syntax highlighting or colouring.

OTOH all the heavy code exists in the webserver in Felix, so if you copy
the webserver and replace the "get a page from a URL" with
"get a file from command line filename" and trap the place where the completed
page is written to a TCP port, all the rest of the code should
work "as is".

The code is modular to some extent, being made up of 
shared libraries.

--
john skaller
skaller@...
http://felix-lang.org

------------------------------------------------------------------------------
john skaller | 3 Jun 00:16 2014
Picon
Picon

Re: [felix] multi-threading exercise


On 03/06/2014, at 1:58 AM, Ryan Gonzalez wrote:

> I didn't see a call to the garbage collector in impl_allocate in flx_collector.cpp. All it does is
increment the statistics. I saw a world_stop call in impl_collect.
> 

That's right. The call is in flx_gc.cpp in gc_profile_t::allocate().
That calls gc_profile_t::actually_collect() which calls gc_collector_t::collect()
which calls flx_ts_collector_t::v_collect().

Which calls impl_collect() WITHOUT locking. There's a comment
in the code specifically noting there is no mutex so that's
deliberate.

impl_collect calls world_stop(), then either does the collection,
or just calls yield if one is already underway. yield should wait
until the collection is complete.

Maybe the code is right and the bug is elsewhere.
But it would be good if someone walked through the code
and verified its correctness.

As an experiment you can write some looping code in which you
do some stuff, and add a pthread which calls the collector
every so many seconds (that's what the web server does).
Set FLX_REPORT_COLLECTIONS=1 to see the activity.
It should  run forever.

--
(Continue reading)

john skaller | 2 Jun 16:02 2014
Picon
Picon

multi-threading exercise

So you think you understand threads?

Felixweb is down again, due to 100% CPU use and continuous swapping.
I am guessing this is because of the way the GC is called: it's called
from a thread. If you look at the collect() routine you will see there's no
lock. This is deliberate: the locking is supposed to be handled by the world
stop routine (only .. I don't think it is called).

So here's the challenge: prove its a bug and fix it :)

The GC must be run in a single thread whilst nothing else is running,
that is, the collection process is not thread safe, so all other threads
have to be stopped before running it.

If the GC is called by allocator calls, this should work.
However if the GC is called by a "manual" call to collect()
I think it does not.

The collect() in the Felix library should call a routine
that does the same as allocate() does when it detects
a collection is required.

--
john skaller
skaller@...
http://felix-lang.org

------------------------------------------------------------------------------
Learn Graph Databases - Download FREE O'Reilly Book
"Graph Databases" is the definitive new guide to graph databases and their 
(Continue reading)

john skaller | 11 May 14:42 2014
Picon
Picon

Re: [felix] Proposed next project


On 11/05/2014, at 9:44 PM, Shayne Fletcher wrote:

> 
> On Sun, May 11, 2014 at 2:58 AM, john skaller
<skaller@...> wrote:
> I think Felix is generating the map in the form of a Hashtbl from
> integers to lists of integers
> 
> To clarify, the table is an adjacency matrix?

To clarify, see src/compiler/flx_frontend/flx_use.ml

In particular there is already a function cal_use_closure.

Maybe there's nothing to do, i don't know.
I need this for the monomorphisation anyhow.

The old routine is driven by the instantiator, but that
instantiates *everything*, i.e. it doesn't trace through 
from the roots.

--
john skaller
skaller@...
http://felix-lang.org

------------------------------------------------------------------------------
Is your legacy SCM system holding you back? Join Perforce May 7 to find out:
&#149; 3 signs your SCM is hindering your productivity
(Continue reading)

john skaller | 11 May 08:58 2014
Picon
Picon

Re: [felix] Proposed next project


On 11/05/2014, at 4:09 PM, Shayne Fletcher wrote:

> 
> On Sun, May 11, 2014 at 2:06 AM, srean <srean.list <at> gmail.com> wrote:
> Sounds fantastic !
> 
> ​A little more reading leads to the Floyd-Warshall​ and a general transitive closure algorithm. I'm
still thinking do depth-first search to warm up and settle on a graph representation then move on to those.

Well the graph representation doesn't have many choices,
its a map from integers to a set of integers, with the roots in the
initial closure set. You basically do a recursive descent,  except that you don't 
if the integer is already in the closure.  It's about 10 lines of code.

I think Felix is generating the map in the form of a Hashtbl from
integers to lists of integers, but I'll check. The hard bit is generating
the dependencies. 

There are some tricky rules: a variable which
is a parameter is always considered used.
Technically this isn't necessary but its very tricky to get 
calls to work right otherwise: for direct calls you can
omit the argument, for calls through a closure you can't
(because you don't know the actual function).

This is a "Srean Uncertainty". An argument with a side-effect
might be omitted if the parameter is lost, and that certainly
happens if the function is inlined. And it certainly doesn't
if the call is via a closure. If you don't know, you can't tell
(Continue reading)

john skaller | 11 May 02:29 2014
Picon
Picon

Proposed next project


On 11/05/2014, at 12:36 AM, Shayne Fletcher wrote:

> Happy days.
> 
> Target was Debian 3.2.54-2 x86_64 GNU/Linux using g++ (Debian 4.7.2-5) 4.7.2.
> 

For my next project I propose to speed up the flxg compiler.

The first step is to rewrite the monomorphisation routine
and run it first, before inlining.

At present, Felix inlines, then monomorphises, then inlines again.

Felix uses a seriously slow hack to garbage collect symbols.
Fixing that will be the next step.

Monomorphisation means instantiating polymorphic entities
replacing the type variables with concrete types. In C++ this
is called specialisation.

In principle the idea is to start with some monomorphic roots
(roots are C functions so have to be monomorphic). Then we
trace all polymorphic entities used recursively keeping a record
to prevent an infinite loop, replacing calls to polymorphic functions
with calls to monomorpised versions. In C++ terms a call like:

	vector<int>,size()

(Continue reading)

john skaller | 8 May 05:00 2014
Picon
Picon

Re: [felix] Webserver up


On 08/05/2014, at 11:55 AM, srean wrote:

> Woohoo! this does answer my question to John "Is victory yours ?"

> However Mark reports his build still doesn't work.

Whilst Mark's build doesn't work, there's still a problem.
It may be in his clang (he's running clang on Linux).

however my build on OSX (clang) and the build on Ubuntu(gcc)
both seem to work, both web servers seem to run ok, and I can
run the tests with FLX_MIN_MEM=0 and FLX_FINALISE=1 which puts
maximum stress on the GC, on both systems.

It's possible Mark has a later clang than me, and it may need
some extra flags to turn off overly eager optimisations not available
on my build. Hard to know.

At some stage I should make a separate directory on the server,
and build the latest clang and use it. However I'm loathe to do that
because of the confusion about which compiler is being used.
I already have a weird setup on my Mac with both gcc 4.2 and
clang 3.3(svn) installed, and I don't know which C++ library the
clang is using. Whilst clang++ supports --std=c++11 the library
it uses definitely is NOT C++11 compliant. Its hard to tell if it
is using libc++ (the clang library) or libstdc++ (the gcc library).

In the mean time there are still some issues. One is that a
direct call to the GC via collect() seems to bypass a mutex lock.
(Continue reading)

john skaller | 8 May 01:41 2014
Picon
Picon

Webserver up

The webserver is now running again. Felix version bumped to 1.1.12.
On both my Mac and Ubuntu server I can run

FLX_MIN_MEM=0 FLX_FINALISE=1 make test

and it all works (except a couple of actual test failures). With the new system
this really hammers the GC.

However Mark reports his build still doesn't work.

--
john skaller
skaller@...
http://felix-lang.org

------------------------------------------------------------------------------
Is your legacy SCM system holding you back? Join Perforce May 7 to find out:
&#149; 3 signs your SCM is hindering your productivity
&#149; Requirements for releasing software faster
&#149; Expert tips and advice for migrating your SCM now
http://p.sf.net/sfu/perforce
john skaller | 5 May 14:34 2014
Picon
Picon

Re: [felix] Re: bugs in bugs


On 05/05/2014, at 7:17 PM, srean wrote:

> Then I found a bug, and I curse that moronic stupid idiotic C++ language.
> 
> here's the constructor for a gc profile:
> 
> gc_profile_t::gc_profile_t (
>   bool debug_driver_,
>   bool debug_allocations_,
>  
> and here the actual initialisation:
> 
>   gcp = new flx::gc::generic::gc_profile_t(
>     c,
>     c->debug_allocations,
> 
> shouldnt the typechecker cried foul when c got passed instead of a bool ?

No, there's an implicit conversion from pointers to bool.
So that this C code remains valid:

	if (p) { printf("Not NULL\n");
	else { printf("NULL\n");

And of course Felix switches off a lot of optional diagnostics
because typical compiler writers assume programmers are idiots
and report all sorts of rubbish as possible mistakes.

Worse, stupid people doing GCC change the diagnostics
(Continue reading)

john skaller | 5 May 23:00 2014
Picon
Picon

Re: [felix] Re: bugs in bugs


On 06/05/2014, at 5:05 AM, srean wrote:

> 
>  Is there a flag to turn the warnings on, they have been very helpful on occasions.

You can add a flag manually when building Felix programs like:

	flx -fsome-flag fred.flx

However there's no way to add flags to the build of Felix itself,
except by editing and recompiling the build code.

So i have made nice change to the compiler so the modulename
prefixes the type name to help debugging .. noticed that there
were two "thread_frame_t" objects with an allocation count
on each of one, and realised that's right, because i loaded a plugin.

But still NO idea how a single print statement crashes a program.

unsigned long gc_profile_t::actually_collect() {
  if(debug_collections || report_collections) 
    fprintf(stderr,"[flx_gc:gc_profile_t] actually_collect\n");

If I comment out the debug flag check here, my test program
crashes. It runs without *apparent* as written provided one of the
flags isn't set, setting the flag also crashes it.

--
john skaller
(Continue reading)

john skaller | 6 May 09:31 2014
Picon
Picon

Re: [felix] Re: bugs in bugs


On 06/05/2014, at 8:40 AM, srean wrote:

> >
> >  Is there a flag to turn the warnings on, they have been very helpful on occasions.
> 
> You can add a flag manually when building Felix programs like:
> 
>         flx -fsome-flag fred.flx
> 
> However there's no way to add flags to the build of Felix itself,
> except by editing and recompiling the build code.
> 
> That seems eminently reasonable.
> 

The basic concept is "no options". More precisely, every set of flags etc
is a separate target, i.e. a complete system. So you can have an optimised
build, a debugging build, a developer build, a build using an alternate
compiler, or any number of cross compilation targets.

Adding flags to just some components would lead to doubt an instability.

--
john skaller
skaller@...
http://felix-lang.org

------------------------------------------------------------------------------
Is your legacy SCM system holding you back? Join Perforce May 7 to find out:
(Continue reading)


Gmane