headers
john skaller | 1 Feb 2012 14:20
Picon

pthread issues

Pthreads are not working right. tools/launcg crashes,
and this program fails to do what I expect:

// pthread test

noinline proc mkfibre(p:int, f:int) {
  spawn_fthread {
    for k in 1 upto 10 do
      eprint$ "Thr " + str p + " fibre " + str f + " step " + str k + "\n";
      //Faio::sleep(sys_clock, 0.01 + f.double / 5.0);
    done
    eprint$ "Thr " + str p + " fibre " + str f + " DEAD " + "="*20+"\n";
  };
}

noinline proc mkthread (x:int) {
  spawn_pthread {
    for var j in 1 upto 10 do
      eprint$ "Thr " + str x + " step " + str j + "\n";
      //Faio::sleep(sys_clock, 0.1 + x.double / 2.0);
      for var f in 0 upto 10 do
        mkfibre(x, f);
      done
    done
    print$ "Thread " + str x + " done" + "*"*20+"\n";
  };
}

for var k in 1 upto 10 do
   mkthread k;
(Continue reading)

Permalink | Reply | 1 comment |
headers
john skaller | 3 Feb 2012 06:03
Picon

vals and vars

Well this surprised me a bit, this is reading pchannels:

    val m1 = read(inp);
    val m2 = read(inp);
    val m3 = read(inp);
    eprintln$ "Mainline Read1="+ m1.str;
    eprintln$ "Mainline Read2="+ m2.str;
    eprintln$ "Mainline Read3="+ m3.str;

Mainline Read1=36
Mainline Read2=16
Mainline Read3=26

These numbers should be in order! Like this: (note: val changed to var):

    var m1 = read(inp);
    var m2 = read(inp);
    var m3 = read(inp);
    eprintln$ "Mainline Read1="+ m1.str;
    eprintln$ "Mainline Read2="+ m2.str;
    eprintln$ "Mainline Read3="+ m3.str;

Mainline Read1=16
Mainline Read2=26
Mainline Read3=36

To see why I looked at the C++:
//PROCEDURE <16106:> _init_::_lam_12704: Resume method
::flx::rtl::con_t *_lam_12704::resume(){
      {
(Continue reading)

Permalink | Reply | 0 comments |
headers
john skaller | 6 Feb 2012 07:24
Picon

Re: pthread issues


On 02/02/2012, at 12:20 AM, john skaller wrote:

> Pthreads are not working right. tools/launcg crashes,

OMFG .. I bow down to the Valgrind God!

Yep, Valgrind found the bug I've been hunting for two week in 2 seconds.

  var out = array_alloc[&char] t.len+1;

is making an array of char* to pass to execve. This is supposed to allocate
an array of the number of args + 1 for the NULL pointer terminator.

But it doesn't.. nope, it doesn't, this does:

  var out = array_alloc[&char] (t.len+1uz);

The first formula made an array 1 too short .. and the proceeded to increment
the pointer to the already too short array by one slot, so the store is actually
two slots too short!

--
john skaller
skaller@...

------------------------------------------------------------------------------
Try before you buy = See our experts in action!
The most comprehensive online learning library for Microsoft developers
is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3,
(Continue reading)

Permalink | Reply | 0 comments |
headers
john skaller | 8 Feb 2012 07:16
Picon

thread model

I'm having a look at the thread model at the moment.

Here's the situation:
===============

Fibres (fthreads) can do blocking blocking ops. The fibres get shuttled
off to the async queue and re-awakened when the op is complete.
Examples: socket I/O and timer waits.

Fibres can do schannel I/O.  The fibres are descheduled and attached
to the schannel until the transaction can be completed, at which time they're
put back on the scheduler queue.

Pthreads can do pchannel I/O. This is a full handshake data exchange by
a monitor, it blocks the whole of a waiting pthread using OS condition
variables. It makes no difference if the read or write is in a fibre.

Pthread get their own private copy of the background event manager
thread (demux), async wait queue, and scheduler queue.

There is a job queue for thread pooling jobs, but no way to create
such jobs from Felix.

Fibres can't be rescheduled in another pthread (enforced by separation
of the queues). There's also no sharing of I/O monitoring or timers.

Fibres can't deadlock, pthreads can.

What we want
===========
(Continue reading)

Permalink | Reply | 0 comments |
headers
john skaller | 10 Feb 2012 07:50
Picon

DSSL fun: Json support in 30 minutes

Wanna use JSON syntax in Felix? Here we go:

(1) Write the grammar

// lib/grammar/json.flxh

syntax json {
  requires expressions;
  x[sapplication_pri] := "json" "(" json_value ")" =># "_3";

  json_object : string = "{" "}";
  json_object : string = "{" json_members "}";
  json_members : string = json_pair ( "," json_pair ) *;
  json_pair : string = strstring ":" json_value;

  json_array : string = "[" "]";
  json_array : string = "[" json_elements "]";
  json_elements : string = json_value ( "," json_value ) *;

  json_value : string = 
     | strstring 
     | json_number
     | json_object
     | json_array
     | "true"
     | "false"
     | "null"
  ;
  json_number : string = 
     | strint
(Continue reading)

Permalink | Reply | 0 comments |
headers
john skaller | 11 Feb 2012 03:17
Picon

Directory stack Concept

I'm looking at integrating this a bit more tightly into Felix: the tools, the compiler,
and of course provide library stuff.

The idea is simple: a "directory stack" is a stack of directories. Duh!

When you open a file for reading:

	open_input(stack, filename)

filename is looked for in the stack, top dow.

When you open a file for writing, you always write in the top of stack directory.

If you do read + write open, the file is searched as for reading, and if found
but not in top of stack it is copied there, and then that is opened.

The effect is a "poor mans src control". Consider the stack

curdir
home
local-mods
felix-library

Then if you compile some library.flx file, the output stuff goes in curdir.
This is good because felix-library and local-mods are read-only anyhow.

With a bit of sex thrown in, you could also recognize filename.patch files
that shadowed a filename file, and generate a patched file "on the fly".

The problem this stuff is meant to solve is this: at present, Felix decides
(Continue reading)

Permalink | Reply | 0 comments |
headers
john skaller | 11 Feb 2012 03:32
Picon

Perl interpolated strings

I have added a restricted version of this back into Felix:

~/felix>cat q.flx
val x = 1;
val y = "World";
val z = 99.76;
val s = q"ab$(x)$(y)$(z)cd";

~/felix>flx --test=build/release q
ab1World99.76cd

The constraint is: you can only use unqualified variable names in the $(varname)
bits, you cannot put a general expression. The reason is that the contents
of $(..) are discovered AFTER parsing.  To reparse them requires a complex
mechanism which will reprocess all the syntax and auto-includes, just to
reload the syntax automaton. This CAN be done I think: it is already
done in the same piece of code for include files. 

The other way to do this is to actually parse the q".." string inside the parser.
This is also possible but tricky! It either involves early recursion of the parser
from the Scheme action codes (which requires some way to call Ocaml functions
from Scheme), or, it requires the actual string to be parsed by grammar
productions. Both are possible but messy.

For expressions there's a simple workaround:

	q"hello $(get_name() + delim), wie gehts?"

can be replaced by
(Continue reading)

Permalink | Reply | 0 comments |
headers
john skaller | 11 Feb 2012 05:59
Picon

Notes on Memory management

Should go in manual but here is good for now.

As you know Felix runs a (rather lame) garbage collector.

It uses a shape object (rtti) to determine the size of an object and where
all the pointers are. The shape handles static arrays with a count.

Dynamic arrays are handled by an externally held per object count
of used entries, and another for the maximum. These two counts allow
an ordinary C pointer to be correctly scanned. The data structure that does
this is called a varray. It is special because it has core support.
In principle all data structures are varrays.

The gc has attached to it a set of roots. When collection is to be done,
all threads are intercepted at the next service call and frozen. The collection
has to wait for this, it's a design problem. [We may add Linux SIGSTOP support
to fix this on Linux, it can't be helped on other OS]

When the world has finally stopped, the stacks of all the threads are
added as roots, considered as pointer arrays. Therefore the stacks
are conservatively scanned, whilst everything else is exactly scanned.

The roots attached to the gc object use an STL map with a root count.

Rooting a pointer is used as a method by the async system. When a request
is queued for servicing, the fthread's root count is increments (usually from 0 to 1).
This ensures the sleeping fthread is not deleted. With schannels, this is not done:
instead the fthread (fibre) is just attached to the schannel. The fibre will be
reaped if there are no other references and the schannel is unreachable.
This is what prevents fibres deadlocking: when they're dead(locked) they're
(Continue reading)

Permalink | Reply | 0 comments |
headers
john skaller | 12 Feb 2012 06:29
Picon

JSON

Ok, I just added a first cut JSON system in web/json.flx.
This is a basic data structure, parser, and pretty printer.

Bugs:  

* Only accepts integers (no floats yet).
* Only accepts strings without weird quotation stuff
* Accepts any C identifier starting with alpha as a name,
  instead of just null, true, false

* The representation is poor: both arrays and dictionaries are just lists.
*  Integers are also just strings

* The pair type will accept any Jvalue, not just a string. However the parser
only parses strings.

The float bug will be fixed, as will the string thing.

The plan: for dictionaries, non-string values will be allowed.
When output is done, the Felix repr of the value will be used as a key.
The converse could be made to happen too. It's not clear why Json
bothered putting this constraint in the language. It is clear why many
representations of dictionaries only accept keys which are strings.

Names:  There are two uses of other names. One is just an extension.
The other is more interesting. We would like to allow a json variable
in a data structure so you can make definitions. Perhaps this should
have a unique syntax:

var x =  '{"a":1}';
(Continue reading)

Permalink | Reply | 0 comments |
headers
john skaller | 13 Feb 2012 08:33
Picon

locale policy

There's a problem with Mike's low-res-time function:

fun rfc1123_date (dt:&tm) => strftime("%a, %d %b %Y %H:%M:%S %Z",dt);

because it uses the %Z specifier and strftime. The result of strftime
and localtime() functions are indeterminate.

My rough view is: local times should never be used.
Time zone names also shouldn't be used.

Instead, the only permitted timezones should be:

UTC
UTC+0230
UTC-0600

i.e. UTC, or UTC+/- 4 digits represent 24 hour clock format offset.

A time without a zone should be considered to mean UTC.
A time for a local timezone can be easily adjusted to UTC this way.

GMT is not a time standard.

The strftime function should never be used with %Z, 
because the time zone is indeterminate. See Posix standard.

In the actual test code, we get a failure because we start with an
absolute (GMT) time, parse it into a tm_ structure thing, and in 
doing so the timezone is lost. Then strftime goes and puts in
the local timezone.
(Continue reading)

Permalink | Reply | 0 comments |

Gmane