Re: [ANN] LuaJIT-2.0.0-beta1

Mark Hamburg wrote:
> Very cool. And do I recall that this no longer needs Coco and hence  
> should be friendlier toward use cases with lots of coroutines?

Yes, it's operating on a single C stack, only switching Lua stacks
(i.e. heap-allocated objects). The coroutine model of LuaJIT 2.x
is more like RVM than Coco in that respect.

LuaJIT 2.0 coroutines use very little memory:

LuaJIT 2.0.0             410 bytes/coroutine (lua_State + default stack)
Lua 5.1.4 (x86/POSIX)    860
Lua 5.1.4 (x86/Win32)   1040
Lua 5.1.4 (x64)         1250
LuaJIT 1.1.5           62550

So you can happily juggle with hundreds of thousands of them (again).

Since there is also less context to update, coroutine yield/resume
is substantially faster (even though the JIT compiler currently
falls back to the interpreter for this).

--Mike

Re: [ANN] LuaJIT-2.0.0-beta1

On Sat, 31 Oct 2009 23:59:59 +0100
Mike Pall <mikelu-0910 <at> mike.de> wrote:

Hello Mike,

> Yes, here it is: the first public release of LuaJIT 2.0!

very nice! I was running some tests and the performance boost is
tremendous. Even though the vast majority of my code runs through
LuaJIT without a noticeable difference, some tests bail out if a
certain function in my C library is getting called. The message is:

/usr/local/bin/luajit-2.0.0-beta1: ./tek/ui/class/drawable.lua:205: bad
argument #-2 to 'drawimage' (number expected, got table)

I got it nailed down to the following line (tek/lib/visual_api.c:694) in
my C code:

   fmt = luaL_checkinteger(L, -1);

This C code fumbles around in nested tables which are passed in by the
caller. The next Lua function on the call stack is this:

function Drawable:drawImage(...)
   self.Visual:drawimage(...)
end

obviously, the same code was working before (and with LuaJIT 1.x). It
only affects the drawimage function, all other C library functions so
far work as expected. Without a spontaneous idea from your side I would

Re: [ANN] LuaJIT-2.0.0-beta1

Rob Kendrick wrote:
> Out of interest, what sort of engineering effort is required to get
> this working natively on AMD64?  From what I've read of your
> descriptions of it in the past, it sounds like it'll be even harder
> than with LuaJIT 1.

Umm, no, I think it will be easier. The code base and the internal
structures of LJ2 have already been designed with a 64 bit port in
mind.

The backend code generator is already pretty close to supporting x64.
It's mainly missing support for the different calling conventions and
stack frame setup. The disassembler included with the debug modules
can already read x64 code.

The real biggie is the interpreter, because it's written in assembler
with the help of DynASM. So one needs to port DynASM to x64 first
(eeek), redesign the C stack layout and then it's mainly cut'n'paste
from the x86 interpreter, modulo x87 vs. SSE2 differences.

There are some general open issues for a 64 bit VM, like where and how
to allocate the machine code and the heap (it needs to be in certain
address ranges). But I already have some ideas for this.

Ok, now, that sounds probably simpler than it is. But it's more a
mechanical translation, not in need of some cutting edge research.
It's still a lot of work, for sure.

> If it's a matter of sponsorship and paying for your time, I wonder if
> it might be worth starting a bounty pot.

Re: [ANN] LuaJIT-2.0.0-beta1

Hi Mike,

On 1/11/2009, at 11:59 AM, Mike Pall wrote:

> It's almost midnight on Halloween here.

That's long gone in this timezone!

> Yes, here it is: the first public release of LuaJIT 2.0!

That's awesome, your work is really appreciated!

Built fine on OS X 10.6.

It didn't like my heap code:

$ time lua heap_test.lua
real	2m4.923s
user	2m4.334s
sys	0m0.381s
$ time luajit -O heap_test.lua
real	1m5.412s
user	1m4.874s
sys	0m0.260s
$ time luajit-2.0.0-beta1 heap_test.lua
real	2m3.326s
user	2m2.565s
sys	0m0.281s

As far as I can tell, it really doesn't like the heap integrity check  

Re: [ANN] LuaJIT-2.0.0-beta1

Geoff Leyland wrote:
> It didn't like my heap code:

Well, yes and no. First, it's easier to analyze when the different
phases are separated out to individual benchmarks. The total
execution time of all of the test is not very meaningful.

The heap integrity checks spend 90% - 95% of their time writing
output. This overwhelms the check itself. And I/O functions are
not recorded (yet).

Comparing test_heap_speed is more interesting:

        binary_heap     skew_heap
------------------------------------
Lua       11.4s         10.7s
LJ1        2.9s          4.2s
LJ2        1.6s          3.0s
         /  50% JIT       25% JIT
Profile (   40% Interp    65% Interp
         \  10% GC        10% GC

So, even though LJ2 is quite a bit faster, it still spends way too
much time in the interpreter. Upon cursory examination it looks
like this is one of the NYI items:

The innermost loops seem to run only for very few iterations. And
side exits from it return back to a lower frame than the loop
itself. This case can't be recorded, yet. So it falls back to the
interpreter instead of creating a nice side trace. Fixing this

Undocumented/Buggy Behavior with Function Definitions

The Manual says that:
function f () body end
is translated to:
f=function() body end,

However, this code doesn't seem to work:
-----------------------------------------------------------------
do
  local cdoc
  local docs
  local function newindex(G,k,v)
    docs[k]=cdoc
    rawset(G,k,v)
    setmetatable(_G,nil)
  end
  function adddoc(s)
    cdoc=s
    setmetatable(_G,{__newindex=newindex})
  end
end
adddoc[[-----------------------
-- Some function doc
-- that has info in it
--Blah foo bar baz
----------------------------------]]
function Foo(arg1)
  doStuff()
end

----------------------------------------------------------------

Re: [ANN] LuaJIT-2.0.0-beta1

Timm S. Mueller wrote:
>    fmt = luaL_checkinteger(L, -1);

Aiiieeee ... you're using negative stack indexes with the
luaL_check*() functions?

Maybe the Lua manual could be a bit more clear about this point,
but it *does* say this at the start of section 4:

  "Because the error message is formatted for arguments (e.g., "bad
  argument #1"), you should not use these functions for other stack
  values."

For me this implies, that only positive values cause defined
behaviour.

Obviously the standard Lua implementation is somewhat more
lenient: it properly checks the type of the stack object at the
relative position, but it would report a pretty meaningless error
if the check ever failed (like "bad argument #-7").

Currently LJ2 happily misinterprets negative indexes for these
functions. Sooo ... what to do? I guess I'll have to bite the
bullet and support some undefined behavior, too. :-/

Patch will follow shortly.

--Mike

Re: Undocumented/Buggy Behavior with Function Definitions

> However, this code doesn't seem to work:

> do
>   local cdoc
>   local docs
>   local function newindex(G,k,v)
>     docs[k]=cdoc

You need to initialize docs:
    local docs={}

Job Offer: Permanent, full-time, work at home, Lua and C developer

Re: [ANN] LuaJIT-2.0.0-beta1

I wrote:
> Currently LJ2 happily misinterprets negative indexes for these
> functions. Sooo ... what to do? I guess I'll have to bite the
> bullet and support some undefined behavior, too. :-/
> 
> Patch will follow shortly.

Ok, here it is. Timm, can you please check whether this fixes
your original problem? Thank you!

--Mike