Re: Task scheduling in Lua

Re: Userdata environment

2008/3/31, Shmuel Zeigerman <shmuz <at> actcom.co.il>:
> Is there some reason behind enforcing userdata environment to be a table?

I'm very surprised by the question, and the rest of the thread. In
some of my modules I use full userdata as the environment to other
full userdata, I never ran into an error specifying that the
environment has to be a table.

More specifically, my userdata A is a pointer to a C structure, and
when I __index one of its field, it returns a userdata B pointing to
the field directly. To avoid the A structure to be collected while I
have a pointer to its field (B), I make A the environment of B. It
seems to work perfectly fine.

How should the restriction manifest itself ? Is that something
specific to the Lua version of setfenv (as opposed to the C function
lua_setfenv) ? Is that something from Lua 5.0 (I'm using 5.1) ?

Re: Userdata environment

Jérôme Vuarand wrote:
> I'm very surprised by the question, and the rest of the thread. In
> some of my modules I use full userdata as the environment to other
> full userdata, I never ran into an error specifying that the
> environment has to be a table.

If you had turned on assertions with LUA_USE_APICHECK, the
  api_check(L, ttistable(L->top - 1));
in lua_setfenv() would have caught it.

> More specifically, my userdata A is a pointer to a C structure, and
> when I __index one of its field, it returns a userdata B pointing to
> the field directly. To avoid the A structure to be collected while I
> have a pointer to its field (B), I make A the environment of B. It
> seems to work perfectly fine.

The GC doesn't care which kind of object is referenced from the
u->env pointer (as long as it's a collectable object). You were
just lucky. But lua_getfenv() will gladly push any environment
object on the stack and tag it as a table (ouch).

> How should the restriction manifest itself ? Is that something
> specific to the Lua version of setfenv (as opposed to the C function
> lua_setfenv) ? Is that something from Lua 5.0 (I'm using 5.1) ?

Nope, it's been there since Lua 5.1 was released. Userdatas had no
environment in Lua 5.0.

--Mike

Re: Userdata environment

2008/3/31, Mike Pall <mikelu-0804 <at> mike.de>:
> Jérôme Vuarand wrote:
>  > I'm very surprised by the question, and the rest of the thread. In
>  > some of my modules I use full userdata as the environment to other
>  > full userdata, I never ran into an error specifying that the
>  > environment has to be a table.
>
>
> If you had turned on assertions with LUA_USE_APICHECK, the
>
>   api_check(L, ttistable(L->top - 1));
>
> in lua_setfenv() would have caught it.

Indeed, it does assert.

>  > More specifically, my userdata A is a pointer to a C structure, and
>  > when I __index one of its field, it returns a userdata B pointing to
>  > the field directly. To avoid the A structure to be collected while I
>  > have a pointer to its field (B), I make A the environment of B. It
>  > seems to work perfectly fine.
>
> The GC doesn't care which kind of object is referenced from the
>  u->env pointer (as long as it's a collectable object). You were
>  just lucky. But lua_getfenv() will gladly push any environment
>  object on the stack and tag it as a table (ouch).

So this means that having a userdata as environment works (as a
collection prevention as in my example above), as long as you don't
try to access it once set ?

It would be nice to have lua_setfenv check the type in a future
version. Alternatively (and I'd favor that alternative), it would be
nice to allow userdata as environment of other userdata. In the
meantime I'll try other methods to store my owner<->owned
relationship.

Re: Task scheduling in Lua

On Tue, Apr 1, 2008 at 3:01 AM, Luís Eduardo Jason Santos
<jasonsantos <at> quantumsatis.net> wrote:

> Have you seen Rings?
> It has an interesting solution for inter-state communication. Maybe taking a
> closer look might help.
> http://www.keplerproject.org/rings/

I have seen it (may be not close enough), but I was under impression
that it does not allow direct data communication -- that is, if one
needs to pass something like table, he have to serialize it.

Alexander.

Re: Task scheduling in Lua

If you need serialization, Pluto library 
(http://luaforge.net/projects/pluto/) is the right way to go. I've been 
using it a lot and its stable. You can serialize practically anything 
with it.

Goran.

On 01-04-08 7:47, Alexander Gladysh wrote:
> On Tue, Apr 1, 2008 at 3:01 AM, Luís Eduardo Jason Santos
> <jasonsantos <at> quantumsatis.net> wrote:
>
>   
>> Have you seen Rings?
>> It has an interesting solution for inter-state communication. Maybe taking a
>> closer look might help.
>> http://www.keplerproject.org/rings/
>>     
>
> I have seen it (may be not close enough), but I was under impression
> that it does not allow direct data communication -- that is, if one
> needs to pass something like table, he have to serialize it.
>
> Alexander.
>
>   

Re: Userdata environment

David Given wrote:
> That sounds very much like a reference leak in your code rather than 
> anything in the Lua core --- tables are used *so* much that any GC bug 
> that keeps references to deleted items would have shown up elsewhere by 
> now. The GC is pretty solid.

As Mike Pall said elsewhere in this thread, it is a known issue, already 
discussed a few times on this list. (I didn't intend to discuss it once 
again: see the starting message of this thread).

--

-- 
Shmuel

Re: Task scheduling in Lua

> If you need serialization, Pluto library
>  (http://luaforge.net/projects/pluto/) is the right way to go. I've been
>  using it a lot and its stable. You can serialize practically anything
>  with it.

Is it fast? If I have to do serialization, I have to do it on each
task switch... Say, 512 tasks with 8 threads, with single update per
second -- that's 64 save/load operations per second if I'm not
mistaken. Looks like too much -- especially considering that some my
tasks are data-heavy.

Alexander.

Re: Userdata environment

From: "Shmuel Zeigerman"
> This is not a leak but that doesn't help. The memory is not freed and as 
> application continues to run, it can eventually consume all the available 
> memory.

I'm afraid I still can't imagine a scenario where that's possible. I mean, 
it doesn't keep dead keys around forever. Just until it needs to grow the 
table, and then it can remove them all.. and in doing so discover that it 
never needed to grow the table in the first place. The only way it could 
consume all available memory would be if at some point you had enough live 
keys in the table to do that.

Either that or I don't understand the problem, which is also quite likely. 
=)

Sorry that there doesn't seem to be an elegant solution though.

- Alex 

Re: Userdata environment

Alex Davies wrote:
> I'm afraid I still can't imagine a scenario where that's possible. I 
> mean, it doesn't keep dead keys around forever. Just until it needs to 
> grow the table, and then it can remove them all.. and in doing so 
> discover that it never needed to grow the table in the first place. The 
> only way it could consume all available memory would be if at some point 
> you had enough live keys in the table to do that.

Here is a pure-Lua test case that is very close to what my application 
is doing. (Change N if needed).
   local N = 6
   local tb = setmetatable ({}, { __mode="k" })
   print(collectgarbage"count")
   for n=1,N do
     for k=1,1e6 do tb[{}] = true end
     print(collectgarbage"count")
   end

--

-- 
Shmuel