Neko intermediate language compiler and virtual machine ()

Adrian Veith | 8 Aug 2010 22:23

problems with GC memory management

 Hi,

I have problems with memory management in our application-server. The
server uses neko modules for custom calculations. This means that I have
conventional and garbage collected management together in a heavy multi
threaded environment. For the conventional memory management I am now
sure that it is clean, because any block that has been allocated is
freed again. For the garbage collected memory I am not sure, because in
theory the calculations should only use a little memory and after that
it should be available again. I crosschecked and disabled the neko
modules inside th calculations -> memory management is at a green level. 
Now I tried to help the garbage collector and nilled any object
allocated inside the calculations when it is out of use - but with no
result.
Next try was, to call Gc.run(true) after each calculation - no result.
Now I think, that something blocks the GC from doing its work, but I
don't know where to look.

Is there a way to have better debugging output from the GC ?

Thanks,

Adrian.

--

-- 
Neko : One VM to run them all
(http://nekovm.org)

headers

Nicolas Cannasse | 9 Aug 2010 10:59

Re: problems with GC memory management

Adrian Veith a écrit :
>  Hi,
> 
> I have problems with memory management in our application-server. The
> server uses neko modules for custom calculations. This means that I have
> conventional and garbage collected management together in a heavy multi
> threaded environment. For the conventional memory management I am now
> sure that it is clean, because any block that has been allocated is
> freed again. For the garbage collected memory I am not sure, because in
> theory the calculations should only use a little memory and after that
> it should be available again. I crosschecked and disabled the neko
> modules inside th calculations -> memory management is at a green level. 
> Now I tried to help the garbage collector and nilled any object
> allocated inside the calculations when it is out of use - but with no
> result.
> Next try was, to call Gc.run(true) after each calculation - no result.
> Now I think, that something blocks the GC from doing its work, but I
> don't know where to look.
> 
> Is there a way to have better debugging output from the GC ?

It is normal that the Boehm GC keep some allocated memory, you can read 
more about the way it works on 
http://www.hpl.hp.com/personal/Hans_Boehm/gc/gcdescr.html

Please note that the GC is also not exact, so it might reach allocated 
memory through some pointer left on the C stack and then not free it.

Check also the results of neko.vm.Gc.stats()

(Continue reading)

headers

Cauê Waneck | 9 Aug 2010 19:35

Neko debugger?

Is there any tool for debugging neko applications? If not, are there any bindings so we can make our own debugger?

Thanks!
Cauê

--

-- 
Neko : One VM to run them all
(http://nekovm.org)

headers

Nicolas Cannasse | 9 Aug 2010 20:32

Re: Neko debugger?

Cauê Waneck a écrit :
> Is there any tool for debugging neko applications? 

If you mean interactive debugging then not yet.

> If not, are there any 
> bindings so we can make our own debugger?

Writing a debugger would be not very hard, it would need to put some 
additional code in main bytecode interpreter loop (in neko/vm/interp.c) 
in order to check for breakpoints, then connect to an external debugger 
on TCP/IP with a small protocol for exchanging informations. Ask me if 
you have any specific question about the implementation.

Best,
Nicolas

--

-- 
Neko : One VM to run them all
(http://nekovm.org)

headers

Hilfer, Eric | 17 Aug 2010 16:07

Re: swhx doesn't support FP 10.1 now?

We are also seeing the same issue – swhx on OS X doesn’t work with Flashplayer 10.1 plugin. We have a product in the field that depends on this, and we could easily contract with someone to get a quick resolution (contributed to the public source, of course).

If anyone is in a position to research and resolve the compatibility of swhx with Flash 10.1 plugin – please let me know.

Thanks for your help,

Eric

--

-- 
Neko : One VM to run them all
(http://nekovm.org)

headers

Konstantin Tcholokachvili | 25 Aug 2010 12:26

Is statically linking NekoVM possible?

Helllo,

In order to make a proof of concept OS kernel based on a VM I would know if it's possible to statically link NekoVM to my kernel.
If I understood correctly until now NekoVM is only dynamically linked.

Can someone give his insight on this subject?

Thanks,

Konstantin Tcholokachvili

--

-- 
Neko : One VM to run them all
(http://nekovm.org)

headers

Lee McColl Sylvester | 25 Aug 2010 12:37

Re: Is statically linking NekoVM possible?

There's no distributed lib of Neko, but the VM code is very elegant and 
clean as well as being well rounded C code, so you could create one with 
some minor changes to the VS project file / makefile.  I did this myself 
back in '06/'07 for a simple desktop app.

Lee

Konstantin Tcholokachvili wrote:
> Helllo,
>
> In order to make a proof of concept OS kernel based on a VM I would 
> know if it's possible to statically link NekoVM to my kernel.
> If I understood correctly until now NekoVM is only dynamically linked.
>
> Can someone give his insight on this subject?
>
> Thanks,
>
> Konstantin Tcholokachvili

--

-- 
Neko : One VM to run them all
(http://nekovm.org)

headers

Nicolas Cannasse | 26 Aug 2010 10:24

Re: Is statically linking NekoVM possible?

Konstantin Tcholokachvili a écrit :
> Helllo,
> 
> In order to make a proof of concept OS kernel based on a VM I would know 
> if it's possible to statically link NekoVM to my kernel.
> If I understood correctly until now NekoVM is only dynamically linked.
> 
> Can someone give his insight on this subject?

You might have a look at xcross (http://code.google.com/p/xcross) which 
does link statically the VM. It compiles neko with NEKO_STANDALONE flag 
which enable you to customize the way the VM is initialized.

However, primitives are still resolved dynamically with dlsym() (after a 
dlopen(NULL)). You might prefer to create yourself a list of primitives 
and have a custom loader resolve them (see code in neko/vm/load.c)

Nicolas

--

-- 
Neko : One VM to run them all
(http://nekovm.org)

headers

Konstantin Tcholokachvili | 29 Aug 2010 23:48

Re: Is statically linking NekoVM possible?

Hello,

I modified the Makefile a little bit in order to add NEKO_STANDALONE
to CFLAGS but I have some compilation problems.

1.

 ld: warning: libgc.so.1, needed by bin/libneko.so, not found (try
using -rpath or -rpath-link)

I have libgc.so.1 in /usr/pkg/lib (I'm using NetBSD), I tried rpath
and -rpath-link but I get this message in all cases.

2.
main.c:(.text+0x532): undefined reference to `neko_standalone_init'
main.c:(.text+0x606): undefined reference to `neko_standalone_loader'
main.c:(.text+0x88c): undefined reference to `neko_standalone_loader'

As I guess I must define these functions somewhere, isn't it?

3.
vm/main.o: In function `report':
main.c:(.text+0x147): undefined reference to `neko_standalone_error'
bin/libneko.so: undefined reference to `GC_no_dls'
bin/libneko.so: undefined reference to `GC_malloc_ignore_off_page'
bin/libneko.so: undefined reference to `GC_malloc_uncollectable'
bin/libneko.so: undefined reference to `GC_gcollect'
bin/libneko.so: undefined reference to `GC_malloc_atomic_ignore_off_page'
bin/libneko.so: undefined reference to `GC_pthread_create'
bin/libneko.so: undefined reference to `GC_get_heap_size'
bin/libneko.so: undefined reference to `GC_get_free_bytes'
bin/libneko.so: undefined reference to `GC_free'
bin/libneko.so: undefined reference to `GC_set_warn_proc'
bin/libneko.so: undefined reference to `GC_malloc_atomic'
bin/libneko.so: undefined reference to `GC_malloc'
bin/libneko.so: undefined reference to `GC_init'
bin/libneko.so: undefined reference to `GC_dlopen'
bin/libneko.so: undefined reference to `GC_register_finalizer_no_order'
bin/libneko.so: undefined reference to `GC_all_interior_pointers'
bin/libneko.so: undefined reference to `GC_clear_roots'
bin/libneko.so: undefined reference to `GC_collect_a_little'
gmake: *** [bin/neko] Error 1

It's probably because of the first error case. But why libgc.so.1
isn't found? I specified it with:

LIBNEKO_LIBS = -L/usr/pkg/lib/ -lgc -lm

And what do you did to compile libneko.so as a static library
(libneko.a)? Addition

2010/8/26, Nicolas Cannasse <ncannasse <at> motion-twin.com>:
> Konstantin Tcholokachvili a écrit :
>> Helllo,
>>
>> In order to make a proof of concept OS kernel based on a VM I would know
>> if it's possible to statically link NekoVM to my kernel.
>> If I understood correctly until now NekoVM is only dynamically linked.
>>
>> Can someone give his insight on this subject?
>
> You might have a look at xcross (http://code.google.com/p/xcross) which
> does link statically the VM. It compiles neko with NEKO_STANDALONE flag
> which enable you to customize the way the VM is initialized.
>
> However, primitives are still resolved dynamically with dlsym() (after a
> dlopen(NULL)). You might prefer to create yourself a list of primitives
> and have a custom loader resolve them (see code in neko/vm/load.c)
>
> Nicolas
>
> --
> Neko : One VM to run them all
> (http://nekovm.org)
>

--

-- 
Neko : One VM to run them all
(http://nekovm.org)

headers

Nicolas Cannasse | 29 Aug 2010 23:54

Re: Is statically linking NekoVM possible?

Le 29/08/2010 23:48, Konstantin Tcholokachvili a écrit :
> Hello,
>
> I modified the Makefile a little bit in order to add NEKO_STANDALONE
> to CFLAGS but I have some compilation problems.

Check the xcross Makefile :

http://code.google.com/p/xcross/source/browse/trunk/Makefile.real

Nicolas

--

-- 
Neko : One VM to run them all
(http://nekovm.org)

Mon	Tue	Wed	Thu	Fri	Sat	Sun

						1
2	3	4	5	6	7	8
9	10	11	12	13	14	15
16	17	18	19	20	21	22
23	24	25	26	27	28	29
30	31

5	March 2013
3	February 2013
8	January 2013
8	December 2012
10	October 2012
5	July 2012
16	May 2012
1	April 2012
33	March 2012
3	January 2012
1	December 2011
7	November 2011
22	October 2011
3	September 2011
5	August 2011
4	May 2011
8	March 2011
6	February 2011
15	January 2011
14	December 2010
5	November 2010
5	October 2010
6	September 2010
13	August 2010
5	July 2010
9	June 2010
31	May 2010
20	April 2010
14	March 2010
28	February 2010
54	January 2010
13	December 2009
12	November 2009
10	October 2009
19	September 2009
10	August 2009
37	July 2009
11	June 2009
18	May 2009
11	April 2009
23	March 2009
15	February 2009
16	January 2009
16	December 2008
21	November 2008
65	October 2008
25	September 2008
75	August 2008
44	July 2008
16	June 2008
10	May 2008
36	April 2008
25	March 2008
37	February 2008
24	January 2008
48	December 2007
64	November 2007
22	October 2007
12	September 2007
81	August 2007
43	July 2007
30	June 2007
48	May 2007
51	April 2007
71	March 2007
79	February 2007
52	January 2007
35	December 2006
72	November 2006
22	October 2006
32	September 2006
56	August 2006
83	July 2006
231	June 2006
111	May 2006
8	April 2006
84	March 2006
79	February 2006
101	January 2006
23	December 2005
25	November 2005
16	October 2005
15	September 2005