Lua and JSON

Re: Lua and JSON

Re: Lua and JSON

On Tue, Feb 9, 2010 at 1:00 PM, Steve Watkins
<steve <at> radicaldevelopment.com> wrote:
*

> Hello,
>
> I'm relatively new to Lua and am looking for a module or script that can
> encode and decode JSON strings inside a .lua script.
>
> I've looked on luaforge and have been unable to get those to work in my
> Scrite environment - thjs was using the
>
> -- JSON4Lua: JSON encoding / decoding support for the Lua language.
> -- json Module.
> -- Author: Craig Mason-Jones
> -- Homepage: http://json.luaforge.net/
> -- Version: 0.9.20
>
>
> I was hoping there was an established solid tested way to encode/decode JSON
> using Lua.
>
> Any info is much appreciated.
>
My LuaJSON projects uses LPeg for parsing and the strict mode passes
all tests provided by the JSON validator project from JSON.org  The
non-strict mode permits some more loose handling of constructs.

null and undefined values are preserved as tag-functions for full
round-trip support, however if you want a more native-friendly
decoding to avoid having to check for these 'null' and 'undefined' tag
values, you can use the
1.1.1 release and the json.decode.simple option-set.

Ex:
json.decode('null', json.decode.simple)
or
json.decode.getDecoder(json.decode.simple)('null')
will both return nil values

json.decode('null') will return a value: json.util.null as a
placeholder to permit re-encoding the 'null's explicitly for
round-tripping...

http://luaforge.net/projects/luajson/
http://github.com/harningt/luajson

The unit tests use lunit and should be easy to check using 'make check'

The makefile is present purely for simplification of some distribution
processes and is not needed for installation (it's a simple recursive
copy to where you need modules stored).

I'm readily available to answer any questions about it.  Any bugs /
behavior problems found are VERY welcome.

--

-- 
Thomas Harning Jr.

Re: Lua and JSON

Re: Lua and JSON

*

On Tue, Feb 9, 2010 at 1:34 PM, Fabio Mascarenhas <mascarenhas <at> acm.org> wrote:
> On Tue, Feb 9, 2010 at 4:17 PM, Thomas Harning Jr. <harningt <at> gmail.com>
> wrote:
>>
>> My LuaJSON projects uses LPeg for parsing and the strict mode passes
>> all tests provided by the JSON validator project from JSON.org  The
>> non-strict mode permits some more loose handling of constructs.
>>
>
> +1 for Thomas' module, it works great. Thomas, have you thought about having
> json.encode honor a "__tojson" metamethod (along with a rawencode function),
> so it is easier to extend the encoder to understand app-specific data types?
Hm... that is a good idea.  I would probably have it be something of the sort:

 __tojson(self, rawencode)

rawencode passed in so that you can take advantage of some of the
infinite recursion avoidance, as well as encoding options.  If I get
it right, I'd expect that rawencode would skip the __tojson field of
the immediate object and handle inner objects.

I'll have to work it out further.  It would be a 1.2 release when
fully released.

Note that there is a preProcess function that you can pass into the
options that will be passed each individual value for optional
replacement... not the best solution, but a workaround if the feature
is needed immediately (though it will do a 'normal' encoding
afterwards, so no super-special constructs allowed)

--

-- 
Thomas Harning Jr.

Re: Lua and JSON

On Tue, Feb 9, 2010 at 4:52 PM, Thomas Harning Jr. <harningt <at> gmail.com> wrote:

On Tue, Feb 9, 2010 at 1:34 PM, Fabio Mascarenhas <mascarenhas <at> acm.org> wrote:
> On Tue, Feb 9, 2010 at 4:17 PM, Thomas Harning Jr. <harningt <at> gmail.com>
> wrote:
>>
>> My LuaJSON projects uses LPeg for parsing and the strict mode passes
>> all tests provided by the JSON validator project from JSON.org  The
>> non-strict mode permits some more loose handling of constructs.
>>
>
> +1 for Thomas' module, it works great. Thomas, have you thought about having
> json.encode honor a "__tojson" metamethod (along with a rawencode function),
> so it is easier to extend the encoder to understand app-specific data types?

Hm... that is a good idea.  I would probably have it be something of the sort:

 __tojson(self, rawencode)

rawencode passed in so that you can take advantage of some of the
infinite recursion avoidance, as well as encoding options.  If I get
it right, I'd expect that rawencode would skip the __tojson field of
the immediate object and handle inner objects.

My idea was to json.rawencode just what json.encode does right now and ignore __tojson metamethods, just like rawget/rawset/raw*. The __json metamethod could receive the options the user passed to json.encode.

 

But it is important that __json is a metamethod instead of a regular field, just to keep things idiomatic, even though having to work with metatables can be a drag sometimes.

--

Fabio

Re: Lua and JSON

On Tue, Feb 9, 2010 at 2:19 PM, Fabio Mascarenhas <mascarenhas <at> acm.org> wrote:
> On Tue, Feb 9, 2010 at 4:52 PM, Thomas Harning Jr. <harningt <at> gmail.com>
> wrote:
>>
>> On Tue, Feb 9, 2010 at 1:34 PM, Fabio Mascarenhas <mascarenhas <at> acm.org>
>> wrote:
>> > On Tue, Feb 9, 2010 at 4:17 PM, Thomas Harning Jr. <harningt <at> gmail.com>
>> > wrote:
>> >>
>> >> My LuaJSON projects uses LPeg for parsing and the strict mode passes
>> >> all tests provided by the JSON validator project from JSON.org  The
>> >> non-strict mode permits some more loose handling of constructs.
>> >>
>> >
>> > +1 for Thomas' module, it works great. Thomas, have you thought about
>> > having
>> > json.encode honor a "__tojson" metamethod (along with a rawencode
>> > function),
>> > so it is easier to extend the encoder to understand app-specific data
>> > types?
>> Hm... that is a good idea.  I would probably have it be something of the
>> sort:
>>
>>  __tojson(self, rawencode)
>>
>> rawencode passed in so that you can take advantage of some of the
>> infinite recursion avoidance, as well as encoding options.  If I get
>> it right, I'd expect that rawencode would skip the __tojson field of
>> the immediate object and handle inner objects.
>>
>
> My idea was to json.rawencode just what json.encode does right now and
> ignore __tojson metamethods, just like rawget/rawset/raw*. The __json
> metamethod could receive the options the user passed to json.encode.
The problem is that it loses certain encoding state, such as the list
of already encoded tables to prevent infinite recursion.

I should have a preliminary version up on the 'next' branch with an
implementation.
>
> But it is important that __json is a metamethod instead of a regular field,
> just to keep things idiomatic, even though having to work with metatables
> can be a drag sometimes. 

--

-- 
Thomas Harning Jr.

Lua optimization (musing)

I was reading one of the Haller and Odersky papers on event-based actors and was thinking about how their
example would translate to Lua. This is what I came up with for their Counter example:

    function makeCounter( value )
        return makeActor( function( mbox ) -- Make an actor passing its mailbox to its handler code
            while true do
                mbox:receive{
                    Increment = function() value = value + 1 end,
                    Value = function( dst, msg ) dst:send( msg or 'Result', value ) end,
                    Lock = function( dst, msg )
                        dst:send( msg or 'Result', value )
                        mbox:receive{
                            Unlock = function( v ) value = v or value end
                        }
                    end
                }
            end
        end )
    end

The Increment message increments the counter. The Value message delivers the value to a particular
mailbox. The Lock message delivers the value to a particular mailbox and waits for an Unlock message. It's
the change in behavior after the Lock message that makes the message handling different from a simple
object model. Haller and Odersky allow unhandled messages to sit in the mailbox, but that's not what I'm
worrying about here.

Rather I'm worrying about code clarity and optimization.

On the clarity front, I think it actually does fairly well. It's a little bit noisy with all of the explicit
uses of "function" for short operations, but the current lambda expression proposals wouldn't really
help with that. The more annoying part is the need for a comma after each "end". All in all, though, it's
probably fairly workable.

The problem is that every iteration of the outer loop generates three new closures and a new table and the
inner receive generates a new closure and a new table. Some of this could be mitigated with a looping
construct for the outer receive that would use the same table repeatedly, but that wouldn't address the
inner receive problem.

The optimized version looks more like:

    function makeCounter_optimized( value )
        return makeActor( function( mbox ) -- Make an actor passing its mailbox to its handler code
            local lockedReceive = {
                Unlock = function( v ) value = v or value end
            }
            local mainReceive = {
                Increment = function() value = value + 1 end,
                Value = function( dst, msg ) dst:send( msg or 'Result', value ) end,
                Lock = function( dst, msg )
                    dst:send( msg or 'Result', value )
                    mbox:receive( lockedReceive )
                end
            }
            while true do
                mbox:receive( mainReceive )
            end
        end )
    end

The control flow is now more scattered and the overall effect seems less clear. It might be approaching a
reasonable format for a state machine, but it feels like it has lost something compared to the first version.

So, the questions I'm musing over are:

	* Is there a better way to write this in stock Lua?
	* If one were to make changes to Lua to try to get both clarity and efficiency, what would those changes be?

I've got a few thoughts on the latter but nothing clear enough to propose at this time.

Mark

Re: Lua optimization (musing)

Re: Lua optimization (musing)

luuid vs. forks

Hi, list!

I see different behaviour of uuid:new() module on my Ubuntu Server
9.10 x86_64 and OS X Snow Leopard 10.6.2 boxes.

I fork a process, then call uuid:new() both in parent and in child. On
Ubuntu I get different results, on OS X generated UUIDs are the same.

I can't remember how I installed libuuid on OS X — I just remember
that I had some big problems with it. So, my OS X luuid installation
may be broken. Or is it the expected behaviour? Is there a workaround
then?

See details below.

I use April 2009 version of luuid.

Alexander.

The test code:

$ cat uuid-fork.lua
local posix = require 'posix'
local uuid = require 'uuid'

if assert(posix.fork()) ~= 0 then
  io.write("parent 1: ", uuid.new(), "\n")
  io.write("parent 2: ", uuid.new(), "\n")
  io.write("parent 3: ", uuid.new(), "\n")
  io.write("parent 4: ", uuid.new(), "\n")
  io.write("parent 5: ", uuid.new(), "\n")
else
  io.write("child  1: ", uuid.new(), "\n")
  io.write("child  2: ", uuid.new(), "\n")
  io.write("child  3: ", uuid.new(), "\n")
  io.write("child  4: ", uuid.new(), "\n")
  io.write("child  5: ", uuid.new(), "\n")
end

The output on OS X box:

$ lua -lluarocks.require uuid-fork.lua
parent 1: DF9C3E77-1ACB-4B15-81C8-D0E3AB0A7B4B
parent 2: 68062AA2-721D-43FF-87D6-9007F9265C2A
parent 3: 39EE9673-BE8D-4EB0-8C1D-2D22CBE94920
parent 4: B895873C-B589-4944-AE16-39D811E28E5A
parent 5: DE5A1755-096B-4656-8001-F280DBB33358
child  1: DF9C3E77-1ACB-4B15-81C8-D0E3AB0A7B4B
child  2: 68062AA2-721D-43FF-87D6-9007F9265C2A
child  3: 39EE9673-BE8D-4EB0-8C1D-2D22CBE94920
child  4: B895873C-B589-4944-AE16-39D811E28E5A
child  5: DE5A1755-096B-4656-8001-F280DBB33358

The output on Ubuntu box:

$ lua -lluarocks.require uuid-fork.lua
parent 1: 43276d7a-ec33-418d-bd99-27bb27ed4ed8
parent 2: 58553e1e-81da-41fc-8dab-50c3d9bf8dcc
parent 3: 461a9247-876a-41f7-add5-87b4572d7740
parent 4: cbe66c54-43cd-4225-b37c-a801503606c8
child  1: 15c32294-add7-4a47-82ee-89c019667cb4
parent 5: c1ec9f2f-b1ba-4851-a718-e0d39073d2c0
child  2: 3ece9152-49f1-4133-803d-5842390d63e4
child  3: fda13f6c-b7ac-407b-abac-5b913b519c70
child  4: 16450b69-33ed-4402-a6e0-869c86477213
child  5: 7d5a4920-8bf9-4c3e-9522-d09574b4aece