How to determine if my lua interpreter support dynamic link ?

Multi-line LPeg CSV parser

I started with the example in the LPeg manual for CSV parsing and tried 
adding support to have it parse an entire buffer of multiple CSV lines:

local field = '"' * lpeg.Cs(((lpeg.P(1) - '"') + lpeg.P'""' / '"')^0) * '"' +
                    lpeg.C((1 - lpeg.S',\r\n"')^0)

local record = lpeg.Ct(field * (',' * field)^0) * (lpeg.P'\r\n' + lpeg.P'\n' + -1)
local all = lpeg.Ct(record^0)

This fails, of course, with a "loop body may accept empty string" error.

What is the proper solution for making that error go away?

Thanks.

Josh

Injecting names & method chaining (was Re: [ANN] Penlight Libraries, First release)

With regard to injection, it's interesting to look at Michael Franz's  
paper on Protocol Extension.

ftp://ftp.inf.ethz.ch/doc/tech-reports/2xx/226.ps.gz

Essentially, he uses the module system in Oberon to avoid name  
conflicts on extensions.

In the context of something like the method chaining wrapper, one  
could write:

	local stringx = require "stringx"

	S "  test  " : [ stringx.trim ] () ()

(This assumes a Lua extension to support calling methods identified by  
variables. The patch to support this looks reasonably simple, but I  
haven't pushed hard on it.)

Or for more efficient execution, one could write:

	local stringx = require "stringx"
	local stringx_trim = stringx.trim

	S "  test " : [ stringx_trim ] () ()

Of course, this then raises the question, why not just name the method  
"stringx_trim" and be done with it:

	S "  test " : stringx_trim() ()

One answer might be that the method chaining wrapper could handle  
functions passed as indices by returning the function itself. Then  
stringx.trim isn't just an identifier for the method, it can actually  
be the method implementation.

Mark

答复: How to determine if my lua interpreter support dynamic link ?

Re: Multi-line LPeg CSV parser

On Fri, May 01, 2009 at 10:44:21AM -0600, Joshua Jensen wrote:
> I started with the example in the LPeg manual for CSV parsing and tried 
> adding support to have it parse an entire buffer of multiple CSV lines:
[...]
> local record = lpeg.Ct(field * (',' * field)^0) * (lpeg.P'\r\n' + 
> lpeg.P'\n' + -1)
> local all = lpeg.Ct(record^0)
>
> This fails, of course, with a "loop body may accept empty string" error.
>
> What is the proper solution for making that error go away?

Hi,

Well, you basically just need to rewrite the expression so you
don't have any empty strings inside loops.  I hope the
following examples will help:

require 'lpeg'

local P  = lpeg.P
local C  = lpeg.C
local Cs = lpeg.Cs
local Ct = lpeg.Ct

local a_qt   = '"'
local a_fsep = ','
local a_rsep = '\n'

local qt   = P( a_qt )
local fsep = P( a_fsep )
local rsep = P( a_rsep ) + P'\r\n'

local unquoted_field = C( (1-(qt + fsep + rsep)) ^0 )
local quoted_field   = qt * Cs( ((P(1)-qt) + (qt*qt)/a_qt) ^0) * qt
local field          = quoted_field + unquoted_field
local record         = Ct( field  * (fsep*field)^0  )
local records        = Ct( record * (rsep*record)^0 )
local all            = records * (rsep + -1)

local txt = [[
1,2,3
a,b,c
"1,a","2""b",""""
]]

local res = lpeg.match( all, txt )

for i, t in ipairs( res ) do
    print( ' <at>  <at>  '..i, t )
    table.foreach( t, print )
end

Note that this might not work quite as expected. If empty
records are allowed, and the string ends with a record
separator, and the last record separator is optional, we don't
know if there should be an intentional empty record between the
last record separator and the end of string, or if the last
record separator just terminates the previous record. So an
alternative is:

local records        = Ct( record * (rsep*#P(1)*record)^0 )

Also note that a syntax error in the string is not detected. The
part after the error is just silently not matched by the lpeg
expression.

A way that more clearly express the other way of thinking is:

local nonemptyrecord = #P(1-rsep) * record
local records        = Ct( (record*rsep)^0 * nonemptyrecord^-1 )
local all            = records * -1

This last version will return nil on errors, since 'all' must
match the end of string.

--

-- 
Tommy Pettersson <ptp <at> lysator.liu.se>

Re: Multi-line LPeg CSV parser

----- Original Message -----
From: Tommy Pettersson
Date: 5/1/2009 2:24 PM
> Well, you basically just need to rewrite the expression so you
> don't have any empty strings inside loops.  I hope the
> following examples will help:
>   
They were very helpful.

It's been a long while since I did anything extensive with LPeg.  I 
think version 0.5 was the last one I really used, and I have forgotten 
much of what I learned then.  I appreciate your willingness to help me out.

Thank you!

Josh

Re: Injecting names & method chaining (was Re: [ANN] Penlight Libraries, First release)

On Fri, May 1, 2009 at 12:48 PM, Mark Hamburg wrote:
> With regard to injection, it's interesting to look at Michael Franz's paper
> on Protocol Extension.
> ftp://ftp.inf.ethz.ch/doc/tech-reports/2xx/226.ps.gz
> Essentially, he uses the module system in Oberon to avoid name conflicts on
> extensions.
> In the context of something like the method chaining wrapper,

i.e. http://lua-users.org/wiki/MethodChainingWrapper

> one could write:
>        local stringx = require "stringx"
>        S "  test  " : [ stringx.trim ] () ()
> (This assumes a Lua extension to support calling methods identified by
> variables.

Such a patch does not in itself allow us to write the expression how
we might really want to:

    ("  test  "):[stringx.trim]()

But we can achieve it if we allow an adjustment to the string metatable:

    debug.getmetatable"".__index = function(self, k) return k end

or even without patching,

  debug.getmetatable"".__index = function(self, k)
      return string[k] or function(...) return k(self, ...) end
  end
  return ("  test  ")[stringx.trim]()

This does, however, lead to the curious behavior that s[v] ~= nil for
all values v and strings s, unless we add further conditionals.  In
Lua, unlike some other languages, calling a method on an object is not
an atomic operation (passing a message) but is defined in terms of an
index and a function call.  s:v(...) is syntactic sugar for s["v"](s,
...).  This imposes restrictions on what we can do with the index
operator (e.g. method s:v exists implies that s["v"] ~= nil).

I also thought it was convenient in the MethodChainingWrapper that the
namespace for operations was held by the wrapper.  Instead of the
above, where stringx.trim is a variable defined outside of the
expression, the operation names are localized to the call chain
expression by the wrapper:

      -S("  test  "):trim()    -- "trim" is a method in the namespace "S"

There may be uses for both approaches.

Maybe the better solution to all this, in a Lua context, is to use
something other than method calls to do the operation chaining.
Execution-wise, we simply want to do

    m.h(m.g(m.f(x, p...), q...), r...)
        -- for argument lists p, q, r, and operations f, g, h in module m.

Syntax-wise, we'd like a less awkward form of expression, including
one that perhaps defines some automatic namespace lookup for the
operations.

Patching the parser or applying Metalua would allow an equivalent syntax like

    x chain m : f(p...) : g(q...) : h(r...)

e.g.

    ("  test  ") chain stringx : trim() : repeatchars(5) : upper()

Or in standard Lua, we could write a helper function like

    chain(stringx, trim, 1,repeatchars,5, "  test  ")

where a number before an operation name (e.g. 1 before repeatchars)
indicates that the operation takes that many number of arguments.

Re: Injecting names & method chaining (was Re: [ANN] Penlight Libraries, First release)

On Fri, May 1, 2009 at 11:31 PM, David Manura wrote:
> Or in standard Lua, we could write a helper function like
>    chain(stringx, trim, 1,repeatchars,5, "  test  ")

Actually, more like chain(stringx, 'trim', 1,'repeatchars',5, 'upper',
nil, '  test  ').

Another idea is chain(stringx):trim():repeatchars(5):upper()('  test
'), which may be preferable to the current approach in
MethodChainingWrapper.  That even allows separation and reuse like

  f = chain(stringx):trim():repeatchars(5):upper()
  print ( f('  test  ') )
  print ( f('  again  ') )

Re: Injecting names & method chaining (was Re: [ANN] Penlight Libraries, First release)

On May 1, 2009, at 8:42 PM, David Manura wrote:

>  f = chain(stringx):trim():repeatchars(5):upper()

This potentially leads to:

	f = chain( string, stringx ):trim():repeatchars(5):upper()

thereby eliminating the need for adding the methods from string to  
stringx.

On a potentially related note, I had previously been thinking about  
ways to do chaining/pipelining for iterator construction. As I recall,  
that had lead to constructs like:

	pipe( pairs, map( function( k, v ) return tostring( k ),  
tostring( v ) end ), filter( function( k, v ) return #k == 3 end ) ) ) 
( t )

Those had never felt as clean as one might like, however, though they  
did encourage saving the pipe for reuse. Perhaps there's something to  
be explored in your chain solution.

Mark

P.S. The obj:[ method ] syntax suggestion actually arose out of  
wanting to make it easier to call variable methods without expanding  
the colon operator and thereby muddying the waters about the intent of  
the code.

Re: Injecting names & method chaining (was Re: [ANN] Penlight Libraries, First release)

On Sat, May 2, 2009 at 7:37 AM, Mark Hamburg <mark <at> grubmah.com> wrote:
> On a potentially related note, I had previously been thinking about ways to
> do chaining/pipelining for iterator construction. As I recall, that had lead
> to constructs like:
>
>        pipe( pairs, map( function( k, v ) return tostring( k ), tostring( v
> ) end ), filter( function( k, v ) return #k == 3 end ) ) )( t )
>
> Those had never felt as clean as one might like, however, though they did
> encourage saving the pipe for reuse. Perhaps there's something to be
> explored in your chain solution.

As an example of convergent thinking, I've been looking at this very
issue. This was inspired by a recent comment of David, and attempts to
make pl.seq more useful and intuitive:

http://lua-users.org/wiki/SequenceAdapters

The basic insight is that we are culturally conditioned to prefer
chains of operations to move from left to right, which is perhaps one
of the hidden strengths of OOP notation.

For an example, this gets all the unique lines in the file and prints them out:

S(io.lines(fname)):unique():printall '\n'

And this puts all the unique identifiers in a Lua code string into a
table result:

ls = S(lexer.lua(str)):filter(Eq(_1,'iden')):map(_2):unique():copy()

Any methods which are not recognized are looked up using the first
value of the sequence:

S{'[one]','[two]','[three]'}:sub(2,-2):upper():printall() ---> ONE TWO THREE

steve d.