Re: Windows SBCL annoyances
Anton Kovalenko <anton <at> sw4me.com>
2011-11-02 00:09:03 GMT
Faré <fahree <at> gmail.com> writes:
> How is the merge going along?
Better than it used to be. David Lichteblau is working on merge, and he
already made some non-revolutionary IO-related things into 1.0.52. That
is, "merge bandwidth" is now greater than "feature creep bandwidth",
which is a great improvement.
> Maybe instead of trying to acheive a complete merge in N perfectly
> separated patches, you could try do one patch at once, get it merged,
> then work on the next patch?
That's how I expected it to be done. Though there are some large
logically inseparable chunks, many things can be separated, and I'm
normally ready to do it when I have time. The latest big thing was a
support for file names longer than MAX_PATH on unicode windows, and I've
backported everything (i.e. cherry-picked with some minor modifications)
to a separate branch [lfn-upstream] that stays ready for merge until it
becomes stale.
The problem here is that I can't really know what can be accepted soon,
which SBCL developers have time to review patches and what areas should
I focus on. Keeping outstanding pieces prepared for integration takes
some resources, and I'm not yet ready to do it on /everything/ just in
case someone suddenly agrees to accept /something/. Discussing things
here does not help /this/ problem too much: I've got some very useful
feedback on long file names, for example, but my impression that it
has good chances to be integrated soon was incorrect.
Since August 2011, SBCL/windows doesn't help me to pay my bills any
longer, so I have to allocate my time more conservatively. I'm
determined to keep it from bit rot no matter what, making a buildable
tree of non-stale SBCL/windows available. Of course, things may change
to the better at any time, as I started to accept donations for ongoing
SBCL/Windows work: <http://www.siftsoft.com/support-sbcl-windows.html>.
I'm kind of skeptical here, 'cause in all my life it never worked for
/windows-specific software/, but users of portable, unix-centric Lisp
implementation are not just a typical group of "Windows people", so
there's some hope.
>>> 2- Can there be a way for run-program to pass arguments unescaped to
>>> cmd.exe and other programs that do not follow the standard argument
>>> parsing convention? Being able to issue system commands via cmd.exe
>>> would really be nice.
>> That's probably a misdiagnosed problem: you attribute a problem in
>> MSVCRT spawn implementation to non-standard argument parsing of cmd.exe.
>> Please provide an example which doesn't work, so I'll know what to test.
> No, no. I'm using xcvb-driver:run-program/read-output-string,
> which does about the same as you do below, except somewhat portably.
> It works well with CCL or Allegro (thanks to a bug (858) in CCL, at that!):
>
> ccl --load driver.lisp --eval "(progn (princ
> (xcvb-driver:run-program/read-output-string "cmd /c echo \"a b c\""))
> (terpri) (quit))"
> ==>
> "a b c"
>
> i.e. the command line is passed as is for cmd to interpret.
[...]
It's a big can of worms, really, with all parties doing their fair share
of bugs and confusion. I'll try to explain (non-exhaustively) what's
going on in SBCL/Windows, both mainline and patched.
Windows is a non-Unix in this department: it has a real notion of
unparsed command line, and that's what is passed to child processes on
the lowest level where it makes sense (kernel32:CreateProcess). On top
of that, Windows has a convention of breaking command line into argv[]:
CommandLineToArgvW is a "reference implementation"; some runtime
libraries and even some programs opted out to roll their own equivalent,
but, with very rare exceptions, any divergence from CommandLineToArgvW
is considered a bug in a program or a runtime library.
ECHO built-in command of cmd.exe is the worst thing we could choose for
"typical example": it's one of those rarest exceptions that doesn't work
on argv[], but rather on raw command line:
$ echo "off"
"off"
$ echo off
(in an interactive session, the /prompt/ disappears here, but line input
is still visible, because it's required for consoles' "cooked" mode
equivalent).
Hence you have to use some other program to test how your parameters are
likely to be interpreted. Some cat.exe [kittens of death again] from
MSYS, or cygwin, or GnuWin32 is helpful here: given a non-existent file
name, it says /which/ file is not found.
CMD /C has some obscure heuristics inside to make it "work" in more
cases. Unless you explicitly want to be able to run built-in commands,
you don't really want to introduce CMD /C into picture.
I believe, however, that for any program which /can/ be found, the
following things are equivalent:
(run-program "cmd" `("/c" ,program , <at> args) :search t)
(run-program program args :search t)
(Though they will escape some args differently in mainline SBCL and in
my fork, these two calls will be bug-for-bug compatible nevertheless).
CommandLineToArgvW convention is documented and well-known, but it's
definitely not what you expect until you read about it. Trivial cases
are unix-shell-style, of course:
two words => {"two","words"}
"one word" => {"one word"}
"one""word" => {"oneword"}
Single-quotes are not special, however:
'single-quoted' => {"'single-quoted'"}
"join"'pieces' => {"join'pieces'"}
Microsoft had to provide some way for embedding double-quotes into
arguments. The obvious Unix-like choice of backslash escaping was
obviously wrong on a system where \ is a filename separator, but when
did it prevent Microsoft from doing anything? So they decided to use \
as a single-escape character and throwed in some additional rules to
keep \-separated filenames usable. Backslashes are only special before
double-quotes; if there is a group of 2n backslashes there, they denote
n "real" backslashes, and 2n+1 backslashes denote n backslashes and
prevent special interpretation of the following double-quote. All other
backslashes denote themselves and don't escape anything.
SB-IMPL::MSWIN-ESCAPE-COMMAND-ARGUMENT in my fork tries to reflect this
logic accurately (it works on a single argument and surrounds it with
double-quotes when it has spaces... wait, there seems to be a subtle bug
here, so I have to look deeper. For "double-quoted-spaceless-word",
quotes will mysteriously disappear in final argv). SB-IMPL::ESCAPE-ARG
from SBCL doesn't do any more than adding double-quotes. There is a
reason for it: underlying primitive used for RUN-PROGRAM on windows is
MSVCRT's _spawnv or _spawnvp, and it pretends to accept argv array, not
a command line. Why escaping is needed at all here?
The problem: spawnvp and spawnv make a command line by throwing in
spaces as argument separators, but they don't apply escaping to
arguments themselves. This behavior is actually /documented/, but you
have to follow a link to _spawn on MSDN: the description of spawnv and
spawnvp contains no such information.
I decided to get rid of spawn* altogether by switching to CreateProcess,
and done it in my fork. In the context of above description, it's easy
to understand why I don't want any more of "native" command-line
construction and interpretation layers than absolutely necessary.
All this experience made me believe that both RUN-PROGRAM and
RUN-SHELL-COMMAND should be provided by implementation, the first
working on argv-style lists and the second on text command line.
--
--
Regards, Anton Kovalenko <http://github.com/akovalenko/sbcl-win32-threads>
+7(916)345-34-02 | Elektrostal' MO, Russia
------------------------------------------------------------------------------
RSA® Conference 2012
Save $700 by Nov 18
Register now!
http://p.sf.net/sfu/rsa-sfdev2dev1
_______________________________________________
Sbcl-devel mailing list
Sbcl-devel <at> lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/sbcl-devel
RSS Feed