General discussion of Steel Bank Common Lisp

nicolas edel | 1 May 2011 11:20

communicating with external process using a pty

Hi,

Consider I'd like to run programs using ptys and interact with them.
So I started considering such code snippet:

;;; open a shell and ask for directory listing
(defvar proc (sb-ext:run-program "sh" nil
                    :search t :input :stream :output :stream :wait nil))
(format (sb-ext:process-input proc) "ls -l /tmp~%echo '###'~%")
(force-output (sb-ext:process-input proc))
(loop
          :with output = (sb-ext:process-output proc)
          :for line = (read-line output nil 'eof)
          :until (or (eq line 'eof) (string= line "###"))
          :do (print line))

That works fine, but as intended, it uses pipes as IPC.
How to do this if I want to use pty ? I tried using :pty :stream, but this didn't work. I can see the process is indeed spawned with pty allocated, but I can't communicate with it while the process is still alive:

CL-USER> (defvar proc (sb-ext:run-program "sh" nil
                    :search t :input :stream :output :stream :pty :stream :wait nil))
PROC
CL-USER> proc
#<SB-IMPL::PROCESS 5195 :RUNNING>
CL-USER> (sb-ext:process-input proc)
#<SB-SYS:FD-STREAM for "descriptor 10" {B9A6671}>
CL-USER> (sb-ext:process-output proc)
#<SB-SYS:FD-STREAM for "descriptor 12" {B9A66F9}>
CL-USER> (format (sb-ext:process-input proc) "ls -l /tmp~%echo '###'~%")
NIL
;;; now flush output:
CL-USER> (force-output (sb-ext:process-input proc))
Couldn't write to #<SB-SYS:FD-STREAM for "descriptor 10" {B9A6671}>:
Broken pipe
   [Condition of type SB-INT:SIMPLE-STREAM-ERROR]
; Evaluation aborted on #<SB-INT:SIMPLE-STREAM-ERROR {AD6B5D9}>.
CL-USER>

and back to shell:
$ ls -l /proc/5195/fd
lrwx------ 1 nicolas users 64 2011-05-01 11:01 0 -> /dev/pts/3
lrwx------ 1 nicolas users 64 2011-05-01 11:05 1 -> /dev/pts/3
lrwx------ 1 nicolas users 64 2011-05-01 11:05 2 -> /dev/pts/3
lrwx------ 1 nicolas users 64 2011-05-01 11:05 255 -> /dev/pts/3

Obviously, I am missing something, but looking at the sbcl documentation didn't help me that much.
Could someone explain ?
Thanks.

:Nicolas

------------------------------------------------------------------------------
WhatsUp Gold - Download Free Network Management Software
The most intuitive, comprehensive, and cost-effective network 
management toolset available today.  Delivers lowest initial 
acquisition cost and overall TCO of any competing solution.
http://p.sf.net/sfu/whatsupgold-sd

_______________________________________________
Sbcl-help mailing list
Sbcl-help <at> lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/sbcl-help

headers

Stig Hemmer | 1 May 2011 19:38

Re: communicating with external process using a pty

On 5/1/11, nicolas edel <nicolas.edel <at> gmail.com> wrote:
| CL-USER> (defvar proc (sb-ext:run-program "sh" nil
|                                                       :search t :input
| :stream :output :stream :pty :stream :wait nil))

The :input and :output arguments establish stdin/out for the process.
The :pty argument ALSO establishes these.

So, when using :pty, don't use :input/:output.
Furthermore, :pty doesn't take :stream as an argument, just give it T
And afterwards, PROCESS-PTY gives the relevant stream.

* (defvar proc (sb-ext:run-program "sh" nil
                       :search t :pty t :wait nil))
PROC

* (format (sb-ext:process-pty proc) "ls -l /tmp~%echo '###'~%")

NIL
* (force-output (sb-ext:process-pty proc))

NIL
* (read-line (sb-ext:process-pty proc))

"sh: can't access tty; job control turned off
NIL
* (read-line (sb-ext:process-pty proc))

"$ totalt 75544
NIL
* (read-line (sb-ext:process-pty proc))

"-rwxrwxrwx 1 gombos   fidi_s       104 2011-04-15 13:58 chat
NIL
* (read-line (sb-ext:process-pty proc))

"drwx------ 2 andrfla  eitk_s      4096 2011-04-14 14:09 dconf.8PPxFp
NIL
* (read-line (sb-ext:process-pty proc))

"drwx------ 2 somaen   fall        4096 2011-04-19 11:57 dconf.A9uC3w
NIL
*
etc etc

I hope that helped a little.  I see that sh complained about not
having a tty, I hope this won't become a problem for you.

Stig Hemmer

------------------------------------------------------------------------------
WhatsUp Gold - Download Free Network Management Software
The most intuitive, comprehensive, and cost-effective network 
management toolset available today.  Delivers lowest initial 
acquisition cost and overall TCO of any competing solution.
http://p.sf.net/sfu/whatsupgold-sd

headers

nicolas edel | 1 May 2011 20:21

Re: communicating with external process using a pty

On Sun, May 1, 2011 at 7:38 PM, Stig Hemmer <stighemmer <at> gmail.com> wrote:

On 5/1/11, nicolas edel <nicolas.edel <at> gmail.com> wrote:
| CL-USER> (defvar proc (sb-ext:run-program "sh" nil
| :search t :input
| :stream :output :stream :pty :stream :wait nil))

The :input and :output arguments establish stdin/out for the process.
The :pty argument ALSO establishes these.

So, when using :pty, don't use :input/:output.
Furthermore, :pty doesn't take :stream as an argument, just give it T
And afterwards, PROCESS-PTY gives the relevant stream.

* (defvar proc (sb-ext:run-program "sh" nil
:search t :pty t :wait nil))
PROC

[snip]
etc etc

I hope that helped a little. I see that sh complained about not
having a tty, I hope this won't become a problem for you.

Stig Hemmer

Yes, this help.
Note:
- sh doesn't complain here
- ssh complains: "ssh_askpass: exec(/usr/bin/ssh-askpass): No such file or directory^M^M "

Looks like wellknown caveats:
- termios are not set properly (termios.c_oflag &= ~ONLCR)
- ioctl (pty, TIOCSCTTY, 1) is missing after fork

But once again, yes that helped. Thanks ;)

:Nicolas

------------------------------------------------------------------------------
WhatsUp Gold - Download Free Network Management Software
The most intuitive, comprehensive, and cost-effective network 
management toolset available today.  Delivers lowest initial 
acquisition cost and overall TCO of any competing solution.
http://p.sf.net/sfu/whatsupgold-sd

_______________________________________________
Sbcl-help mailing list
Sbcl-help <at> lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/sbcl-help

headers

Roman Marynchak | 10 May 2011 11:08

run-program and bat files on Windows

Hello,

I have strange problems with launching bat scripts on Windows. The idea is to use cmd.exe and a bat file as an argument. During the process of rewriting the elegant solutions into the stupid ones, I found that even this all-hardcoded variant is not working:

(sb-ext:run-program "C:/Windows/System32/cmd.exe" '("/c" "D:/run.bat"))

It executes without problems, but run.bat does nothing (or cmd does not run it at all). I have tried both SBCL 1.0.48 and Win64 fork.

Can something better be done here? I will appreciate any advices.

Thank you,

Roman

------------------------------------------------------------------------------
Achieve unprecedented app performance and reliability
What every C/C++ and Fortran developer should know.
Learn how Intel has extended the reach of its next-generation tools
to help boost performance applications - inlcuding clusters.
http://p.sf.net/sfu/intel-dev2devmay

_______________________________________________
Sbcl-help mailing list
Sbcl-help <at> lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/sbcl-help

headers

Nikodemus Siivola | 10 May 2011 23:55

Re: [Sbcl-bugs] sbcl x86 binary distribution v1.0.48

2011/5/10 José María García Pérez <josemaria.alkala <at> gmail.com>:

(Moving to sbcl-help.)

> I am trying to produce a package for StumpWM for TinyCoreLinux. So please,
> bear in mind I am not a Lisp user. To compile it, I first need to have
> "sbcl" working in the distribution. I have tried the existing "sbcl" package
> which doesn't seem to work. I have created a new one based on the binary
> file:
> http://prdownloads.sourceforge.net/sbcl/sbcl-1.0.48-x86-linux-binary.tar.bz2
>
> Then I tried to download the "clx" package by executing:
>  $ sbcl
>  * (require 'asdf)
>  * (asdf:oos 'asdf:load-op :asdf-install)
>  * (asdf-install:install 'clx)
>
> But there seem to be lot of problems when I execute: (asdf:oos 'asdf:load-op
> :asdf-install).
>
> Could somebody help me to find what I am doing wrong?

Without any more details as to what goes wrong it's hard to say.

However, ASDF-INSTALL as an installation mechanism leaves a lot to be
desired. (It's defects are too numerous to fit into this metaphorical
margin, and besides the point.)

You might have better luck using Quicklisp -- which is pretty much the
successor of CLX.

 wget http://beta.quicklisp.org/quicklisp.lisp
 sbcl --load quicklisp.lisp
 # follow prompts, then new a new SBCL session
 (ql:quickload "clx")

Cheers,

 -- Nikodemus

------------------------------------------------------------------------------
Achieve unprecedented app performance and reliability
What every C/C++ and Fortran developer should know.
Learn how Intel has extended the reach of its next-generation tools
to help boost performance applications - inlcuding clusters.
http://p.sf.net/sfu/intel-dev2devmay
_______________________________________________
Sbcl-help mailing list
Sbcl-help <at> lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/sbcl-help

headers

Nikodemus Siivola | 11 May 2011 01:06

Re: [Sbcl-bugs] sbcl x86 binary distribution v1.0.48

2011/5/11 José María García Pérez <josemaria.alkala <at> gmail.com>:

> tc <at> box:/tmp$ sbcl --load quicklisp.lisp > build.log
> ASDF could not load sb-bsd-sockets because failed to find the TRUENAME of
> /home/tc/.cache/common-lisp/sbcl-1.0.48-linux-x86/tmp/tcloop/sbcl-x86-bin/usr/local/lib/sbcl/sb-bsd-sockets/constants.lisp-temp:
>                                             
No such file or directory.

I'm betting the same happens if you do just

  (require :sb-bsd-sockets)

?

Something is messed up in the environment or the installation: all
sb-bsd-sockets related files should be built by make.sh and installed
into /usr/local/lib/sbcl/.. by install.sh.

From the error it looks like that has happened, but something is causing
the system to think they need to be recompiled ... and badness happens.

My first thought is that something is funny about the timestamps.

What does

(list
 (file-write-date "/usr/local/lib/sbcl/sb-bsd-sockets/constants.lisp")
 (file-write-date "/usr/local/lib/sbcl/sb-bsd-sockets/constants.lisp-temp")
 (file-write-date "/usr/local/lib/sbcl/sb-bsd-sockets/constants.fasl")
 (get-universal-time))

return? The numbers should be in increasing order.

Cheers,

 -- Nikodemus

------------------------------------------------------------------------------
Achieve unprecedented app performance and reliability
What every C/C++ and Fortran developer should know.
Learn how Intel has extended the reach of its next-generation tools
to help boost performance applications - inlcuding clusters.
http://p.sf.net/sfu/intel-dev2devmay
_______________________________________________
Sbcl-help mailing list
Sbcl-help <at> lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/sbcl-help

headers

Sean Bryant | 11 May 2011 02:00

Creating a coverage operation in asdf

Hello sbcl,

First time posting and a fairly new user. I'm a fairly new lisper and
I'm running into trouble trying to generate coverage reports (using
sb-cover, obviously) for a specific package in my project.

I tried to just add the (declaim (optimization
sb-cover:store-coverage-data)) after loading the system, but then I
realized I made a horrible mistake as sbcl uses that optimization
declaration to add tracing data to the compiled code. I then naively
decided it would be a good idea to just force a recompile of the
system. To my horror, I ended up recompiling everything sbcl touched,
cl-sql, bordeaux threads, and more. The good news is that I managed to
generate the coverage report but only with tons of external dependencies
being included in the report as well. As much as I like knowing these
things, it's not exactly useful. 

The solution I'm trying to come up with is creating a new type of
operation in asdf conveniently called cover-op. I haven't had much luck
in getting this working (I've only spent an hour on this, so not really
surprising given my exposure to asdf). Currently, I like the setup with
asdf:test-system and then just specialize perform to run the tests. I
would like to do the same thing to generate coverage reports, but the
compilation step is proving the be a problem. 

I'm wondering what other sbcl users do to ensure coverage reports are
generating for only the "interesting" aspects of your applications? 

Thanks,
Sean

------------------------------------------------------------------------------
Achieve unprecedented app performance and reliability
What every C/C++ and Fortran developer should know.
Learn how Intel has extended the reach of its next-generation tools
to help boost performance applications - inlcuding clusters.
http://p.sf.net/sfu/intel-dev2devmay

headers

Nikodemus Siivola | 11 May 2011 18:49

Deadlock detection has landed

I just committed deadlock detection code for spinlocks and mutexes into CVS.

Please let me know if you see slowdowns in applications due to this:
currently deadlocks are checked always on contested locks, but I can
make it optional if there are performance issues.

Cheers,

 -- Nikodemus

------------------------------------------------------------------------------
Achieve unprecedented app performance and reliability
What every C/C++ and Fortran developer should know.
Learn how Intel has extended the reach of its next-generation tools
to help boost performance applications - inlcuding clusters.
http://p.sf.net/sfu/intel-dev2devmay

headers

Jim Wise | 11 May 2011 20:20

Re: [Sbcl-devel] Deadlock detection has landed

Nikodemus Siivola <nikodemus <at> random-state.net> writes:

> I just committed deadlock detection code for spinlocks and mutexes into CVS.
>
> Please let me know if you see slowdowns in applications due to this:
> currently deadlocks are checked always on contested locks, but I can
> make it optional if there are performance issues.

FWIW, the new test DEADLOCK-DETECTION.3 hangs indefinitely on Solaris/x86
(.1 and .2 run successfully).  I'll test later today or early tomorrow
on MacOS, to see if this affects other lutex builds.

I interrupted the run manually after some time, yielding the following
stack trace (I've trimmed to just the bits from when the test is begun
until the SIGINT begins to be handled):

14: ((FLET SB-UNIX::RUN-HANDLER))[:EXTERNAL]
15: ("foreign function: call_into_lisp")
16: ("foreign function: funcall3")
17: ("foreign function: interrupt_handle_now")
18: ("foreign function: maybe_defer_handler")
19: ("foreign function: __moddi3")
20: ("foreign function: _sema_post")
21: ("foreign function: _sema_post")
22: ("bogus stack frame")
23: ("foreign function: _thr_schedctl")
24: ("foreign function: mutex_lock")
25: ("foreign function: lutex_lock")
26: (GET-MUTEX
     #<unavailable argument>
     #<unavailable argument>
     #<unavailable argument>
     #<unavailable argument>)
27: ((FLET #:WITHOUT-INTERRUPTS-BODY-[CALL-WITH-MUTEX]301))
28: (SB-THREAD::CALL-WITH-MUTEX
     #<FUNCTION (FLET SB-THREAD::WITH-MUTEX-THUNK) {4AACFDCD}>
     #<MUTEX "M1"
         owner: #1=#<THREAD "T1" #2=waiting for:
                        #<MUTEX "M2" owner: #>
                      {4AADA649}>>
     #<THREAD "initial thread" waiting for:
          #<MUTEX "M1" owner: #1=#<THREAD "T1" #2=waiting for: # {4AADA649}>>
          timeout:
          0.1
        {49A8B1A9}>
     T)
29: ((FLET SB-THREAD::WITH-MUTEX-THUNK))
30: ((FLET #:WITHOUT-INTERRUPTS-BODY-[CALL-WITH-MUTEX]301))
31: (SB-THREAD::CALL-WITH-MUTEX
     #<CLOSURE (FLET SB-THREAD::WITH-MUTEX-THUNK) {FE9FF715}>
     #<MUTEX "M2"
         owner: #1=#<THREAD "initial thread" #2=waiting for:
                        #<MUTEX "M1" owner: #>
                        timeout:
                        0.1
                      {49A8B1A9}>>
     #<THREAD "initial thread" waiting for:
          #<MUTEX "M1" owner: #1=#<THREAD "T1" #2=waiting for: # {4AADA649}>>
          timeout:
          0.1
        {49A8B1A9}>
     T)
32: (#:EVAL-THUNK)
33: (SB-INT:SIMPLE-EVAL-IN-LEXENV
     (WITH-TEST (:NAME DEADLOCK-DETECTION.3)
       (LET* ((M1 (MAKE-MUTEX :NAME "M1"))
              (M2 (MAKE-MUTEX :NAME "M2"))
              (S1 (MAKE-SEMAPHORE :NAME "S1"))
              (S2 (MAKE-SEMAPHORE :NAME "S2"))
              (T1
               (MAKE-THREAD
                (LAMBDA ()
                  (WITH-MUTEX (M1)
                    (SIGNAL-SEMAPHORE S1)
                    (WAIT-ON-SEMAPHORE S2)
                    (WITH-MUTEX (M2)
                      :OK)))
                :NAME "T1")))
         (ASSERT
          (EQ :DEADLINE
              (HANDLER-CASE
               (WITH-MUTEX (M2)
                 (SIGNAL-SEMAPHORE S2)
                 (WAIT-ON-SEMAPHORE S1)
                 (SLEEP 1)
                 (SB-SYS:WITH-DEADLINE (:SECONDS 0.1)
                   (WITH-MUTEX (M1)
                     :OK)))
               (SB-SYS:DEADLINE-TIMEOUT NIL :DEADLINE))))
         (ASSERT (EQ :OK (JOIN-THREAD T1)))))
     #<NULL-LEXENV>)
34: (SB-EXT:EVAL-TLF
     (WITH-TEST (:NAME DEADLOCK-DETECTION.3)
       (LET* ((M1 (MAKE-MUTEX :NAME "M1"))
              (M2 (MAKE-MUTEX :NAME "M2"))
              (S1 (MAKE-SEMAPHORE :NAME "S1"))
              (S2 (MAKE-SEMAPHORE :NAME "S2"))
              (T1
               (MAKE-THREAD
                (LAMBDA ()
                  (WITH-MUTEX (M1)
                    (SIGNAL-SEMAPHORE S1)
                    (WAIT-ON-SEMAPHORE S2)
                    (WITH-MUTEX (M2)
                      :OK)))
                :NAME "T1")))
         (ASSERT
          (EQ :DEADLINE
              (HANDLER-CASE
               (WITH-MUTEX (M2)
                 (SIGNAL-SEMAPHORE S2)
                 (WAIT-ON-SEMAPHORE S1)
                 (SLEEP 1)
                 (SB-SYS:WITH-DEADLINE (:SECONDS 0.1)
                   (WITH-MUTEX (M1)
                     :OK)))
               (SB-SYS:DEADLINE-TIMEOUT NIL :DEADLINE))))
         (ASSERT (EQ :OK (JOIN-THREAD T1)))))
     21
     #<NULL-LEXENV>)

--

-- 
				Jim Wise
				jwise <at> draga.com

------------------------------------------------------------------------------
Achieve unprecedented app performance and reliability
What every C/C++ and Fortran developer should know.
Learn how Intel has extended the reach of its next-generation tools
to help boost performance applications - inlcuding clusters.
http://p.sf.net/sfu/intel-dev2devmay

_______________________________________________
Sbcl-help mailing list
Sbcl-help <at> lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/sbcl-help

headers

Nikodemus Siivola | 11 May 2011 20:24

Re: [Sbcl-devel] Deadlock detection has landed

On 11 May 2011 21:20, Jim Wise <jwise <at> draga.com> wrote:

> FWIW, the new test DEADLOCK-DETECTION.3 hangs indefinitely on Solaris/x86
> (.1 and .2 run successfully).  I'll test later today or early tomorrow
> on MacOS, to see if this affects other lutex builds.

I'm a moron. No mutex deadlines lutex builds. I'll check in a fix ...
tomorrow, I think. This day is running late already. (Just realized
there's a related bug in the deadlock detection code as well when
considering deadlines.)

Cheers,

 -- Nikodemus

------------------------------------------------------------------------------
Achieve unprecedented app performance and reliability
What every C/C++ and Fortran developer should know.
Learn how Intel has extended the reach of its next-generation tools
to help boost performance applications - inlcuding clusters.
http://p.sf.net/sfu/intel-dev2devmay
_______________________________________________
Sbcl-help mailing list
Sbcl-help <at> lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/sbcl-help

10	April 2013
17	March 2013
8	February 2013
9	January 2013
4	December 2012
14	November 2012
10	October 2012
10	September 2012
12	August 2012
40	July 2012
13	June 2012
58	May 2012
16	April 2012
17	March 2012
22	February 2012
9	January 2012
21	December 2011
21	November 2011
25	October 2011
21	September 2011
38	August 2011
15	July 2011
28	June 2011
17	May 2011
31	April 2011
5	March 2011
31	February 2011
17	January 2011
19	December 2010
41	November 2010
33	October 2010
43	September 2010
31	August 2010
43	July 2010
24	June 2010
8	May 2010
59	April 2010
55	March 2010
44	February 2010
26	January 2010
31	December 2009
46	November 2009
59	October 2009
52	September 2009
35	August 2009
48	July 2009
58	June 2009
29	May 2009
15	April 2009
18	March 2009
31	February 2009
33	January 2009
83	December 2008
42	November 2008
52	October 2008
19	September 2008
52	August 2008
24	July 2008
25	June 2008
1	May 2008
13	April 2008
28	March 2008
17	February 2008
36	January 2008
59	December 2007
9	November 2007
16	October 2007
11	September 2007
24	August 2007
31	July 2007
31	June 2007
60	May 2007
40	April 2007
50	March 2007
37	February 2007
41	January 2007
69	December 2006
20	November 2006
39	October 2006
51	September 2006
23	August 2006
11	July 2006
29	June 2006
19	May 2006
26	April 2006
19	March 2006
37	February 2006
45	January 2006
24	December 2005
30	November 2005
17	October 2005
28	September 2005
26	August 2005
33	July 2005
37	June 2005
69	May 2005
65	April 2005
39	March 2005
32	February 2005
24	January 2005
27	December 2004
80	November 2004
20	October 2004
38	September 2004
14	August 2004
23	July 2004
15	June 2004
18	May 2004
69	April 2004
21	March 2004
15	February 2004
32	January 2004
19	December 2003
14	November 2003
3	October 2003
27	September 2003
28	August 2003
3	July 2003
10	June 2003
27	May 2003
14	March 2003
4	February 2003
5	January 2003
8	December 2002
3	September 2002
2	August 2002
5	May 2002

Mon	Tue	Wed	Thu	Fri	Sat	Sun

						1
2	3	4	5	6	7	8
9	10	11	12	13	14	15
16	17	18	19	20	21	22
23	24	25	26	27	28	29
30	31