Elliott Slaughter | 1 May 05:44 2011
Picon

Re: How to tell size of data waiting in a TCP socket?

I have another minimal testcase. It actually doesn't exhibit a freeze like I see in my application, but it's definitely not working, so I think it's worth looking at.

The code:

(defun receive-each (connections)
  (let ((ready (usocket:wait-for-input connections :timeout 0 :ready-only t)))
    (loop for connection in ready
       collect (read-line (usocket:socket-stream connection)))))

(defun receive-all (connections)
  (loop for messages = (receive-each connections)
     then (receive-each connections)
     while messages append messages))

(defun send (connection message)
  (format (usocket:socket-stream connection) "~a~%" message)
  (force-output (usocket:socket-stream connection)))

(defun server ()
  (let* ((listen (usocket:socket-listen usocket:*wildcard-host* 12345))
         (connection (usocket:socket-accept listen)))
    (loop for messages = (receive-all connection) then (receive-all connection)
       do (format t "Got messages:~%~s~%" messages)
       do (sleep 1/50))))

(defun client ()
  (let ((connection (usocket:socket-connect "localhost" 12345)))
    (loop for i from 0
       do (send connection (format nil "This is message ~a." i))
       do (sleep 1/100))))

Here is what the output looks like on everything I've tested except SBCL on Windows. (Tested on SBCL on Mac, and Clozure CL on Windows.)

* (server)
Got messages:
("This is message 0." "This is message 1." "This is message 2.")
Got messages:
("This is message 3." "This is message 4.")
...

Here is the output on SBCL on Windows.

* (server)
Got messages:
NIL
Got messages:
NIL
...

I think in this case wait-for-input never returns anything as ready so I never read from the socket.

On Sat, Apr 30, 2011 at 1:13 AM, Elliott Slaughter <elliottslaughter-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
Hi,

I'm wondering if there is a good way to figure out how much data is available on a socket. Right now I have a situation where my program accepts messages over a TCP socket. To check to see if a single message is available, I can just call wait-for-input with :timeout 0. But in order to check for multiple messages on a single socket, I need multiple calls to wait-for-input. So something like

(iter (for ready next (wait-for-input socket :timeout 0))
      (while (read-message socket)))

I'm avoiding multithreading to preserve compatibility with Lisps that don't support (or have unstable support for) threads.

The above works on most implementations, but breaks on SBCL/Windows, where subsequent calls to wait-for-input appear to block, despite the :timeout 0 parameter.

There may very well be a bug in the SBCL/Windows implementation in usocket to chase down here. Alternatively, is there a better way to do the above (assuming that multithread is unavailable in the implementation I want to run this on)?

Thanks.

--
Elliott Slaughter

"Don't worry about what anybody else is going to do. The best way to predict the future is to invent it." - Alan Kay



--
Elliott Slaughter

"Don't worry about what anybody else is going to do. The best way to predict the future is to invent it." - Alan Kay
_______________________________________________
usocket-devel mailing list
usocket-devel@...
http://common-lisp.net/cgi-bin/mailman/listinfo/usocket-devel
Chun Tian (binghe | 1 May 07:41 2011
Picon

Re: SBCL wait-for-input bug on attempt at second connection

Hi, Elliott

Thank you, I've installed this SBCL binary and merged your test code into usocket's unit test [1]. And Yes, I can see the BAD-FILE-DESCRIPTOR-ERROR now. This happens only on SBCL/Windows but other platforms.  I also tried your test code on other CL implementations, it passed on most backends on Mac OS (CMUCL, SBCL, Allegro CL, Clozure CL) but strangely failed on LispWorks/Mac in another form, and hang forever on ABCL ...  Any way, let me try to fix SBCL/Windows first.

Regards,

Chun Tian (binghe)


在 2011-5-1,01:48, Elliott Slaughter 写道:

By the way (in case you have any trouble reproducing), the version of SBCL used to test this can be obtained here. (It's just a more up-to-date version than what you can get on the SBCL download page that I compiled myself.


2011/4/30 Chun Tian (binghe) <binghe.lisp-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
Hi, Elliott

Sorry for late responses, I'm a little busy in the past week.

It seems that current implementation of WAIT-FOR-INPUT on SBCL/Windows have some bugs, I'll look into it in next two days and try to fix it.  Whatever bug it has, it shuold NOT be any bug in SBCL itself, because this function was completely written using SB-ALIEN and underlying Win32 API.

I'll put your code as part of usocket's unit test and see if it also affects other backends.

Regards,

Chun Tian (binghe)

在 2011-4-25,09:17, Elliott Slaughter 写道:

First, the documentation for wait-for-input says that only integer are accepted, but code seems to think reals should work. (Tested on SBCL/Windows.)

Second, I have found a bug in wait-for-input in SBCL on Windows. The problem is that a wait-for-input on a TCP server socket which has already accepted one client will continue to return that the socket is ready to accept more clients even though it isn't, and thus cause an error when I try to call socket-accept. Here is the code that breaks it:

(defvar *port* 12345)
(defvar *socket-server-listen*
  (socket-listen *wildcard-host* *port* :element-type '(unsigned-byte 8)))

(defvar *socket-server-connection*)
(setf
 *socket-server-connection*
 (when (wait-for-input *socket-server-listen* :timeout 0 :ready-only t)
   (socket-accept *socket-server-listen*)))
(format t "First time (before client connects) is ~s.~%"
        *socket-server-connection*)

(defvar *socket-client-connection*)
(setf *socket-client-connection*
      (socket-connect "localhost" *port* :protocol :stream
                      :element-type '(unsigned-byte 8) :timeout 0))

(setf
 *socket-server-connection*
 (when (wait-for-input *socket-server-listen* :timeout 0 :ready-only t)
   (socket-accept *socket-server-listen*)))
(format t "Second time (after client connects) is ~s.~%"
        *socket-server-connection*)

(setf
 *socket-server-connection*
 (when (wait-for-input *socket-server-listen* :timeout 0 :ready-only t)
   (socket-accept *socket-server-listen*)))
(format t "Third time (before second client) is ~s.~%"
        *socket-server-connection*)

The output on Windows/SBCL looks like:

First time (before client connects) is NIL.
Second time (after client connects) is #<STREAM-USOCKET {24757F91}>.

It doesn't get to the first format call, because it hits the following error first:

Condition BAD-FILE-DESCRIPTOR-ERROR was signalled.
   [Condition of type BAD-FILE-DESCRIPTOR-ERROR]

Restarts:
 0: [RETRY] Retry SLIME REPL evaluation request.
 1: [ABORT] Return to SLIME's top level.
 2: [ABORT] Abort
 3: [CLOSE-CONNECTION] Close SLIME connection
 4: [ABORT] Exit debugger, returning to top level.

Backtrace:
  0: (USOCKET::HANDLE-CONDITION #<SB-BSD-SOCKETS:SOCKET-ERROR {2475C889}> #<STREAM-SERVER-USOCKET {256F7C19}>)
  1: (SIGNAL #<SB-BSD-SOCKETS:SOCKET-ERROR {2475C889}>)
  2: (ERROR SB-BSD-SOCKETS:SOCKET-ERROR :ERRNO 9 :SYSCALL "accept")
  3: (SB-BSD-SOCKETS:SOCKET-ERROR "accept")
  4: ((SB-PCL::FAST-METHOD SB-BSD-SOCKETS:SOCKET-ACCEPT (SB-BSD-SOCKETS:SOCKET)) #<unavailable argument> #<unavailable argument> #<SB-BSD-SOCKETS:INET-SOCKET 0.0.0.0:12345, fd: 6 {2576DD91}>)
  5: ((SB-PCL::FAST-METHOD SOCKET-ACCEPT (STREAM-SERVER-USOCKET)) #<unused argument> #<unused argument> #<STREAM-SERVER-USOCKET {256F7C19}> :ELEMENT-TYPE NIL)
  6: (SB-INT:SIMPLE-EVAL-IN-LEXENV (SOCKET-ACCEPT *SOCKET-SERVER-LISTEN*) #<NULL-LEXENV>)
  7: (SB-INT:SIMPLE-EVAL-IN-LEXENV (PROGN (SOCKET-ACCEPT *SOCKET-SERVER-LISTEN*)) #<NULL-LEXENV>)
  8: (SB-INT:SIMPLE-EVAL-IN-LEXENV (WHEN (WAIT-FOR-INPUT *SOCKET-SERVER-LISTEN* :TIMEOUT 0 :READY-ONLY T) (SOCKET-ACCEPT *SOCKET-SERVER-LISTEN*)) #<NULL-LEXENV>)
  9: (SB-INT:SIMPLE-EVAL-IN-LEXENV (SETF *SOCKET-SERVER-CONNECTION* (WHEN # #)) #<NULL-LEXENV>)

--
Elliott Slaughter

"Don't worry about what anybody else is going to do. The best way to predict the future is to invent it." - Alan Kay
_______________________________________________
usocket-devel mailing list
usocket-devel-F1HGIaG5STRyXAeb93iumQ@public.gmane.org
http://common-lisp.net/cgi-bin/mailman/listinfo/usocket-devel




--
Elliott Slaughter

"Don't worry about what anybody else is going to do. The best way to predict the future is to invent it." - Alan Kay

_______________________________________________
usocket-devel mailing list
usocket-devel@...
http://common-lisp.net/cgi-bin/mailman/listinfo/usocket-devel
Chun Tian (binghe | 1 May 13:36 2011
Picon

Re: SBCL wait-for-input bug on attempt at second connection

Hi, Elliott

I believe this bug was now fixed.  The third WAIT-FOR-INPUT in your test code shouldn't return a non-NIL value, but a bug in function WAIT-FOR-INPUT caused the "last" status of sockets didn't been reset correctly.

This bug only affect SBCL and ECL on Windows, and was fixed in r653 on USOCKET 0.5.x branch [1].

Please update your USOCKET and confirm if it works for you (and report back, if possible), and I'll see your another bug report in a moment.

Regards,

Chun Tian (binghe)

P. S. Your test code also let me find a bug in LispWorks' version of SOCKET-ACCEPT, and I've done a fix based on completely different theory. Thanks.


在 2011-4-25,09:17, Elliott Slaughter 写道:

First, the documentation for wait-for-input says that only integer are accepted, but code seems to think reals should work. (Tested on SBCL/Windows.)

Second, I have found a bug in wait-for-input in SBCL on Windows. The problem is that a wait-for-input on a TCP server socket which has already accepted one client will continue to return that the socket is ready to accept more clients even though it isn't, and thus cause an error when I try to call socket-accept. Here is the code that breaks it:

(defvar *port* 12345)
(defvar *socket-server-listen*
  (socket-listen *wildcard-host* *port* :element-type '(unsigned-byte 8)))

(defvar *socket-server-connection*)
(setf
 *socket-server-connection*
 (when (wait-for-input *socket-server-listen* :timeout 0 :ready-only t)
   (socket-accept *socket-server-listen*)))
(format t "First time (before client connects) is ~s.~%"
        *socket-server-connection*)

(defvar *socket-client-connection*)
(setf *socket-client-connection*
      (socket-connect "localhost" *port* :protocol :stream
                      :element-type '(unsigned-byte 8) :timeout 0))

(setf
 *socket-server-connection*
 (when (wait-for-input *socket-server-listen* :timeout 0 :ready-only t)
   (socket-accept *socket-server-listen*)))
(format t "Second time (after client connects) is ~s.~%"
        *socket-server-connection*)

(setf
 *socket-server-connection*
 (when (wait-for-input *socket-server-listen* :timeout 0 :ready-only t)
   (socket-accept *socket-server-listen*)))
(format t "Third time (before second client) is ~s.~%"
        *socket-server-connection*)

The output on Windows/SBCL looks like:

First time (before client connects) is NIL.
Second time (after client connects) is #<STREAM-USOCKET {24757F91}>.

It doesn't get to the first format call, because it hits the following error first:

Condition BAD-FILE-DESCRIPTOR-ERROR was signalled.
   [Condition of type BAD-FILE-DESCRIPTOR-ERROR]

Restarts:
 0: [RETRY] Retry SLIME REPL evaluation request.
 1: [ABORT] Return to SLIME's top level.
 2: [ABORT] Abort
 3: [CLOSE-CONNECTION] Close SLIME connection
 4: [ABORT] Exit debugger, returning to top level.

Backtrace:
  0: (USOCKET::HANDLE-CONDITION #<SB-BSD-SOCKETS:SOCKET-ERROR {2475C889}> #<STREAM-SERVER-USOCKET {256F7C19}>)
  1: (SIGNAL #<SB-BSD-SOCKETS:SOCKET-ERROR {2475C889}>)
  2: (ERROR SB-BSD-SOCKETS:SOCKET-ERROR :ERRNO 9 :SYSCALL "accept")
  3: (SB-BSD-SOCKETS:SOCKET-ERROR "accept")
  4: ((SB-PCL::FAST-METHOD SB-BSD-SOCKETS:SOCKET-ACCEPT (SB-BSD-SOCKETS:SOCKET)) #<unavailable argument> #<unavailable argument> #<SB-BSD-SOCKETS:INET-SOCKET 0.0.0.0:12345, fd: 6 {2576DD91}>)
  5: ((SB-PCL::FAST-METHOD SOCKET-ACCEPT (STREAM-SERVER-USOCKET)) #<unused argument> #<unused argument> #<STREAM-SERVER-USOCKET {256F7C19}> :ELEMENT-TYPE NIL)
  6: (SB-INT:SIMPLE-EVAL-IN-LEXENV (SOCKET-ACCEPT *SOCKET-SERVER-LISTEN*) #<NULL-LEXENV>)
  7: (SB-INT:SIMPLE-EVAL-IN-LEXENV (PROGN (SOCKET-ACCEPT *SOCKET-SERVER-LISTEN*)) #<NULL-LEXENV>)
  8: (SB-INT:SIMPLE-EVAL-IN-LEXENV (WHEN (WAIT-FOR-INPUT *SOCKET-SERVER-LISTEN* :TIMEOUT 0 :READY-ONLY T) (SOCKET-ACCEPT *SOCKET-SERVER-LISTEN*)) #<NULL-LEXENV>)
  9: (SB-INT:SIMPLE-EVAL-IN-LEXENV (SETF *SOCKET-SERVER-CONNECTION* (WHEN # #)) #<NULL-LEXENV>)

--
Elliott Slaughter

"Don't worry about what anybody else is going to do. The best way to predict the future is to invent it." - Alan Kay
_______________________________________________
usocket-devel mailing list
usocket-devel-F1HGIaG5STTwUBNsjRvlJw@public.gmane.orgt
http://common-lisp.net/cgi-bin/mailman/listinfo/usocket-devel

_______________________________________________
usocket-devel mailing list
usocket-devel@...
http://common-lisp.net/cgi-bin/mailman/listinfo/usocket-devel
Elliott Slaughter | 1 May 18:07 2011
Picon

Re: SBCL wait-for-input bug on attempt at second connection

Yes, the test suite now passes the tests for this.


My application still breaks as reported in my other emails, so it's a little hard to test this on my real code.

Thanks!

2011/5/1 Chun Tian (binghe) <binghe.lisp-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
Hi, Elliott

I believe this bug was now fixed.  The third WAIT-FOR-INPUT in your test code shouldn't return a non-NIL value, but a bug in function WAIT-FOR-INPUT caused the "last" status of sockets didn't been reset correctly.

This bug only affect SBCL and ECL on Windows, and was fixed in r653 on USOCKET 0.5.x branch [1].

Please update your USOCKET and confirm if it works for you (and report back, if possible), and I'll see your another bug report in a moment.

Regards,

Chun Tian (binghe)

P. S. Your test code also let me find a bug in LispWorks' version of SOCKET-ACCEPT, and I've done a fix based on completely different theory. Thanks.
在 2011-4-25,09:17, Elliott Slaughter 写道:

First, the documentation for wait-for-input says that only integer are accepted, but code seems to think reals should work. (Tested on SBCL/Windows.)

Second, I have found a bug in wait-for-input in SBCL on Windows. The problem is that a wait-for-input on a TCP server socket which has already accepted one client will continue to return that the socket is ready to accept more clients even though it isn't, and thus cause an error when I try to call socket-accept. Here is the code that breaks it:

(defvar *port* 12345)
(defvar *socket-server-listen*
  (socket-listen *wildcard-host* *port* :element-type '(unsigned-byte 8)))

(defvar *socket-server-connection*)
(setf
 *socket-server-connection*
 (when (wait-for-input *socket-server-listen* :timeout 0 :ready-only t)
   (socket-accept *socket-server-listen*)))
(format t "First time (before client connects) is ~s.~%"
        *socket-server-connection*)

(defvar *socket-client-connection*)
(setf *socket-client-connection*
      (socket-connect "localhost" *port* :protocol :stream
                      :element-type '(unsigned-byte 8) :timeout 0))

(setf
 *socket-server-connection*
 (when (wait-for-input *socket-server-listen* :timeout 0 :ready-only t)
   (socket-accept *socket-server-listen*)))
(format t "Second time (after client connects) is ~s.~%"
        *socket-server-connection*)

(setf
 *socket-server-connection*
 (when (wait-for-input *socket-server-listen* :timeout 0 :ready-only t)
   (socket-accept *socket-server-listen*)))
(format t "Third time (before second client) is ~s.~%"
        *socket-server-connection*)

The output on Windows/SBCL looks like:

First time (before client connects) is NIL.
Second time (after client connects) is #<STREAM-USOCKET {24757F91}>.

It doesn't get to the first format call, because it hits the following error first:

Condition BAD-FILE-DESCRIPTOR-ERROR was signalled.
   [Condition of type BAD-FILE-DESCRIPTOR-ERROR]

Restarts:
 0: [RETRY] Retry SLIME REPL evaluation request.
 1: [ABORT] Return to SLIME's top level.
 2: [ABORT] Abort
 3: [CLOSE-CONNECTION] Close SLIME connection
 4: [ABORT] Exit debugger, returning to top level.

Backtrace:
  0: (USOCKET::HANDLE-CONDITION #<SB-BSD-SOCKETS:SOCKET-ERROR {2475C889}> #<STREAM-SERVER-USOCKET {256F7C19}>)
  1: (SIGNAL #<SB-BSD-SOCKETS:SOCKET-ERROR {2475C889}>)
  2: (ERROR SB-BSD-SOCKETS:SOCKET-ERROR :ERRNO 9 :SYSCALL "accept")
  3: (SB-BSD-SOCKETS:SOCKET-ERROR "accept")
  4: ((SB-PCL::FAST-METHOD SB-BSD-SOCKETS:SOCKET-ACCEPT (SB-BSD-SOCKETS:SOCKET)) #<unavailable argument> #<unavailable argument> #<SB-BSD-SOCKETS:INET-SOCKET 0.0.0.0:12345, fd: 6 {2576DD91}>)
  5: ((SB-PCL::FAST-METHOD SOCKET-ACCEPT (STREAM-SERVER-USOCKET)) #<unused argument> #<unused argument> #<STREAM-SERVER-USOCKET {256F7C19}> :ELEMENT-TYPE NIL)
  6: (SB-INT:SIMPLE-EVAL-IN-LEXENV (SOCKET-ACCEPT *SOCKET-SERVER-LISTEN*) #<NULL-LEXENV>)
  7: (SB-INT:SIMPLE-EVAL-IN-LEXENV (PROGN (SOCKET-ACCEPT *SOCKET-SERVER-LISTEN*)) #<NULL-LEXENV>)
  8: (SB-INT:SIMPLE-EVAL-IN-LEXENV (WHEN (WAIT-FOR-INPUT *SOCKET-SERVER-LISTEN* :TIMEOUT 0 :READY-ONLY T) (SOCKET-ACCEPT *SOCKET-SERVER-LISTEN*)) #<NULL-LEXENV>)
  9: (SB-INT:SIMPLE-EVAL-IN-LEXENV (SETF *SOCKET-SERVER-CONNECTION* (WHEN # #)) #<NULL-LEXENV>)

--
Elliott Slaughter

"Don't worry about what anybody else is going to do. The best way to predict the future is to invent it." - Alan Kay




--
Elliott Slaughter

"Don't worry about what anybody else is going to do. The best way to predict the future is to invent it." - Alan Kay
_______________________________________________
usocket-devel mailing list
usocket-devel@...
http://common-lisp.net/cgi-bin/mailman/listinfo/usocket-devel
Chun Tian (binghe | 10 May 18:08 2011
Picon

Re: How to tell size of data waiting in a TCP socket?

Hi, Elliott

Sorry again for late response, but I think I've found the reason about this issue: WAIT-FOR-INPUT correctly detected that there're inputs on the server side socket, but due to a subtle bug when updating the STATE slot of the socket object, the socket wasn't marked as READY, so when W-F-I was called by (READY-ONLY T), a empty list returned (and your code have no chance to get touch with the stream behind the socket, and nothing could be output ...)

Due to some problems on USOCKET SVN, I cannot commit my work right now, but I have a patch as .diff file in attach, which could solve all your needs.  I hope you can take a little download it and apply to your local USOCKET directory and see if your application work well this time.

The bug you reported also affect ECL on Windows, but I don't sync my patch to ECL until I get your positive confirmation.

Regards,

Chun Tian (binghe)

Attachment (elliott-2.diff): application/octet-stream, 5398 bytes



在 2011-5-1,11:44, Elliott Slaughter 写道:

I have another minimal testcase. It actually doesn't exhibit a freeze like I see in my application, but it's definitely not working, so I think it's worth looking at.

The code:

(defun receive-each (connections)
  (let ((ready (usocket:wait-for-input connections :timeout 0 :ready-only t)))
    (loop for connection in ready
       collect (read-line (usocket:socket-stream connection)))))

(defun receive-all (connections)
  (loop for messages = (receive-each connections)
     then (receive-each connections)
     while messages append messages))

(defun send (connection message)
  (format (usocket:socket-stream connection) "~a~%" message)
  (force-output (usocket:socket-stream connection)))

(defun server ()
  (let* ((listen (usocket:socket-listen usocket:*wildcard-host* 12345))
         (connection (usocket:socket-accept listen)))
    (loop for messages = (receive-all connection) then (receive-all connection)
       do (format t "Got messages:~%~s~%" messages)
       do (sleep 1/50))))

(defun client ()
  (let ((connection (usocket:socket-connect "localhost" 12345)))
    (loop for i from 0
       do (send connection (format nil "This is message ~a." i))
       do (sleep 1/100))))

Here is what the output looks like on everything I've tested except SBCL on Windows. (Tested on SBCL on Mac, and Clozure CL on Windows.)

* (server)
Got messages:
("This is message 0." "This is message 1." "This is message 2.")
Got messages:
("This is message 3." "This is message 4.")
...

Here is the output on SBCL on Windows.

* (server)
Got messages:
NIL
Got messages:
NIL
...

I think in this case wait-for-input never returns anything as ready so I never read from the socket.

On Sat, Apr 30, 2011 at 1:13 AM, Elliott Slaughter <elliottslaughter-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
Hi,

I'm wondering if there is a good way to figure out how much data is available on a socket. Right now I have a situation where my program accepts messages over a TCP socket. To check to see if a single message is available, I can just call wait-for-input with :timeout 0. But in order to check for multiple messages on a single socket, I need multiple calls to wait-for-input. So something like

(iter (for ready next (wait-for-input socket :timeout 0))
      (while (read-message socket)))

I'm avoiding multithreading to preserve compatibility with Lisps that don't support (or have unstable support for) threads.

The above works on most implementations, but breaks on SBCL/Windows, where subsequent calls to wait-for-input appear to block, despite the :timeout 0 parameter.

There may very well be a bug in the SBCL/Windows implementation in usocket to chase down here. Alternatively, is there a better way to do the above (assuming that multithread is unavailable in the implementation I want to run this on)?

Thanks.

--
Elliott Slaughter

"Don't worry about what anybody else is going to do. The best way to predict the future is to invent it." - Alan Kay



--
Elliott Slaughter

"Don't worry about what anybody else is going to do. The best way to predict the future is to invent it." - Alan Kay
_______________________________________________
usocket-devel mailing list
usocket-devel-F1HGIaG5STTwUBNsjRvlJw@public.gmane.orgt
http://common-lisp.net/cgi-bin/mailman/listinfo/usocket-devel

_______________________________________________
usocket-devel mailing list
usocket-devel@...
http://common-lisp.net/cgi-bin/mailman/listinfo/usocket-devel
Chun Tian (binghe | 10 May 18:15 2011
Picon

Re: How to tell size of data waiting in a TCP socket?

Hi again, Elliott

To find "how much (bytes) data is available on a socket", you can use a internal function called
USOCKET::BYTES-AVAILABLE-FOR-READ, the only argument is the STREAM-SOCKET object, and it returns the
number of bytes.

But if you want to know how many "messages" is available, I'm afraid that unless you know exactly the length
of your messages, above function still cannot let you cut down the number of calls to WAIT-FOR-INPUT.

And this function is currently only available on Windows for LispWorks, SBCL and ECL.

Hope this helps.

Regards,

Chun Tian (binghe)

在 2011-4-30,16:13, Elliott Slaughter 写道:

> Hi,
> 
> I'm wondering if there is a good way to figure out how much data is available on a socket. Right now I have a
situation where my program accepts messages over a TCP socket. To check to see if a single message is
available, I can just call wait-for-input with :timeout 0. But in order to check for multiple messages on a
single socket, I need multiple calls to wait-for-input. So something like
> 
> (iter (for ready next (wait-for-input socket :timeout 0))
>       (while (read-message socket)))
> 
> I'm avoiding multithreading to preserve compatibility with Lisps that don't support (or have unstable
support for) threads.
> 
> The above works on most implementations, but breaks on SBCL/Windows, where subsequent calls to
wait-for-input appear to block, despite the :timeout 0 parameter.
> 
> There may very well be a bug in the SBCL/Windows implementation in usocket to chase down here.
Alternatively, is there a better way to do the above (assuming that multithread is unavailable in the
implementation I want to run this on)?
> 
> Thanks.
> 
> -- 
> Elliott Slaughter
> 
> "Don't worry about what anybody else is going to do. The best way to predict the future is to invent it." -
Alan Kay
> _______________________________________________
> usocket-devel mailing list
> usocket-devel <at> common-lisp.net
> http://common-lisp.net/cgi-bin/mailman/listinfo/usocket-devel

_______________________________________________
usocket-devel mailing list
usocket-devel <at> common-lisp.net
http://common-lisp.net/cgi-bin/mailman/listinfo/usocket-devel
Elliott Slaughter | 10 May 20:49 2011
Picon

Re: How to tell size of data waiting in a TCP socket?

2011/5/10 Chun Tian (binghe) <binghe.lisp-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>

Hi, Elliott

Sorry again for late response, but I think I've found the reason about this issue: WAIT-FOR-INPUT correctly detected that there're inputs on the server side socket, but due to a subtle bug when updating the STATE slot of the socket object, the socket wasn't marked as READY, so when W-F-I was called by (READY-ONLY T), a empty list returned (and your code have no chance to get touch with the stream behind the socket, and nothing could be output ...)

Due to some problems on USOCKET SVN, I cannot commit my work right now, but I have a patch as .diff file in attach, which could solve all your needs.  I hope you can take a little download it and apply to your local USOCKET directory and see if your application work well this time.

The bug you reported also affect ECL on Windows, but I don't sync my patch to ECL until I get your positive confirmation.

Yes, it works! Thanks!

Regards,

Chun Tian (binghe)




在 2011-5-1,11:44, Elliott Slaughter 写道:

I have another minimal testcase. It actually doesn't exhibit a freeze like I see in my application, but it's definitely not working, so I think it's worth looking at.

The code:

(defun receive-each (connections)
  (let ((ready (usocket:wait-for-input connections :timeout 0 :ready-only t)))
    (loop for connection in ready
       collect (read-line (usocket:socket-stream connection)))))

(defun receive-all (connections)
  (loop for messages = (receive-each connections)
     then (receive-each connections)
     while messages append messages))

(defun send (connection message)
  (format (usocket:socket-stream connection) "~a~%" message)
  (force-output (usocket:socket-stream connection)))

(defun server ()
  (let* ((listen (usocket:socket-listen usocket:*wildcard-host* 12345))
         (connection (usocket:socket-accept listen)))
    (loop for messages = (receive-all connection) then (receive-all connection)
       do (format t "Got messages:~%~s~%" messages)
       do (sleep 1/50))))

(defun client ()
  (let ((connection (usocket:socket-connect "localhost" 12345)))
    (loop for i from 0
       do (send connection (format nil "This is message ~a." i))
       do (sleep 1/100))))

Here is what the output looks like on everything I've tested except SBCL on Windows. (Tested on SBCL on Mac, and Clozure CL on Windows.)

* (server)
Got messages:
("This is message 0." "This is message 1." "This is message 2.")
Got messages:
("This is message 3." "This is message 4.")
...

Here is the output on SBCL on Windows.

* (server)
Got messages:
NIL
Got messages:
NIL
...

I think in this case wait-for-input never returns anything as ready so I never read from the socket.

On Sat, Apr 30, 2011 at 1:13 AM, Elliott Slaughter <elliottslaughter-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
Hi,

I'm wondering if there is a good way to figure out how much data is available on a socket. Right now I have a situation where my program accepts messages over a TCP socket. To check to see if a single message is available, I can just call wait-for-input with :timeout 0. But in order to check for multiple messages on a single socket, I need multiple calls to wait-for-input. So something like

(iter (for ready next (wait-for-input socket :timeout 0))
      (while (read-message socket)))

I'm avoiding multithreading to preserve compatibility with Lisps that don't support (or have unstable support for) threads.

The above works on most implementations, but breaks on SBCL/Windows, where subsequent calls to wait-for-input appear to block, despite the :timeout 0 parameter.

There may very well be a bug in the SBCL/Windows implementation in usocket to chase down here. Alternatively, is there a better way to do the above (assuming that multithread is unavailable in the implementation I want to run this on)?

Thanks.

--
Elliott Slaughter

"Don't worry about what anybody else is going to do. The best way to predict the future is to invent it." - Alan Kay



--
Elliott Slaughter

"Don't worry about what anybody else is going to do. The best way to predict the future is to invent it." - Alan Kay
_______________________________________________
usocket-devel mailing list
usocket-devel-F1HGIaG5STRyXAeb93iumQ@public.gmane.org
http://common-lisp.net/cgi-bin/mailman/listinfo/usocket-devel





--
Elliott Slaughter

"Don't worry about what anybody else is going to do. The best way to predict the future is to invent it." - Alan Kay
_______________________________________________
usocket-devel mailing list
usocket-devel@...
http://common-lisp.net/cgi-bin/mailman/listinfo/usocket-devel
Chun Tian (binghe | 11 May 15:25 2011
Picon

[ANNOUNCE] usocket 0.5.2

Hello everyone,

I'm pleased to release USOCKET 0.5.2, a new version of the universal socket library for Common Lisp.

In this release, the core API function WAIT-FOR-INPUT was much improved on SBCL, ECL, CLISP; there're some
other fixes for ABCL and LispWorks. I heard, with this release, Hunchentoot can run on ABCL now.

Major changes since 0.5.1:

* General: [SBCL] SOCKET-CONNECT's TIMEOUT argument was limited on non-Windows platforms.
* Bugfix: [CLISP] WAIT-FOR-INPUT now functions right (with/without READY-ONLY), this made Hunchentoot
working on CLISP. (Thanks to Anton Vodonosov <avodonosov@...>)
* Bugfix: [ABCL] Fix SOCKET-ACCEPT to follow the documented API so that when called without an
:ELEMENT-TYPE argument. (Thanks to Mark Evenson, the ABCL developer)
* Bugfix: [LispWorks] Fixed SOCKET-ACCEPT (Windows only) on WAIT-FOR-INPUTed sockets.
* Bugfix: [SBCL, ECL] Fixed wrongly STATE set/unset for WAIT-FOR-INPUT on Windows (report by Elliott Slaughter)
* Enhancement: Additional NAME keyword argument for SOCKET-SERVER for setting the server thread name.
* Enhancement: [ABCL] GET-ADDRESS now works with underlying IP6 addresses.
* Enhancement: [CLISP] missing GET-LOCAL-* methods for STREAM-SERVER-USOCKET was now added.

We give special thanks to:

	Elliott Slaughter <elliottslaughter@...>

for his two bug reports on WAIT-FOR-INPUT (SBCL/Windows) with unit test code, and these code are now part of USOCKET-TEST.

If you want to download this release, please checkout

	http://common-lisp.net/project/usocket/releases/

The API documentation page is here:

	http://common-lisp.net/project/usocket/api-docs.shtml

As usual, I hope more and more people use USOCKET as part of their Lisp applications, and any feedback - bugs
or hugs - is greatly appreciated. If you use USOCKET, please let us know what for.

And, if there's any historical issue which haven't been solved, please feel free to remind me.

And below is a roadmap for future USOCKET releases (ignore the order):

* Add UDP (Datagram) support for Digitool MCL;
* The DFFI version of ECL backend without calling C compilers;
* New backends: Symbolics Lisp Machine, XCL and IOlib;
* New API: SOCKET-OPTION for setting/getting various socket options (already exist in trunk (0.6))

Let me know if you have other ideas.

Regards,

Chun Tian (binghe)

Gmane