Celelibi | 10 Apr 01:46 2012
Picon

close socket and TCP RST

Hello,

I'm not sure about the mailing list I should ask this. I would try
linux-net if it was still alive.

I spent a few days searching about a weird bug I had.
It is actually similar to this one:
http://marc.info/?l=linux-net&m=127651583824851&w=2 .

Summary for lazy people ^^: calling close(2) on a socket with a
non-empty receive kernel-buffer cause the connection to be ReSeT and
the send buffer discarded and not sent.

I have a server program (code included below)  that wait for a client.
When a client connect, the server:
- sent "Greet\n"
- read one char
- send "Hello "
- send "World\n"
- send "Quit\n"
- close the socket

And it happens that sometimes (quite often actually) that the client
does not recieve anything after "Hello " and the connection is just
closed.
Actually, after I dumped the TCP trafic with wireshark, I saw that
"World\n" and "Quit\n" were NOT sent, despite the send(2) succeded.
The server just send a packet with the RST flag to interrupt the
connection.
Even more strange: This only occurs when the client send more data to
(Continue reading)

Bogdan Cristea | 10 Apr 23:01 2012
Picon

Re: close socket and TCP RST

On Tuesday 10 April 2012 01:46:45 you wrote:
> Hello,
> 
> I'm not sure about the mailing list I should ask this. I would try
> linux-net if it was still alive.
> 
> I spent a few days searching about a weird bug I had.
> It is actually similar to this one:
> http://marc.info/?l=linux-net&m=127651583824851&w=2 .
> 
> Summary for lazy people ^^: calling close(2) on a socket with a
> non-empty receive kernel-buffer cause the connection to be ReSeT and
> the send buffer discarded and not sent.
> 
> 
> I have a server program (code included below)  that wait for a client.
> When a client connect, the server:
> - sent "Greet\n"
> - read one char
> - send "Hello "
> - send "World\n"
> - send "Quit\n"
> - close the socket
> 
> And it happens that sometimes (quite often actually) that the client
> does not recieve anything after "Hello " and the connection is just
> closed.
> Actually, after I dumped the TCP trafic with wireshark, I saw that
> "World\n" and "Quit\n" were NOT sent, despite the send(2) succeded.
> The server just send a packet with the RST flag to interrupt the
(Continue reading)

Celelibi | 11 Apr 02:38 2012
Picon

Re: close socket and TCP RST

2012/4/10, Bogdan Cristea <cristeab <at> gmail.com>:
> On Tuesday 10 April 2012 01:46:45 you wrote:
>> Hello,
>>
>> I'm not sure about the mailing list I should ask this. I would try
>> linux-net if it was still alive.
>>
>> I spent a few days searching about a weird bug I had.
>> It is actually similar to this one:
>> http://marc.info/?l=linux-net&m=127651583824851&w=2 .
>>
>> Summary for lazy people ^^: calling close(2) on a socket with a
>> non-empty receive kernel-buffer cause the connection to be ReSeT and
>> the send buffer discarded and not sent.
>>
>>
>> I have a server program (code included below)  that wait for a client.
>> When a client connect, the server:
>> - sent "Greet\n"
>> - read one char
>> - send "Hello "
>> - send "World\n"
>> - send "Quit\n"
>> - close the socket
>>
>> And it happens that sometimes (quite often actually) that the client
>> does not recieve anything after "Hello " and the connection is just
>> closed.
>> Actually, after I dumped the TCP trafic with wireshark, I saw that
>> "World\n" and "Quit\n" were NOT sent, despite the send(2) succeded.
(Continue reading)

Glynn Clements | 11 Apr 15:33 2012

Re: close socket and TCP RST


Celelibi wrote:

> Summary for lazy people ^^: calling close(2) on a socket with a
> non-empty receive kernel-buffer cause the connection to be ReSeT and
> the send buffer discarded and not sent.

Yes; this is all as it should be.

> 1) Is this a standard behavior?

Yes.

> Doesn't the RFC state that every pending data is sent when the
> connection is closed?

The RFCs describe the TCP protocol, not the sockets API.

> 2) Shouldn't that behavior be documented somewhere? I didn't found any
> information about that anywhere. I looked at the man close(2),
> shutdown(2), socket(7), tcp(7).
>
> >From this I deduce that shutdown must be called everytime we want to
> close a socket. But this is not taught anywhere. :p

In many cases, shutdown() is not necessary. Normally, one side knows
whether the other side will send more data. E.g. for (non-pipelined)
HTTP, the client sends a request, the server sends a response, then
closes the connection. At that point, the client sees EOF then
close()s the socket (or it could just close the socket once the amount
(Continue reading)

Bogdan Cristea | 11 Apr 20:42 2012
Picon

Re: close socket and TCP RST

On Wednesday 11 April 2012 14:33:37 you wrote:
> In many cases, shutdown() is not necessary. Normally, one side knows
> whether the other side will send more data. E.g. for (non-pipelined)
> HTTP, the client sends a request, the server sends a response, then
> closes the connection.

It is exactly what he does, but the question is how to close the connection so 
that the client receives the last message. 
He is using for this:

 err = setsockopt(sockfd, SOL_SOCKET, SO_LINGER, &lin, sizeof(lin));

but for some reason it does not work as it should.
--

-- 
Bogdan 
--
To unsubscribe from this list: send the line "unsubscribe linux-c-programming" in
the body of a message to majordomo <at> vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Glynn Clements | 12 Apr 01:31 2012

Re: close socket and TCP RST


Bogdan Cristea wrote:

> > In many cases, shutdown() is not necessary. Normally, one side knows
> > whether the other side will send more data. E.g. for (non-pipelined)
> > HTTP, the client sends a request, the server sends a response, then
> > closes the connection.
> 
> It is exactly what he does, but the question is how to close the connection so 
> that the client receives the last message. 
> He is using for this:
> 
>  err = setsockopt(sockfd, SOL_SOCKET, SO_LINGER, &lin, sizeof(lin));
> 
> but for some reason it does not work as it should.

IIUC, he close()s a socket which has unread data. Whether the data had
already been received when close() was called or whether it arrived
afterwards doesn't matter.

Normal behaviour is simply not to use close() or shutdown(SHUT_RD) if
you expect to receive more data, e.g. wait until you have seen EOF
(i.e. read(), recv() etc return a zero count) before closing the read
side of the socket.

Current Linux behaviour is that receiving data on a close()ed socket
sends a RST. No data can be sent after a RST.

SO_LINGER doesn't affect this; it just affects whether close() or
shutdown(SHUT_WR) wait until the data has been sent (i.e. for the FIN
(Continue reading)

Celelibi | 13 Apr 03:37 2012
Picon

Re: close socket and TCP RST

2012/4/11, Glynn Clements <glynn <at> gclements.plus.com>:
>
> Celelibi wrote:
>
>> Summary for lazy people ^^: calling close(2) on a socket with a
>> non-empty receive kernel-buffer cause the connection to be ReSeT and
>> the send buffer discarded and not sent.
>
> Yes; this is all as it should be.
>
>> 1) Is this a standard behavior?
>
> Yes.
>
>> Doesn't the RFC state that every pending data is sent when the
>> connection is closed?
>
> The RFCs describe the TCP protocol, not the sockets API.
>

I agree with that. But I didn't find any information about the
berkeley socket API standard.
Then, I assume that the close() syscall close the file descriptor and
close the socket (both ways) if it was the last file descriptor for
that socket.

However, when I look at the TCP's RFC 793 (a.k.a STD7):
http://www.rfc-editor.org/rfc/std/std7.txt
Section 3.5 "Closing a Connection" says:
"A TCP will reliably deliver all buffers SENT before the connection was CLOSED"
(Continue reading)

Celelibi | 13 Apr 03:54 2012
Picon

Re: close socket and TCP RST

2012/4/12, Glynn Clements <glynn <at> gclements.plus.com>:
> IIUC, he close()s a socket which has unread data. Whether the data had
> already been received when close() was called or whether it arrived
> afterwards doesn't matter.

Right.

>
> Normal behaviour is simply not to use close() or shutdown(SHUT_RD) if
> you expect to receive more data, e.g. wait until you have seen EOF
> (i.e. read(), recv() etc return a zero count) before closing the read
> side of the socket.

I don't care about these data. I don't want to read them. If there are
too much data after the client sent a QUIT command, it have to receive
a RST.

> AFAIK, there is no way to make close() or shutdown(SHUT_RD) silently
> discard subsequent inbound data.

And that's ok because that's not what I want. :)

> Alternatives include:
>
> 1. ACK and discard the data. But then there would be no way for the
> sender to identify that it's writing to a closed socket.

Right.

>
(Continue reading)

Glynn Clements | 13 Apr 09:32 2012

Re: close socket and TCP RST


Celelibi wrote:

> However, when I look at the TCP's RFC 793 (a.k.a STD7):
> http://www.rfc-editor.org/rfc/std/std7.txt
> Section 3.5 "Closing a Connection" says:
> "A TCP will reliably deliver all buffers SENT before the connection was CLOSED"
> 
> And RFC 1122 ("host requirement"): http://www.rfc-editor.org/rfc/rfc1122.txt
> Section 4.2.2.13 "Closing a Connection" says:
> "If such a host issues a CLOSE call while received data is still
> pending in TCP, or if new data is received after CLOSE is called, its
> TCP SHOULD send a RST to show that data was lost."
> 
> IMO both are not incompatible and should be done. But I may have
> missed something somewhere.

First, the RFC's "close" doesn't necessarily equate to the Unix
close() call.

The host can't send outstanding data after a RST has been sent (if it
did, the receiver would just reply with an RST rather than an ACK, it
the sender definitely shouldn't send data after an RST has been
received).

Sending outstanding data then sending the RST afterwards is
potentially problematic as the receiver may not be able or willing to
read the data until its own data has been ACK'd, which would result in
deadlock.

(Continue reading)

Celelibi | 14 Apr 17:37 2012
Picon

Re: close socket and TCP RST

2012/4/13, Glynn Clements <glynn <at> gclements.plus.com>:
> Sending outstanding data then sending the RST afterwards is
> potentially problematic as the receiver may not be able or willing to
> read the data until its own data has been ACK'd, which would result in
> deadlock.

Could you tell me more about that?
I can't see a case which would be problematic.
Moreover shutdown(SHUT_WR) can actually send the outstanding data.

>> The client send "QUIT;\n", the server read 'Q', 'U', 'I', 'T', ';',
>> process "QUIT", send to the client Something that say "Ok, I quit" and
>> close() the connection.
>> However, a "\n" is still in the receive buffer of the server! That
>> makes the server RST the connection ASAP and discard the messages that
>> were not sent yet... Namely the "Ok, I quit" message which was delayed
>> because of Nagle's algorithm or whatever.
>> Annoying isn't it?
>
> If the server sends a response to the quit command, the client should
> read the response then close() the socket. Provided that the server's
> response is well-formed, there shouldn't be a problem.

The client is ok. As soon as it receive "Ok, I quit" from the server,
it close the connection. Although the server should also work with
basic clients like netcat or telnet, which won't close the connection
by themselves. Then the server just has to send its last message and
close the connection. That's the protocol we expect:
The server read the command "QUIT;", send its last message saying it's
going to close the connection, and close the connection. The client is
(Continue reading)


Gmane