Amos Jeffries | 25 Jul 06:37 2014
Picon

Squid 3.5 release timetable

As luck would have it today is exactly 1 year since the first patch was
held in trunk for 3.5 series release.

Below is my current plan. Any objections please speak up.

1) Branching:

I hope this can be done next weekend. August 1-3, maybe the week after
if there are delays.

We have enough features to make it useful even though some of the larger
projects have not made it in.

However, to minimize work in stage-2 trunk needs to be relatively stable
before this happens. If any of you have patches lined up for commit or
about to be, please reply to this mail with details so we can triage
what gets in and what can hold off in audit.

Note that patches applied after branching may still get to 3.5, but will
have to be stable in trunk first.

Patches that are welcome any time:
 - documentation updates
 - security bug fixes

2) Documentation and stability testing

After branching we need to do as much testing as we can throw at the new
branch and update any missing documentation.

(Continue reading)

noc | 23 Jul 03:45 2014

Build failed in Jenkins: 3.HEAD-amd64-FreeBSD-9.1-clang #615

See <http://build.squid-cache.org/job/3.HEAD-amd64-FreeBSD-9.1-clang/615/changes>

Changes:

[Amos Jeffries] Fix unit test linker issues in testHttpReply and testStore

Also, update STUB files for comm.cc, event.cc, and libssl-squid.la

[Automatic source maintenance] SourceFormat Enforcement

[Christos Tsantilas] Fix tcp outgoing tos bugs

The tcp_outgoing_tos is buggy in trunk:
- The ToS is never set for packets of the first request of a TCP connection.
- The ToS is never set for HTTPS traffic no matter whether requests are bumped
or not.
- The ToS value is not set for ftp data connections

This patch solve the above problems:
- It moves the codes which sets the TOS value for a new connection from the
the comm_openex to a higher-level code, where the connection protocol
(IPv4 or IPv6) is known.
- Add code to set TOS value for ftp data connections.
- Add a check on parsing code to warn users if the configured ToS value has the
ECN bits set, and adjust the value to a correct one.

Notes
Currently squid support only passive ftp data connections. If squid in the
future supports active ftp connections, then some work required to TcpAcceptor
class to allow setting ToS values for connections established on squid listening
(Continue reading)

noc | 23 Jul 03:42 2014

Build failed in Jenkins: 3.HEAD-amd64-OpenBSD-5.4 #121

See <http://build.squid-cache.org/job/3.HEAD-amd64-OpenBSD-5.4/121/changes>

Changes:

[Amos Jeffries] Fix unit test linker issues in testHttpReply and testStore

Also, update STUB files for comm.cc, event.cc, and libssl-squid.la

[Automatic source maintenance] SourceFormat Enforcement

[Christos Tsantilas] Fix tcp outgoing tos bugs

The tcp_outgoing_tos is buggy in trunk:
- The ToS is never set for packets of the first request of a TCP connection.
- The ToS is never set for HTTPS traffic no matter whether requests are bumped
or not.
- The ToS value is not set for ftp data connections

This patch solve the above problems:
- It moves the codes which sets the TOS value for a new connection from the
the comm_openex to a higher-level code, where the connection protocol
(IPv4 or IPv6) is known.
- Add code to set TOS value for ftp data connections.
- Add a check on parsing code to warn users if the configured ToS value has the
ECN bits set, and adjust the value to a correct one.

Notes
Currently squid support only passive ftp data connections. If squid in the
future supports active ftp connections, then some work required to TcpAcceptor
class to allow setting ToS values for connections established on squid listening
(Continue reading)

noc | 23 Jul 03:19 2014

Build failed in Jenkins: 3.HEAD-amd64-FreeBSD-9.1 #715

See <http://build.squid-cache.org/job/3.HEAD-amd64-FreeBSD-9.1/715/changes>

Changes:

[Amos Jeffries] Fix unit test linker issues in testHttpReply and testStore

Also, update STUB files for comm.cc, event.cc, and libssl-squid.la

[Automatic source maintenance] SourceFormat Enforcement

[Christos Tsantilas] Fix tcp outgoing tos bugs

The tcp_outgoing_tos is buggy in trunk:
- The ToS is never set for packets of the first request of a TCP connection.
- The ToS is never set for HTTPS traffic no matter whether requests are bumped
or not.
- The ToS value is not set for ftp data connections

This patch solve the above problems:
- It moves the codes which sets the TOS value for a new connection from the
the comm_openex to a higher-level code, where the connection protocol
(IPv4 or IPv6) is known.
- Add code to set TOS value for ftp data connections.
- Add a check on parsing code to warn users if the configured ToS value has the
ECN bits set, and adjust the value to a correct one.

Notes
Currently squid support only passive ftp data connections. If squid in the
future supports active ftp connections, then some work required to TcpAcceptor
class to allow setting ToS values for connections established on squid listening
(Continue reading)

noc | 23 Jul 02:54 2014

Build failed in Jenkins: 3.HEAD-amd64-FreeBSD-7.2 #2240

See <http://build.squid-cache.org/job/3.HEAD-amd64-FreeBSD-7.2/2240/changes>

Changes:

[Amos Jeffries] Fix unit test linker issues in testHttpReply and testStore

Also, update STUB files for comm.cc, event.cc, and libssl-squid.la

[Automatic source maintenance] SourceFormat Enforcement

[Christos Tsantilas] Fix tcp outgoing tos bugs

The tcp_outgoing_tos is buggy in trunk:
- The ToS is never set for packets of the first request of a TCP connection.
- The ToS is never set for HTTPS traffic no matter whether requests are bumped
or not.
- The ToS value is not set for ftp data connections

This patch solve the above problems:
- It moves the codes which sets the TOS value for a new connection from the
the comm_openex to a higher-level code, where the connection protocol
(IPv4 or IPv6) is known.
- Add code to set TOS value for ftp data connections.
- Add a check on parsing code to warn users if the configured ToS value has the
ECN bits set, and adjust the value to a correct one.

Notes
Currently squid support only passive ftp data connections. If squid in the
future supports active ftp connections, then some work required to TcpAcceptor
class to allow setting ToS values for connections established on squid listening
(Continue reading)

Amos Jeffries | 19 Jul 04:20 2014
Picon

[RFC] remove JobCallback macro

I have once again been foiled by brokenness in the JobCallback macro
provided apparently for convenience generating Job based AsyncCalls.

This time I seem to have found the problem though. It stems from
JobCalback requiring a Dialer+Job+method - with zero parameter objects
accepted. If one attempts to use it with UnaryMemFunT<> the compile
breaks with some fairly cryptic errors.

The resolution to this is to use the asyncCall() template function
directly which JobCallback was supposedly making easier. However, the
asyncCall is not particularly hard to use in the first place and has
lots of code examples to look at now.

Give that JobCallback() is only usable (and used) by the
NullaryMemFunT<> and where the CommCalls parameter hack/workaround means
the parameters variable need not be provided to the dialer constructor.
I am believing that this particualar macro is providing almost no
benefit outside of CommCalls, so we should do one of the following:

 a) move it to the CommCalls.h header and rename it specific to that API
to avoid others hitting the same confusion in future.

 b) remove JobCallback macro completely in favour of asyncCall.

 c) replace existing JobCallback with overloaded inline template
equivalents to asyncCall() so that UnaryMemFunT parameter can be provided.

I favour b or c.
 * b makes the code cleaner and more consistent regarding Call creation.
 * c offers the chance to hide lots of Unary/Nullary-MemFunT definitions
(Continue reading)

Amos Jeffries | 18 Jul 09:32 2014
Picon

[RFC] bandwidth savigns via header eliding

Some of the statisticas being brought up in the IETF HTTP/2 discussions
is highlighting certain garbage headers which are unfortunately quite
common.

I have wondered about creating a registry of known garbage and simply
dropping those headers on arrival in the parser. This would be in
addition to the header registry lookup and masking process we have for
hop-by-hop headers.

Any other thoughts on this?

Amos

Tsantilas Christos | 16 Jul 19:39 2014
Picon
Picon

TOS values

Hi all,
  Squid currently does not make a check for the TOS values used in squid 
configuration file. Squid will accept 8bit numbers as TOS values, however:
  1) TOS values with 1st ad 2nd bit set can not be used. These bits used 
by the ECN. For Linux if someone try to set the 0x23 value as TOS value 
(which sets 1st and 1nd bits), the 0x20 will be used instead, without 
any warn for the user.

  2) Some of the TOS values are already reserved for example those which 
are reserved from RFC2597.

The above may confuse squid users.

Maybe it is not bad idea to:
  - Warn users when try to use TOS value which uses the ECN bits
  - Warn users when try to use TOS values which are not reserved. The 
user will know that this value is free for use.

Opinions?

Rainer Weikusat | 16 Jul 19:11 2014

Comm::TcpAcceptor::doAccept fd limit handling is broken

NB: This occurred in the real world on some 'customer machine' using
3.3.12. Since the code is unchanged in 3.HEAD, I assume it is affected,
too.

The method mentioned in the subject is

void
Comm::TcpAcceptor::doAccept(int fd, void *data)
{
    try {
        debugs(5, 2, HERE << "New connection on FD " << fd);

        Must(isOpen(fd));
        TcpAcceptor *afd = static_cast<TcpAcceptor*>(data);

        if (!okToAccept()) {
            AcceptLimiter::Instance().defer(afd);
        } else {
            afd->acceptNext();
        }
        SetSelect(fd, COMM_SELECT_READ, Comm::TcpAcceptor::doAccept, afd, 0);

    } catch (const std::exception &e) {
        fatalf("FATAL: error while accepting new client connection: %s\n", e.what());
    } catch (...) {
        fatal("FATAL: error while accepting new client connection: [unkown]\n");
    }
}

This is broken because it re-enables readable notifications even if no
(Continue reading)

Alex Rousskov | 14 Jul 19:02 2014

Re: r13497: Converts the PortCfgPointer to reference counted

On 07/14/2014 03:48 AM, Amos Jeffries wrote:
>   Converts the PortCfgPointer to reference counted

This change should have gone through an audit IMO, but I appreciate
having an entire Sunday to react to the "will commit shortly" notice for
a not yet posted patch.

> -    if (!s.valid()) {
> +    if (!s) {
>          // it is possible the call or accept() was still queued when the port was reconfigured
>          debugs(33, 2, "HTTP accept failure: port reconfigured.");
>          return;

While the port pointer could get invalidated in the [fixed] old code, it
cannot become nil in the new code. The above comment is no longer valid
and the code itself no longer works as intended. Please fix both
instances of the above logic: one in httpAccept() and one in httpsAccept().

AFAICT, if you simply replace the above code with a comment warning that
the port may no longer be in the current configuration, we will continue
to start transactions previously accepted on no longer configured ports.
Hopefully, this will not cause any new problems, but there may be a
better solution.

Thank you,

Alex.

noc | 14 Jul 12:09 2014

Build failed in Jenkins: 3.HEAD-amd64-centos-6 #398

See <http://build.squid-cache.org/job/3.HEAD-amd64-centos-6/398/changes>

Changes:

[Amos Jeffries] Converts the PortCfgPointer to reference counted

This allows long-lived connections to retain access to their original
receiving port configuration even after squid has been reconfigured.
Reference counting prevents some leaking of these port configuration
details and associated state by removing locking uncertainties.

Also, fixes all parsing errors resulting from the change. Most of
the issues were due to use of raw-pointers and explicit
cbdataReference*() API.

------------------------------------------
[...truncated 16574 lines...]
make[2]: Leaving directory `<http://build.squid-cache.org/job/3.HEAD-amd64-centos-6/ws/btlayer-01-minimal/squid-3.HEAD-BZR/_build/errors'>
Making all in doc
make[2]: Entering directory `<http://build.squid-cache.org/job/3.HEAD-amd64-centos-6/ws/btlayer-01-minimal/squid-3.HEAD-BZR/_build/doc'>
Making all in manuals
make[3]: Entering directory `<http://build.squid-cache.org/job/3.HEAD-amd64-centos-6/ws/btlayer-01-minimal/squid-3.HEAD-BZR/_build/doc/manuals'>
make[3]: Nothing to be done for `all'.
make[3]: Leaving directory `<http://build.squid-cache.org/job/3.HEAD-amd64-centos-6/ws/btlayer-01-minimal/squid-3.HEAD-BZR/_build/doc/manuals'>
make[3]: Entering directory `<http://build.squid-cache.org/job/3.HEAD-amd64-centos-6/ws/btlayer-01-minimal/squid-3.HEAD-BZR/_build/doc'>
make[3]: Nothing to be done for `all-am'.
make[3]: Leaving directory `<http://build.squid-cache.org/job/3.HEAD-amd64-centos-6/ws/btlayer-01-minimal/squid-3.HEAD-BZR/_build/doc'>
make[2]: Leaving directory `<http://build.squid-cache.org/job/3.HEAD-amd64-centos-6/ws/btlayer-01-minimal/squid-3.HEAD-BZR/_build/doc'>
Making all in helpers
make[2]: Entering directory `<http://build.squid-cache.org/job/3.HEAD-amd64-centos-6/ws/btlayer-01-minimal/squid-3.HEAD-BZR/_build/helpers'>
(Continue reading)


Gmane