Re: [PATCH] add ap_sendfile_enabled

On 9/30/08 1:15 PM, "William A. Rowe, Jr." <wrowe <at> rowe-clan.net> wrote:

> should we look
> at something closer to ap_mpm_query (ap_core_query?) and start adding it a
> whole lot of stuff?

+1

> Something to ponder; injecting this as-is for 2.2.10 seems rushed.

+1

--

-- 
Brian Akins
Chief Operations Engineer
Turner Digital Media Technologies

Re: [PATCH] add ap_sendfile_enabled

On Sep 30, 2008, at 1:15 PM, William A. Rowe, Jr. wrote:
>
> Rather than many symbols that become slow to bind, would it make  
> more sense
> to either pull aside the core config stuff so that anyone with a  
> legitimate
> reason can inspect it all without CORE_PRIVATE_EVERYTHING, or should  
> we look
> at something closer to ap_mpm_query (ap_core_query?) and start  
> adding it a
> whole lot of stuff?
>

Me like.

> Something to ponder; injecting this as-is for 2.2.10 seems rushed.
>

?? Who said anything about this being in 2.2.10?

Re: 2.2.10 (Was: Re: STATUS file, 2.2.10 and voting)

On Sep 30, 2008, at 7:59 AM, Jim Jagielski wrote:

> I'm looking to T&R 2.2.10 this week, for a release early next
> week... Complain now :)
>

There is one small proposed patch in STATUS that is lacking
1 single vote to be folded into 2.2.10... Anyone have cycles
to look, test and vote?

[PATCH] UNIX domain socket listener and file descriptor passing

Hello,

Please reply directly to me as I am not subscribed.

I created a patch for Apache 1.3.34 from debian etch which adds the
ability to set an absolute path for a Listen address.  With this path a
different bind & listen (make_sock()) routine is used to establish a
UNIX domain socket listener in the filesystem.

After accepting a connection on this new type of listener, a recvmsg() is
entered to receive ancillary data for the purposes of passing a TCP socket
descriptor to Apache from an unrelated process.  Upon receiving such a
message, the UNIX domain socket stream is closed and the passed socket
descriptor is treated as if it were simply accepted through a normal TCP
listener.

The potential implications for this new way of getting TCP sockets into
the Apache process surely are more than I can imagine.

For me the reason for creating this patch was to integrate my module httpx
found here:
http://serverkit.org/modules/httpx/

This module does some wacky things to snoop the HTTP/1.1 Host: request line
before passing the socket to it's final destination httpd process via
a named UNIX domain socket.  The goal here was to allow total freedom in
Apache configuration for HTTP/1.1 name-based vhosts, mod_whatever, user
access to configuration file, runs as their user and group from the start,
no suexec needed, etc.  It's understood that httpx potentially violates the
rfc and is potentially quite rude.

Re: 2.2.10 (Was: Re: STATUS file, 2.2.10 and voting)

On Oct 2, 2008, at 10:21 AM, Jim Jagielski wrote:

>
> On Sep 30, 2008, at 7:59 AM, Jim Jagielski wrote:
>
>> I'm looking to T&R 2.2.10 this week, for a release early next
>> week... Complain now :)
>>
>
> There is one small proposed patch in STATUS that is lacking
> 1 single vote to be folded into 2.2.10... Anyone have cycles
> to look, test and vote?
>

FWIW, I'm hoping to say a 1/2 today, so the T&R will likely
happen Monday. This gives us the weekend to review the
remaining proposed backport as well as do some prelim testing
before the tag. Note which versions of APR/APU I intend
to bundle (1.3.3/1.3.4)

Building minimal (static) httpd

Re: svn commit: r699481 - /httpd/httpd/trunk/server/mpm/winnt/child.c

William A. Rowe, Jr. wrote:
> Mladen Turk wrote:
>> wrowe <at> apache.org wrote:
>>> Author: wrowe
>>> Date: Fri Sep 26 13:15:10 2008
>>> New Revision: 699481
>>>
>>> URL: http://svn.apache.org/viewvc?rev=699481&view=rev
>>> Log:
>>> Reimplement ThreadStackSize to behave as on unix for any
>>> Windows 2003/2008 (XP/Vista) servers.  Virtual allocations
>>> will only consume pages once referenced, while the page
>>> alignment will vary by ThreadStackSize setting so that the
>>> maximum number of threads and minimum stack VM profile will
>>> be wasted.
>>>
>>> -            child_handles[i] = (HANDLE) _beginthreadex(
>>> +            child_handles[i] = CreateThread(NULL, ap_thread_stacksize,
>> Why did you change the _beginthreadex to CreateThread?
>> Won't that bring back CRT local Tls problems that forced using
>> _beginthreadex at the first place?
> 
> Please site the Tls problems.  There is one I can think of, which is
> the behavior of the locale (inherit from parent v.s. inherit from global)
> but that's not even a portable contract between unix and windows modules.
> Otherwise, TLS is allocated on first-reference.

OK... here we go, sorry it wasn't on a convenient machine when I posted this
message in the first place;

  http://blogs.msdn.com/michkap/archive/2008/01/02/6950506.aspx

It appears that only if you started playing with _ENABLE_PER_THREAD_LOCALE
(something unwise for httpd or even apr, since it's far from portable) will
you observe some interesting side effects.

> _beginthreadex() is further dangerous because returning from that thread,
> before return, the handle is *closed*.  Elsewhere we are testing those
> handles to see if the thread has terminated, potentially what it's return
> code is.  So closing the handle before we do is invalid behavior.

This was my reason.  In the mpm (not in apr) I deliberately choose CreateThread
for every handle we utilize and close beyond the lifespan of the thread, while
I left _beginthreadex for those threads which are simply started and forgotten.

The _beginthreadex pragma will do the thread handle cleanup for us on end thread.

>> Also I have concerns with chosen 64K thread stack size for
>> some other threads in this patch series. Perhaps a 256K will
>> be more safe (particularly for winnt_accept), because 64K on
>> stack overflow OS will allocate 1M actually, so this either needs to
>> get carefully calculated or traced by some tool, cause larger
>> initial commit size might eventually use less memory.
> 
> Are you sure?  Please validate your assumptions with actual testing
> before postulating; the original behavior and assumptions of the entire
> ThreadStackSize behavior on windows was so off base as to be laughable.
> These changes are all based on direct observation, on Windows XP (2003)
> to be confirmed on 2008 before I propose for backport.

AFAICT, there's one chance.  An overflow can't extend the stack when there
is another stack right afterwards, so on overflow httpd can and should fault.

Note that with the flag changes, these are true virtual allocation sizes,
no longer preallocation within the oversized thread spaces.  That's the unix
behavior, so that should be the win32 behavior.  I've managed to crank up to
the 15,000 threadlimit hard limit.

Why, for that matter, do we have a hardcoded ThreadLimit cap?  Since we have
a default ThreadLimit cap, the hardcode cap seems silly, as ThreadLimit now
controls storage and reservations, and is tuned as needed by the user.
This seems like now-useless, legacy cruft.

> Can you look at those threads and please justify a larger stack size
> rather than just kicking them up for the sake of it?  Based on rev 88498
> I'm pretty sure that several choices were made without any real basis
> to get the code adjusted, and they never were revisited.

I have one major test before this is allowed to remain.  The stacks I checked
out and my initial review was all based on 32 bit win32.  On 64 bit, obviously
that stack expands in size due to pointer sizes and alignment.

"Apache 2 Pocket Reference" now published by O'Reilly

O'Reilly have now published "Apache 2 Pocket Reference".  It is
available for purchase on O'Reilly's web site, as well as from Amazon
and other online bookstores.  O'Reilly also have the book available to
purchase from their site as a downloadable PDF file.  The book is priced
at US$14.99 (£8.99) and the PDF US$11.99.

This book started off as the second edition of my "Apache Pocket
Reference", but as I worked on it it became clear that it was not
feasible to cover Apache 1.3 and 2.x in the same book.  As it is, the
number of directives covered has increased from something like 150 to
about 350, and the number of pages from 110 to 208.  I have tried to
succinctly cover all the directives and modules included in the 2.2.9
release of Apache.

I would welcome any feedback, corrections, or suggestions for improvements.

Regards
Andrew

--

-- 
Andrew Ford                     
South Wing Compton House, Compton Green,
Redmarley, Gloucestershire, GL19 3JB, UK
Tel:    +44 1531 829900
Mobile: +44 7785 258278
Email:  A.Ford <at> ford-mason.co.uk

Re: mod_ssl, SSL_CLIENT_CERT_CHAIN, mod_proxy_ajp and full chain of client certificates

Yes, while mod_jk has an option to send the cert chain (added a little over 
18 months ago by mturk), no Tomcat connector has an option to read it.  As a 
result, Tomcat will read the end certificate and ignore the rest of the 
chain.

This is because the AJP/1.3 protocol was created back in the days of 
Servlet-2.2 (corresponding to Tomcat 3.x) and back then only the end 
certificate was exposed by the Servlet-API.

Mladen's patch to mod_jk is simplier than this one, so I would prefer it to 
this one.  But I have no voting rights on this list :).

"Bruno Harbulot" <Bruno.Harbulot <at> manchester.ac.uk> wrote in message 
news:gbt26i$9p2$1 <at> ger.gmane.org...
> Hello,
>
> I'm trying to use mod_proxy_ajp instead of mod_jk, but I'd like to be
> able to pass the whole client certificate chain, instead of only the end
> certificate. The servlet specification allows for a chain of
> certificates to be presented and this is indeed possible with mod_jk,
> using "JkOptions +ForwardSSLCertChain".
>
> This doesn't seem to be possible using mod_proxy_ajp, which uses the
> content of the SSL_CLIENT_CERT variable only.
>
> I thought I would be able to pass the chain using mod_headers.
> Unfortunately, there doesn't seem to be a mod_ssl variable that
> represents the whole chain. There is a set of variables called
> SSL_CLIENT_CERT_CHAIN_n (where n is an integer), but they have to be
> named individually.
>
> I'm attaching the patch I've written to provide a variable called
> SSL_CLIENT_CERT_CHAIN, which is the concatenation of all the
> certificates in the chain, in PEM format. (It also sets
> SSL_CLIENT_CERT_CHAIN_0 when there's no chain available but just one
> certificate.)
>
> A few tests with mod_headers "RequestHeader set X-ClientCertChain
> %{SSL_CLIENT_CERT_CHAIN}s" seem to indicate that it works.
>
> However, I've also tried to modify mod_proxy_ajp to send the whole
> chain, but this doesn't work:
>
> --- a/modules/proxy/ajp.h
> +++ b/modules/proxy/ajp.h
>  <at>  <at>  -60,7 +60,7  <at>  <at> 
>
>  /* The following environment variables match mod_ssl! */
>  #define AJP13_HTTPS_INDICATOR           "HTTPS"
> -#define AJP13_SSL_CLIENT_CERT_INDICATOR "SSL_CLIENT_CERT"
> +#define AJP13_SSL_CLIENT_CERT_INDICATOR "SSL_CLIENT_CERT_CHAIN"
>  #define AJP13_SSL_CIPHER_INDICATOR      "SSL_CIPHER"
>  #define AJP13_SSL_SESSION_INDICATOR     "SSL_SESSION_ID"
>  #define AJP13_SSL_KEY_SIZE_INDICATOR    "SSL_CIPHER_USEKEYSIZE"
>
> This patch has been made against the svn trunk, rev 695234.
>
>
> I'm aware that my knowledge of the Apache Httpd code is limited, so this
> patch is likely to need improvements (there's obviously something wrong
> since my modification to mod_proxy_ajp doesn't work).
> I'd appreciate any comments and suggestions.
>
>
> Best wishes,
>
> Bruno.
>

--------------------------------------------------------------------------------

> diff --git a/modules/ssl/ssl_engine_kernel.c 
> b/modules/ssl/ssl_engine_kernel.c
> index e938d05..894bef8 100644
> --- a/modules/ssl/ssl_engine_kernel.c
> +++ b/modules/ssl/ssl_engine_kernel.c
>  <at>  <at>  -1157,6 +1157,11  <at>  <at>  int ssl_hook_Fixup(request_rec *r)
>                     apr_table_setn(env, var, val);
>                 }
>             }
> +        } else {
> +            val = ssl_var_lookup(r->pool, r->server, r->connection,
> +                                 r, "SSL_CLIENT_CERT");
> +
> +            apr_table_setn(env, "SSL_CLIENT_CERT_CHAIN_0", val);
>         }
>     }
>
> diff --git a/modules/ssl/ssl_engine_vars.c b/modules/ssl/ssl_engine_vars.c
> index 3d688cd..e191ddd 100644
> --- a/modules/ssl/ssl_engine_vars.c
> +++ b/modules/ssl/ssl_engine_vars.c
>  <at>  <at>  -46,6 +46,7  <at>  <at>  static char *ssl_var_lookup_ssl_cert_remain(apr_pool_t 
> *p, ASN1_UTCTIME *tm);
> static char *ssl_var_lookup_ssl_cert_serial(apr_pool_t *p, X509 *xs);
> static char *ssl_var_lookup_ssl_cert_chain(apr_pool_t *p, STACK_OF(X509) 
> *sk, char *var);
> static char *ssl_var_lookup_ssl_cert_PEM(apr_pool_t *p, X509 *xs);
> +static char *ssl_var_lookup_ssl_cert_chain_PEM(apr_pool_t *p, 
> STACK_OF(X509) *sk);
> static char *ssl_var_lookup_ssl_cert_verify(apr_pool_t *p, conn_rec *c);
> static char *ssl_var_lookup_ssl_cipher(apr_pool_t *p, conn_rec *c, char 
> *var);
> static void  ssl_var_lookup_ssl_cipher_bits(SSL *ssl, int *usekeysize, int 
> *algkeysize);
>  <at>  <at>  -300,6 +301,10  <at>  <at>  static char *ssl_var_lookup_ssl(apr_pool_t *p, 
> conn_rec *c, char *var)
>     else if (ssl != NULL && strlen(var) >= 6 && strcEQn(var, "CIPHER", 6)) 
> {
>         result = ssl_var_lookup_ssl_cipher(p, c, var+6);
>     }
> +    else if (ssl != NULL && strcEQ(var, "CLIENT_CERT_CHAIN")) {
> +        sk = SSL_get_peer_cert_chain(ssl);
> +        result = ssl_var_lookup_ssl_cert_chain(p, sk, NULL);
> +    }
>     else if (ssl != NULL && strlen(var) > 18 && strcEQn(var, 
> "CLIENT_CERT_CHAIN_", 18)) {
>         sk = SSL_get_peer_cert_chain(ssl);
>         result = ssl_var_lookup_ssl_cert_chain(p, sk, var+18);
>  <at>  <at>  -550,13 +555,18  <at>  <at>  static char 
> *ssl_var_lookup_ssl_cert_chain(apr_pool_t *p, STACK_OF(X509) *sk, ch
>
>     result = NULL;
>
> -    if (strspn(var, "0123456789") == strlen(var)) {
> -        n = atoi(var);
> -        if (n < sk_X509_num(sk)) {
> -            xs = sk_X509_value(sk, n);
> -            result = ssl_var_lookup_ssl_cert_PEM(p, xs);
> +    if (var != NULL) {
> +        if (strspn(var, "0123456789") == strlen(var)) {
> +            n = atoi(var);
> +            if (n < sk_X509_num(sk)) {
> +                xs = sk_X509_value(sk, n);
> +                result = ssl_var_lookup_ssl_cert_PEM(p, xs);
> +            }
>         }
>     }
> +    else {
> +        result = ssl_var_lookup_ssl_cert_chain_PEM(p, sk);
> +    }
>
>     return result;
> }
>  <at>  <at>  -578,6 +588,28  <at>  <at>  static char *ssl_var_lookup_ssl_cert_PEM(apr_pool_t 
> *p, X509 *xs)
>     return result;
> }
>
> +static char *ssl_var_lookup_ssl_cert_chain_PEM(apr_pool_t *p, 
> STACK_OF(X509) *sk)
> +{
> +    char *result;
> +    BIO *bio;
> +    X509 *xs;
> +    int n;
> +    int i;
> +
> +    if ((bio = BIO_new(BIO_s_mem())) == NULL)
> +        return NULL;
> +    for (i=0; i < sk_X509_num(sk); i++) {
> +        xs = sk_X509_value(sk, i);
> +        PEM_write_bio_X509(bio, xs);
> +    }
> +    n = BIO_pending(bio);
> +    result = apr_pcalloc(p, n+1);
> +    n = BIO_read(bio, result, n);
> +    result[n] = NUL;
> +    BIO_free(bio);
> +    return result;
> +}
> +
> static char *ssl_var_lookup_ssl_cert_verify(apr_pool_t *p, conn_rec *c)
> {
>     SSLConnRec *sslconn = myConnConfig(c);
> 

Re: mod_ssl, SSL_CLIENT_CERT_CHAIN, mod_proxy_ajp and full chain of client certificates

Bill Barker wrote:
> 
> Mladen's patch to mod_jk is simplier than this one, so I would prefer it to 
> this one.  But I have no voting rights on this list :).
>

Right, I'll prepare something for mod_proxy as well.
It is on my TODO list for a long time.

Regards
--

-- 
^(TM)