headers
Jim Jagielski | 1 Nov 2011 21:23
Favicon

prefetch proxy

In mod_proxy_http we have:

    /* Prefetch MAX_MEM_SPOOL bytes
     *
     * This helps us avoid any election of C-L v.s. T-E
     * request bodies, since we are willing to keep in
     * memory this much data, in any case.  This gives
     * us an instant C-L election if the body is of some
     * reasonable size.
     */
    temp_brigade = apr_brigade_create(p, bucket_alloc);
    do {
        status = ap_get_brigade(r->input_filters, temp_brigade,
                                AP_MODE_READBYTES, APR_BLOCK_READ,
                                MAX_MEM_SPOOL - bytes_read);
        if (status != APR_SUCCESS) {
            ap_log_error(APLOG_MARK, APLOG_ERR, status, r->server,
                         "proxy: prefetch request body failed to %pI (%s)"
                         " from %s (%s)",
                         p_conn->addr, p_conn->hostname ? p_conn->hostname: "",
                         c->remote_ip, c->remote_host ? c->remote_host: "");
            return HTTP_BAD_REQUEST;
        }

        apr_brigade_length(temp_brigade, 1, &bytes);
        bytes_read += bytes;

However, I see times when status could be APR_EAGAIN. IMO, it doesn't
make sense to error out here in that case. Right?
(Continue reading)

Permalink | Reply | 27 comments |
headers
Rüdiger Plüm | 2 Nov 2011 10:44

Re: prefetch proxy


Am 01.11.2011 21:23, schrieb Jim Jagielski:
> In mod_proxy_http we have:
>
>      /* Prefetch MAX_MEM_SPOOL bytes
>       *
>       * This helps us avoid any election of C-L v.s. T-E
>       * request bodies, since we are willing to keep in
>       * memory this much data, in any case.  This gives
>       * us an instant C-L election if the body is of some
>       * reasonable size.
>       */
>      temp_brigade = apr_brigade_create(p, bucket_alloc);
>      do {
>          status = ap_get_brigade(r->input_filters, temp_brigade,
>                                  AP_MODE_READBYTES, APR_BLOCK_READ,
>                                  MAX_MEM_SPOOL - bytes_read);
>          if (status != APR_SUCCESS) {
>              ap_log_error(APLOG_MARK, APLOG_ERR, status, r->server,
>                           "proxy: prefetch request body failed to %pI (%s)"
>                           " from %s (%s)",
>                           p_conn->addr, p_conn->hostname ? p_conn->hostname: "",
>                           c->remote_ip, c->remote_host ? c->remote_host: "");
>              return HTTP_BAD_REQUEST;
>          }
>
>          apr_brigade_length(temp_brigade, 1,&bytes);
>          bytes_read += bytes;
>
> However, I see times when status could be APR_EAGAIN. IMO, it doesn't
(Continue reading)

Permalink | Reply | 20 comments |
headers
Jim Jagielski | 2 Nov 2011 12:22
Favicon

Re: prefetch proxy


On Nov 2, 2011, at 5:44 AM, Rüdiger Plüm wrote:

> 
> 
> Am 01.11.2011 21:23, schrieb Jim Jagielski:
>> In mod_proxy_http we have:
>> 
>>     /* Prefetch MAX_MEM_SPOOL bytes
>>      *
>>      * This helps us avoid any election of C-L v.s. T-E
>>      * request bodies, since we are willing to keep in
>>      * memory this much data, in any case.  This gives
>>      * us an instant C-L election if the body is of some
>>      * reasonable size.
>>      */
>>     temp_brigade = apr_brigade_create(p, bucket_alloc);
>>     do {
>>         status = ap_get_brigade(r->input_filters, temp_brigade,
>>                                 AP_MODE_READBYTES, APR_BLOCK_READ,
>>                                 MAX_MEM_SPOOL - bytes_read);
>>         if (status != APR_SUCCESS) {
>>             ap_log_error(APLOG_MARK, APLOG_ERR, status, r->server,
>>                          "proxy: prefetch request body failed to %pI (%s)"
>>                          " from %s (%s)",
>>                          p_conn->addr, p_conn->hostname ? p_conn->hostname: "",
>>                          c->remote_ip, c->remote_host ? c->remote_host: "");
>>             return HTTP_BAD_REQUEST;
>>         }
>>
(Continue reading)

Permalink | Reply | 19 comments |
headers
Plüm, Rüdiger, VF-Group | 2 Nov 2011 12:40

RE: prefetch proxy


> -----Original Message-----
> From: Jim Jagielski [mailto:jim <at> jaguNET.com] 
> Sent: Mittwoch, 2. November 2011 12:22
> To: dev <at> httpd.apache.org
> Subject: Re: prefetch proxy
> 
> 
> On Nov 2, 2011, at 5:44 AM, Rüdiger Plüm wrote:
> 
> > 
> > 
> > Am 01.11.2011 21:23, schrieb Jim Jagielski:
> >> In mod_proxy_http we have:
> >> 
> >>     /* Prefetch MAX_MEM_SPOOL bytes
> >>      *
> >>      * This helps us avoid any election of C-L v.s. T-E
> >>      * request bodies, since we are willing to keep in
> >>      * memory this much data, in any case.  This gives
> >>      * us an instant C-L election if the body is of some
> >>      * reasonable size.
> >>      */
> >>     temp_brigade = apr_brigade_create(p, bucket_alloc);
> >>     do {
> >>         status = ap_get_brigade(r->input_filters, temp_brigade,
> >>                                 AP_MODE_READBYTES, APR_BLOCK_READ,
> >>                                 MAX_MEM_SPOOL - bytes_read);
> >>         if (status != APR_SUCCESS) {
> >>             ap_log_error(APLOG_MARK, APLOG_ERR, status, r->server,
(Continue reading)

Permalink | Reply | 18 comments |
headers
Gregg L. Smith | 3 Nov 2011 00:21

RE: svn commit: r1183475 - in /httpd/httpd/trunk: include/ap_mmn.h

Hi Stefan,

I am getting type errors on these in MSVC, can we please move the * after
PROXY_DECLARE_DATA?

Cheers,

Gregg

--- httpd/httpd/trunk/modules/proxy/proxy_util.c	2011/10/14 19:48:03	1183474
+++ httpd/httpd/trunk/modules/proxy/proxy_util.c	2011/10/14 19:51:17	1183475
 <at>  <at>  -65,6 +65,8  <at>  <at> 
  /* Global balancer counter */
  int PROXY_DECLARE_DATA proxy_lb_workers = 0;
  static int lb_workers_limit = 0;
+const apr_strmatch_pattern * PROXY_DECLARE_DATA ap_proxy_strmatch_path;
+const apr_strmatch_pattern * PROXY_DECLARE_DATA ap_proxy_strmatch_domain;

  static int proxy_match_ipaddr(struct dirconn_entry *This, request_rec *r);
  static int proxy_match_domainname(struct dirconn_entry *This, request_rec *r);

--- httpd/httpd/trunk/modules/proxy/mod_proxy.h	2011/10/14 19:48:03	1183474
+++ httpd/httpd/trunk/modules/proxy/mod_proxy.h	2011/10/14 19:51:17	1183475
 <at>  <at>  -907,6 +905,8  <at>  <at> 
  extern module PROXY_DECLARE_DATA proxy_module;

  extern int PROXY_DECLARE_DATA proxy_lb_workers;
+extern const apr_strmatch_pattern * PROXY_DECLARE_DATA ap_proxy_strmatch_path;
+extern const apr_strmatch_pattern * PROXY_DECLARE_DATA ap_proxy_strmatch_domain;
(Continue reading)

Permalink | Reply | 1 comment |
headers
Stefan Fritsch | 3 Nov 2011 06:37
Picon
Favicon

RE: svn commit: r1183475 - in /httpd/httpd/trunk: include/ap_mmn.h

On Wed, 2 Nov 2011, Gregg L. Smith wrote:
> I am getting type errors on these in MSVC, can we please move the * after
> PROXY_DECLARE_DATA?

Sure, fixed in r1196946.

Cheers,
Stefan
Permalink | Reply | 0 comments |
headers
Stefan Fritsch | 3 Nov 2011 07:29
Picon
Favicon

Re: svn commit: r1189985 - in /httpd/httpd/trunk: include/httpd.h include/util_varbuf.h modules/filters/mod_substitute.c server/util.c

> On 10/27/2011 3:15 PM, sf <at> apache.org wrote:
>> Use apr_pregsub_ex() and maxlen = 0 for unlimited in mod_substitute.
>
> Uhm... wha?

This was not intended as a final solution. Besides, one case already had 
unlimited (but using argument APR_SIZE_MAX instead of 0).

> Not acceptable.  Choose a bound.  unlimited is not acceptable.
>
> Yes, that is a veto.

What can a content filter do to handle the error? Does returning 
AP_FILTER_ERROR do any good? The headers may have been sent already.
Permalink | Reply | 1 comment |
headers
Stefan Fritsch | 3 Nov 2011 07:43
Picon
Favicon

Re: 2.0.65 and 2.3.15-dev

On Fri, 28 Oct 2011, William A. Rowe Jr. wrote:
> I think 2.4.0 GA at AC is destined to be a "dot zero" release (most
> emphasis on the zero, as in null).  Not suggesting we don't do it,
> simply that nobody, including yourself, will be at liberty to change
> scoreboard datum, api interfaces, etc.
>
> Are we anywhere near that level of commitment to trunk?  It would be
> rather sad if 2.4.0 was ditched for 2.6.0.  Just look to the recent
> perl releases for examples of such instability.

What are opinions about adding some fields to structs for extensions that 
seem like a good idea but have not been written yet. This would then allow 
to add them during the lifetime of 2.4.x with only a minor MMN bump.

I could imagine:

- add some fields to the global scoreboard, for benefit of mod_status:
   * timestamp of last time max worker number was reached
   * timestamp of last time all workers were busy
   * timestamp of last time a child crashed
   * number of child crashes since start

- add a void or apr_array_header_t pointer to ap_logconf, so that we may 
add filtering of error log messages (there was some interest for this in a 
recent thread on -dev).

I don't think any of these features should delay 2.4.0, but I think it 
would be nice to have them before 2.6/3.0.
(Continue reading)

Permalink | Reply | 11 comments |
headers
Jim Jagielski | 3 Nov 2011 12:52
Favicon

Re: prefetch proxy


On Nov 2, 2011, at 7:40 AM, Plüm, Rüdiger, VF-Group wrote:

> 
> I think a timeout should be handled like it is now as failing on
> a slow client is IMHO a desired action by the admin. If he wants to give
> the client more time he should configure a higher timeout.
> For other errors like 'Resource temporarily unavailable' we should log and possibly
> retry. But the question is: Why does this error happen in the first place?
> What is the root cause of it and what can be done to remove it?
> Is this error something that can just happen and we should gracefully live
> with by retrying? And if we are retrying how often do we do this to avoid an endless loop?
> 

I'm fine with with having a set number of retries with EAGAIN and
treating a timeout as an error. If we exhaust the retries, we
simply break out of the prefetch loop and continue on, and let
a continued Resource temporarily unavailable be handled as it
normally would.

Sound OK?
Permalink | Reply | 17 comments |
headers
Jim Jagielski | 3 Nov 2011 12:55
Favicon

Re: 2.0.65 and 2.3.15-dev


On Nov 3, 2011, at 2:43 AM, Stefan Fritsch wrote:

> On Fri, 28 Oct 2011, William A. Rowe Jr. wrote:
>> I think 2.4.0 GA at AC is destined to be a "dot zero" release (most
>> emphasis on the zero, as in null).  Not suggesting we don't do it,
>> simply that nobody, including yourself, will be at liberty to change
>> scoreboard datum, api interfaces, etc.
>> 
>> Are we anywhere near that level of commitment to trunk?  It would be
>> rather sad if 2.4.0 was ditched for 2.6.0.  Just look to the recent
>> perl releases for examples of such instability.
> 
> What are opinions about adding some fields to structs for extensions that seem like a good idea but have not
been written yet. This would then allow to add them during the lifetime of 2.4.x with only a minor MMN bump.

Not sure what you mean... we've added additional fields, always at the
end of the struct, during releases with just a minor MMN bump before.
Permalink | Reply | 10 comments |

Gmane