Robby Pedrica | 1 Feb 2006 10:32
Picon

Re: AW: proxy failover/load balance

Ruediger Pluem wrote:
On 01/31/2006 08:13 PM, Robby Pedrica wrote:
Hi Rudiger Thanks for the reply: a. any idea on when a fix will be available or whether there are patches available for these items? b. Yes, if you manually disable a member in balancer-manager, then it becomes enabled ( OK ) after a minute or 2 Regards Robby
Could you please give the attached patch a try and let me know the results. Regards Rüdiger

Hi Rudiger,

I've applied patches and recompiled. My results are as follows:

1. apache starts up with the member 'b' disabled now
2. if I shutdown the working member 'a' httpd, then manager shows the change only when you try and access the web site. Moves from state ok to err
3. If I bring the member 'a' back up again ( start httpd ) then manager shows the move from err to ok only after trying to access the site.

This is definitely an improvement even if failover isn't working because at least we can manually failover ...

Thanks Rudiger!!!

Regards, Robby


Plüm, Rüdiger, VIS | 1 Feb 2006 10:52

AW: AW: proxy failover/load balance


-----Ursprüngliche Nachricht-----
Von: Robby Pedrica 

> Hi Rudiger,
>
> I've applied patches and recompiled. My results are as follows:
> 
> 1. apache starts up with the member 'b' disabled now
> 2. if I shutdown the working member 'a' httpd, then manager shows the change only when you try and access the
web site. Moves from state ok to err
> 3. If I bring the member 'a' back up again ( start httpd ) then manager shows the move from err to ok only after
trying to access the site.

2. and 3. work as designed. httpd only changes the status for a worker to err if a request failed in the middle
or if it tried to assign a request
to this worker and that failed. Once the worker is in error state httpd will not try to assign a new request to
this worker for "retry" seconds
(see parameter explanation at http://httpd.apache.org/docs/2.2/mod/mod_proxy.html#proxypass).
After that grace period the worker gets new requests
assigned again and will switch back to ok once it processed the assigned request successfully.

> 
> This is definitely an improvement even if failover isn't working because at least we can manually
failover ...

Considering my explanation above, what do you mean by "failover isn't working"?

Regards

Rüdiger

Robby Pedrica | 1 Feb 2006 10:55
Picon

Re: AW: AW: proxy failover/load balance

Plüm wrote:
-----Ursprüngliche Nachricht----- Von: Robby Pedrica
Hi Rudiger, I've applied patches and recompiled. My results are as follows: 1. apache starts up with the member 'b' disabled now 2. if I shutdown the working member 'a' httpd, then manager shows the change only when you try and access the web site. Moves from state ok to err 3. If I bring the member 'a' back up again ( start httpd ) then manager shows the move from err to ok only after trying to access the site.
2. and 3. work as designed. httpd only changes the status for a worker to err if a request failed in the middle or if it tried to assign a request to this worker and that failed. Once the worker is in error state httpd will not try to assign a new request to this worker for "retry" seconds (see parameter explanation at http://httpd.apache.org/docs/2.2/mod/mod_proxy.html#proxypass). After that grace period the worker gets new requests assigned again and will switch back to ok once it processed the assigned request successfully.
This is definitely an improvement even if failover isn't working because at least we can manually failover ...
Considering my explanation above, what do you mean by "failover isn't working"? Regards Rüdiger
Thanks Rudiger,

I've set redirect and route values - not sure if these are correct:

<Proxy balancer://mycluster>
BalancerMember http://192.168.4.2:80 redirect=1
BalancerMember http://192.168.4.3:80 route=1 status=D
</Proxy>

I'm assuming that if member a. fails then requests will be redirected to member b. - this is not happening currently.

Regards

Robby
Plüm, Rüdiger, VIS | 1 Feb 2006 11:24

AW: AW: AW: proxy failover/load balance


-----Ursprüngliche Nachricht-----
Von: Robby Pedrica 

  
> Thanks Rudiger,
> 
> I've set redirect and route values - not sure if these are correct:
> 
> <Proxy balancer://mycluster>
> BalancerMember http://192.168.4.2:80 redirect=1
> BalancerMember http://192.168.4.3:80 route=1 status=D
> </Proxy>
> 
> I'm assuming that if member a. fails then requests will be redirected to member b. - this is not happening currently.

This only works with session stickyness. So

ProxyPass / balancer://mycluster stickysession=SESSION_COOKIE
<Proxy balancer://mycluster>
BalancerMember http://192.168.4.2:80 route=a redirect=b
BalancerMember http://192.168.4.3:80 route=b status=Disabled
</Proxy>

Furthermore your backend should deliver a session cookie

SESSION_COOKIE which has .a / .b added to the session id.
If you don't have an application on the backend that does this (from previous statements
I assume that you are using a httpd as backend) you can use
mod_headers on the backend servers to set these cookies.

Header add Set-Cookie "SESSION_COOKIE=anything.a;Path=/"

Regards

Rüdiger

Plüm, Rüdiger, VIS | 1 Feb 2006 12:03

AW: AW: AW: proxy failover/load balance


> -----Ursprüngliche Nachricht-----
> Von: Plüm, Rüdiger,
> 
> This only works with session stickyness. So
> 
> ProxyPass / balancer://mycluster stickysession=SESSION_COOKIE 
> <Proxy balancer://mycluster> BalancerMember 
> http://192.168.4.2:80 route=a redirect=b BalancerMember 
> http://192.168.4.3:80 route=b status=Disabled </Proxy>
> 
> Furthermore your backend should deliver a session cookie
> 
> SESSION_COOKIE which has .a / .b added to the session id.
> If you don't have an application on the backend that does 
> this (from previous statements I assume that you are using a 
> httpd as backend) you can use mod_headers on the backend 
> servers to set these cookies.
> 
> Header add Set-Cookie "SESSION_COOKIE=anything.a;Path=/"

I fear that the configuration above is not sufficient without the attached
patch.

Regards

Rüdiger

Attachment (disabled_worker.diff): application/octet-stream, 1215 bytes
Jim Jagielski | 1 Feb 2006 14:42
Favicon

Re: AW: AW: proxy failover/load balance

I think we're in agreement that the current "failover" does
not work as it should with HTTP, and is quite
cumbersome to get it to work. :)

I hope to later on this week work on code that has
a real "hot standby" status, and avoids the requirement
for sticky sessions. It won't replace what's in
there now (for AJP) but will make it easier
to implement failover for simple tasks.

Jim Jagielski | 1 Feb 2006 14:57
Favicon

Re: AW: proxy failover/load balance


Why the breaks? Certainly we still want to continue the
for loop even if we see a valid setting. For example,
to set a worker in DISABLED and STOPPED mode.

On Jan 31, 2006, at 4:32 PM, Ruediger Pluem wrote:
>
> Index: modules/proxy/mod_proxy.c
> ===================================================================
> --- modules/proxy/mod_proxy.c	(Revision 371134)
> +++ modules/proxy/mod_proxy.c	(Arbeitskopie)
>  <at>  <at>  -200,18 +200,21  <at>  <at> 
>                      worker->status |= PROXY_WORKER_DISABLED;
>                  else
>                      worker->status &= ~PROXY_WORKER_DISABLED;
> +                break;
>              }
>              else if (*v == 'S' || *v == 's') {
>                  if (mode)
>                      worker->status |= PROXY_WORKER_STOPPED;
>                  else
>                      worker->status &= ~PROXY_WORKER_STOPPED;
> +                break;
>              }
>              else if (*v == 'E' || *v == 'e') {
>                  if (mode)
>                      worker->status |= PROXY_WORKER_IN_ERROR;
>                  else
>                      worker->status &= ~PROXY_WORKER_IN_ERROR;
> +                break;
>              }
>              else {
>                  return "Unknow status parameter option";

Jim Jagielski | 1 Feb 2006 15:01
Favicon

Re: AW: proxy failover/load balance

I mean, of course, having status be simple flags +d+s
for example, rather than the whole word. The code looks
to be designed with that in mind.

On Feb 1, 2006, at 8:57 AM, Jim Jagielski wrote:

>
> Why the breaks? Certainly we still want to continue the
> for loop even if we see a valid setting. For example,
> to set a worker in DISABLED and STOPPED mode.
>
> On Jan 31, 2006, at 4:32 PM, Ruediger Pluem wrote:
>>
>> Index: modules/proxy/mod_proxy.c
>> ===================================================================
>> --- modules/proxy/mod_proxy.c	(Revision 371134)
>> +++ modules/proxy/mod_proxy.c	(Arbeitskopie)
>>  <at>  <at>  -200,18 +200,21  <at>  <at> 
>>                      worker->status |= PROXY_WORKER_DISABLED;
>>                  else
>>                      worker->status &= ~PROXY_WORKER_DISABLED;
>> +                break;
>>              }
>>              else if (*v == 'S' || *v == 's') {
>>                  if (mode)
>>                      worker->status |= PROXY_WORKER_STOPPED;
>>                  else
>>                      worker->status &= ~PROXY_WORKER_STOPPED;
>> +                break;
>>              }
>>              else if (*v == 'E' || *v == 'e') {
>>                  if (mode)
>>                      worker->status |= PROXY_WORKER_IN_ERROR;
>>                  else
>>                      worker->status &= ~PROXY_WORKER_IN_ERROR;
>> +                break;
>>              }
>>              else {
>>                  return "Unknow status parameter option";
>

Plüm, Rüdiger, VIS | 1 Feb 2006 15:02

AW: AW: AW: proxy failover/load balance


> -----Ursprüngliche Nachricht-----
> Von: Jim Jagielski 
> 
> I think we're in agreement that the current "failover" does
> not work as it should with HTTP, and is quite
> cumbersome to get it to work. :)

Apart from the fact that it currently does not even work without
patches :-).
So I am keen on feedback by Robby. I hope to find time to commit these
changes to the trunk tonight, so that it works at least in the cumbersome way :-).

> 
> I hope to later on this week work on code that has
> a real "hot standby" status, and avoids the requirement
> for sticky sessions. It won't replace what's in
> there now (for AJP) but will make it easier
> to implement failover for simple tasks.
> 

Sounds good.

Regards

Rüdiger

Plüm, Rüdiger, VIS | 1 Feb 2006 15:15

AW: AW: proxy failover/load balance

> -----Ursprüngliche Nachricht-----
> Von: Jim Jagielski 
> 
> Why the breaks? Certainly we still want to continue the
> for loop even if we see a valid setting. For example,
> to set a worker in DISABLED and STOPPED mode.

1. Currently there is no clear separation letter.
2. Setting status=disabled will result in "Unknow status parameter option"

But maybe I just understand the syntax of status wrong and status should
not be compiled of clear text words but rather of something cryptic like

[eEdDsS\+-]+ 

where 

e,E is error
d,D is disabled
s,S is stopped

But as it is not documented I don't know :-)

Regards

Rüdiger


Gmane