Small question on mod_jk load balancing methon "Next"

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Small question on mod_jk load balancing methon "Next"

Martin Knoblauch-4
Hi,

 just need some clarification on the mod_jk load blanacing method "Next".
The documentation states:

"If method is set to N[ext] the balancer will again use the number of
sessions to find the best worker. All remarks concerning the Session method
apply as well. The difference to the Session method is how the session
count is handled in the sliding time window. The Next method does not
divide by 2, instead it subtracts the current minimum number. This should
effectively result in a round-robin session balancing, thus the name Next.
Under high load, the two session balancing methods will result in a similar
distribution, but Next will be better if you need to distribute small
numbers of sessions. "

 What exactly is the "current minimum number"? How is the minimum taken?
From all workers in the balancer set, or only the ACTive ones? I know, I
should look it up in the code :-)

Thanks in advance
Martin
--
------------------------------------------------------
Martin Knoblauch
email: k n o b i AT knobisoft DOT de
www: http://www.knobisoft.de
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Small question on mod_jk load balancing methon "Next"

Rainer Jung-3
Hi Martin,

Am 04.08.2017 um 10:53 schrieb Martin Knoblauch:

> Hi,
>
>  just need some clarification on the mod_jk load blanacing method "Next".
> The documentation states:
>
> "If method is set to N[ext] the balancer will again use the number of
> sessions to find the best worker. All remarks concerning the Session method
> apply as well. The difference to the Session method is how the session
> count is handled in the sliding time window. The Next method does not
> divide by 2, instead it subtracts the current minimum number. This should
> effectively result in a round-robin session balancing, thus the name Next.
> Under high load, the two session balancing methods will result in a similar
> distribution, but Next will be better if you need to distribute small
> numbers of sessions. "
>
>  What exactly is the "current minimum number"? How is the minimum taken?
> From all workers in the balancer set, or only the ACTive ones? I know, I
> should look it up in the code :-)

I looked up the code I wrote 6 years ago.

First: when using the session base lb methods, mod_jk needs to estimate
session counts. No lb method of mod_jk contacts the backends to get real
data, instead mod_jk uses the request info it sees to estimate the
backend situation.

For session based methods, mod_jk counts requests, that do not include a
session id assuming that those are exactly the ones that create new
sessions. Of course:

a) a session id can be outdated, meaning mod_jk would not count the
request as session creating but in fact it would create a new one. One
can at least configure mod_jk to be aware of login pages which will
always create a new session (see
http://tomcat.apache.org/connectors-doc/reference/uriworkermap.html and
http://tomcat.apache.org/connectors-doc/reference/apache.html and there
look for "sticky_ignore").

b) a request without a session ID might not actually create a session,
depending on app details. There are additional config options to teach
mod_jk which URIs do not create sessions (see
http://tomcat.apache.org/connectors-doc/reference/uriworkermap.html and
http://tomcat.apache.org/connectors-doc/reference/apache.html and there
look for "stateless").

c) sessions time out in backends and users can log out. mod_jk does not
track that. One can remove the session cookie during the logout, so that
the "new" requests from that user will be counted by the mod_jk session
counter.

Because of these problems I typically recommend to stick to the default
lb method (request counting, not session counting). But sometimes apps
have resource usage dominated by sessions and then a "session" based lb
method can help, especially if you find a configuration which keeps the
effect of a)-c) above small.

Since all counting methods, not only session based ones, would count
stuff since the last restart of mod_jk, but the current backend load
situation depends much more on stuff that happened recently, we try to
get rid of past counts by reducing the counters regularly. By default
this happens once per minute and is done in a way that the counters are
divided by 2 once per minute. That way old counter increases contribute
less and less to the current counter value. For the session based method
this would mean we assume half of the counted sessions die after one
minute, 50% of the rest during the next minute etc. Note that the
counters are integers, so e.g. a counter value of 1 will after division
by 2 result in a new value 0. Most often that is no problem, because on
a loaded system numbers are big and rounding down doesn't change a lot.

The next request without session id will be send to the worker with the
smallest such "session" counter.

The "Next" message varies that procedure by not dividing by 2 every
minute, but instead subtracting the minimum value of the backend
counters. Assume after the first minute, your 4 backends have "session"
counters 2, 3, 3 and 2. Then the minimum is 2, so after the minute we
correct the values to 0, 1, 1 and 0. Then we add for the next minute new
sessions to that counter and again subtract the new minimum etc.

When would that be helpful? It was for an application with really huge
sessions but small session numbers. There was a risk that if for a
minute only 0 or one sessions were created on the backends, after
dividing by 2 all workers were again 0.

You can actually track the counters via the status worker, were they are
exposed as column "V" (load balancer value).

Regards,

Rainer

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Small question on mod_jk load balancing methon "Next"

Christopher Schultz-2
In reply to this post by Martin Knoblauch-4
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Martin,

On 8/4/17 4:53 AM, Martin Knoblauch wrote:

> Hi,
>
> just need some clarification on the mod_jk load blanacing method
> "Next". The documentation states:
>
> "If method is set to N[ext] the balancer will again use the number
> of sessions to find the best worker. All remarks concerning the
> Session method apply as well. The difference to the Session method
> is how the session count is handled in the sliding time window. The
> Next method does not divide by 2, instead it subtracts the current
> minimum number. This should effectively result in a round-robin
> session balancing, thus the name Next. Under high load, the two
> session balancing methods will result in a similar distribution,
> but Next will be better if you need to distribute small numbers of
> sessions. "
>
> What exactly is the "current minimum number"? How is the minimum
> taken?
>> From all workers in the balancer set, or only the ACTive ones?

I don't know for sure, but just in case this is a language issue, I
believe it means it will subtract the "lowest number of sessions
amongst all active workers". So if you had workers 1, 2, and 3 with
45, 50, and 55 sessions respectively, it will subtract the value 45
from all worker session counts to determine the priority for which
node will get the next session.

> I know, I should look it up in the code :-)

The mod_jk code is some of the most difficult to read (within the
Tomcat project) for a number of reasons.

Hope that helps,
- -chris
-----BEGIN PGP SIGNATURE-----
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQIcBAEBCAAGBQJZhOv+AAoJEBzwKT+lPKRYN2UQAKUFGQTSP4FKAGQym14SfBhe
ubtSXYyy8AWcEFax6VNJ61PKs6HDLzZWGrZU2A5T0q49/odTPXEOtFDls71quxvj
T++Nnq7fQrZPWT7x4agOzlnna6lz4FXZBGXoOBemQ86NDuU2tG2Q4J4V+2ZMErK8
kDPXhQ4UoVXTEUuIFfCSOXsPKg4SoIwJb0F/icrypFLtTIvd/De4DRZgNXbhf/CZ
9SRjDIniJ01xUcaUK+qyyD/KX91Gp80LVp5p1ma6fzXlVf2rQe7CLR7OLLUxjHFc
3ggWc+u6jipI9JiDNsuq/YkIUIIMGhKok4d/MaE9ydZEI9u3TZqtOzPwWiFZDY8j
n2ws38v7T8GHKhwRUsOEYlq/F9PaB410OgxMVLU0DO5Swdn1V5M2g6fMYTyMJTsa
wE7Ur4dh8R5Yrlhh9lYaRtt+9QpQBPpkYFPbY1F6PFOs0+zWwkEEQdjn3b2HPiVm
ot66WRSa5mXoEhJNIUeezxrQC4BugFte+bGcoTEkei7hPZp4msgV8zt2kMvSp0c7
hK9I0lZlnNeb3DosiFkWMABSwkLrSrLYiJb+AdC8gAhGcwFRsseRYh3gSP+yaEr2
0VV5iWYuaIhCkq1h/uVMnqId4PJjOAgB/NYD4AE2bXwbTR/qGpjMZDsWefnU6AM/
7TnD5f8GbgGpUun0C/pg
=Saq3
-----END PGP SIGNATURE-----

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Small question on mod_jk load balancing methon "Next"

Martin Knoblauch-4
On Fri, Aug 4, 2017 at 11:49 PM, Christopher Schultz <
[hidden email]> wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
>
> Martin,
>
> On 8/4/17 4:53 AM, Martin Knoblauch wrote:
> > Hi,
> >
> > just need some clarification on the mod_jk load blanacing method
> > "Next". The documentation states:
> >
> > "If method is set to N[ext] the balancer will again use the number
> > of sessions to find the best worker. All remarks concerning the
> > Session method apply as well. The difference to the Session method
> > is how the session count is handled in the sliding time window. The
> > Next method does not divide by 2, instead it subtracts the current
> > minimum number. This should effectively result in a round-robin
> > session balancing, thus the name Next. Under high load, the two
> > session balancing methods will result in a similar distribution,
> > but Next will be better if you need to distribute small numbers of
> > sessions. "
> >
> > What exactly is the "current minimum number"? How is the minimum
> > taken?
> >> From all workers in the balancer set, or only the ACTive ones?
>
> I don't know for sure, but just in case this is a language issue, I
> believe it means it will subtract the "lowest number of sessions
> amongst all active workers". So if you had workers 1, 2, and 3 with
> 45, 50, and 55 sessions respectively, it will subtract the value 45
> from all worker session counts to determine the priority for which
> node will get the next session.
>
>
 Now my question was: which workers are considered for the lowest number of
sessions. All of them, or only the ones in ACT state?

Thanks
Martin


>
>


--
------------------------------------------------------
Martin Knoblauch
email: k n o b i AT knobisoft DOT de
www: http://www.knobisoft.de
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Small question on mod_jk load balancing methon "Next"

Martin Knoblauch-4
In reply to this post by Rainer Jung-3
On Fri, Aug 4, 2017 at 11:47 PM, Rainer Jung <[hidden email]>
wrote:

> Hi Martin,
>
> Am 04.08.2017 um 10:53 schrieb Martin Knoblauch:
>
>> Hi,
>>
>>  just need some clarification on the mod_jk load blanacing method "Next".
>> The documentation states:
>>
>> "If method is set to N[ext] the balancer will again use the number of
>> sessions to find the best worker. All remarks concerning the Session
>> method
>> apply as well. The difference to the Session method is how the session
>> count is handled in the sliding time window. The Next method does not
>> divide by 2, instead it subtracts the current minimum number. This should
>> effectively result in a round-robin session balancing, thus the name Next.
>> Under high load, the two session balancing methods will result in a
>> similar
>> distribution, but Next will be better if you need to distribute small
>> numbers of sessions. "
>>
>>  What exactly is the "current minimum number"? How is the minimum taken?
>> From all workers in the balancer set, or only the ACTive ones? I know, I
>> should look it up in the code :-)
>>
>
> I looked up the code I wrote 6 years ago.
>
> First: when using the session base lb methods, mod_jk needs to estimate
> session counts. No lb method of mod_jk contacts the backends to get real
> data, instead mod_jk uses the request info it sees to estimate the backend
> situation.
>
> For session based methods, mod_jk counts requests, that do not include a
> session id assuming that those are exactly the ones that create new
> sessions. Of course:
>
> a) a session id can be outdated, meaning mod_jk would not count the
> request as session creating but in fact it would create a new one. One can
> at least configure mod_jk to be aware of login pages which will always
> create a new session (see http://tomcat.apache.org/conne
> ctors-doc/reference/uriworkermap.html and http://tomcat.apache.org/conne
> ctors-doc/reference/apache.html and there look for "sticky_ignore").
>
> b) a request without a session ID might not actually create a session,
> depending on app details. There are additional config options to teach
> mod_jk which URIs do not create sessions (see
> http://tomcat.apache.org/connectors-doc/reference/uriworkermap.html and
> http://tomcat.apache.org/connectors-doc/reference/apache.html and there
> look for "stateless").
>
> c) sessions time out in backends and users can log out. mod_jk does not
> track that. One can remove the session cookie during the logout, so that
> the "new" requests from that user will be counted by the mod_jk session
> counter.
>
> Because of these problems I typically recommend to stick to the default lb
> method (request counting, not session counting). But sometimes apps have
> resource usage dominated by sessions and then a "session" based lb method
> can help, especially if you find a configuration which keeps the effect of
> a)-c) above small.
>
> Since all counting methods, not only session based ones, would count stuff
> since the last restart of mod_jk, but the current backend load situation
> depends much more on stuff that happened recently, we try to get rid of
> past counts by reducing the counters regularly. By default this happens
> once per minute and is done in a way that the counters are divided by 2
> once per minute. That way old counter increases contribute less and less to
> the current counter value. For the session based method this would mean we
> assume half of the counted sessions die after one minute, 50% of the rest
> during the next minute etc. Note that the counters are integers, so e.g. a
> counter value of 1 will after division by 2 result in a new value 0. Most
> often that is no problem, because on a loaded system numbers are big and
> rounding down doesn't change a lot.
>
> The next request without session id will be send to the worker with the
> smallest such "session" counter.
>
> The "Next" message varies that procedure by not dividing by 2 every
> minute, but instead subtracting the minimum value of the backend counters.
> Assume after the first minute, your 4 backends have "session" counters 2,
> 3, 3 and 2. Then the minimum is 2, so after the minute we correct the
> values to 0, 1, 1 and 0. Then we add for the next minute new sessions to
> that counter and again subtract the new minimum etc.
>
> When would that be helpful? It was for an application with really huge
> sessions but small session numbers. There was a risk that if for a minute
> only 0 or one sessions were created on the backends, after dividing by 2
> all workers were again 0.
>
> You can actually track the counters via the status worker, were they are
> exposed as column "V" (load balancer value).
>
> Regards,
>
> Rainer
>
>
Hi Rainer,

 thanks a lot for the comprehensive write-up. Very useful. Just it does not
answer my question on which workers are considered when determining the
"minimum number" :-) Will all workers be considered, or only those in ACT
state?

 The reason why I am interested in the session based methods is exactely
that the application has a relatively small number of "sessions", which
tend to be heavy weight (memory, I/O and CPU). The request methods tend to
not lead to a good distribution of load here.

 What I really would be interested in is a balancer method that actually
looks at the worker backends themselves to determine the load and state
they are in. And I did not find a lot (any) pointers. I imagine that this
is a difficult issue that may lead to its own problems (bad latency, ...).

Cheers
Martin
--
------------------------------------------------------
Martin Knoblauch
email: k n o b i AT knobisoft DOT de
www: http://www.knobisoft.de
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Small question on mod_jk load balancing methon "Next"

Rainer Jung-3
Am 09.08.2017 um 11:50 schrieb Martin Knoblauch:

> On Fri, Aug 4, 2017 at 11:47 PM, Rainer Jung <[hidden email]>
> wrote:
>
>> Hi Martin,
>>
>> Am 04.08.2017 um 10:53 schrieb Martin Knoblauch:
>>
>>> Hi,
>>>
>>>  just need some clarification on the mod_jk load blanacing method "Next".
>>> The documentation states:
>>>
>>> "If method is set to N[ext] the balancer will again use the number of
>>> sessions to find the best worker. All remarks concerning the Session
>>> method
>>> apply as well. The difference to the Session method is how the session
>>> count is handled in the sliding time window. The Next method does not
>>> divide by 2, instead it subtracts the current minimum number. This should
>>> effectively result in a round-robin session balancing, thus the name Next.
>>> Under high load, the two session balancing methods will result in a
>>> similar
>>> distribution, but Next will be better if you need to distribute small
>>> numbers of sessions. "
>>>
>>>  What exactly is the "current minimum number"? How is the minimum taken?
>>> From all workers in the balancer set, or only the ACTive ones? I know, I
>>> should look it up in the code :-)
>>>
>>
>> I looked up the code I wrote 6 years ago.
>>
>> First: when using the session base lb methods, mod_jk needs to estimate
>> session counts. No lb method of mod_jk contacts the backends to get real
>> data, instead mod_jk uses the request info it sees to estimate the backend
>> situation.
>>
>> For session based methods, mod_jk counts requests, that do not include a
>> session id assuming that those are exactly the ones that create new
>> sessions. Of course:
>>
>> a) a session id can be outdated, meaning mod_jk would not count the
>> request as session creating but in fact it would create a new one. One can
>> at least configure mod_jk to be aware of login pages which will always
>> create a new session (see http://tomcat.apache.org/conne
>> ctors-doc/reference/uriworkermap.html and http://tomcat.apache.org/conne
>> ctors-doc/reference/apache.html and there look for "sticky_ignore").
>>
>> b) a request without a session ID might not actually create a session,
>> depending on app details. There are additional config options to teach
>> mod_jk which URIs do not create sessions (see
>> http://tomcat.apache.org/connectors-doc/reference/uriworkermap.html and
>> http://tomcat.apache.org/connectors-doc/reference/apache.html and there
>> look for "stateless").
>>
>> c) sessions time out in backends and users can log out. mod_jk does not
>> track that. One can remove the session cookie during the logout, so that
>> the "new" requests from that user will be counted by the mod_jk session
>> counter.
>>
>> Because of these problems I typically recommend to stick to the default lb
>> method (request counting, not session counting). But sometimes apps have
>> resource usage dominated by sessions and then a "session" based lb method
>> can help, especially if you find a configuration which keeps the effect of
>> a)-c) above small.
>>
>> Since all counting methods, not only session based ones, would count stuff
>> since the last restart of mod_jk, but the current backend load situation
>> depends much more on stuff that happened recently, we try to get rid of
>> past counts by reducing the counters regularly. By default this happens
>> once per minute and is done in a way that the counters are divided by 2
>> once per minute. That way old counter increases contribute less and less to
>> the current counter value. For the session based method this would mean we
>> assume half of the counted sessions die after one minute, 50% of the rest
>> during the next minute etc. Note that the counters are integers, so e.g. a
>> counter value of 1 will after division by 2 result in a new value 0. Most
>> often that is no problem, because on a loaded system numbers are big and
>> rounding down doesn't change a lot.
>>
>> The next request without session id will be send to the worker with the
>> smallest such "session" counter.
>>
>> The "Next" message varies that procedure by not dividing by 2 every
>> minute, but instead subtracting the minimum value of the backend counters.
>> Assume after the first minute, your 4 backends have "session" counters 2,
>> 3, 3 and 2. Then the minimum is 2, so after the minute we correct the
>> values to 0, 1, 1 and 0. Then we add for the next minute new sessions to
>> that counter and again subtract the new minimum etc.
>>
>> When would that be helpful? It was for an application with really huge
>> sessions but small session numbers. There was a risk that if for a minute
>> only 0 or one sessions were created on the backends, after dividing by 2
>> all workers were again 0.
>>
>> You can actually track the counters via the status worker, were they are
>> exposed as column "V" (load balancer value).
>>
>> Regards,
>>
>> Rainer
>>
>>
> Hi Rainer,
>
>  thanks a lot for the comprehensive write-up. Very useful. Just it does not
> answer my question on which workers are considered when determining the
> "minimum number" :-) Will all workers be considered, or only those in ACT
> state?

Ah, I didn't get that question, because you didn't mention the worker
states.

The current minimum will be taken over all workers which are in
activation state active and are also not in error.

The subtraction of the minimum from the lb value will be done for every
worker. Workers who are not active or in error and whose value is
already smaller than the minimum taken over the possibly smaller set of
workers will have their value set to 0 instead of becoming negative.

>  The reason why I am interested in the session based methods is exactely
> that the application has a relatively small number of "sessions", which
> tend to be heavy weight (memory, I/O and CPU). The request methods tend to
> not lead to a good distribution of load here.

Understood.

>  What I really would be interested in is a balancer method that actually
> looks at the worker backends themselves to determine the load and state
> they are in. And I did not find a lot (any) pointers. I imagine that this
> is a difficult issue that may lead to its own problems (bad latency, ...).

There would be two approaches for this:

- polling the load situation in intervals. For this one would need to
define (configure) the poll URL and the format of the expected response,
the format could be fixed, like an integer number. One would also have
to think about how to extrapolate the numbers between the poll
intervals, and code concurrent HTTP(S) requests with timeouts etc.

- piggy-back the info via responses, e.g. in a custom HTTP response
header (configurable header name), that the module would strip, with a
fixed format, e.g. an integer value. This approach would mean generating
the load value must be cheap in the back end. It would be much simpler
to implement, but whenever a node does not get requests for some time,
we again have no idea about the current load situation, so some
extrapolation is still needed.

Note that "extrapolation" would also be needed because if we have a farm
of reverse proxies in front of our backends, it would be nice to get a
consolidated view of load.

There is currently no code to support any of the two attempts. A naive
implementation of the second attempt wouldn't be hard, but the problem
of extrapolating would not be solved in the naive impl. With such an
impl you would then add a custom response header, e.g. X-SESSION-COUNT
and mod_jk would update it lb value whenever (and only if) it sees such
a response header. Would that actually help?

Regards,

Rainer



---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Loading...