Tomcat 8/NIO performance discrepancies

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
23 messages Options
12
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Tomcat 8/NIO performance discrepancies

Piyush Kumar Nayak
We have been testing different connectors of tomcat for performance.
We have tested the following server versions:

-          Tomcat 7.077

-          Tomcat 8.0.44

-          Tomcat 8.0.32

-          Tomcat 8.5.14

with the following HTTP connectors:

-          BIO

-          BIO with Executor

-          NIO

-          NIO with Executor

Our observation is that NIO is up to 15% faster than BIO. But BIO with Executor is up to 65% faster than BIO. Using a shared executor thread pool with NIO does not improve the performance. We have tried some variations of connector attribute with Tomcat 8 (maxThreads, acceptorThreadCount), but nothing brings us close to the performance that we can get with BIO with Executor.

We have upgraded to Tomcat 8.5, and can't seem to get it to perform as well as Tomcat 7 using BIO with Executor, which is what we were using earlier.

I would appreciate any help with the following:

-          Why is the executor not boosting the performance for NIO, as it is for BIO?

-          BIO is not available as an option with Tomcat 8.5. Why has it been removed? Is there any way to get it back ?

I'd be happy to share any other details from my testing for any clarification.

Regards,
Piyush.





Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Tomcat 8/NIO performance discrepancies

Mark Thomas-2
On 06/06/17 07:30, Piyush Kumar Nayak wrote:

> We have been testing different connectors of tomcat for performance.
> We have tested the following server versions:
>
> -          Tomcat 7.077
>
> -          Tomcat 8.0.44
>
> -          Tomcat 8.0.32
>
> -          Tomcat 8.5.14
>
> with the following HTTP connectors:
>
> -          BIO
>
> -          BIO with Executor
>
> -          NIO
>
> -          NIO with Executor
>
> Our observation is that NIO is up to 15% faster than BIO. But BIO with Executor is up to 65% faster than BIO. Using a shared executor thread pool with NIO does not improve the performance. We have tried some variations of connector attribute with Tomcat 8 (maxThreads, acceptorThreadCount), but nothing brings us close to the performance that we can get with BIO with Executor.
>
> We have upgraded to Tomcat 8.5, and can't seem to get it to perform as well as Tomcat 7 using BIO with Executor, which is what we were using earlier.
>
> I would appreciate any help with the following:
>
> -          Why is the executor not boosting the performance for NIO, as it is for BIO?

Without knowing the details of your tests - no idea. The numbers are a
long way from what I'd expect.

>
> -          BIO is not available as an option with Tomcat 8.5. Why has it been removed? Is there any way to get it back ?

Because it can't support the non-blocking requirements of the WebSocket
API or the Servlet API and while you can fake non-blocking support, the
result is liable to enter a deadlock.

> I'd be happy to share any other details from my testing for any clarification.

You'll need to explain your testing methodology - in detail - before
anyone can provide any useful input.

Mark

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

RE: Tomcat 8/NIO performance discrepancies

Piyush Kumar Nayak
Thanks, Mark.

Here are some additional details.

I am using Apache JMeter to inject load. I am using a simple hello-world JSP.

The test JMX simulates 100 concurrent user threads with a ramp-up of 5 secs and uses an HTTP Cookie Manager, that reuses cookies for each user thread.

I have disabled access log in tomcat. All the other server.xml settings are the default.


The BIO with Executor configuration we are using is:
<Executor name="tomcatThreadPool" namePrefix="catalina-exec-"
        maxThreads="150" minSpareThreads="4"/>

<Connector port="8080" protocol="org.apache.coyote.http11.Http11Protocol"
               connectionTimeout="20000"
               redirectPort="8443" />



It's the same for NIO. Just the protocol changes.



The client(jMeter) and server (Tomcat) are on 2 separate physical machines with the same configuration:

RHEL 7.1 x64

16 core CPU;

32 GB RAM;

1Gbps NIC



------------------------------------------------------------- JSP test page --------------------------------------------

<%@ page language="java" contentType="text/html; charset=ISO-8859-1"

    pageEncoding="ISO-8859-1"%>

<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">

<html>

<head>

<meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">

<title>Hello World - JSP tomcat test</title>

</head>

<body>

    <%= "Hello World! - JSP on Tomcat 8.15" %>

</body>

</html>

-------------------------------------------------------------  JSP test page -------------------------------------------------------------



Test result:
helloWorld.JSP



BIO

NIO

BIO w/ Exec

NIO w/ Exec

Throughput

ART

Throughput

ART

Throughput

ART

Throughput

ART

Tomcat 7.077

38600

2

42664

2

57104

1

43660

2

Tomcat 8.0.44

38585

2

44752

2

63000

1

44341

2

Tomcat 8.0.32









41211







Tomcat 8.5.14

N/A

45600

1.67

N/A

46000

1.67






Regards,

Piyush.



-----Original Message-----

From: Mark Thomas [mailto:[hidden email]]

Sent: Tuesday, June 06, 2017 12:50 PM

To: Tomcat Users List <[hidden email]>

Subject: Re: Tomcat 8/NIO performance discrepancies



On 06/06/17 07:30, Piyush Kumar Nayak wrote:

> We have been testing different connectors of tomcat for performance.

> We have tested the following server versions:

>

> -          Tomcat 7.077

>

> -          Tomcat 8.0.44

>

> -          Tomcat 8.0.32

>

> -          Tomcat 8.5.14

>

> with the following HTTP connectors:

>

> -          BIO

>

> -          BIO with Executor

>

> -          NIO

>

> -          NIO with Executor

>

> Our observation is that NIO is up to 15% faster than BIO. But BIO with Executor is up to 65% faster than BIO. Using a shared executor thread pool with NIO does not improve the performance. We have tried some variations of connector attribute with Tomcat 8 (maxThreads, acceptorThreadCount), but nothing brings us close to the performance that we can get with BIO with Executor.

>

> We have upgraded to Tomcat 8.5, and can't seem to get it to perform as well as Tomcat 7 using BIO with Executor, which is what we were using earlier.

>

> I would appreciate any help with the following:

>

> -          Why is the executor not boosting the performance for NIO, as it is for BIO?



Without knowing the details of your tests - no idea. The numbers are a long way from what I'd expect.



>

> -          BIO is not available as an option with Tomcat 8.5. Why has it been removed? Is there any way to get it back ?



Because it can't support the non-blocking requirements of the WebSocket API or the Servlet API and while you can fake non-blocking support, the result is liable to enter a deadlock.



> I'd be happy to share any other details from my testing for any clarification.



You'll need to explain your testing methodology - in detail - before anyone can provide any useful input.



Mark



---------------------------------------------------------------------

To unsubscribe, e-mail: [hidden email]

For additional commands, e-mail: [hidden email]


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Tomcat 8/NIO performance discrepancies

violetagg
Hi,

2017-06-06 11:10 GMT+03:00 Piyush Kumar Nayak <[hidden email]>:
>
> Thanks, Mark.
>
> Here are some additional details.
>
> I am using Apache JMeter to inject load. I am using a simple hello-world
JSP.
>
> The test JMX simulates 100 concurrent user threads with a ramp-up of 5
secs and uses an HTTP Cookie Manager, that reuses cookies for each user
thread.

Do you specify "Loop Count" for the number of iterations or you specify
time duration?

Thanks,
Violeta

>
> I have disabled access log in tomcat. All the other server.xml settings
are the default.

>
>
> The BIO with Executor configuration we are using is:
> <Executor name="tomcatThreadPool" namePrefix="catalina-exec-"
>         maxThreads="150" minSpareThreads="4"/>
>
> <Connector port="8080" protocol="org.apache.coyote.http11.Http11Protocol"
>                connectionTimeout="20000"
>                redirectPort="8443" />
>
>
>
> It's the same for NIO. Just the protocol changes.
>
>
>
> The client(jMeter) and server (Tomcat) are on 2 separate physical
machines with the same configuration:

>
> RHEL 7.1 x64
>
> 16 core CPU;
>
> 32 GB RAM;
>
> 1Gbps NIC
>
>
>
> ------------------------------------------------------------- JSP test
page --------------------------------------------
>
> <%@ page language="java" contentType="text/html; charset=ISO-8859-1"
>
>     pageEncoding="ISO-8859-1"%>
>
> <!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "
http://www.w3.org/TR/html4/loose.dtd">

>
> <html>
>
> <head>
>
> <meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">
>
> <title>Hello World - JSP tomcat test</title>
>
> </head>
>
> <body>
>
>     <%= "Hello World! - JSP on Tomcat 8.15" %>
>
> </body>
>
> </html>
>
> -------------------------------------------------------------  JSP test
page -------------------------------------------------------------

>
>
>
> Test result:
> helloWorld.JSP
>
>
>
> BIO
>
> NIO
>
> BIO w/ Exec
>
> NIO w/ Exec
>
> Throughput
>
> ART
>
> Throughput
>
> ART
>
> Throughput
>
> ART
>
> Throughput
>
> ART
>
> Tomcat 7.077
>
> 38600
>
> 2
>
> 42664
>
> 2
>
> 57104
>
> 1
>
> 43660
>
> 2
>
> Tomcat 8.0.44
>
> 38585
>
> 2
>
> 44752
>
> 2
>
> 63000
>
> 1
>
> 44341
>
> 2
>
> Tomcat 8.0.32
>
>
>
>
>
>
>
>
>
> 41211
>
>
>
>
>
>
>
> Tomcat 8.5.14
>
> N/A
>
> 45600
>
> 1.67
>
> N/A
>
> 46000
>
> 1.67
>
>
>
>
>
>
> Regards,
>
> Piyush.
>
>
>
> -----Original Message-----
>
> From: Mark Thomas [mailto:[hidden email]]
>
> Sent: Tuesday, June 06, 2017 12:50 PM
>
> To: Tomcat Users List <[hidden email]>
>
> Subject: Re: Tomcat 8/NIO performance discrepancies
>
>
>
> On 06/06/17 07:30, Piyush Kumar Nayak wrote:
>
> > We have been testing different connectors of tomcat for performance.
>
> > We have tested the following server versions:
>
> >
>
> > -          Tomcat 7.077
>
> >
>
> > -          Tomcat 8.0.44
>
> >
>
> > -          Tomcat 8.0.32
>
> >
>
> > -          Tomcat 8.5.14
>
> >
>
> > with the following HTTP connectors:
>
> >
>
> > -          BIO
>
> >
>
> > -          BIO with Executor
>
> >
>
> > -          NIO
>
> >
>
> > -          NIO with Executor
>
> >
>
> > Our observation is that NIO is up to 15% faster than BIO. But BIO with
Executor is up to 65% faster than BIO. Using a shared executor thread pool
with NIO does not improve the performance. We have tried some variations of
connector attribute with Tomcat 8 (maxThreads, acceptorThreadCount), but
nothing brings us close to the performance that we can get with BIO with
Executor.
>
> >
>
> > We have upgraded to Tomcat 8.5, and can't seem to get it to perform as
well as Tomcat 7 using BIO with Executor, which is what we were using
earlier.
>
> >
>
> > I would appreciate any help with the following:
>
> >
>
> > -          Why is the executor not boosting the performance for NIO, as
it is for BIO?
>
>
>
> Without knowing the details of your tests - no idea. The numbers are a
long way from what I'd expect.
>
>
>
> >
>
> > -          BIO is not available as an option with Tomcat 8.5. Why has
it been removed? Is there any way to get it back ?
>
>
>
> Because it can't support the non-blocking requirements of the WebSocket
API or the Servlet API and while you can fake non-blocking support, the
result is liable to enter a deadlock.
>
>
>
> > I'd be happy to share any other details from my testing for any
clarification.
>
>
>
> You'll need to explain your testing methodology - in detail - before
anyone can provide any useful input.

>
>
>
> Mark
>
>
>
> ---------------------------------------------------------------------
>
> To unsubscribe, e-mail: [hidden email]
>
> For additional commands, e-mail: [hidden email]
>
>
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

RE: Tomcat 8/NIO performance discrepancies

Piyush Kumar Nayak
Violeta,

The data you see in an average of 3 samples of 180 second duration each.

Missed out on mentioning it before, but the thoughput is in no. of requests per second, and ART is in milliseconds.



Thanks

Piyush.



-----Original Message-----
From: Violeta Georgieva [mailto:[hidden email]]
Sent: Tuesday, June 06, 2017 7:09 PM
To: Tomcat Users List <[hidden email]>
Subject: Re: Tomcat 8/NIO performance discrepancies



Hi,



2017-06-06 11:10 GMT+03:00 Piyush Kumar Nayak <[hidden email]<mailto:[hidden email]>>:

>

> Thanks, Mark.

>

> Here are some additional details.

>

> I am using Apache JMeter to inject load. I am using a simple

> hello-world

JSP.

>

> The test JMX simulates 100 concurrent user threads with a ramp-up of 5

secs and uses an HTTP Cookie Manager, that reuses cookies for each user thread.



Do you specify "Loop Count" for the number of iterations or you specify time duration?



Thanks,

Violeta



>

> I have disabled access log in tomcat. All the other server.xml

> settings

are the default.

>

>

> The BIO with Executor configuration we are using is:

> <Executor name="tomcatThreadPool" namePrefix="catalina-exec-"

>         maxThreads="150" minSpareThreads="4"/>

>

> <Connector port="8080" protocol="org.apache.coyote.http11.Http11Protocol"

>                connectionTimeout="20000"

>                redirectPort="8443" />

>

>

>

> It's the same for NIO. Just the protocol changes.

>

>

>

> The client(jMeter) and server (Tomcat) are on 2 separate physical

machines with the same configuration:

>

> RHEL 7.1 x64

>

> 16 core CPU;

>

> 32 GB RAM;

>

> 1Gbps NIC

>

>

>

> ------------------------------------------------------------- JSP test

page --------------------------------------------

>

> <%@ page language="java" contentType="text/html; charset=ISO-8859-1"

>

>     pageEncoding="ISO-8859-1"%>

>

> <!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "

https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.w3.org%2FTR%2Fhtml4%2Floose.dtd&data=02%7C01%7C%7Ce541b882239d42e880f508d4ace1739f%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C636323531683671518&sdata=AwpErTK9ClLCwDZ23QnOqj195rlITTyAeG4JX0BFv0o%3D&reserved=0">

>

> <html>

>

> <head>

>

> <meta http-equiv="Content-Type" content="text/html;

> charset=ISO-8859-1">

>

> <title>Hello World - JSP tomcat test</title>

>

> </head>

>

> <body>

>

>     <%= "Hello World! - JSP on Tomcat 8.15" %>

>

> </body>

>

> </html>

>

> -------------------------------------------------------------  JSP

> test

page -------------------------------------------------------------

>

>

>

> Test result:

> helloWorld.JSP

>

>

>

> BIO

>

> NIO

>

> BIO w/ Exec

>

> NIO w/ Exec

>

> Throughput

>

> ART

>

> Throughput

>

> ART

>

> Throughput

>

> ART

>

> Throughput

>

> ART

>

> Tomcat 7.077

>

> 38600

>

> 2

>

> 42664

>

> 2

>

> 57104

>

> 1

>

> 43660

>

> 2

>

> Tomcat 8.0.44

>

> 38585

>

> 2

>

> 44752

>

> 2

>

> 63000

>

> 1

>

> 44341

>

> 2

>

> Tomcat 8.0.32

>

>

>

>

>

>

>

>

>

> 41211

>

>

>

>

>

>

>

> Tomcat 8.5.14

>

> N/A

>

> 45600

>

> 1.67

>

> N/A

>

> 46000

>

> 1.67

>

>

>

>

>

>

> Regards,

>

> Piyush.

>

>

>

> -----Original Message-----

>

> From: Mark Thomas [mailto:[hidden email]]

>

> Sent: Tuesday, June 06, 2017 12:50 PM

>

> To: Tomcat Users List <[hidden email]<mailto:[hidden email]>>

>

> Subject: Re: Tomcat 8/NIO performance discrepancies

>

>

>

> On 06/06/17 07:30, Piyush Kumar Nayak wrote:

>

> > We have been testing different connectors of tomcat for performance.

>

> > We have tested the following server versions:

>

> >

>

> > -          Tomcat 7.077

>

> >

>

> > -          Tomcat 8.0.44

>

> >

>

> > -          Tomcat 8.0.32

>

> >

>

> > -          Tomcat 8.5.14

>

> >

>

> > with the following HTTP connectors:

>

> >

>

> > -          BIO

>

> >

>

> > -          BIO with Executor

>

> >

>

> > -          NIO

>

> >

>

> > -          NIO with Executor

>

> >

>

> > Our observation is that NIO is up to 15% faster than BIO. But BIO

> > with

Executor is up to 65% faster than BIO. Using a shared executor thread pool with NIO does not improve the performance. We have tried some variations of connector attribute with Tomcat 8 (maxThreads, acceptorThreadCount), but nothing brings us close to the performance that we can get with BIO with Executor.

>

> >

>

> > We have upgraded to Tomcat 8.5, and can't seem to get it to perform

> > as

well as Tomcat 7 using BIO with Executor, which is what we were using earlier.

>

> >

>

> > I would appreciate any help with the following:

>

> >

>

> > -          Why is the executor not boosting the performance for NIO, as

it is for BIO?

>

>

>

> Without knowing the details of your tests - no idea. The numbers are a

long way from what I'd expect.

>

>

>

> >

>

> > -          BIO is not available as an option with Tomcat 8.5. Why has

it been removed? Is there any way to get it back ?

>

>

>

> Because it can't support the non-blocking requirements of the

> WebSocket

API or the Servlet API and while you can fake non-blocking support, the result is liable to enter a deadlock.

>

>

>

> > I'd be happy to share any other details from my testing for any

clarification.

>

>

>

> You'll need to explain your testing methodology - in detail - before

anyone can provide any useful input.

>

>

>

> Mark

>

>

>

> ---------------------------------------------------------------------

>

> To unsubscribe, e-mail: [hidden email]<mailto:[hidden email]>

>

> For additional commands, e-mail: [hidden email]<mailto:[hidden email]>

>

>
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

RE: Tomcat 8/NIO performance discrepancies

Piyush Kumar Nayak
Just realized that the comparison matrix formatting was lost earlier, so sharing it again.
                                                               
---------------------------------------------------------------------------------------------------------------------------------------------
helloWorld.JSP BIO    NIO   BIO w/ Exec NIO w/ Exec
                Throughput ART Throughput  ART Throughput ART Throughput ART
---------------------------------------------------------------------------------------------------------------------------------------------
Tomcat 7.077 38600        2 42664   2 57104       1 43660        2
Tomcat 8.0.44 38585        2 44752   2 63000       1 44341        2
Tomcat 8.0.32     -- 41211   - --
Tomcat 8.5.14   N/A 45600         1.67 N/A 46000        1.67
---------------------------------------------------------------------------------------------------------------------------------------------
Throughput: requests per sec.
ART: msecs

Also, forgot to add earlier, the jMeter client has keep-alive header enabled, and the server is left to use the default keepalive values.


-----Original Message-----
From: Piyush Kumar Nayak [mailto:[hidden email]]
Sent: Tuesday, June 06, 2017 7:59 PM
To: Tomcat Users List <[hidden email]>
Subject: RE: Tomcat 8/NIO performance discrepancies

Violeta,

The data you see in an average of 3 samples of 180 second duration each.

Missed out on mentioning it before, but the thoughput is in no. of requests per second, and ART is in milliseconds.



Thanks

Piyush.



-----Original Message-----
From: Violeta Georgieva [mailto:[hidden email]]
Sent: Tuesday, June 06, 2017 7:09 PM
To: Tomcat Users List <[hidden email]>
Subject: Re: Tomcat 8/NIO performance discrepancies



Hi,



2017-06-06 11:10 GMT+03:00 Piyush Kumar Nayak <[hidden email]<mailto:[hidden email]>>:

>

> Thanks, Mark.

>

> Here are some additional details.

>

> I am using Apache JMeter to inject load. I am using a simple

> hello-world

JSP.

>

> The test JMX simulates 100 concurrent user threads with a ramp-up of 5

secs and uses an HTTP Cookie Manager, that reuses cookies for each user thread.



Do you specify "Loop Count" for the number of iterations or you specify time duration?



Thanks,

Violeta



>

> I have disabled access log in tomcat. All the other server.xml

> settings

are the default.

>

>

> The BIO with Executor configuration we are using is:

> <Executor name="tomcatThreadPool" namePrefix="catalina-exec-"

>         maxThreads="150" minSpareThreads="4"/>

>

> <Connector port="8080" protocol="org.apache.coyote.http11.Http11Protocol"

>                connectionTimeout="20000"

>                redirectPort="8443" />

>

>

>

> It's the same for NIO. Just the protocol changes.

>

>

>

> The client(jMeter) and server (Tomcat) are on 2 separate physical

machines with the same configuration:

>

> RHEL 7.1 x64

>

> 16 core CPU;

>

> 32 GB RAM;

>

> 1Gbps NIC

>

>

>

> ------------------------------------------------------------- JSP test

page --------------------------------------------

>

> <%@ page language="java" contentType="text/html; charset=ISO-8859-1"

>

>     pageEncoding="ISO-8859-1"%>

>

> <!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "

https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.w3.org%2FTR%2Fhtml4%2Floose.dtd&data=02%7C01%7C%7Ce541b882239d42e880f508d4ace1739f%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C636323531683671518&sdata=AwpErTK9ClLCwDZ23QnOqj195rlITTyAeG4JX0BFv0o%3D&reserved=0">

>

> <html>

>

> <head>

>

> <meta http-equiv="Content-Type" content="text/html;

> charset=ISO-8859-1">

>

> <title>Hello World - JSP tomcat test</title>

>

> </head>

>

> <body>

>

>     <%= "Hello World! - JSP on Tomcat 8.15" %>

>

> </body>

>

> </html>

>

> -------------------------------------------------------------  JSP

> test

page -------------------------------------------------------------

>

>

>

> Test result:

> helloWorld.JSP

>

>

>

> BIO

>

> NIO

>

> BIO w/ Exec

>

> NIO w/ Exec

>

> Throughput

>

> ART

>

> Throughput

>

> ART

>

> Throughput

>

> ART

>

> Throughput

>

> ART

>

> Tomcat 7.077

>

> 38600

>

> 2

>

> 42664

>

> 2

>

> 57104

>

> 1

>

> 43660

>

> 2

>

> Tomcat 8.0.44

>

> 38585

>

> 2

>

> 44752

>

> 2

>

> 63000

>

> 1

>

> 44341

>

> 2

>

> Tomcat 8.0.32

>

>

>

>

>

>

>

>

>

> 41211

>

>

>

>

>

>

>

> Tomcat 8.5.14

>

> N/A

>

> 45600

>

> 1.67

>

> N/A

>

> 46000

>

> 1.67

>

>

>

>

>

>

> Regards,

>

> Piyush.

>

>

>

> -----Original Message-----

>

> From: Mark Thomas [mailto:[hidden email]]

>

> Sent: Tuesday, June 06, 2017 12:50 PM

>

> To: Tomcat Users List <[hidden email]<mailto:[hidden email]>>

>

> Subject: Re: Tomcat 8/NIO performance discrepancies

>

>

>

> On 06/06/17 07:30, Piyush Kumar Nayak wrote:

>

> > We have been testing different connectors of tomcat for performance.

>

> > We have tested the following server versions:

>

> >

>

> > -          Tomcat 7.077

>

> >

>

> > -          Tomcat 8.0.44

>

> >

>

> > -          Tomcat 8.0.32

>

> >

>

> > -          Tomcat 8.5.14

>

> >

>

> > with the following HTTP connectors:

>

> >

>

> > -          BIO

>

> >

>

> > -          BIO with Executor

>

> >

>

> > -          NIO

>

> >

>

> > -          NIO with Executor

>

> >

>

> > Our observation is that NIO is up to 15% faster than BIO. But BIO

> > with

Executor is up to 65% faster than BIO. Using a shared executor thread pool with NIO does not improve the performance. We have tried some variations of connector attribute with Tomcat 8 (maxThreads, acceptorThreadCount), but nothing brings us close to the performance that we can get with BIO with Executor.

>

> >

>

> > We have upgraded to Tomcat 8.5, and can't seem to get it to perform

> > as

well as Tomcat 7 using BIO with Executor, which is what we were using earlier.

>

> >

>

> > I would appreciate any help with the following:

>

> >

>

> > -          Why is the executor not boosting the performance for NIO, as

it is for BIO?

>

>

>

> Without knowing the details of your tests - no idea. The numbers are a

long way from what I'd expect.

>

>

>

> >

>

> > -          BIO is not available as an option with Tomcat 8.5. Why has

it been removed? Is there any way to get it back ?

>

>

>

> Because it can't support the non-blocking requirements of the

> WebSocket

API or the Servlet API and while you can fake non-blocking support, the result is liable to enter a deadlock.

>

>

>

> > I'd be happy to share any other details from my testing for any

clarification.

>

>

>

> You'll need to explain your testing methodology - in detail - before

anyone can provide any useful input.

>

>

>

> Mark

>

>

>

> ---------------------------------------------------------------------

>

> To unsubscribe, e-mail: [hidden email]<mailto:[hidden email]>

>

> For additional commands, e-mail: [hidden email]<mailto:[hidden email]>

>

>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Tomcat 8/NIO performance discrepancies

Mark Thomas-2
In reply to this post by Piyush Kumar Nayak
On 06/06/17 09:10, Piyush Kumar Nayak wrote:
> Thanks, Mark.
>
> Here are some additional details.

Thanks. These help a lot to make sure we are testing (at least roughly)
the same thing.

> I am using Apache JMeter to inject load. I am using a simple hello-world JSP.
>
> The test JMX simulates 100 concurrent user threads with a ramp-up of 5 secs and uses an HTTP Cookie Manager, that reuses cookies for each user thread.

I've disabled session in the JSP for my test and used a simpler JSP.
That should amplify any difference in connector performance.

I'm also testing on localhost.

> I have disabled access log in tomcat. All the other server.xml settings are the default.
>
>
> The BIO with Executor configuration we are using is:
> <Executor name="tomcatThreadPool" namePrefix="catalina-exec-"
>         maxThreads="150" minSpareThreads="4"/>
>
> <Connector port="8080" protocol="org.apache.coyote.http11.Http11Protocol"
>                connectionTimeout="20000"
>                redirectPort="8443" />

FYI for anyone following along. That is a non-executor config. The
connector needs to specify the executor it wants to used.

I do see odd behaviour with BIO + executor. It locks up very easily. It
appears that the client isn't closing the connections. I don't see this
problem with BIO but that may be related to how BIO without an executor
handles large numbers of connections.

If I lower the client thread count, the odd behaviour stops and I do see
notably higher throughput with BIO + executor. That is unexpected.

My initial thoughts are maybe a contention issue related to the thread
pool. I'm continuing to investigate. I plan to look at performance first
and then the locking up.

Mark

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Tomcat 8/NIO performance discrepancies

Christopher Schultz-2
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Mark,

On 6/7/17 10:57 AM, Mark Thomas wrote:
> I do see odd behaviour with BIO + executor. It locks up very
> easily. It appears that the client isn't closing the connections. I
> don't see this problem with BIO but that may be related to how BIO
> without an executor handles large numbers of connections.

Was one of those BIOs supposed to be an "NIO"?

- -chris
-----BEGIN PGP SIGNATURE-----
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQIcBAEBCAAGBQJZOCq+AAoJEBzwKT+lPKRYguwQAIgksfcD9loUKPQoR5LZsjgg
0VNrWVSmn5o4enqjIA5RLOt55XzflK7H4drBN2IB/QfJ/5Z40d9HYAHh9x5xdFst
kyY+3kFveS0GG8lxcutsl2sr2zetV9D7/vYsjfBcvgTB+5SPcwlL9oQjk0SKSO1I
lA6pQCTdEvNfXl+gYuY9SJnDQz6SXxsLXGjUNu7CN5xesjoTGyDdX43DwhGRmrRC
rn00X2sYOlELr2jQ6K7jEPKNM0O2gkIBqlbrYOMY4hbas8K9UI+m4apsk+jPoFeG
iSGUIl7bgLIymawFyWcjhAjkjm6qrzEorBkKbO+HqSr85SsvtyfCax39RmtzqTR/
VQZka/0NFSO57pScAeWz1hZxC92Iyxj6o3LLtWtm2vWFzyw8t9QhSMvtHxok5Moj
hENhDaz+bOv5s+yWjrvWJB7zHOD8mmjrZka8tmvTdcqoESFggXG1yYPwb3NrPHoc
p7/Sy97g21t8lCbPV2LUfa8mpEf0L6I98SpwxJ3dGuXS3r9FqdsTPPEIOXjD/fsv
/0GkupemYMk9AycIbCapK1xklj3qUDKuCsePtl3A/InZlPxiKMhQ357xz/sH1AT7
ZPCoItvp+gsEGb01moIX1TwaxQ8ubiXtHYgXAAAHhVeMUIrEmuW8b7nC4DXyMJUe
cmSZQy23IkBUaqVDcY/d
=gPB6
-----END PGP SIGNATURE-----

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Tomcat 8/NIO performance discrepancies

Mark Thomas-2
On 7 June 2017 17:33:02 BST, Christopher Schultz <[hidden email]> wrote:
>Mark,
>
>On 6/7/17 10:57 AM, Mark Thomas wrote:
>> I do see odd behaviour with BIO + executor. It locks up very
>> easily. It appears that the client isn't closing the connections. I
>> don't see this problem with BIO but that may be related to how BIO
>> without an executor handles large numbers of connections.
>
>Was one of those BIOs supposed to be an "NIO"?

No. The second BIO should have been BIO without an executor.

Mark

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

RE: Tomcat 8/NIO performance discrepancies

Piyush Kumar Nayak
In reply to this post by Mark Thomas-2
Mark,

It may not be important, but just to be clear, I'm not testing on localhost. My JMeter and Tomcat are on different machines, with 1 Gbps NIC interfaces between them.

Can you also please provide some pointers on the following:
- How did you disable session on your JSP?
- Did you observe anything that suggests that the client is not closing the connections?
I did not specify the implementation for the HTTP sampler that I configured with JMeter, and it is not set in my jmeter.properties file. So, going by JMeter's documentation, Apache HttpComponents HttpClient 4.x. should be in use.

Thanks
Piyush.

-----Original Message-----
From: Mark Thomas [mailto:[hidden email]]
Sent: Wednesday, June 07, 2017 8:27 PM
To: Tomcat Users List <[hidden email]>
Subject: Re: Tomcat 8/NIO performance discrepancies

On 06/06/17 09:10, Piyush Kumar Nayak wrote:
> Thanks, Mark.
>
> Here are some additional details.

Thanks. These help a lot to make sure we are testing (at least roughly) the same thing.

> I am using Apache JMeter to inject load. I am using a simple hello-world JSP.
>
> The test JMX simulates 100 concurrent user threads with a ramp-up of 5 secs and uses an HTTP Cookie Manager, that reuses cookies for each user thread.

I've disabled session in the JSP for my test and used a simpler JSP.
That should amplify any difference in connector performance.

I'm also testing on localhost.

> I have disabled access log in tomcat. All the other server.xml settings are the default.
>
>
> The BIO with Executor configuration we are using is:
> <Executor name="tomcatThreadPool" namePrefix="catalina-exec-"
>         maxThreads="150" minSpareThreads="4"/>
>
> <Connector port="8080" protocol="org.apache.coyote.http11.Http11Protocol"
>                connectionTimeout="20000"
>                redirectPort="8443" />

FYI for anyone following along. That is a non-executor config. The connector needs to specify the executor it wants to used.

I do see odd behaviour with BIO + executor. It locks up very easily. It appears that the client isn't closing the connections. I don't see this problem with BIO but that may be related to how BIO without an executor handles large numbers of connections.

If I lower the client thread count, the odd behaviour stops and I do see notably higher throughput with BIO + executor. That is unexpected.

My initial thoughts are maybe a contention issue related to the thread pool. I'm continuing to investigate. I plan to look at performance first and then the locking up.

Mark

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Tomcat 8/NIO performance discrepancies

Mark Thomas-2
On 07/06/17 20:25, Piyush Kumar Nayak wrote:
> Mark,
>
> It may not be important, but just to be clear, I'm not testing on localhost. My JMeter and Tomcat are on different machines, with 1 Gbps NIC interfaces between them.

I'm aware you were using two machines but that doesn't appear to be a
significant factor in the observed differences.

> Can you also please provide some pointers on the following:
> - How did you disable session on your JSP?

Look in the index.jsp for the ROOT web application that ships with
Tomcat for an example. Full details are in the JSP specification.

> - Did you observe anything that suggests that the client is not closing the connections?

Yes. This looks like a JMeter issue. It isn't the first time JMeter has
given strange results under load. I've switched to ab which I have found
to be generally much more reliable for these sorts of tests.

I'm still seeing significant differences in BIO with and without an
executor.

I'm still looking for the root cause. I have noticed that without an
executor I see far fewer keep-alive requests than I'd expect. That could
explain the difference, or some of it at least.

Mark


> I did not specify the implementation for the HTTP sampler that I configured with JMeter, and it is not set in my jmeter.properties file. So, going by JMeter's documentation, Apache HttpComponents HttpClient 4.x. should be in use.
>
> Thanks
> Piyush.
>
> -----Original Message-----
> From: Mark Thomas [mailto:[hidden email]]
> Sent: Wednesday, June 07, 2017 8:27 PM
> To: Tomcat Users List <[hidden email]>
> Subject: Re: Tomcat 8/NIO performance discrepancies
>
> On 06/06/17 09:10, Piyush Kumar Nayak wrote:
>> Thanks, Mark.
>>
>> Here are some additional details.
>
> Thanks. These help a lot to make sure we are testing (at least roughly) the same thing.
>
>> I am using Apache JMeter to inject load. I am using a simple hello-world JSP.
>>
>> The test JMX simulates 100 concurrent user threads with a ramp-up of 5 secs and uses an HTTP Cookie Manager, that reuses cookies for each user thread.
>
> I've disabled session in the JSP for my test and used a simpler JSP.
> That should amplify any difference in connector performance.
>
> I'm also testing on localhost.
>
>> I have disabled access log in tomcat. All the other server.xml settings are the default.
>>
>>
>> The BIO with Executor configuration we are using is:
>> <Executor name="tomcatThreadPool" namePrefix="catalina-exec-"
>>         maxThreads="150" minSpareThreads="4"/>
>>
>> <Connector port="8080" protocol="org.apache.coyote.http11.Http11Protocol"
>>                connectionTimeout="20000"
>>                redirectPort="8443" />
>
> FYI for anyone following along. That is a non-executor config. The connector needs to specify the executor it wants to used.
>
> I do see odd behaviour with BIO + executor. It locks up very easily. It appears that the client isn't closing the connections. I don't see this problem with BIO but that may be related to how BIO without an executor handles large numbers of connections.
>
> If I lower the client thread count, the odd behaviour stops and I do see notably higher throughput with BIO + executor. That is unexpected.
>
> My initial thoughts are maybe a contention issue related to the thread pool. I'm continuing to investigate. I plan to look at performance first and then the locking up.
>
> Mark
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Tomcat 8/NIO performance discrepancies

Astrat Goe

> On Jun 8, 2017, at 4:28 AM, Mark Thomas <[hidden email]> wrote:
>
> On 07/06/17 20:25, Piyush Kumar Nayak wrote:
>> Mark,
>>
>> It may not be important, but just to be clear, I'm not testing on localhost. My JMeter and Tomcat are on different machines, with 1 Gbps NIC interfaces between them.
>
> I'm aware you were using two machines but that doesn't appear to be a
> significant factor in the observed differences.
>
>> Can you also please provide some pointers on the following:
>> - How did you disable session on your JSP?
>
> Look in the index.jsp for the ROOT web application that ships with
> Tomcat for an example. Full details are in the JSP specification.
>
>> - Did you observe anything that suggests that the client is not closing the connections?
>
> Yes. This looks like a JMeter issue. It isn't the first time JMeter has
> given strange results under load. I've switched to ab which I have found
> to be generally much more reliable for these sorts of tests.
>
> I'm still seeing significant differences in BIO with and without an
> executor.
>
> I'm still looking for the root cause. I have noticed that without an
> executor I see far fewer keep-alive requests than I'd expect. That could
> explain the difference, or some of it at least.
>
> Mark
>
>
>> I did not specify the implementation for the HTTP sampler that I configured with JMeter, and it is not set in my jmeter.properties file. So, going by JMeter's documentation, Apache HttpComponents HttpClient 4.x. should be in use.
>>
>> Thanks
>> Piyush.
>>
>> -----Original Message-----
>> From: Mark Thomas [mailto:[hidden email]]
>> Sent: Wednesday, June 07, 2017 8:27 PM
>> To: Tomcat Users List <[hidden email]>
>> Subject: Re: Tomcat 8/NIO performance discrepancies
>>
>> On 06/06/17 09:10, Piyush Kumar Nayak wrote:
>>> Thanks, Mark.
>>>
>>> Here are some additional details.
>>
>> Thanks. These help a lot to make sure we are testing (at least roughly) the same thing.
>>
>>> I am using Apache JMeter to inject load. I am using a simple hello-world JSP.
>>>
>>> The test JMX simulates 100 concurrent user threads with a ramp-up of 5 secs and uses an HTTP Cookie Manager, that reuses cookies for each user thread.
>>
>> I've disabled session in the JSP for my test and used a simpler JSP.
>> That should amplify any difference in connector performance.
>>
>> I'm also testing on localhost.
>>
>>> I have disabled access log in tomcat. All the other server.xml settings are the default.
>>>
>>>
>>> The BIO with Executor configuration we are using is:
>>> <Executor name="tomcatThreadPool" namePrefix="catalina-exec-"
>>>        maxThreads="150" minSpareThreads="4"/>
>>>
>>> <Connector port="8080" protocol="org.apache.coyote.http11.Http11Protocol"
>>>               connectionTimeout="20000"
>>>               redirectPort="8443" />
>>
>> FYI for anyone following along. That is a non-executor config. The connector needs to specify the executor it wants to used.
>>
>> I do see odd behaviour with BIO + executor. It locks up very easily. It appears that the client isn't closing the connections. I don't see this problem with BIO but that may be related to how BIO without an executor handles large numbers of connections.
>>
>> If I lower the client thread count, the odd behaviour stops and I do see notably higher throughput with BIO + executor. That is unexpected.
>>
>> My initial thoughts are maybe a contention issue related to the thread pool. I'm continuing to investigate. I plan to look at performance first and then the locking up.
>>
>> Mark
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Tomcat 8/NIO performance discrepancies

Astrat Goe
In reply to this post by Mark Thomas-2

> On Jun 8, 2017, at 4:28 AM, Mark Thomas <[hidden email]> wrote:
>
> On 07/06/17 20:25, Piyush Kumar Nayak wrote:
>> Mark,
>>
>> It may not be important, but just to be clear, I'm not testing on localhost. My JMeter and Tomcat are on different machines, with 1 Gbps NIC interfaces between them.
>
> I'm aware you were using two machines but that doesn't appear to be a
> significant factor in the observed differences.
>
>> Can you also please provide some pointers on the following:
>> - How did you disable session on your JSP?
>
> Look in the index.jsp for the ROOT web application that ships with
> Tomcat for an example. Full details are in the JSP specification.
>
>> - Did you observe anything that suggests that the client is not closing the connections?
>
> Yes. This looks like a JMeter issue. It isn't the first time JMeter has
> given strange results under load. I've switched to ab which I have found
> to be generally much more reliable for these sorts of tests.
>
> I'm still seeing significant differences in BIO with and without an
> executor.
>
> I'm still looking for the root cause. I have noticed that without an
> executor I see far fewer keep-alive requests than I'd expect. That could
> explain the difference, or some of it at least.
>
> Mark
>
>
>> I did not specify the implementation for the HTTP sampler that I configured with JMeter, and it is not set in my jmeter.properties file. So, going by JMeter's documentation, Apache HttpComponents HttpClient 4.x. should be in use.
>>
>> Thanks
>> Piyush.
>>
>> -----Original Message-----
>> From: Mark Thomas [mailto:[hidden email]]
>> Sent: Wednesday, June 07, 2017 8:27 PM
>> To: Tomcat Users List <[hidden email]>
>> Subject: Re: Tomcat 8/NIO performance discrepancies
>>
>> On 06/06/17 09:10, Piyush Kumar Nayak wrote:
>>> Thanks, Mark.
>>>
>>> Here are some additional details.
>>
>> Thanks. These help a lot to make sure we are testing (at least roughly) the same thing.
>>
>>> I am using Apache JMeter to inject load. I am using a simple hello-world JSP.
>>>
>>> The test JMX simulates 100 concurrent user threads with a ramp-up of 5 secs and uses an HTTP Cookie Manager, that reuses cookies for each user thread.
>>
>> I've disabled session in the JSP for my test and used a simpler JSP.
>> That should amplify any difference in connector performance.
>>
>> I'm also testing on localhost.
>>
>>> I have disabled access log in tomcat. All the other server.xml settings are the default.
>>>
>>>
>>> The BIO with Executor configuration we are using is:
>>> <Executor name="tomcatThreadPool" namePrefix="catalina-exec-"
>>>        maxThreads="150" minSpareThreads="4"/>
>>>
>>> <Connector port="8080" protocol="org.apache.coyote.http11.Http11Protocol"
>>>               connectionTimeout="20000"
>>>               redirectPort="8443" />
>>
>> FYI for anyone following along. That is a non-executor config. The connector needs to specify the executor it wants to used.
>>
>> I do see odd behaviour with BIO + executor. It locks up very easily. It appears that the client isn't closing the connections. I don't see this problem with BIO but that may be related to how BIO without an executor handles large numbers of connections.
>>
>> If I lower the client thread count, the odd behaviour stops and I do see notably higher throughput with BIO + executor. That is unexpected.
>>
>> My initial thoughts are maybe a contention issue related to the thread pool. I'm continuing to investigate. I plan to look at performance first and then the locking up.
>>
>> Mark
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Tomcat 8/NIO performance discrepancies

Astrat Goe
In reply to this post by Mark Thomas-2

> On Jun 8, 2017, at 4:28 AM, Mark Thomas <[hidden email]> wrote:
>
> On 07/06/17 20:25, Piyush Kumar Nayak wrote:
>> Mark,
>>
>> It may not be important, but just to be clear, I'm not testing on localhost. My JMeter and Tomcat are on different machines, with 1 Gbps NIC interfaces between them.
>
> I'm aware you were using two machines but that doesn't appear to be a
> significant factor in the observed differences.
>
>> Can you also please provide some pointers on the following:
>> - How did you disable session on your JSP?
>
> Look in the index.jsp for the ROOT web application that ships with
> Tomcat for an example. Full details are in the JSP specification.
>
>> - Did you observe anything that suggests that the client is not closing the connections?
>
> Yes. This looks like a JMeter issue. It isn't the first time JMeter has
> given strange results under load. I've switched to ab which I have found
> to be generally much more reliable for these sorts of tests.
>
> I'm still seeing significant differences in BIO with and without an
> executor.
>
> I'm still looking for the root cause. I have noticed that without an
> executor I see far fewer keep-alive requests than I'd expect. That could
> explain the difference, or some of it at least.
>
> Mark
>
>
>> I did not specify the implementation for the HTTP sampler that I configured with JMeter, and it is not set in my jmeter.properties file. So, going by JMeter's documentation, Apache HttpComponents HttpClient 4.x. should be in use.
>>
>> Thanks
>> Piyush.
>>
>> -----Original Message-----
>> From: Mark Thomas [mailto:[hidden email]]
>> Sent: Wednesday, June 07, 2017 8:27 PM
>> To: Tomcat Users List <[hidden email]>
>> Subject: Re: Tomcat 8/NIO performance discrepancies
>>
>> On 06/06/17 09:10, Piyush Kumar Nayak wrote:
>>> Thanks, Mark.
>>>
>>> Here are some additional details.
>>
>> Thanks. These help a lot to make sure we are testing (at least roughly) the same thing.
>>
>>> I am using Apache JMeter to inject load. I am using a simple hello-world JSP.
>>>
>>> The test JMX simulates 100 concurrent user threads with a ramp-up of 5 secs and uses an HTTP Cookie Manager, that reuses cookies for each user thread.
>>
>> I've disabled session in the JSP for my test and used a simpler JSP.
>> That should amplify any difference in connector performance.
>>
>> I'm also testing on localhost.
>>
>>> I have disabled access log in tomcat. All the other server.xml settings are the default.
>>>
>>>
>>> The BIO with Executor configuration we are using is:
>>> <Executor name="tomcatThreadPool" namePrefix="catalina-exec-"
>>>        maxThreads="150" minSpareThreads="4"/>
>>>
>>> <Connector port="8080" protocol="org.apache.coyote.http11.Http11Protocol"
>>>               connectionTimeout="20000"
>>>               redirectPort="8443" />
>>
>> FYI for anyone following along. That is a non-executor config. The connector needs to specify the executor it wants to used.
>>
>> I do see odd behaviour with BIO + executor. It locks up very easily. It appears that the client isn't closing the connections. I don't see this problem with BIO but that may be related to how BIO without an executor handles large numbers of connections.
>>
>> If I lower the client thread count, the odd behaviour stops and I do see notably higher throughput with BIO + executor. That is unexpected.
>>
>> My initial thoughts are maybe a contention issue related to the thread pool. I'm continuing to investigate. I plan to look at performance first and then the locking up.
>>
>> Mark
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Tomcat 8/NIO performance discrepancies

Mark Thomas-2
In reply to this post by Mark Thomas-2
On 07/06/17 22:28, Mark Thomas wrote:

<snip/>

> I'm still seeing significant differences in BIO with and without an
> executor.
>
> I'm still looking for the root cause. I have noticed that without an
> executor I see far fewer keep-alive requests than I'd expect. That could
> explain the difference, or some of it at least.

Further testing indicates that most, if not all, of the performance
difference is down to the fewer number of requests processed using HTTP
keep-alive. This is because BIO without an executor is subject to
disableKeepAlivePercentage whereas BIO with an executor is not.

If the number of concurrent clients is kept below 75% of maxThreads,
performance remains very similar.

The problems you are likely to experience in the real world without
disableKeepAlivePercentage (i.e. if you use BIO with an executor) are
described in this thread:

http://tomcat.markmail.org/thread/mcfdk2273v2fsh5k

The better tuning options are probably:
- increase maxThreads to max expected client concurrency + headroom
  or if your memory/CPU won't permit that, as high as you can
- increase disableKeepAlivePercentage

I'll look at BIO vs NIO next.

Mark

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Tomcat 8/NIO performance discrepancies

Mark Thomas-2
On 08/06/17 14:42, Mark Thomas wrote:
> I'll look at BIO vs NIO next.

At the sort of request rates and response sizes we are talking about, we
are a long way from real-world applications and the differences you see
between BIO and NIO are largely down to the differences between doing a
blocking read for the next request (BIO) and polling (NIO).

What you will notice is that as the number of concurrent clients rises,
BIO performance will drop considerably as soon as the
disableKeepAlivePercentage is exceeded and will continue to fall as
client concurrency rises. NIO, on the other hand, will retain broadly
the same performance even when client concurrency >> threads.

I have spent some time looking at this in detail with a profiler but the
timing differences we are talking about are so small that often the
profiler overhead ends up dominating.

I think the general lesson here is that benchmarks like this can be
interesting but the further removed they are from real-world examples,
the less useful they are likely to be.

As you move towards more real-world examples, what you tend to find is
that I/O and/or application logic become far more dominant.

Mark


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Tomcat 8/NIO performance discrepancies

kmaxwilliams43
In reply to this post by Astrat Goe
Gvhuf

Envoyé de mon smartphone BlackBerry 10.
  Message d'origine  
De: Astrat Goe
Envoyé: mercredi 7 juin 2017 21:29
À: Tomcat Users List
Répondre à: Tomcat Users List
Objet: Re: Tomcat 8/NIO performance discrepancies


> On Jun 8, 2017, at 4:28 AM, Mark Thomas <[hidden email]> wrote:
>
> On 07/06/17 20:25, Piyush Kumar Nayak wrote:
>> Mark,
>>
>> It may not be important, but just to be clear, I'm not testing on localhost. My JMeter and Tomcat are on different machines, with 1 Gbps NIC interfaces between them.
>
> I'm aware you were using two machines but that doesn't appear to be a
> significant factor in the observed differences.
>
>> Can you also please provide some pointers on the following:
>> - How did you disable session on your JSP?
>
> Look in the index.jsp for the ROOT web application that ships with
> Tomcat for an example. Full details are in the JSP specification.
>
>> - Did you observe anything that suggests that the client is not closing the connections?
>
> Yes. This looks like a JMeter issue. It isn't the first time JMeter has
> given strange results under load. I've switched to ab which I have found
> to be generally much more reliable for these sorts of tests.
>
> I'm still seeing significant differences in BIO with and without an
> executor.
>
> I'm still looking for the root cause. I have noticed that without an
> executor I see far fewer keep-alive requests than I'd expect. That could
> explain the difference, or some of it at least.
>
> Mark
>
>
>> I did not specify the implementation for the HTTP sampler that I configured with JMeter, and it is not set in my jmeter.properties file. So, going by JMeter's documentation, Apache HttpComponents HttpClient 4.x. should be in use.
>>
>> Thanks
>> Piyush.
>>
>> -----Original Message-----
>> From: Mark Thomas [mailto:[hidden email]]
>> Sent: Wednesday, June 07, 2017 8:27 PM
>> To: Tomcat Users List <[hidden email]>
>> Subject: Re: Tomcat 8/NIO performance discrepancies
>>
>> On 06/06/17 09:10, Piyush Kumar Nayak wrote:
>>> Thanks, Mark.
>>>
>>> Here are some additional details.
>>
>> Thanks. These help a lot to make sure we are testing (at least roughly) the same thing.
>>
>>> I am using Apache JMeter to inject load. I am using a simple hello-world JSP.
>>>
>>> The test JMX simulates 100 concurrent user threads with a ramp-up of 5 secs and uses an HTTP Cookie Manager, that reuses cookies for each user thread.
>>
>> I've disabled session in the JSP for my test and used a simpler JSP.
>> That should amplify any difference in connector performance.
>>
>> I'm also testing on localhost.
>>
>>> I have disabled access log in tomcat. All the other server.xml settings are the default.
>>>
>>>
>>> The BIO with Executor configuration we are using is:
>>> <Executor name="tomcatThreadPool" namePrefix="catalina-exec-"
>>> maxThreads="150" minSpareThreads="4"/>
>>>
>>> <Connector port="8080" protocol="org.apache.coyote.http11.Http11Protocol"
>>> connectionTimeout="20000"
>>> redirectPort="8443" />
>>
>> FYI for anyone following along. That is a non-executor config. The connector needs to specify the executor it wants to used.
>>
>> I do see odd behaviour with BIO + executor. It locks up very easily. It appears that the client isn't closing the connections. I don't see this problem with BIO but that may be related to how BIO without an executor handles large numbers of connections.
>>
>> If I lower the client thread count, the odd behaviour stops and I do see notably higher throughput with BIO + executor. That is unexpected.
>>
>> My initial thoughts are maybe a contention issue related to the thread pool. I'm continuing to investigate. I plan to look at performance first and then the locking up.
>>
>> Mark
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Tomcat 8/NIO performance discrepancies

kmaxwilliams43


Envoyé de mon smartphone BlackBerry 10.
  Message d'origine  
De: [hidden email]
Envoyé: vendredi 9 juin 2017 00:12
À: Astrat Goe; Tomcat Users List
Objet: Re: Tomcat 8/NIO performance discrepancies

Gvhuf

Envoyé de mon smartphone BlackBerry 10.
  Message d'origine  
De: Astrat Goe
Envoyé: mercredi 7 juin 2017 21:29
À: Tomcat Users List
Répondre à: Tomcat Users List
Objet: Re: Tomcat 8/NIO performance discrepancies
Ggfggthgd
Vid

> On Jun 8, 2017, at 4:28 AM, Mark Thomas <[hidden email]> wrote:
>
> On 07/06/17 20:25, Piyush Kumar Nayak wrote:
>> Mark,
>>
>> It may not be important, but just to be clear, I'm not testing on localhost. My JMeter and Tomcat are on different machines, with 1 Gbps NIC interfaces between them.
>
> I'm aware you were using two machines but that doesn't appear to be a
> significant factor in the observed differences.
>
>> Can you also please provide some pointers on the following:
>> - How did you disable session on your JSP?
>
> Look in the index.jsp for the ROOT web application that ships with
> Tomcat for an example. Full details are in the JSP specification.
>
>> - Did you observe anything that suggests that the client is not closing the connections?
>
> Yes. This looks like a JMeter issue. It isn't the first time JMeter has
> given strange results under load. I've switched to ab which I have found
> to be generally much more reliable for these sorts of tests.
>
> I'm still seeing significant differences in BIO with and without an
> executor.
>
> I'm still looking for the root cause. I have noticed that without an
> executor I see far fewer keep-alive requests than I'd expect. That could
> explain the difference, or some of it at least.
>
> Mark
>
>
>> I did not specify the implementation for the HTTP sampler that I configured with JMeter, and it is not set in my jmeter.properties file. So, going by JMeter's documentation, Apache HttpComponents HttpClient 4.x. should be in use.
>>
>> Thanks
>> Piyush.
>>
>> -----Original Message-----
>> From: Mark Thomas [mailto:[hidden email]]
>> Sent: Wednesday, June 07, 2017 8:27 PM
>> To: Tomcat Users List <[hidden email]>
>> Subject: Re: Tomcat 8/NIO performance discrepancies
>>
>> On 06/06/17 09:10, Piyush Kumar Nayak wrote:
>>> Thanks, Mark.
>>>
>>> Here are some additional details.
>>
>> Thanks. These help a lot to make sure we are testing (at least roughly) the same thing.
>>
>>> I am using Apache JMeter to inject load. I am using a simple hello-world JSP.
>>>
>>> The test JMX simulates 100 concurrent user threads with a ramp-up of 5 secs and uses an HTTP Cookie Manager, that reuses cookies for each user thread.
>>
>> I've disabled session in the JSP for my test and used a simpler JSP.
>> That should amplify any difference in connector performance.
>>
>> I'm also testing on localhost.
>>
>>> I have disabled access log in tomcat. All the other server.xml settings are the default.
>>>
>>>
>>> The BIO with Executor configuration we are using is:
>>> <Executor name="tomcatThreadPool" namePrefix="catalina-exec-"
>>> maxThreads="150" minSpareThreads="4"/>
>>>
>>> <Connector port="8080" protocol="org.apache.coyote.http11.Http11Protocol"
>>> connectionTimeout="20000"
>>> redirectPort="8443" />
>>
>> FYI for anyone following along. That is a non-executor config. The connector needs to specify the executor it wants to used.
>>
>> I do see odd behaviour with BIO + executor. It locks up very easily. It appears that the client isn't closing the connections. I don't see this problem with BIO but that may be related to how BIO without an executor handles large numbers of connections.
>>
>> If I lower the client thread count, the odd behaviour stops and I do see notably higher throughput with BIO + executor. That is unexpected.
>>
>> My initial thoughts are maybe a contention issue related to the thread pool. I'm continuing to investigate. I plan to look at performance first and then the locking up.
>>
>> Mark
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

RE: Tomcat 8/NIO performance discrepancies

Piyush Kumar Nayak
In reply to this post by Mark Thomas-2
Thanks, Mark.
I have been testing with 100 concurrent users. While the executor had the maxThreads set to 150, the connector without the executor was left to the default value (which should be 200). I did not have the disableKeepAlivePercentage attribute set, which means that it should take to the default value of 75.

So, the number of threads in use before the keepalive is disabled in case of non-executor connector should be 150. At 100 users, the no. of clients is at 50% of the maxThread count. Unless I'm missing something here, I don't see why keep-alive would be disabled.
I'd appreciate if you can give me some pointers on how you can tell if requests are processed at the server end with keep-alive.

I'll try setting disableKeepAlivePercentage to 100, and see if that improves the performance.

-----Original Message-----
From: Mark Thomas [mailto:[hidden email]]
Sent: Thursday, June 08, 2017 7:13 PM
To: Tomcat Users List <[hidden email]>
Subject: Re: Tomcat 8/NIO performance discrepancies

On 07/06/17 22:28, Mark Thomas wrote:

<snip/>

> I'm still seeing significant differences in BIO with and without an
> executor.
>
> I'm still looking for the root cause. I have noticed that without an
> executor I see far fewer keep-alive requests than I'd expect. That
> could explain the difference, or some of it at least.

Further testing indicates that most, if not all, of the performance difference is down to the fewer number of requests processed using HTTP keep-alive. This is because BIO without an executor is subject to disableKeepAlivePercentage whereas BIO with an executor is not.

If the number of concurrent clients is kept below 75% of maxThreads, performance remains very similar.

The problems you are likely to experience in the real world without disableKeepAlivePercentage (i.e. if you use BIO with an executor) are described in this thread:

https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Ftomcat.markmail.org%2Fthread%2Fmcfdk2273v2fsh5k&data=02%7C01%7C%7C6816158122aa4914b20208d4ae744a1e%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C636325261860429837&sdata=8t1j6lxgc3aTCDq45mXbH7LqedtIuRgPsrgqUKXyCrc%3D&reserved=0

The better tuning options are probably:
- increase maxThreads to max expected client concurrency + headroom
  or if your memory/CPU won't permit that, as high as you can
- increase disableKeepAlivePercentage

I'll look at BIO vs NIO next.

Mark

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Tomcat 8/NIO performance discrepancies

Mark Thomas-2
On 12/06/17 20:27, Piyush Kumar Nayak wrote:
> Thanks, Mark.
> I have been testing with 100 concurrent users. While the executor had the maxThreads set to 150, the connector without the executor was left to the default value (which should be 200). I did not have the disableKeepAlivePercentage attribute set, which means that it should take to the default value of 75.
>
> So, the number of threads in use before the keepalive is disabled in case of non-executor connector should be 150. At 100 users, the no. of clients is at 50% of the maxThread count. Unless I'm missing something here, I don't see why keep-alive would be disabled.
> I'd appreciate if you can give me some pointers on how you can tell if requests are processed at the server end with keep-alive.

I suggest you use ab rather than JMeter. I've seen all sorts of odd
behaviour with JMeter when processing requests very quickly that I
haven't seen with ab. Also, ab tells you how many of your requests used
keep-alive.

Mark


>
> I'll try setting disableKeepAlivePercentage to 100, and see if that improves the performance.
>
> -----Original Message-----
> From: Mark Thomas [mailto:[hidden email]]
> Sent: Thursday, June 08, 2017 7:13 PM
> To: Tomcat Users List <[hidden email]>
> Subject: Re: Tomcat 8/NIO performance discrepancies
>
> On 07/06/17 22:28, Mark Thomas wrote:
>
> <snip/>
>
>> I'm still seeing significant differences in BIO with and without an
>> executor.
>>
>> I'm still looking for the root cause. I have noticed that without an
>> executor I see far fewer keep-alive requests than I'd expect. That
>> could explain the difference, or some of it at least.
>
> Further testing indicates that most, if not all, of the performance difference is down to the fewer number of requests processed using HTTP keep-alive. This is because BIO without an executor is subject to disableKeepAlivePercentage whereas BIO with an executor is not.
>
> If the number of concurrent clients is kept below 75% of maxThreads, performance remains very similar.
>
> The problems you are likely to experience in the real world without disableKeepAlivePercentage (i.e. if you use BIO with an executor) are described in this thread:
>
> https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Ftomcat.markmail.org%2Fthread%2Fmcfdk2273v2fsh5k&data=02%7C01%7C%7C6816158122aa4914b20208d4ae744a1e%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C636325261860429837&sdata=8t1j6lxgc3aTCDq45mXbH7LqedtIuRgPsrgqUKXyCrc%3D&reserved=0
>
> The better tuning options are probably:
> - increase maxThreads to max expected client concurrency + headroom
>   or if your memory/CPU won't permit that, as high as you can
> - increase disableKeepAlivePercentage
>
> I'll look at BIO vs NIO next.
>
> Mark
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

12
Loading...