BIO: Async servlet with it's own thread pool; but get connection refused!

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

BIO: Async servlet with it's own thread pool; but get connection refused!

Yasser Zamani
Hi there,

I'm studying Servlet 3's async API using tomcat. I see following strange
behavior from tomcat in a very simple test app!

I have following JMeter test plan:
Number of threads (users): 700
Ramp-Up period (in seconds): 23
Loop count: 1

So JMeter generates 30.43 requests per second and 304.3 requests per 10
seconds. I'm planning to full tomcat's BIO pool and accept buffer :)

I have an always async test servlet which on destroy, I print tomcat
container max used threads count. It prints 187 for me which is lower
than 200 (tomcat default pool size) so I should not get any "connection
refuse" but I get!

I have simulated a blocking operation by a sleep for 10 seconds. When my
servlet gets a request, it quickly starts an async and add further
processing to my own thread pool (container thread comes back to pool
quickly, right). My own thread pool size is 310 which is greater than
304.3 (requests in 10 seconds) so never full.

I've tested several times. Tomcat successfully returns from all requests
below 326th but fails 102 requests from 326th to 700th with "connection
refuse" and afew with "connection reset".

Why?! My own thread pool does the jobs and Tomcat's pool is free (my
servlet uses 187 threads of tomcat at max).

Thanks in advance!

JMETER RESULT of RESPONSE TIMES:
Max: 60 seconds (lower then tomcat and asyncContext timeout)
MIN: 10 seconds
AVG: 37 seconds
ERR: 15%

CONFIGURATIONS:

Server.xml
<Connector port="7780" protocol="org.apache.coyote.http11.Http11Protocol"
                connectionTimeout="120000"
                redirectPort="7743" />

Async.java

package com.sstr.example;

import javax.servlet.*;
import javax.servlet.annotation.WebInitParam;
import javax.servlet.annotation.WebServlet;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import java.io.IOException;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.ThreadFactory;

@WebServlet(
         name = "async",
         value = {"/async"},
         asyncSupported = true,
         initParams = {
                 @WebInitParam(name = "JobPoolSize", value = "310")
         }
)
public class Async extends HttpServlet {

     public final int REQUEST_TIMEOUT = 120000;
     private ExecutorService exe;

     @Override
     public void init() throws ServletException {
         int size = Integer.parseInt(getInitParameter("JobPoolSize"));
         exe = Executors.newFixedThreadPool(
                 size,
                 new ThreadFactory() {
                     @Override
                     public Thread newThread(Runnable r) {
                         return new Thread(r, "Async Processor");
                     }
                 }
         );
     }

     @Override
     protected void doGet(HttpServletRequest req, HttpServletResponse
resp) throws ServletException, IOException {
         final AsyncContext context = req.startAsync();
         context.setTimeout(REQUEST_TIMEOUT);
         exe.execute(new ContextExecution(context,
Thread.currentThread().getName()));
     }

     @Override
     public void destroy() {
         System.out.println("Container MAX used threads: " + threadCount);
         exe.shutdown();
     }

     int threadCount = 0;
     class ContextExecution implements Runnable {

         final AsyncContext context;
         final String containerThreadName;

         public ContextExecution(AsyncContext context, String
containerThreadName) {
             this.context = context;
             this.containerThreadName = containerThreadName;
         }

         @Override
         public void run() {
             try {
                 int threadNumber =
Integer.parseInt(containerThreadName.substring(
                         containerThreadName.lastIndexOf('-')+1));
                 if(threadNumber > threadCount) {
                     threadCount = threadNumber;
                 }

                 // Simulate Time Consuming Task
                 Thread.sleep(10000);

                 ServletResponse resp = context.getResponse();
                 if (resp != null) {
                     resp.getWriter().write("Ok");
                 }

                 context.complete();
             } catch (Exception e) {
                 // Handle ?
             }
         }
     }
}

OUTPUT:

Container MAX used threads: 187

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: BIO: Async servlet with it's own thread pool; but get connection refused!

Guang Chao
On Thu, Sep 7, 2017 at 3:59 AM, Yasser Zamani <[hidden email]>
wrote:

> Hi there,
>
> I'm studying Servlet 3's async API using tomcat. I see following strange
> behavior from tomcat in a very simple test app!
>
> I have following JMeter test plan:
> Number of threads (users): 700
> Ramp-Up period (in seconds): 23
> Loop count: 1
>
> So JMeter generates 30.43 requests per second and 304.3 requests per 10
> seconds. I'm planning to full tomcat's BIO pool and accept buffer :)
>
> I have an always async test servlet which on destroy, I print tomcat
> container max used threads count. It prints 187 for me which is lower
> than 200 (tomcat default pool size) so I should not get any "connection
> refuse" but I get!
>
> I have simulated a blocking operation by a sleep for 10 seconds. When my
> servlet gets a request, it quickly starts an async and add further
> processing to my own thread pool (container thread comes back to pool
> quickly, right). My own thread pool size is 310 which is greater than
> 304.3 (requests in 10 seconds) so never full.
>
> I've tested several times. Tomcat successfully returns from all requests
> below 326th but fails 102 requests from 326th to 700th with "connection
> refuse" and afew with "connection reset".
>
> Why?! My own thread pool does the jobs and Tomcat's pool is free (my
> servlet uses 187 threads of tomcat at max).
>
> Thanks in advance!
>
> JMETER RESULT of RESPONSE TIMES:
> Max: 60 seconds (lower then tomcat and asyncContext timeout)
> MIN: 10 seconds
> AVG: 37 seconds
> ERR: 15%
>
> CONFIGURATIONS:
>
> Server.xml
> <Connector port="7780" protocol="org.apache.coyote.http11.Http11Protocol"
>                 connectionTimeout="120000"
>                 redirectPort="7743" />
>
> Async.java
>
> package com.sstr.example;
>
> import javax.servlet.*;
> import javax.servlet.annotation.WebInitParam;
> import javax.servlet.annotation.WebServlet;
> import javax.servlet.http.HttpServlet;
> import javax.servlet.http.HttpServletRequest;
> import javax.servlet.http.HttpServletResponse;
> import java.io.IOException;
> import java.util.concurrent.ExecutorService;
> import java.util.concurrent.Executors;
> import java.util.concurrent.ThreadFactory;
>
> @WebServlet(
>          name = "async",
>          value = {"/async"},
>          asyncSupported = true,
>          initParams = {
>                  @WebInitParam(name = "JobPoolSize", value = "310")
>          }
> )
> public class Async extends HttpServlet {
>
>      public final int REQUEST_TIMEOUT = 120000;
>      private ExecutorService exe;
>
>      @Override
>      public void init() throws ServletException {
>          int size = Integer.parseInt(getInitParameter("JobPoolSize"));
>          exe = Executors.newFixedThreadPool(
>                  size,
>                  new ThreadFactory() {
>                      @Override
>                      public Thread newThread(Runnable r) {
>                          return new Thread(r, "Async Processor");
>                      }
>                  }
>          );
>      }
>
>      @Override
>      protected void doGet(HttpServletRequest req, HttpServletResponse
> resp) throws ServletException, IOException {
>          final AsyncContext context = req.startAsync();
>          context.setTimeout(REQUEST_TIMEOUT);
>          exe.execute(new ContextExecution(context,
> Thread.currentThread().getName()));
>      }
>

I'm not 100% sure, but it seems the doGet method code here is not correct.


>
>      @Override
>      public void destroy() {
>          System.out.println("Container MAX used threads: " + threadCount);
>          exe.shutdown();
>      }
>
>      int threadCount = 0;
>      class ContextExecution implements Runnable {
>
>          final AsyncContext context;
>          final String containerThreadName;
>
>          public ContextExecution(AsyncContext context, String
> containerThreadName) {
>              this.context = context;
>              this.containerThreadName = containerThreadName;
>          }
>
>          @Override
>          public void run() {
>              try {
>                  int threadNumber =
> Integer.parseInt(containerThreadName.substring(
>                          containerThreadName.lastIndexOf('-')+1));
>                  if(threadNumber > threadCount) {
>                      threadCount = threadNumber;
>                  }
>
>                  // Simulate Time Consuming Task
>                  Thread.sleep(10000);
>
>                  ServletResponse resp = context.getResponse();
>                  if (resp != null) {
>                      resp.getWriter().write("Ok");
>                  }
>
>                  context.complete();
>              } catch (Exception e) {
>                  // Handle ?
>              }
>          }
>      }
> }
>
> OUTPUT:
>
> Container MAX used threads: 187
>



--
Guang <http://javadevnotes.com/java-long-to-string-examples/>
Reply | Threaded
Open this post in threaded view
|

Re: BIO: Async servlet with it's own thread pool; but get connection refused!

Mark Thomas-2
In reply to this post by Yasser Zamani
On 06/09/17 20:59, Yasser Zamani wrote:
> Hi there,
>
> I'm studying Servlet 3's async API using tomcat. I see following strange
> behavior from tomcat in a very simple test app!

You are also using the BIO connector which, since it is blocking,
doesn't offer any benefits when using async. You'd be better off with
the NIO connector.

You haven't told us which Tomcat version you are using. Since you are
using BIO that narrows it down a bit to 7.0.x or 8.0.x but that is still
a lot of possibilities.

Neither have you told us what operating system you are using. My
experience of JMeter under load, particularly on Windows, is that you
see strange behaviour and it can be hard to figure out the interaction
of JMeter, the OS network stack and Tomcat.

You also haven't told us what hardware this test is running on.
Particularly the number of cores available.

> I have following JMeter test plan:
> Number of threads (users): 700
> Ramp-Up period (in seconds): 23
> Loop count: 1
>
> So JMeter generates 30.43 requests per second and 304.3 requests per 10
> seconds. I'm planning to full tomcat's BIO pool and accept buffer :)
>
> I have an always async test servlet which on destroy, I print tomcat
> container max used threads count. It prints 187 for me which is lower
> than 200 (tomcat default pool size) so I should not get any "connection
> refuse" but I get!

There are all sorts of possible reasons for that. I'd suggest scaling
down the test. Limit Tomcat to 20 threads. Reduce the load similarly.
Increase the sleep time. You want to ensure that the only limit you are
hitting is the one you are trying to hit.

Mark


> I have simulated a blocking operation by a sleep for 10 seconds. When my
> servlet gets a request, it quickly starts an async and add further
> processing to my own thread pool (container thread comes back to pool
> quickly, right). My own thread pool size is 310 which is greater than
> 304.3 (requests in 10 seconds) so never full.
>
> I've tested several times. Tomcat successfully returns from all requests
> below 326th but fails 102 requests from 326th to 700th with "connection
> refuse" and afew with "connection reset".
>
> Why?! My own thread pool does the jobs and Tomcat's pool is free (my
> servlet uses 187 threads of tomcat at max).
>
> Thanks in advance!
>
> JMETER RESULT of RESPONSE TIMES:
> Max: 60 seconds (lower then tomcat and asyncContext timeout)
> MIN: 10 seconds
> AVG: 37 seconds
> ERR: 15%
>
> CONFIGURATIONS:
>
> Server.xml
> <Connector port="7780" protocol="org.apache.coyote.http11.Http11Protocol"
>                 connectionTimeout="120000"
>                 redirectPort="7743" />
>
> Async.java
>
> package com.sstr.example;
>
> import javax.servlet.*;
> import javax.servlet.annotation.WebInitParam;
> import javax.servlet.annotation.WebServlet;
> import javax.servlet.http.HttpServlet;
> import javax.servlet.http.HttpServletRequest;
> import javax.servlet.http.HttpServletResponse;
> import java.io.IOException;
> import java.util.concurrent.ExecutorService;
> import java.util.concurrent.Executors;
> import java.util.concurrent.ThreadFactory;
>
> @WebServlet(
>          name = "async",
>          value = {"/async"},
>          asyncSupported = true,
>          initParams = {
>                  @WebInitParam(name = "JobPoolSize", value = "310")
>          }
> )
> public class Async extends HttpServlet {
>
>      public final int REQUEST_TIMEOUT = 120000;
>      private ExecutorService exe;
>
>      @Override
>      public void init() throws ServletException {
>          int size = Integer.parseInt(getInitParameter("JobPoolSize"));
>          exe = Executors.newFixedThreadPool(
>                  size,
>                  new ThreadFactory() {
>                      @Override
>                      public Thread newThread(Runnable r) {
>                          return new Thread(r, "Async Processor");
>                      }
>                  }
>          );
>      }
>
>      @Override
>      protected void doGet(HttpServletRequest req, HttpServletResponse
> resp) throws ServletException, IOException {
>          final AsyncContext context = req.startAsync();
>          context.setTimeout(REQUEST_TIMEOUT);
>          exe.execute(new ContextExecution(context,
> Thread.currentThread().getName()));
>      }
>
>      @Override
>      public void destroy() {
>          System.out.println("Container MAX used threads: " + threadCount);
>          exe.shutdown();
>      }
>
>      int threadCount = 0;
>      class ContextExecution implements Runnable {
>
>          final AsyncContext context;
>          final String containerThreadName;
>
>          public ContextExecution(AsyncContext context, String
> containerThreadName) {
>              this.context = context;
>              this.containerThreadName = containerThreadName;
>          }
>
>          @Override
>          public void run() {
>              try {
>                  int threadNumber =
> Integer.parseInt(containerThreadName.substring(
>                          containerThreadName.lastIndexOf('-')+1));
>                  if(threadNumber > threadCount) {
>                      threadCount = threadNumber;
>                  }
>
>                  // Simulate Time Consuming Task
>                  Thread.sleep(10000);
>
>                  ServletResponse resp = context.getResponse();
>                  if (resp != null) {
>                      resp.getWriter().write("Ok");
>                  }
>
>                  context.complete();
>              } catch (Exception e) {
>                  // Handle ?
>              }
>          }
>      }
> }
>
> OUTPUT:
>
> Container MAX used threads: 187
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: BIO: Async servlet with it's own thread pool; but get connection refused!

Yasser Zamani
At first thanks a lot for your reply!

On 9/7/2017 1:43 PM, Mark Thomas wrote:
> On 06/09/17 20:59, Yasser Zamani wrote:
>> Hi there,
>>
>> I'm studying Servlet 3's async API using tomcat. I see following strange
>> behavior from tomcat in a very simple test app!
>
> You are also using the BIO connector which, since it is blocking,
> doesn't offer any benefits when using async. You'd be better off with
> the NIO connector.

Yes I know but currently it's not important for me. I am studying
Servlet 3's async API and BIO keeps it simpler to study and focus only
on it (with NIO I cannot know if something is because of Servlet 3's
async API or Tomcat's NIO).

>
> You haven't told us which Tomcat version you are using. Since you are
> using BIO that narrows it down a bit to 7.0.x or 8.0.x but that is still
> a lot of possibilities.

It's 7.0.47

>
> Neither have you told us what operating system you are using. My
> experience of JMeter under load, particularly on Windows, is that you
> see strange behaviour and it can be hard to figure out the interaction
> of JMeter, the OS network stack and Tomcat.
>
> You also haven't told us what hardware this test is running on.
> Particularly the number of cores available.
>

OS Name Microsoft Windows 8.1 Enterprise
System Type x64-based PC
Processor Intel(R) Core(TM) i3-4130 CPU @ 3.40GHz, 3400 Mhz, 2 Core(s),
4 Logical Processor(s)


>> I have following JMeter test plan:
>> Number of threads (users): 700
>> Ramp-Up period (in seconds): 23
>> Loop count: 1
>>
>> So JMeter generates 30.43 requests per second and 304.3 requests per 10
>> seconds. I'm planning to full tomcat's BIO pool and accept buffer :)
>>
>> I have an always async test servlet which on destroy, I print tomcat
>> container max used threads count. It prints 187 for me which is lower
>> than 200 (tomcat default pool size) so I should not get any "connection
>> refuse" but I get!
>
> There are all sorts of possible reasons for that. I'd suggest scaling
> down the test. Limit Tomcat to 20 threads. Reduce the load similarly.
> Increase the sleep time. You want to ensure that the only limit you are
> hitting is the one you are trying to hit.
>

Previously I also tested very low loads but again as you suggested I
tested following low load configuration and get even worse results!!
Tomcat successfully returns from request below 29th but fails 25
requests of 29th to 70th. However this time all fails are "connection
refuse" and there are not any "connection reset". Whilst the program
output is "Container MAX used threads: 10" !!

CONFIGURATION #2:

Server.xml
<Connector port="7780" protocol="org.apache.coyote.http11.Http11Protocol"
                connectionTimeout="120000"
           maxThreads="20" maxConnections="20" acceptCount="10"
                redirectPort="7743" />

JMeter
Number of threads (users): 70
Ramp-Up period (in seconds): 35 (40 requests per 20 seconds)
Loop count: 1

My async servlet
Async Sleep Time: 20 seconds (ensures 40 concurrent requests)
It's own thread pool size: 41 (lower than 40 so never fulls)

JMETER RESULT of RESPONSE TIMES #2:
Max: 38 seconds (lower then tomcat and asyncContext timeout)
MIN: 20 seconds
AVG: 18 seconds (because of fails)
ERR: 36%

UTPUT:

Container MAX used threads: 10

Thanks in advance!

> Mark
>
>
>> I have simulated a blocking operation by a sleep for 10 seconds. When my
>> servlet gets a request, it quickly starts an async and add further
>> processing to my own thread pool (container thread comes back to pool
>> quickly, right). My own thread pool size is 310 which is greater than
>> 304.3 (requests in 10 seconds) so never full.
>>
>> I've tested several times. Tomcat successfully returns from all requests
>> below 326th but fails 102 requests from 326th to 700th with "connection
>> refuse" and afew with "connection reset".
>>
>> Why?! My own thread pool does the jobs and Tomcat's pool is free (my
>> servlet uses 187 threads of tomcat at max).
>>
>> Thanks in advance!
>>
>> JMETER RESULT of RESPONSE TIMES:
>> Max: 60 seconds (lower then tomcat and asyncContext timeout)
>> MIN: 10 seconds
>> AVG: 37 seconds
>> ERR: 15%
>>
>> CONFIGURATIONS:
>>
>> Server.xml
>> <Connector port="7780" protocol="org.apache.coyote.http11.Http11Protocol"
>>                 connectionTimeout="120000"
>>                 redirectPort="7743" />
>>
>> Async.java
>>
>> package com.sstr.example;
>>
>> import javax.servlet.*;
>> import javax.servlet.annotation.WebInitParam;
>> import javax.servlet.annotation.WebServlet;
>> import javax.servlet.http.HttpServlet;
>> import javax.servlet.http.HttpServletRequest;
>> import javax.servlet.http.HttpServletResponse;
>> import java.io.IOException;
>> import java.util.concurrent.ExecutorService;
>> import java.util.concurrent.Executors;
>> import java.util.concurrent.ThreadFactory;
>>
>> @WebServlet(
>>          name = "async",
>>          value = {"/async"},
>>          asyncSupported = true,
>>          initParams = {
>>                  @WebInitParam(name = "JobPoolSize", value = "310")
>>          }
>> )
>> public class Async extends HttpServlet {
>>
>>      public final int REQUEST_TIMEOUT = 120000;
>>      private ExecutorService exe;
>>
>>      @Override
>>      public void init() throws ServletException {
>>          int size = Integer.parseInt(getInitParameter("JobPoolSize"));
>>          exe = Executors.newFixedThreadPool(
>>                  size,
>>                  new ThreadFactory() {
>>                      @Override
>>                      public Thread newThread(Runnable r) {
>>                          return new Thread(r, "Async Processor");
>>                      }
>>                  }
>>          );
>>      }
>>
>>      @Override
>>      protected void doGet(HttpServletRequest req, HttpServletResponse
>> resp) throws ServletException, IOException {
>>          final AsyncContext context = req.startAsync();
>>          context.setTimeout(REQUEST_TIMEOUT);
>>          exe.execute(new ContextExecution(context,
>> Thread.currentThread().getName()));
>>      }
>>
>>      @Override
>>      public void destroy() {
>>          System.out.println("Container MAX used threads: " + threadCount);
>>          exe.shutdown();
>>      }
>>
>>      int threadCount = 0;
>>      class ContextExecution implements Runnable {
>>
>>          final AsyncContext context;
>>          final String containerThreadName;
>>
>>          public ContextExecution(AsyncContext context, String
>> containerThreadName) {
>>              this.context = context;
>>              this.containerThreadName = containerThreadName;
>>          }
>>
>>          @Override
>>          public void run() {
>>              try {
>>                  int threadNumber =
>> Integer.parseInt(containerThreadName.substring(
>>                          containerThreadName.lastIndexOf('-')+1));
>>                  if(threadNumber > threadCount) {
>>                      threadCount = threadNumber;
>>                  }
>>
>>                  // Simulate Time Consuming Task
>>                  Thread.sleep(10000);
>>
>>                  ServletResponse resp = context.getResponse();
>>                  if (resp != null) {
>>                      resp.getWriter().write("Ok");
>>                  }
>>
>>                  context.complete();
>>              } catch (Exception e) {
>>                  // Handle ?
>>              }
>>          }
>>      }
>> }
>>
>> OUTPUT:
>>
>> Container MAX used threads: 187
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: BIO: Async servlet with it's own thread pool; but get connection refused!

Mark Thomas-2
On 07/09/17 22:22, Yasser Zamani wrote:

> At first thanks a lot for your reply!
>
> On 9/7/2017 1:43 PM, Mark Thomas wrote:
>> On 06/09/17 20:59, Yasser Zamani wrote:
>>> Hi there,
>>>
>>> I'm studying Servlet 3's async API using tomcat. I see following strange
>>> behavior from tomcat in a very simple test app!
>>
>> You are also using the BIO connector which, since it is blocking,
>> doesn't offer any benefits when using async. You'd be better off with
>> the NIO connector.
>
> Yes I know but currently it's not important for me. I am studying
> Servlet 3's async API and BIO keeps it simpler to study and focus only
> on it (with NIO I cannot know if something is because of Servlet 3's
> async API or Tomcat's NIO).
>
>>
>> You haven't told us which Tomcat version you are using. Since you are
>> using BIO that narrows it down a bit to 7.0.x or 8.0.x but that is still
>> a lot of possibilities.
>
> It's 7.0.47

You are unlikely to get much interest on this list until you upgrade to
the latest stable 7.0.x (or 8.0.x). So much has changed in the ~4 years
since 7.0.47 that there isn't much value in investigating this. If you
see the same or similar issues with 7.0.81, that would be more interesting.

Mark


>
>>
>> Neither have you told us what operating system you are using. My
>> experience of JMeter under load, particularly on Windows, is that you
>> see strange behaviour and it can be hard to figure out the interaction
>> of JMeter, the OS network stack and Tomcat.
>>
>> You also haven't told us what hardware this test is running on.
>> Particularly the number of cores available.
>>
>
> OS Name Microsoft Windows 8.1 Enterprise
> System Type x64-based PC
> Processor Intel(R) Core(TM) i3-4130 CPU @ 3.40GHz, 3400 Mhz, 2 Core(s),
> 4 Logical Processor(s)
>
>
>>> I have following JMeter test plan:
>>> Number of threads (users): 700
>>> Ramp-Up period (in seconds): 23
>>> Loop count: 1
>>>
>>> So JMeter generates 30.43 requests per second and 304.3 requests per 10
>>> seconds. I'm planning to full tomcat's BIO pool and accept buffer :)
>>>
>>> I have an always async test servlet which on destroy, I print tomcat
>>> container max used threads count. It prints 187 for me which is lower
>>> than 200 (tomcat default pool size) so I should not get any "connection
>>> refuse" but I get!
>>
>> There are all sorts of possible reasons for that. I'd suggest scaling
>> down the test. Limit Tomcat to 20 threads. Reduce the load similarly.
>> Increase the sleep time. You want to ensure that the only limit you are
>> hitting is the one you are trying to hit.
>>
>
> Previously I also tested very low loads but again as you suggested I
> tested following low load configuration and get even worse results!!
> Tomcat successfully returns from request below 29th but fails 25
> requests of 29th to 70th. However this time all fails are "connection
> refuse" and there are not any "connection reset". Whilst the program
> output is "Container MAX used threads: 10" !!
>
> CONFIGURATION #2:
>
> Server.xml
> <Connector port="7780" protocol="org.apache.coyote.http11.Http11Protocol"
>                 connectionTimeout="120000"
>   maxThreads="20" maxConnections="20" acceptCount="10"
>                 redirectPort="7743" />
>
> JMeter
> Number of threads (users): 70
> Ramp-Up period (in seconds): 35 (40 requests per 20 seconds)
> Loop count: 1
>
> My async servlet
> Async Sleep Time: 20 seconds (ensures 40 concurrent requests)
> It's own thread pool size: 41 (lower than 40 so never fulls)
>
> JMETER RESULT of RESPONSE TIMES #2:
> Max: 38 seconds (lower then tomcat and asyncContext timeout)
> MIN: 20 seconds
> AVG: 18 seconds (because of fails)
> ERR: 36%
>
> UTPUT:
>
> Container MAX used threads: 10
>
> Thanks in advance!
>
>> Mark
>>
>>
>>> I have simulated a blocking operation by a sleep for 10 seconds. When my
>>> servlet gets a request, it quickly starts an async and add further
>>> processing to my own thread pool (container thread comes back to pool
>>> quickly, right). My own thread pool size is 310 which is greater than
>>> 304.3 (requests in 10 seconds) so never full.
>>>
>>> I've tested several times. Tomcat successfully returns from all requests
>>> below 326th but fails 102 requests from 326th to 700th with "connection
>>> refuse" and afew with "connection reset".
>>>
>>> Why?! My own thread pool does the jobs and Tomcat's pool is free (my
>>> servlet uses 187 threads of tomcat at max).
>>>
>>> Thanks in advance!
>>>
>>> JMETER RESULT of RESPONSE TIMES:
>>> Max: 60 seconds (lower then tomcat and asyncContext timeout)
>>> MIN: 10 seconds
>>> AVG: 37 seconds
>>> ERR: 15%
>>>
>>> CONFIGURATIONS:
>>>
>>> Server.xml
>>> <Connector port="7780" protocol="org.apache.coyote.http11.Http11Protocol"
>>>                 connectionTimeout="120000"
>>>                 redirectPort="7743" />
>>>
>>> Async.java
>>>
>>> package com.sstr.example;
>>>
>>> import javax.servlet.*;
>>> import javax.servlet.annotation.WebInitParam;
>>> import javax.servlet.annotation.WebServlet;
>>> import javax.servlet.http.HttpServlet;
>>> import javax.servlet.http.HttpServletRequest;
>>> import javax.servlet.http.HttpServletResponse;
>>> import java.io.IOException;
>>> import java.util.concurrent.ExecutorService;
>>> import java.util.concurrent.Executors;
>>> import java.util.concurrent.ThreadFactory;
>>>
>>> @WebServlet(
>>>          name = "async",
>>>          value = {"/async"},
>>>          asyncSupported = true,
>>>          initParams = {
>>>                  @WebInitParam(name = "JobPoolSize", value = "310")
>>>          }
>>> )
>>> public class Async extends HttpServlet {
>>>
>>>      public final int REQUEST_TIMEOUT = 120000;
>>>      private ExecutorService exe;
>>>
>>>      @Override
>>>      public void init() throws ServletException {
>>>          int size = Integer.parseInt(getInitParameter("JobPoolSize"));
>>>          exe = Executors.newFixedThreadPool(
>>>                  size,
>>>                  new ThreadFactory() {
>>>                      @Override
>>>                      public Thread newThread(Runnable r) {
>>>                          return new Thread(r, "Async Processor");
>>>                      }
>>>                  }
>>>          );
>>>      }
>>>
>>>      @Override
>>>      protected void doGet(HttpServletRequest req, HttpServletResponse
>>> resp) throws ServletException, IOException {
>>>          final AsyncContext context = req.startAsync();
>>>          context.setTimeout(REQUEST_TIMEOUT);
>>>          exe.execute(new ContextExecution(context,
>>> Thread.currentThread().getName()));
>>>      }
>>>
>>>      @Override
>>>      public void destroy() {
>>>          System.out.println("Container MAX used threads: " + threadCount);
>>>          exe.shutdown();
>>>      }
>>>
>>>      int threadCount = 0;
>>>      class ContextExecution implements Runnable {
>>>
>>>          final AsyncContext context;
>>>          final String containerThreadName;
>>>
>>>          public ContextExecution(AsyncContext context, String
>>> containerThreadName) {
>>>              this.context = context;
>>>              this.containerThreadName = containerThreadName;
>>>          }
>>>
>>>          @Override
>>>          public void run() {
>>>              try {
>>>                  int threadNumber =
>>> Integer.parseInt(containerThreadName.substring(
>>>                          containerThreadName.lastIndexOf('-')+1));
>>>                  if(threadNumber > threadCount) {
>>>                      threadCount = threadNumber;
>>>                  }
>>>
>>>                  // Simulate Time Consuming Task
>>>                  Thread.sleep(10000);
>>>
>>>                  ServletResponse resp = context.getResponse();
>>>                  if (resp != null) {
>>>                      resp.getWriter().write("Ok");
>>>                  }
>>>
>>>                  context.complete();
>>>              } catch (Exception e) {
>>>                  // Handle ?
>>>              }
>>>          }
>>>      }
>>> }
>>>
>>> OUTPUT:
>>>
>>> Container MAX used threads: 187
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: [hidden email]
>>> For additional commands, e-mail: [hidden email]
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: BIO: Async servlet with it's own thread pool; but get connection refused!

Yasser Zamani
Thanks for your attention.

Now I downloaded a fresh apache-tomcat-7.0.81-windows-x64 and chenged
it's connector in same way (BIO,20,20,10). I get same result, fortunately :)

OUTPUT:

Using CATALINA_BASE:
"C:\Users\user\.IntelliJIdea2016.3\system\tomcat\Unnamed_Async-Servlet-Example_2"
Using CATALINA_HOME:
"C:\Users\user\Downloads\apache-tomcat-7.0.81-windows-x64-IJ\apache-tomcat-7.0.81"
Using CATALINA_TMPDIR:
"C:\Users\user\Downloads\apache-tomcat-7.0.81-windows-x64-IJ\apache-tomcat-7.0.81\temp"
Using JRE_HOME:        "E:\jdk1.7.0_79"
INFO: Server version:        Apache Tomcat/7.0.81
INFO: Server built:          Aug 11 2017 10:21:27 UTC
INFO: Server number:         7.0.81.0
INFO: OS Name:               Windows 8.1
INFO: OS Version:            6.3
INFO: Architecture:          amd64
INFO: Java Home:             E:\jdk1.7.0_79\jre
INFO: JVM Version:           1.7.0_79-b15
INFO: JVM Vendor:            Oracle Corporation
INFO: CATALINA_BASE:
C:\Users\user\.IntelliJIdea2016.3\system\tomcat\Unnamed_Async-Servlet-Example_2

Container MAX used threads: 10

Sincerely Yours,
Yasser.

On 9/8/2017 2:30 AM, Mark Thomas wrote:

> On 07/09/17 22:22, Yasser Zamani wrote:
>> At first thanks a lot for your reply!
>>
>> On 9/7/2017 1:43 PM, Mark Thomas wrote:
>>> On 06/09/17 20:59, Yasser Zamani wrote:
>>>> Hi there,
>>>>
>>>> I'm studying Servlet 3's async API using tomcat. I see following strange
>>>> behavior from tomcat in a very simple test app!
>>>
>>> You are also using the BIO connector which, since it is blocking,
>>> doesn't offer any benefits when using async. You'd be better off with
>>> the NIO connector.
>>
>> Yes I know but currently it's not important for me. I am studying
>> Servlet 3's async API and BIO keeps it simpler to study and focus only
>> on it (with NIO I cannot know if something is because of Servlet 3's
>> async API or Tomcat's NIO).
>>
>>>
>>> You haven't told us which Tomcat version you are using. Since you are
>>> using BIO that narrows it down a bit to 7.0.x or 8.0.x but that is still
>>> a lot of possibilities.
>>
>> It's 7.0.47
>
> You are unlikely to get much interest on this list until you upgrade to
> the latest stable 7.0.x (or 8.0.x). So much has changed in the ~4 years
> since 7.0.47 that there isn't much value in investigating this. If you
> see the same or similar issues with 7.0.81, that would be more interesting.
>
> Mark
>
>
>>
>>>
>>> Neither have you told us what operating system you are using. My
>>> experience of JMeter under load, particularly on Windows, is that you
>>> see strange behaviour and it can be hard to figure out the interaction
>>> of JMeter, the OS network stack and Tomcat.
>>>
>>> You also haven't told us what hardware this test is running on.
>>> Particularly the number of cores available.
>>>
>>
>> OS Name Microsoft Windows 8.1 Enterprise
>> System Type x64-based PC
>> Processor Intel(R) Core(TM) i3-4130 CPU @ 3.40GHz, 3400 Mhz, 2 Core(s),
>> 4 Logical Processor(s)
>>
>>
>>>> I have following JMeter test plan:
>>>> Number of threads (users): 700
>>>> Ramp-Up period (in seconds): 23
>>>> Loop count: 1
>>>>
>>>> So JMeter generates 30.43 requests per second and 304.3 requests per 10
>>>> seconds. I'm planning to full tomcat's BIO pool and accept buffer :)
>>>>
>>>> I have an always async test servlet which on destroy, I print tomcat
>>>> container max used threads count. It prints 187 for me which is lower
>>>> than 200 (tomcat default pool size) so I should not get any "connection
>>>> refuse" but I get!
>>>
>>> There are all sorts of possible reasons for that. I'd suggest scaling
>>> down the test. Limit Tomcat to 20 threads. Reduce the load similarly.
>>> Increase the sleep time. You want to ensure that the only limit you are
>>> hitting is the one you are trying to hit.
>>>
>>
>> Previously I also tested very low loads but again as you suggested I
>> tested following low load configuration and get even worse results!!
>> Tomcat successfully returns from request below 29th but fails 25
>> requests of 29th to 70th. However this time all fails are "connection
>> refuse" and there are not any "connection reset". Whilst the program
>> output is "Container MAX used threads: 10" !!
>>
>> CONFIGURATION #2:
>>
>> Server.xml
>> <Connector port="7780" protocol="org.apache.coyote.http11.Http11Protocol"
>>                 connectionTimeout="120000"
>>   maxThreads="20" maxConnections="20" acceptCount="10"
>>                 redirectPort="7743" />
>>
>> JMeter
>> Number of threads (users): 70
>> Ramp-Up period (in seconds): 35 (40 requests per 20 seconds)
>> Loop count: 1
>>
>> My async servlet
>> Async Sleep Time: 20 seconds (ensures 40 concurrent requests)
>> It's own thread pool size: 41 (lower than 40 so never fulls)
>>
>> JMETER RESULT of RESPONSE TIMES #2:
>> Max: 38 seconds (lower then tomcat and asyncContext timeout)
>> MIN: 20 seconds
>> AVG: 18 seconds (because of fails)
>> ERR: 36%
>>
>> UTPUT:
>>
>> Container MAX used threads: 10
>>
>> Thanks in advance!
>>
>>> Mark
>>>
>>>
>>>> I have simulated a blocking operation by a sleep for 10 seconds. When my
>>>> servlet gets a request, it quickly starts an async and add further
>>>> processing to my own thread pool (container thread comes back to pool
>>>> quickly, right). My own thread pool size is 310 which is greater than
>>>> 304.3 (requests in 10 seconds) so never full.
>>>>
>>>> I've tested several times. Tomcat successfully returns from all requests
>>>> below 326th but fails 102 requests from 326th to 700th with "connection
>>>> refuse" and afew with "connection reset".
>>>>
>>>> Why?! My own thread pool does the jobs and Tomcat's pool is free (my
>>>> servlet uses 187 threads of tomcat at max).
>>>>
>>>> Thanks in advance!
>>>>
>>>> JMETER RESULT of RESPONSE TIMES:
>>>> Max: 60 seconds (lower then tomcat and asyncContext timeout)
>>>> MIN: 10 seconds
>>>> AVG: 37 seconds
>>>> ERR: 15%
>>>>
>>>> CONFIGURATIONS:
>>>>
>>>> Server.xml
>>>> <Connector port="7780" protocol="org.apache.coyote.http11.Http11Protocol"
>>>>                 connectionTimeout="120000"
>>>>                 redirectPort="7743" />
>>>>
>>>> Async.java
>>>>
>>>> package com.sstr.example;
>>>>
>>>> import javax.servlet.*;
>>>> import javax.servlet.annotation.WebInitParam;
>>>> import javax.servlet.annotation.WebServlet;
>>>> import javax.servlet.http.HttpServlet;
>>>> import javax.servlet.http.HttpServletRequest;
>>>> import javax.servlet.http.HttpServletResponse;
>>>> import java.io.IOException;
>>>> import java.util.concurrent.ExecutorService;
>>>> import java.util.concurrent.Executors;
>>>> import java.util.concurrent.ThreadFactory;
>>>>
>>>> @WebServlet(
>>>>          name = "async",
>>>>          value = {"/async"},
>>>>          asyncSupported = true,
>>>>          initParams = {
>>>>                  @WebInitParam(name = "JobPoolSize", value = "310")
>>>>          }
>>>> )
>>>> public class Async extends HttpServlet {
>>>>
>>>>      public final int REQUEST_TIMEOUT = 120000;
>>>>      private ExecutorService exe;
>>>>
>>>>      @Override
>>>>      public void init() throws ServletException {
>>>>          int size = Integer.parseInt(getInitParameter("JobPoolSize"));
>>>>          exe = Executors.newFixedThreadPool(
>>>>                  size,
>>>>                  new ThreadFactory() {
>>>>                      @Override
>>>>                      public Thread newThread(Runnable r) {
>>>>                          return new Thread(r, "Async Processor");
>>>>                      }
>>>>                  }
>>>>          );
>>>>      }
>>>>
>>>>      @Override
>>>>      protected void doGet(HttpServletRequest req, HttpServletResponse
>>>> resp) throws ServletException, IOException {
>>>>          final AsyncContext context = req.startAsync();
>>>>          context.setTimeout(REQUEST_TIMEOUT);
>>>>          exe.execute(new ContextExecution(context,
>>>> Thread.currentThread().getName()));
>>>>      }
>>>>
>>>>      @Override
>>>>      public void destroy() {
>>>>          System.out.println("Container MAX used threads: " + threadCount);
>>>>          exe.shutdown();
>>>>      }
>>>>
>>>>      int threadCount = 0;
>>>>      class ContextExecution implements Runnable {
>>>>
>>>>          final AsyncContext context;
>>>>          final String containerThreadName;
>>>>
>>>>          public ContextExecution(AsyncContext context, String
>>>> containerThreadName) {
>>>>              this.context = context;
>>>>              this.containerThreadName = containerThreadName;
>>>>          }
>>>>
>>>>          @Override
>>>>          public void run() {
>>>>              try {
>>>>                  int threadNumber =
>>>> Integer.parseInt(containerThreadName.substring(
>>>>                          containerThreadName.lastIndexOf('-')+1));
>>>>                  if(threadNumber > threadCount) {
>>>>                      threadCount = threadNumber;
>>>>                  }
>>>>
>>>>                  // Simulate Time Consuming Task
>>>>                  Thread.sleep(10000);
>>>>
>>>>                  ServletResponse resp = context.getResponse();
>>>>                  if (resp != null) {
>>>>                      resp.getWriter().write("Ok");
>>>>                  }
>>>>
>>>>                  context.complete();
>>>>              } catch (Exception e) {
>>>>                  // Handle ?
>>>>              }
>>>>          }
>>>>      }
>>>> }
>>>>
>>>> OUTPUT:
>>>>
>>>> Container MAX used threads: 187
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: [hidden email]
>>>> For additional commands, e-mail: [hidden email]
>>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: [hidden email]
>>> For additional commands, e-mail: [hidden email]
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: BIO: Async servlet with it's own thread pool; but get connection refused!

Yasser Zamani
In reply to this post by Guang Chao


On 9/7/2017 12:15 PM, Guang Chao wrote:

> On Thu, Sep 7, 2017 at 3:59 AM, Yasser Zamani <[hidden email]>
> wrote:
>
>> Hi there,
>>
>> I'm studying Servlet 3's async API using tomcat. I see following strange
>> behavior from tomcat in a very simple test app!
>>
>> I have following JMeter test plan:
>> Number of threads (users): 700
>> Ramp-Up period (in seconds): 23
>> Loop count: 1
>>
>> So JMeter generates 30.43 requests per second and 304.3 requests per 10
>> seconds. I'm planning to full tomcat's BIO pool and accept buffer :)
>>
>> I have an always async test servlet which on destroy, I print tomcat
>> container max used threads count. It prints 187 for me which is lower
>> than 200 (tomcat default pool size) so I should not get any "connection
>> refuse" but I get!
>>
>> I have simulated a blocking operation by a sleep for 10 seconds. When my
>> servlet gets a request, it quickly starts an async and add further
>> processing to my own thread pool (container thread comes back to pool
>> quickly, right). My own thread pool size is 310 which is greater than
>> 304.3 (requests in 10 seconds) so never full.
>>
>> I've tested several times. Tomcat successfully returns from all requests
>> below 326th but fails 102 requests from 326th to 700th with "connection
>> refuse" and afew with "connection reset".
>>
>> Why?! My own thread pool does the jobs and Tomcat's pool is free (my
>> servlet uses 187 threads of tomcat at max).
>>
>> Thanks in advance!
>>
>> JMETER RESULT of RESPONSE TIMES:
>> Max: 60 seconds (lower then tomcat and asyncContext timeout)
>> MIN: 10 seconds
>> AVG: 37 seconds
>> ERR: 15%
>>
>> CONFIGURATIONS:
>>
>> Server.xml
>> <Connector port="7780" protocol="org.apache.coyote.http11.Http11Protocol"
>>                 connectionTimeout="120000"
>>                 redirectPort="7743" />
>>
>> Async.java
>>
>> package com.sstr.example;
>>
>> import javax.servlet.*;
>> import javax.servlet.annotation.WebInitParam;
>> import javax.servlet.annotation.WebServlet;
>> import javax.servlet.http.HttpServlet;
>> import javax.servlet.http.HttpServletRequest;
>> import javax.servlet.http.HttpServletResponse;
>> import java.io.IOException;
>> import java.util.concurrent.ExecutorService;
>> import java.util.concurrent.Executors;
>> import java.util.concurrent.ThreadFactory;
>>
>> @WebServlet(
>>          name = "async",
>>          value = {"/async"},
>>          asyncSupported = true,
>>          initParams = {
>>                  @WebInitParam(name = "JobPoolSize", value = "310")
>>          }
>> )
>> public class Async extends HttpServlet {
>>
>>      public final int REQUEST_TIMEOUT = 120000;
>>      private ExecutorService exe;
>>
>>      @Override
>>      public void init() throws ServletException {
>>          int size = Integer.parseInt(getInitParameter("JobPoolSize"));
>>          exe = Executors.newFixedThreadPool(
>>                  size,
>>                  new ThreadFactory() {
>>                      @Override
>>                      public Thread newThread(Runnable r) {
>>                          return new Thread(r, "Async Processor");
>>                      }
>>                  }
>>          );
>>      }
>>
>>      @Override
>>      protected void doGet(HttpServletRequest req, HttpServletResponse
>> resp) throws ServletException, IOException {
>>          final AsyncContext context = req.startAsync();
>>          context.setTimeout(REQUEST_TIMEOUT);
>>          exe.execute(new ContextExecution(context,
>> Thread.currentThread().getName()));
>>      }
>>
>
> I'm not 100% sure, but it seems the doGet method code here is not correct.

Maybe you mean I should startAsync() inside Servlet's thread pool rather
than tomcat's thread?

>
>
>>
>>      @Override
>>      public void destroy() {
>>          System.out.println("Container MAX used threads: " + threadCount);
>>          exe.shutdown();
>>      }
>>
>>      int threadCount = 0;
>>      class ContextExecution implements Runnable {
>>
>>          final AsyncContext context;
>>          final String containerThreadName;
>>
>>          public ContextExecution(AsyncContext context, String
>> containerThreadName) {
>>              this.context = context;
>>              this.containerThreadName = containerThreadName;
>>          }
>>
>>          @Override
>>          public void run() {
>>              try {
>>                  int threadNumber =
>> Integer.parseInt(containerThreadName.substring(
>>                          containerThreadName.lastIndexOf('-')+1));
>>                  if(threadNumber > threadCount) {
>>                      threadCount = threadNumber;
>>                  }
>>
>>                  // Simulate Time Consuming Task
>>                  Thread.sleep(10000);
>>
>>                  ServletResponse resp = context.getResponse();
>>                  if (resp != null) {
>>                      resp.getWriter().write("Ok");
>>                  }
>>
>>                  context.complete();
>>              } catch (Exception e) {
>>                  // Handle ?
>>              }
>>          }
>>      }
>> }
>>
>> OUTPUT:
>>
>> Container MAX used threads: 187
>>
>
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: BIO: Async servlet with it's own thread pool; but get connection refused!

Mark Thomas-2
In reply to this post by Yasser Zamani
On 07/09/17 23:30, Yasser Zamani wrote:

> Thanks for your attention.
>
> Now I downloaded a fresh apache-tomcat-7.0.81-windows-x64 and chenged
> it's connector in same way (BIO,20,20,10). I get same result, fortunately :)
>
> OUTPUT:
>
> Using CATALINA_BASE:
> "C:\Users\user\.IntelliJIdea2016.3\system\tomcat\Unnamed_Async-Servlet-Example_2"
> Using CATALINA_HOME:
> "C:\Users\user\Downloads\apache-tomcat-7.0.81-windows-x64-IJ\apache-tomcat-7.0.81"
> Using CATALINA_TMPDIR:
> "C:\Users\user\Downloads\apache-tomcat-7.0.81-windows-x64-IJ\apache-tomcat-7.0.81\temp"
> Using JRE_HOME:        "E:\jdk1.7.0_79"
> INFO: Server version:        Apache Tomcat/7.0.81
> INFO: Server built:          Aug 11 2017 10:21:27 UTC
> INFO: Server number:         7.0.81.0
> INFO: OS Name:               Windows 8.1
> INFO: OS Version:            6.3
> INFO: Architecture:          amd64
> INFO: Java Home:             E:\jdk1.7.0_79\jre
> INFO: JVM Version:           1.7.0_79-b15
> INFO: JVM Vendor:            Oracle Corporation
> INFO: CATALINA_BASE:
> C:\Users\user\.IntelliJIdea2016.3\system\tomcat\Unnamed_Async-Servlet-Example_2
>
> Container MAX used threads: 10

I see similar results.

There looks to be things going on either in JMeter or at the network
level I don't understand. I had to resort to drawing it out to get my
head around what is happening.

The first 20 requests (10 seconds) are accepted and processed by Tomcat.
The 21st request is accepted but then the acceptor blocks waiting for
the connection count to reduce below 20 before proceeding.

Requests 22 to 31 are placed in the accept queue. We are now 15.5s into
the test and the first request accepted won't finish processing for
another 4.5 seconds.

Requests 32 to 40 are dropped since the request queue is full. We are
now 20s into the test and the first request is about to complete
processing. Oddly, JMeter doesn't report these as failed until some 35
seconds later.

Request 1 completes. This allows request 21 to proceed. The acceptor
takes a connection from the accept queue (this appears to be FIFO).
Request 41 enters the accept queue.

The continues until request 10 completes, 30 starts processing and 50
enters the accept queue.

Next 11 completes, 41 starts processing and 51 enters the accept queue.
This continues until 20 completes, 50 starts processing and 60 enters
the accept queue.

At this point there are 20 threads processing, 10 in the accept queue
and no thread due to complete for anther 10s.

I'd expected requests 61 to 70 to be rejected. However, 65 to 70 are
processed. It looks like there is some sort of timeout for acceptance or
rejection in the accept queue.

That explains the rejected requests.

The other question is why maxThreads is reported as it is.

The answer is that the thread pool never grows beyond its initial size
of 10. A request comes in, it is processed by a container thread,
dispatched to an async thread and then the container thread is returned
to the pool to await the next request. Tomcat is able to do this because
the container doesn't perform any I/O on the connection once it enters
async mode until it is dispatched back to the container. The default
thread pool implementation cycles through the threads so the max value
you see is 10 which is the initial size.

HTH,

Mark

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: BIO: Async servlet with it's own thread pool; but get connection refused!

Yasser Zamani


On 9/12/2017 1:17 AM, Mark Thomas wrote:

> On 07/09/17 23:30, Yasser Zamani wrote:
>> Thanks for your attention.
>>
>> Now I downloaded a fresh apache-tomcat-7.0.81-windows-x64 and chenged
>> it's connector in same way (BIO,20,20,10). I get same result, fortunately :)
>>
>> OUTPUT:
>>
>> Using CATALINA_BASE:
>> "C:\Users\user\.IntelliJIdea2016.3\system\tomcat\Unnamed_Async-Servlet-Example_2"
>> Using CATALINA_HOME:
>> "C:\Users\user\Downloads\apache-tomcat-7.0.81-windows-x64-IJ\apache-tomcat-7.0.81"
>> Using CATALINA_TMPDIR:
>> "C:\Users\user\Downloads\apache-tomcat-7.0.81-windows-x64-IJ\apache-tomcat-7.0.81\temp"
>> Using JRE_HOME:        "E:\jdk1.7.0_79"
>> INFO: Server version:        Apache Tomcat/7.0.81
>> INFO: Server built:          Aug 11 2017 10:21:27 UTC
>> INFO: Server number:         7.0.81.0
>> INFO: OS Name:               Windows 8.1
>> INFO: OS Version:            6.3
>> INFO: Architecture:          amd64
>> INFO: Java Home:             E:\jdk1.7.0_79\jre
>> INFO: JVM Version:           1.7.0_79-b15
>> INFO: JVM Vendor:            Oracle Corporation
>> INFO: CATALINA_BASE:
>> C:\Users\user\.IntelliJIdea2016.3\system\tomcat\Unnamed_Async-Servlet-Example_2
>>
>> Container MAX used threads: 10
>
> I see similar results.
>
> There looks to be things going on either in JMeter or at the network
> level I don't understand. I had to resort to drawing it out to get my
> head around what is happening.
>

Sorry for bothering you,

To examine if things going on either in JMeter or at the network, I
tested same config (BIO,20,20,10) on Jetty. All 70 requests returned
successfully and response time was ~20 seconds for all, as I expects.

Then to make myself sure, I tested same servlet but a sync one (removed
my own thread pool and asyncStart etc) on Jetty. Average response time
increased to 95s, as I expected.

> The first 20 requests (10 seconds) are accepted and processed by Tomcat.
> The 21st request is accepted but then the acceptor blocks waiting for
> the connection count to reduce below 20 before proceeding.
>

You have forgot. My configuration is maxThreads=maxConnections=20 (not
10) and acceptCount=10. As it prints "Container MAX used threads: 10" so
it never reaches maxThreads. So why acceptor blocks ?! I think although
I have an async servlet and own thread pool, and although time consuming
process (Thread.sleep) is inside my own thread pool, but Tomcat's
container thread wrongly does not back to thread pool and fails to
satisfy Servlet 3's async API!

> Requests 22 to 31 are placed in the accept queue. We are now 15.5s into
> the test and the first request accepted won't finish processing for
> another 4.5 seconds.
>
> Requests 32 to 40 are dropped since the request queue is full. We are
> now 20s into the test and the first request is about to complete
> processing. Oddly, JMeter doesn't report these as failed until some 35
> seconds later.
>
> Request 1 completes. This allows request 21 to proceed. The acceptor
> takes a connection from the accept queue (this appears to be FIFO).
> Request 41 enters the accept queue.
>
> The continues until request 10 completes, 30 starts processing and 50
> enters the accept queue.
>
> Next 11 completes, 41 starts processing and 51 enters the accept queue.
> This continues until 20 completes, 50 starts processing and 60 enters
> the accept queue.
>
> At this point there are 20 threads processing, 10 in the accept queue
> and no thread due to complete for anther 10s.
>
> I'd expected requests 61 to 70 to be rejected. However, 65 to 70 are
> processed. It looks like there is some sort of timeout for acceptance or
> rejection in the accept queue.
>
> That explains the rejected requests.
>

I'm not such smart to understand your analysis :) I just understand when
my servlet is completely async, then I should not have any rejected
requests (specially in such low load) like Jetty result. As I said, max
Tomcat container's threads used by my APP is 10 (half of initial size
20) so why I see rejected requests while thread pool has 10 free threads
to accept new requests ?!

> The other question is why maxThreads is reported as it is.
>
> The answer is that the thread pool never grows beyond its initial size
> of 10. A request comes in, it is processed by a container thread,
> dispatched to an async thread and then the container thread is returned
> to the pool to await the next request. Tomcat is able to do this because
> the container doesn't perform any I/O on the connection once it enters
> async mode until it is dispatched back to the container. The default
> thread pool implementation cycles through the threads so the max value
> you see is 10 which is the initial size.
>

My config was maxThreads=20 (not 10).

Sincerely Yours,
Yasser.

> HTH,
>
> Mark
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: BIO: Async servlet with it's own thread pool; but get connection refused!

Mark Thomas-2
On 12/09/17 10:00, Yasser Zamani wrote:

>
>
> On 9/12/2017 1:17 AM, Mark Thomas wrote:
>> On 07/09/17 23:30, Yasser Zamani wrote:
>>> Thanks for your attention.
>>>
>>> Now I downloaded a fresh apache-tomcat-7.0.81-windows-x64 and chenged
>>> it's connector in same way (BIO,20,20,10). I get same result, fortunately :)
>>>
>>> OUTPUT:
>>>
>>> Using CATALINA_BASE:
>>> "C:\Users\user\.IntelliJIdea2016.3\system\tomcat\Unnamed_Async-Servlet-Example_2"
>>> Using CATALINA_HOME:
>>> "C:\Users\user\Downloads\apache-tomcat-7.0.81-windows-x64-IJ\apache-tomcat-7.0.81"
>>> Using CATALINA_TMPDIR:
>>> "C:\Users\user\Downloads\apache-tomcat-7.0.81-windows-x64-IJ\apache-tomcat-7.0.81\temp"
>>> Using JRE_HOME:        "E:\jdk1.7.0_79"
>>> INFO: Server version:        Apache Tomcat/7.0.81
>>> INFO: Server built:          Aug 11 2017 10:21:27 UTC
>>> INFO: Server number:         7.0.81.0
>>> INFO: OS Name:               Windows 8.1
>>> INFO: OS Version:            6.3
>>> INFO: Architecture:          amd64
>>> INFO: Java Home:             E:\jdk1.7.0_79\jre
>>> INFO: JVM Version:           1.7.0_79-b15
>>> INFO: JVM Vendor:            Oracle Corporation
>>> INFO: CATALINA_BASE:
>>> C:\Users\user\.IntelliJIdea2016.3\system\tomcat\Unnamed_Async-Servlet-Example_2
>>>
>>> Container MAX used threads: 10
>>
>> I see similar results.
>>
>> There looks to be things going on either in JMeter or at the network
>> level I don't understand. I had to resort to drawing it out to get my
>> head around what is happening.
>>
>
> Sorry for bothering you,
>
> To examine if things going on either in JMeter or at the network, I
> tested same config (BIO,20,20,10) on Jetty. All 70 requests returned
> successfully and response time was ~20 seconds for all, as I expects.

I'm fairly sure Jetty uses NIO, not BIO which would explain the
differences you are observing.

> Then to make myself sure, I tested same servlet but a sync one (removed
> my own thread pool and asyncStart etc) on Jetty. Average response time
> increased to 95s, as I expected.
>
>> The first 20 requests (10 seconds) are accepted and processed by Tomcat.
>> The 21st request is accepted but then the acceptor blocks waiting for
>> the connection count to reduce below 20 before proceeding.
>>
>
> You have forgot.

No I haven't.

> My configuration is maxThreads=maxConnections=20 (not
> 10) and acceptCount=10. As it prints "Container MAX used threads: 10" so
> it never reaches maxThreads. So why acceptor blocks ?!

After the first 20 requests you have 20 connections so you hit the
maxConnection limit. That is why the acceptor blocks and subsequent
requests go into the accept queue.

> I think although
> I have an async servlet and own thread pool, and although time consuming
> process (Thread.sleep) is inside my own thread pool, but Tomcat's
> container thread wrongly does not back to thread pool

Incorrect. The container thread does return to the container thread
pool. It does so almost immediately. Given that there are 0.5 seconds
between requests and that the time taken to process an incoming request,
dispatch the request to your thread pool and return the container thread
to the container thread pool is almost certainly less than 0.5 it is
very likely that there is never more than one container thread active at
any one point.

> and fails to> satisfy Servlet 3's async API!

Also incorrect.

>> Requests 22 to 31 are placed in the accept queue. We are now 15.5s into
>> the test and the first request accepted won't finish processing for
>> another 4.5 seconds.
>>
>> Requests 32 to 40 are dropped since the request queue is full. We are
>> now 20s into the test and the first request is about to complete
>> processing. Oddly, JMeter doesn't report these as failed until some 35
>> seconds later.
>>
>> Request 1 completes. This allows request 21 to proceed. The acceptor
>> takes a connection from the accept queue (this appears to be FIFO).
>> Request 41 enters the accept queue.
>>
>> The continues until request 10 completes, 30 starts processing and 50
>> enters the accept queue.
>>
>> Next 11 completes, 41 starts processing and 51 enters the accept queue.
>> This continues until 20 completes, 50 starts processing and 60 enters
>> the accept queue.
>>
>> At this point there are 20 threads processing, 10 in the accept queue
>> and no thread due to complete for anther 10s.
>>
>> I'd expected requests 61 to 70 to be rejected. However, 65 to 70 are
>> processed. It looks like there is some sort of timeout for acceptance or
>> rejection in the accept queue.
>>
>> That explains the rejected requests.
>>
>
> I'm not such smart to understand your analysis :) I just understand when
> my servlet is completely async, then I should not have any rejected
> requests (specially in such low load) like Jetty result.

You are failing to take account of the maxConnections configuration.
With a new request being made every 0.5 seconds and a wait time of 20
seconds Tomcat needs to be able to handle at least 40 concurrent
connections to prevent dropped connections. You have configured Tomcat
to allow 30 (20 from maxConnections and 10 in the accept queue). Hence
you are going to see dropped connections.


> As I said, max
> Tomcat container's threads used by my APP is 10 (half of initial size
> 20) so why I see rejected requests while thread pool has 10 free threads
> to accept new requests ?!
>
>> The other question is why maxThreads is reported as it is.
>>
>> The answer is that the thread pool never grows beyond its initial size
>> of 10. A request comes in, it is processed by a container thread,
>> dispatched to an async thread and then the container thread is returned
>> to the pool to await the next request. Tomcat is able to do this because
>> the container doesn't perform any I/O on the connection once it enters
>> async mode until it is dispatched back to the container. The default
>> thread pool implementation cycles through the threads so the max value
>> you see is 10 which is the initial size.
>>
>
> My config was maxThreads=20 (not 10).

Irrelevant. The container threads aren't doing very much. As I explained
above it is unlikely that there is ever more than one container thread
doing work at any one time. Tomcat's thread pool has an INITIAL size of
10 that only grows if more than 10 concurrent threads are required.
Given that your test case only requires one concurrent container thread
(NOT 20 because you are using async so the container thread is freed as
soon as the async dispatch completes) the thread pool never grows. What
happens is:

The thread pool is a FIFO pool.
At the start:
Pool: 1 2 3 4 5 6 7 8 9 10
Thread 1 processes request 1 and returns to the pool.
Pool: 2 3 4 5 6 7 8 9 10 1
Thread 2 processes request 2 and returns to the pool.
Pool: 3 4 5 6 7 8 9 10 1 2
...
Pool: 10 1 2 3 4 5 6 7 8 9
Thread 10 processes request 10 and returns to the pool.
Pool: 1 2 3 4 5 6 7 8 9 10
Thread 1 processes request 11 and returns to the pool.
...

Hence the highest container thread number you see in your test is 10.

You are making the incorrect assumption that the highest thread number
you see always represents the highest concurrency. That assumption is
not true when maximum concurrency is less than the initial size of the
thread pool.

Mark

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: BIO: Async servlet with it's own thread pool; but get connection refused!

Yasser Zamani
Ouch! maxConnections! You're right. I failed to take account of it
correctly. Actually I was confused connections like threads (because of
maxThreads/Connections) and thought they release when their thread
releases :[ while a connection keeps alive until response and thread
pool is shared between all of them to service them on demand.

I'm really sorry that I bothered you a lot!

Thanks a tone for your replies!

Regards.

On 9/12/2017 1:51 PM, Mark Thomas wrote:

> On 12/09/17 10:00, Yasser Zamani wrote:
>>
>>
>> On 9/12/2017 1:17 AM, Mark Thomas wrote:
>>> On 07/09/17 23:30, Yasser Zamani wrote:
>>>> Thanks for your attention.
>>>>
>>>> Now I downloaded a fresh apache-tomcat-7.0.81-windows-x64 and chenged
>>>> it's connector in same way (BIO,20,20,10). I get same result, fortunately :)
>>>>
>>>> OUTPUT:
>>>>
>>>> Using CATALINA_BASE:
>>>> "C:\Users\user\.IntelliJIdea2016.3\system\tomcat\Unnamed_Async-Servlet-Example_2"
>>>> Using CATALINA_HOME:
>>>> "C:\Users\user\Downloads\apache-tomcat-7.0.81-windows-x64-IJ\apache-tomcat-7.0.81"
>>>> Using CATALINA_TMPDIR:
>>>> "C:\Users\user\Downloads\apache-tomcat-7.0.81-windows-x64-IJ\apache-tomcat-7.0.81\temp"
>>>> Using JRE_HOME:        "E:\jdk1.7.0_79"
>>>> INFO: Server version:        Apache Tomcat/7.0.81
>>>> INFO: Server built:          Aug 11 2017 10:21:27 UTC
>>>> INFO: Server number:         7.0.81.0
>>>> INFO: OS Name:               Windows 8.1
>>>> INFO: OS Version:            6.3
>>>> INFO: Architecture:          amd64
>>>> INFO: Java Home:             E:\jdk1.7.0_79\jre
>>>> INFO: JVM Version:           1.7.0_79-b15
>>>> INFO: JVM Vendor:            Oracle Corporation
>>>> INFO: CATALINA_BASE:
>>>> C:\Users\user\.IntelliJIdea2016.3\system\tomcat\Unnamed_Async-Servlet-Example_2
>>>>
>>>> Container MAX used threads: 10
>>>
>>> I see similar results.
>>>
>>> There looks to be things going on either in JMeter or at the network
>>> level I don't understand. I had to resort to drawing it out to get my
>>> head around what is happening.
>>>
>>
>> Sorry for bothering you,
>>
>> To examine if things going on either in JMeter or at the network, I
>> tested same config (BIO,20,20,10) on Jetty. All 70 requests returned
>> successfully and response time was ~20 seconds for all, as I expects.
>
> I'm fairly sure Jetty uses NIO, not BIO which would explain the
> differences you are observing.
>
>> Then to make myself sure, I tested same servlet but a sync one (removed
>> my own thread pool and asyncStart etc) on Jetty. Average response time
>> increased to 95s, as I expected.
>>
>>> The first 20 requests (10 seconds) are accepted and processed by Tomcat.
>>> The 21st request is accepted but then the acceptor blocks waiting for
>>> the connection count to reduce below 20 before proceeding.
>>>
>>
>> You have forgot.
>
> No I haven't.
>
>> My configuration is maxThreads=maxConnections=20 (not
>> 10) and acceptCount=10. As it prints "Container MAX used threads: 10" so
>> it never reaches maxThreads. So why acceptor blocks ?!
>
> After the first 20 requests you have 20 connections so you hit the
> maxConnection limit. That is why the acceptor blocks and subsequent
> requests go into the accept queue.
>
>> I think although
>> I have an async servlet and own thread pool, and although time consuming
>> process (Thread.sleep) is inside my own thread pool, but Tomcat's
>> container thread wrongly does not back to thread pool
>
> Incorrect. The container thread does return to the container thread
> pool. It does so almost immediately. Given that there are 0.5 seconds
> between requests and that the time taken to process an incoming request,
> dispatch the request to your thread pool and return the container thread
> to the container thread pool is almost certainly less than 0.5 it is
> very likely that there is never more than one container thread active at
> any one point.
>
>> and fails to> satisfy Servlet 3's async API!
>
> Also incorrect.
>
>>> Requests 22 to 31 are placed in the accept queue. We are now 15.5s into
>>> the test and the first request accepted won't finish processing for
>>> another 4.5 seconds.
>>>
>>> Requests 32 to 40 are dropped since the request queue is full. We are
>>> now 20s into the test and the first request is about to complete
>>> processing. Oddly, JMeter doesn't report these as failed until some 35
>>> seconds later.
>>>
>>> Request 1 completes. This allows request 21 to proceed. The acceptor
>>> takes a connection from the accept queue (this appears to be FIFO).
>>> Request 41 enters the accept queue.
>>>
>>> The continues until request 10 completes, 30 starts processing and 50
>>> enters the accept queue.
>>>
>>> Next 11 completes, 41 starts processing and 51 enters the accept queue.
>>> This continues until 20 completes, 50 starts processing and 60 enters
>>> the accept queue.
>>>
>>> At this point there are 20 threads processing, 10 in the accept queue
>>> and no thread due to complete for anther 10s.
>>>
>>> I'd expected requests 61 to 70 to be rejected. However, 65 to 70 are
>>> processed. It looks like there is some sort of timeout for acceptance or
>>> rejection in the accept queue.
>>>
>>> That explains the rejected requests.
>>>
>>
>> I'm not such smart to understand your analysis :) I just understand when
>> my servlet is completely async, then I should not have any rejected
>> requests (specially in such low load) like Jetty result.
>
> You are failing to take account of the maxConnections configuration.
> With a new request being made every 0.5 seconds and a wait time of 20
> seconds Tomcat needs to be able to handle at least 40 concurrent
> connections to prevent dropped connections. You have configured Tomcat
> to allow 30 (20 from maxConnections and 10 in the accept queue). Hence
> you are going to see dropped connections.
>
>
>> As I said, max
>> Tomcat container's threads used by my APP is 10 (half of initial size
>> 20) so why I see rejected requests while thread pool has 10 free threads
>> to accept new requests ?!
>>
>>> The other question is why maxThreads is reported as it is.
>>>
>>> The answer is that the thread pool never grows beyond its initial size
>>> of 10. A request comes in, it is processed by a container thread,
>>> dispatched to an async thread and then the container thread is returned
>>> to the pool to await the next request. Tomcat is able to do this because
>>> the container doesn't perform any I/O on the connection once it enters
>>> async mode until it is dispatched back to the container. The default
>>> thread pool implementation cycles through the threads so the max value
>>> you see is 10 which is the initial size.
>>>
>>
>> My config was maxThreads=20 (not 10).
>
> Irrelevant. The container threads aren't doing very much. As I explained
> above it is unlikely that there is ever more than one container thread
> doing work at any one time. Tomcat's thread pool has an INITIAL size of
> 10 that only grows if more than 10 concurrent threads are required.
> Given that your test case only requires one concurrent container thread
> (NOT 20 because you are using async so the container thread is freed as
> soon as the async dispatch completes) the thread pool never grows. What
> happens is:
>
> The thread pool is a FIFO pool.
> At the start:
> Pool: 1 2 3 4 5 6 7 8 9 10
> Thread 1 processes request 1 and returns to the pool.
> Pool: 2 3 4 5 6 7 8 9 10 1
> Thread 2 processes request 2 and returns to the pool.
> Pool: 3 4 5 6 7 8 9 10 1 2
> ...
> Pool: 10 1 2 3 4 5 6 7 8 9
> Thread 10 processes request 10 and returns to the pool.
> Pool: 1 2 3 4 5 6 7 8 9 10
> Thread 1 processes request 11 and returns to the pool.
> ...
>
> Hence the highest container thread number you see in your test is 10.
>
> You are making the incorrect assumption that the highest thread number
> you see always represents the highest concurrency. That assumption is
> not true when maximum concurrency is less than the initial size of the
> thread pool.
>
> Mark
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]