native connector, server problems with "No data received", what could be causing it?
Locked
native connector, server problems with "No data received", what could be causing it?
 
|
Hi,
We have experienced a large number of HTTP requests with the problem "No data received". this is the example using curl: --2020-12-15 15:38:13-- https://www.numbeo.com/admin/DownloadLastDbBackupServlet?numbeoPass=xxx Loaded CA certificate '/etc/ssl/certs/ca-certificates.crt' Resolving www.numbeo.com (www.numbeo.com)... 209.126.119.66 Connecting to www.numbeo.com (www.numbeo.com)|209.126.119.66|:443... connected. HTTP request sent, awaiting response... No data received. Retrying. --2020-12-15 15:38:15-- (try: 2) https://www.numbeo.com/admin/DownloadLastDbBackupServlet?numbeoPass=xxx Connecting to www.numbeo.com (www.numbeo.com)|209.126.119.66|:443... connected. HTTP request sent, awaiting response... No data received. Retrying. We have a self-monitoring script which runs on server and when the server is not working properly it does a log save and the service restart. We are using currently Apache 8.5.5 and the Connector is set-up as follows (using Tomcat native): <Connector port="8181" protocol="org.apache.coyote.http11.Http11AprProtocol" SSLCertificateFile="/etc/letsencrypt/live/numbeo.com/cert.pem" SSLCertificateKeyFile="/etc/letsencrypt/live/ numbeo.com/privkey.pem" SSLCertificateChainFile="/etc/letsencrypt/live/ numbeo.com/chain.pem" SSLVerifyClient="optional" SSLProtocol="TLSv1+TLSv1.1+TLSv1.2" connectionTimeout="20000" acceptCount="30000" acceptorThreadCount="4" compression="on" maxConnections="50000" maxThreads="500" compressableMimeType="text/html,text/xml,text/plain,text/css,text/javascript,application/javascript,image/svg+xml,image/svg,image/png,image/jpeg" useSendfile="false" maxHttpHeaderSize="16392" SSLEnabled="true" enableLookups="false" scheme="https" secure="true" clientAuth="false" useBodyEncodingForURI="true" URIEncoding="UTF-8" /> *What would be the next steps how to identify the problem and perhaps solve it?* If we should migrate to the new version, should we try first the last 8.5 version or the last 9 Tomcat version? Our JSTACK when the last problem occurred is as follows: root@condor1796 ~ # cat jstack.log Attaching to process ID 27753, please wait... Debugger attached successfully. Server compiler detected. JVM version is 25.101-b13 Deadlock Detection: No deadlocks found. Thread 1237: (state = BLOCKED) - java.lang.Thread.sleep(long) @bci=0 (Compiled frame; information may be imprecise) - common.MemoryCleanerAndLogger.run() @bci=3, line=20 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 516: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) @bci=20, line=215 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(long) @bci=78, line=2078 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=62, line=467 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 515: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, line=175 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt() @bci=1, line=836 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireInterruptibly(int) @bci=66, line=897 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(int) @bci=24, line=1222 (Compiled frame) - java.util.concurrent.locks.ReentrantLock.lockInterruptibly() @bci=5, line=335 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=27, line=462 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 514: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, line=175 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt() @bci=1, line=836 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireInterruptibly(int) @bci=66, line=897 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(int) @bci=24, line=1222 (Compiled frame) - java.util.concurrent.locks.ReentrantLock.lockInterruptibly() @bci=5, line=335 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=27, line=462 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 512: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) @bci=20, line=215 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(long) @bci=78, line=2078 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=62, line=467 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 511: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) @bci=20, line=215 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(long) @bci=78, line=2078 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=62, line=467 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 510: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, line=175 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt() @bci=1, line=836 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireInterruptibly(int) @bci=66, line=897 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(int) @bci=24, line=1222 (Compiled frame) - java.util.concurrent.locks.ReentrantLock.lockInterruptibly() @bci=5, line=335 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=27, line=462 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 509: (state = IN_NATIVE) - org.apache.tomcat.jni.SSLSocket.handshake(long) @bci=0 (Compiled frame; information may be imprecise) - org.apache.tomcat.util.net.AprEndpoint.setSocketOptions(org.apache.tomcat.util.net.SocketWrapperBase) @bci=119, line=726 (Compiled frame) - org.apache.tomcat.util.net.AprEndpoint$SocketWithOptionsProcessor.run() @bci=102, line=2204 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=95, line=1142 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 508: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) @bci=20, line=215 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(long) @bci=78, line=2078 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=62, line=467 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 507: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, line=175 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt() @bci=1, line=836 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireInterruptibly(int) @bci=66, line=897 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(int) @bci=24, line=1222 (Compiled frame) - java.util.concurrent.locks.ReentrantLock.lockInterruptibly() @bci=5, line=335 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=27, line=462 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 456: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) @bci=20, line=215 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(long) @bci=78, line=2078 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=62, line=467 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 455: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) @bci=20, line=215 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(long) @bci=78, line=2078 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=62, line=467 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 450: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) @bci=20, line=215 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(long) @bci=78, line=2078 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=62, line=467 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 412: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) @bci=20, line=215 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(long) @bci=78, line=2078 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=62, line=467 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 332: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) @bci=20, line=215 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(long) @bci=78, line=2078 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=62, line=467 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 300: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) @bci=20, line=215 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(long) @bci=78, line=2078 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=62, line=467 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 32691: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, line=175 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt() @bci=1, line=836 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireInterruptibly(int) @bci=66, line=897 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(int) @bci=24, line=1222 (Compiled frame) - java.util.concurrent.locks.ReentrantLock.lockInterruptibly() @bci=5, line=335 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=27, line=462 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 32690: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) @bci=20, line=215 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(long) @bci=78, line=2078 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=62, line=467 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 32621: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) @bci=20, line=215 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(long) @bci=78, line=2078 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=62, line=467 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 32608: (state = IN_NATIVE) - org.apache.tomcat.jni.SSLSocket.handshake(long) @bci=0 (Compiled frame; information may be imprecise) - org.apache.tomcat.util.net.AprEndpoint.setSocketOptions(org.apache.tomcat.util.net.SocketWrapperBase) @bci=119, line=726 (Compiled frame) - org.apache.tomcat.util.net.AprEndpoint$SocketWithOptionsProcessor.run() @bci=102, line=2204 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=95, line=1142 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 32607: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, line=175 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt() @bci=1, line=836 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireInterruptibly(int) @bci=66, line=897 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(int) @bci=24, line=1222 (Compiled frame) - java.util.concurrent.locks.ReentrantLock.lockInterruptibly() @bci=5, line=335 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=27, line=462 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 32606: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) @bci=20, line=215 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(long) @bci=78, line=2078 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=62, line=467 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 32581: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, line=175 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt() @bci=1, line=836 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireInterruptibly(int) @bci=66, line=897 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(int) @bci=24, line=1222 (Compiled frame) - java.util.concurrent.locks.ReentrantLock.lockInterruptibly() @bci=5, line=335 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=27, line=462 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 32488: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) @bci=20, line=215 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(long) @bci=78, line=2078 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=62, line=467 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 32487: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, line=175 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt() @bci=1, line=836 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireInterruptibly(int) @bci=66, line=897 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(int) @bci=24, line=1222 (Compiled frame) - java.util.concurrent.locks.ReentrantLock.lockInterruptibly() @bci=5, line=335 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=27, line=462 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 32310: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) @bci=20, line=215 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(long) @bci=78, line=2078 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=62, line=467 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 32309: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) @bci=20, line=215 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(long) @bci=78, line=2078 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=62, line=467 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 32308: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) @bci=20, line=215 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(long) @bci=78, line=2078 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=62, line=467 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 32307: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) @bci=20, line=215 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(long) @bci=78, line=2078 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=62, line=467 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 32306: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) @bci=20, line=215 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(long) @bci=78, line=2078 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=62, line=467 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 32305: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) @bci=20, line=215 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(long) @bci=78, line=2078 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=62, line=467 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 32304: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) @bci=20, line=215 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(long) @bci=78, line=2078 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=62, line=467 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 32303: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) @bci=20, line=215 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(long) @bci=78, line=2078 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=62, line=467 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 32247: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) @bci=20, line=215 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(long) @bci=78, line=2078 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=62, line=467 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 32056: (state = BLOCKED) - java.lang.Object.wait(long) @bci=0 (Compiled frame; information may be imprecise) - sun.net.www.http.KeepAliveStreamCleaner.run() @bci=30, line=101 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 30294: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, line=175 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt() @bci=1, line=836 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireInterruptibly(int) @bci=66, line=897 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(int) @bci=24, line=1222 (Compiled frame) - java.util.concurrent.locks.ReentrantLock.lockInterruptibly() @bci=5, line=335 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=27, line=462 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 30293: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) @bci=20, line=215 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(long) @bci=78, line=2078 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=62, line=467 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 30292: (state = IN_NATIVE) - org.apache.tomcat.jni.Socket.sendb(long, java.nio.ByteBuffer, int, int) @bci=0 (Compiled frame; information may be imprecise) - org.apache.tomcat.util.net.AprEndpoint$AprSocketWrapper.doWriteInternal() @bci=89, line=2517 (Compiled frame) - org.apache.tomcat.util.net.AprEndpoint$AprSocketWrapper.doWrite(boolean) @bci=85, line=2464 (Compiled frame) - org.apache.tomcat.util.net.SocketWrapperBase.writeBlocking(byte[], int, int) @bci=47, line=378 (Compiled frame) - org.apache.tomcat.util.net.SocketWrapperBase.write(boolean, byte[], int, int) @bci=19, line=347 (Compiled frame) - org.apache.coyote.http11.Http11OutputBuffer$SocketOutputBuffer.doWrite(org.apache.tomcat.util.buf.ByteChunk) @bci=34, line=561 (Compiled frame) - org.apache.coyote.http11.filters.ChunkedOutputFilter.doWrite(org.apache.tomcat.util.buf.ByteChunk) @bci=90, line=112 (Compiled frame) - org.apache.coyote.http11.filters.GzipOutputFilter$FakeOutputStream.write(byte[], int, int) @bci=21, line=168 (Compiled frame) - java.util.zip.DeflaterOutputStream.deflate() @bci=32, line=253 (Compiled frame) - java.util.zip.DeflaterOutputStream.write(byte[], int, int) @bci=81, line=211 (Compiled frame) - java.util.zip.GZIPOutputStream.write(byte[], int, int) @bci=4, line=145 (Compiled frame) - org.apache.coyote.http11.filters.GzipOutputFilter.doWrite(org.apache.tomcat.util.buf.ByteChunk) @bci=39, line=70 (Compiled frame) - org.apache.coyote.http11.Http11OutputBuffer.doWrite(org.apache.tomcat.util.buf.ByteChunk) @bci=50, line=206 (Compiled frame) - org.apache.coyote.Response.doWrite(org.apache.tomcat.util.buf.ByteChunk) @bci=5, line=499 (Compiled frame) - org.apache.catalina.connector.OutputBuffer.realWriteBytes(byte[], int, int) @bci=38, line=375 (Compiled frame) - org.apache.tomcat.util.buf.ByteChunk.flushBuffer() @bci=71, line=420 (Compiled frame) - org.apache.tomcat.util.buf.ByteChunk.append(byte[], int, int) @bci=152, line=345 (Compiled frame) - org.apache.catalina.connector.OutputBuffer.writeBytes(byte[], int, int) @bci=15, line=405 (Compiled frame) - org.apache.catalina.connector.OutputBuffer.write(byte[], int, int) @bci=12, line=393 (Compiled frame) - org.apache.catalina.connector.CoyoteOutputStream.write(byte[], int, int) @bci=13, line=96 (Compiled frame) - org.apache.catalina.servlets.DefaultServlet.copyRange(java.io.InputStream, javax.servlet.ServletOutputStream) @bci=38, line=2281 (Compiled frame) - org.apache.catalina.servlets.DefaultServlet.copy(org.apache.catalina.WebResource, java.io.InputStream, javax.servlet.ServletOutputStream) @bci=21, line=2124 (Compiled frame) - org.apache.catalina.servlets.DefaultServlet.serveResource(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse, boolean, java.lang.String) @bci=1277, line=1006 (Compiled frame) - org.apache.catalina.servlets.DefaultServlet.doGet(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse) @bci=8, line=425 (Compiled frame) - javax.servlet.http.HttpServlet.service(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse) @bci=35, line=622 (Compiled frame) - javax.servlet.http.HttpServlet.service(javax.servlet.ServletRequest, javax.servlet.ServletResponse) @bci=30, line=729 (Compiled frame) - org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(javax.servlet.ServletRequest, javax.servlet.ServletResponse) @bci=304, line=230 (Compiled frame) - org.apache.catalina.core.ApplicationFilterChain.doFilter(javax.servlet.ServletRequest, javax.servlet.ServletResponse) @bci=101, line=165 (Compiled frame) - org.apache.tomcat.websocket.server.WsFilter.doFilter(javax.servlet.ServletRequest, javax.servlet.ServletResponse, javax.servlet.FilterChain) @bci=21, line=52 (Compiled frame) - org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(javax.servlet.ServletRequest, javax.servlet.ServletResponse) @bci=135, line=192 (Compiled frame) - org.apache.catalina.core.ApplicationFilterChain.doFilter(javax.servlet.ServletRequest, javax.servlet.ServletResponse) @bci=101, line=165 (Compiled frame) - org.apache.catalina.filters.HttpHeaderSecurityFilter.doFilter(javax.servlet.ServletRequest, javax.servlet.ServletResponse, javax.servlet.FilterChain) @bci=126, line=120 (Compiled frame) - org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(javax.servlet.ServletRequest, javax.servlet.ServletResponse) @bci=135, line=192 (Compiled frame) - org.apache.catalina.core.ApplicationFilterChain.doFilter(javax.servlet.ServletRequest, javax.servlet.ServletResponse) @bci=101, line=165 (Compiled frame) - common.ExpiresFilter.doFilter(javax.servlet.ServletRequest, javax.servlet.ServletResponse, javax.servlet.FilterChain) @bci=90, line=44 (Compiled frame) - org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(javax.servlet.ServletRequest, javax.servlet.ServletResponse) @bci=135, line=192 (Compiled frame) - org.apache.catalina.core.ApplicationFilterChain.doFilter(javax.servlet.ServletRequest, javax.servlet.ServletResponse) @bci=101, line=165 (Compiled frame) - org.apache.catalina.core.StandardWrapperValve.invoke(org.apache.catalina.connector.Request, org.apache.catalina.connector.Response) @bci=688, line=198 (Compiled frame) - org.apache.catalina.core.StandardContextValve.invoke(org.apache.catalina.connector.Request, org.apache.catalina.connector.Response) @bci=166, line=108 (Compiled frame) - org.apache.catalina.authenticator.AuthenticatorBase.invoke(org.apache.catalina.connector.Request, org.apache.catalina.connector.Response) @bci=275, line=472 (Compiled frame) - org.apache.catalina.core.StandardHostValve.invoke(org.apache.catalina.connector.Request, org.apache.catalina.connector.Response) @bci=135, line=140 (Compiled frame) - org.apache.catalina.valves.ErrorReportValve.invoke(org.apache.catalina.connector.Request, org.apache.catalina.connector.Response) @bci=6, line=79 (Compiled frame) - org.apache.catalina.valves.AbstractAccessLogValve.invoke(org.apache.catalina.connector.Request, org.apache.catalina.connector.Response) @bci=6, line=620 (Compiled frame) - org.apache.catalina.core.StandardEngineValve.invoke(org.apache.catalina.connector.Request, org.apache.catalina.connector.Response) @bci=71, line=87 (Compiled frame) - org.apache.catalina.connector.CoyoteAdapter.service(org.apache.coyote.Request, org.apache.coyote.Response) @bci=199, line=349 (Compiled frame) - org.apache.coyote.http11.Http11Processor.service(org.apache.tomcat.util.net.SocketWrapperBase) @bci=806, line=784 (Compiled frame) - org.apache.coyote.AbstractProcessorLight.process(org.apache.tomcat.util.net.SocketWrapperBase, org.apache.tomcat.util.net.SocketEvent) @bci=106, line=66 (Compiled frame) - org.apache.coyote.AbstractProtocol$ConnectionHandler.process(org.apache.tomcat.util.net.SocketWrapperBase, org.apache.tomcat.util.net.SocketEvent) @bci=295, line=802 (Compiled frame) - org.apache.tomcat.util.net.AprEndpoint$SocketProcessor.doRun() @bci=15, line=2241 (Compiled frame) - org.apache.tomcat.util.net.SocketProcessorBase.run() @bci=21, line=49 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=95, line=1142 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 30291: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, line=175 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt() @bci=1, line=836 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireInterruptibly(int) @bci=66, line=897 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(int) @bci=24, line=1222 (Compiled frame) - java.util.concurrent.locks.ReentrantLock.lockInterruptibly() @bci=5, line=335 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=27, line=462 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 30290: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, line=175 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt() @bci=1, line=836 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireInterruptibly(int) @bci=66, line=897 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(int) @bci=24, line=1222 (Compiled frame) - java.util.concurrent.locks.ReentrantLock.lockInterruptibly() @bci=5, line=335 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=27, line=462 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 30289: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, line=175 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt() @bci=1, line=836 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireInterruptibly(int) @bci=66, line=897 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(int) @bci=24, line=1222 (Compiled frame) - java.util.concurrent.locks.ReentrantLock.lockInterruptibly() @bci=5, line=335 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=27, line=462 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27951: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, line=175 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await() @bci=42, line=2039 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.take() @bci=29, line=442 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.take() @bci=36, line=103 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.take() @bci=1, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=149, line=1067 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Interpreted frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27950: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, line=175 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await() @bci=42, line=2039 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.take() @bci=29, line=442 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.take() @bci=36, line=103 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.take() @bci=1, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=149, line=1067 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Interpreted frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27929: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, line=175 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await() @bci=42, line=2039 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.take() @bci=29, line=442 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.take() @bci=36, line=103 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.take() @bci=1, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=149, line=1067 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Interpreted frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27927: (state = BLOCKED) - java.lang.Object.wait(long) @bci=0 (Compiled frame; information may be imprecise) - java.lang.ref.ReferenceQueue.remove(long) @bci=59, line=143 (Compiled frame) - com.mysql.jdbc.AbandonedConnectionCleanupThread.run() @bci=23, line=41 (Compiled frame) Thread 27919: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, line=175 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await() @bci=42, line=2039 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.take() @bci=29, line=442 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.take() @bci=36, line=103 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.take() @bci=1, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=149, line=1067 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Interpreted frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27918: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, line=175 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await() @bci=42, line=2039 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.take() @bci=29, line=442 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.take() @bci=36, line=103 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.take() @bci=1, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=149, line=1067 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Interpreted frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27915: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, line=175 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await() @bci=42, line=2039 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.take() @bci=29, line=442 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.take() @bci=36, line=103 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.take() @bci=1, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=149, line=1067 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Interpreted frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27914: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, line=175 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await() @bci=42, line=2039 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.take() @bci=29, line=442 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.take() @bci=36, line=103 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.take() @bci=1, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=149, line=1067 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Interpreted frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27913: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, line=175 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await() @bci=42, line=2039 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.take() @bci=29, line=442 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.take() @bci=36, line=103 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.take() @bci=1, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=149, line=1067 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Interpreted frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27910: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, line=175 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await() @bci=42, line=2039 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.take() @bci=29, line=442 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.take() @bci=36, line=103 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.take() @bci=1, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=149, line=1067 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Interpreted frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27909: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, line=175 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await() @bci=42, line=2039 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.take() @bci=29, line=442 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.take() @bci=36, line=103 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.take() @bci=1, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=149, line=1067 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Interpreted frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27908: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, line=175 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await() @bci=42, line=2039 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.take() @bci=29, line=442 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.take() @bci=36, line=103 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.take() @bci=1, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=149, line=1067 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Interpreted frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27904: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, line=175 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await() @bci=42, line=2039 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.take() @bci=29, line=442 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.take() @bci=36, line=103 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.take() @bci=1, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=149, line=1067 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Interpreted frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27903: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, line=175 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await() @bci=42, line=2039 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.take() @bci=29, line=442 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.take() @bci=36, line=103 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.take() @bci=1, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=149, line=1067 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Interpreted frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27889: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, line=175 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await() @bci=42, line=2039 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.take() @bci=29, line=442 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.take() @bci=36, line=103 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.take() @bci=1, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=149, line=1067 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Interpreted frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27886: (state = BLOCKED) - java.lang.Thread.sleep(long) @bci=0 (Compiled frame; information may be imprecise) - org.apache.coyote.AbstractProtocol$AsyncTimeout.run() @bci=10, line=1137 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27885: (state = IN_NATIVE) - sun.nio.ch.ServerSocketChannelImpl.accept0(java.io.FileDescriptor, java.io.FileDescriptor, java.net.InetSocketAddress[]) @bci=0 (Compiled frame; information may be imprecise) - sun.nio.ch.ServerSocketChannelImpl.accept(java.io.FileDescriptor, java.io.FileDescriptor, java.net.InetSocketAddress[]) @bci=4, line=422 (Compiled frame) - sun.nio.ch.ServerSocketChannelImpl.accept() @bci=130, line=250 (Compiled frame) - org.apache.tomcat.util.net.NioEndpoint$Acceptor.run() @bci=88, line=456 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27883: (state = BLOCKED) - sun.nio.ch.ServerSocketChannelImpl.accept() @bci=6, line=233 (Compiled frame) - org.apache.tomcat.util.net.NioEndpoint$Acceptor.run() @bci=88, line=456 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27882: (state = IN_NATIVE) - sun.nio.ch.EPollArrayWrapper.epollWait(long, int, long, int) @bci=0 (Compiled frame; information may be imprecise) - sun.nio.ch.EPollArrayWrapper.poll(long) @bci=18, line=269 (Compiled frame) - sun.nio.ch.EPollSelectorImpl.doSelect(long) @bci=28, line=93 (Compiled frame) - sun.nio.ch.SelectorImpl.lockAndDoSelect(long) @bci=37, line=86 (Compiled frame) - sun.nio.ch.SelectorImpl.select(long) @bci=30, line=97 (Compiled frame) - org.apache.tomcat.util.net.NioEndpoint$Poller.run() @bci=55, line=790 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27881: (state = IN_NATIVE) - sun.nio.ch.EPollArrayWrapper.epollWait(long, int, long, int) @bci=0 (Compiled frame; information may be imprecise) - sun.nio.ch.EPollArrayWrapper.poll(long) @bci=18, line=269 (Compiled frame) - sun.nio.ch.EPollSelectorImpl.doSelect(long) @bci=28, line=93 (Compiled frame) - sun.nio.ch.SelectorImpl.lockAndDoSelect(long) @bci=37, line=86 (Compiled frame) - sun.nio.ch.SelectorImpl.select(long) @bci=30, line=97 (Compiled frame) - org.apache.tomcat.util.net.NioEndpoint$Poller.run() @bci=55, line=790 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27880: (state = BLOCKED) - java.lang.Thread.sleep(long) @bci=0 (Compiled frame; information may be imprecise) - org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.run() @bci=46, line=1339 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27877: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) @bci=20, line=215 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(long) @bci=78, line=2078 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=62, line=467 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27876: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) @bci=20, line=215 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(long) @bci=78, line=2078 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=62, line=467 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27875: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, line=175 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt() @bci=1, line=836 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireInterruptibly(int) @bci=66, line=897 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(int) @bci=24, line=1222 (Compiled frame) - java.util.concurrent.locks.ReentrantLock.lockInterruptibly() @bci=5, line=335 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=27, line=462 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27874: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, line=175 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt() @bci=1, line=836 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireInterruptibly(int) @bci=66, line=897 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(int) @bci=24, line=1222 (Compiled frame) - java.util.concurrent.locks.ReentrantLock.lockInterruptibly() @bci=5, line=335 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=27, line=462 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27873: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) @bci=20, line=215 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(long) @bci=78, line=2078 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=62, line=467 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27872: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, line=175 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt() @bci=1, line=836 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireInterruptibly(int) @bci=66, line=897 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(int) @bci=24, line=1222 (Compiled frame) - java.util.concurrent.locks.ReentrantLock.lockInterruptibly() @bci=5, line=335 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=27, line=462 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27871: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) @bci=20, line=215 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(long) @bci=78, line=2078 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=62, line=467 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27870: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, line=175 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt() @bci=1, line=836 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireInterruptibly(int) @bci=66, line=897 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(int) @bci=24, line=1222 (Compiled frame) - java.util.concurrent.locks.ReentrantLock.lockInterruptibly() @bci=5, line=335 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=27, line=462 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27869: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) @bci=20, line=215 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(long) @bci=78, line=2078 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=62, line=467 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27868: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) @bci=20, line=215 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(long) @bci=78, line=2078 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=62, line=467 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27867: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) @bci=20, line=215 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(long) @bci=78, line=2078 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=62, line=467 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27866: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) @bci=20, line=215 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(long) @bci=78, line=2078 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=62, line=467 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27865: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, line=175 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt() @bci=1, line=836 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireInterruptibly(int) @bci=66, line=897 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(int) @bci=24, line=1222 (Compiled frame) - java.util.concurrent.locks.ReentrantLock.lockInterruptibly() @bci=5, line=335 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=27, line=462 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27864: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) @bci=20, line=215 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(long) @bci=78, line=2078 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=62, line=467 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27863: (state = IN_NATIVE) - org.apache.tomcat.jni.SSLSocket.handshake(long) @bci=0 (Compiled frame; information may be imprecise) - org.apache.tomcat.util.net.AprEndpoint.setSocketOptions(org.apache.tomcat.util.net.SocketWrapperBase) @bci=119, line=726 (Compiled frame) - org.apache.tomcat.util.net.AprEndpoint$SocketWithOptionsProcessor.run() @bci=102, line=2204 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=95, line=1142 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27862: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) @bci=20, line=215 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(long) @bci=78, line=2078 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=62, line=467 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27861: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) @bci=20, line=215 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(long) @bci=78, line=2078 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=62, line=467 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27860: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, line=175 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt() @bci=1, line=836 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireInterruptibly(int) @bci=66, line=897 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(int) @bci=24, line=1222 (Compiled frame) - java.util.concurrent.locks.ReentrantLock.lockInterruptibly() @bci=5, line=335 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=27, line=462 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27859: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) @bci=20, line=215 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(long) @bci=78, line=2078 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=62, line=467 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27858: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) @bci=20, line=215 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(long) @bci=78, line=2078 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=62, line=467 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27857: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) @bci=20, line=215 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(long) @bci=78, line=2078 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=62, line=467 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27856: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, line=175 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt() @bci=1, line=836 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireInterruptibly(int) @bci=66, line=897 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(int) @bci=24, line=1222 (Compiled frame) - java.util.concurrent.locks.ReentrantLock.lockInterruptibly() @bci=5, line=335 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=27, line=462 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27855: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) @bci=20, line=215 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(long) @bci=78, line=2078 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=62, line=467 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27854: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) @bci=20, line=215 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(long) @bci=78, line=2078 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=62, line=467 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27853: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, line=175 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt() @bci=1, line=836 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireInterruptibly(int) @bci=66, line=897 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(int) @bci=24, line=1222 (Compiled frame) - java.util.concurrent.locks.ReentrantLock.lockInterruptibly() @bci=5, line=335 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=27, line=462 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27852: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) @bci=20, line=215 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(long) @bci=78, line=2078 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=62, line=467 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27851: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) @bci=20, line=215 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(long) @bci=78, line=2078 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=62, line=467 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27850: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) @bci=20, line=215 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(long) @bci=78, line=2078 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=62, line=467 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27849: (state = IN_NATIVE) - org.apache.tomcat.jni.SSLSocket.handshake(long) @bci=0 (Compiled frame; information may be imprecise) - org.apache.tomcat.util.net.AprEndpoint.setSocketOptions(org.apache.tomcat.util.net.SocketWrapperBase) @bci=119, line=726 (Compiled frame) - org.apache.tomcat.util.net.AprEndpoint$SocketWithOptionsProcessor.run() @bci=102, line=2204 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=95, line=1142 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27848: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) @bci=20, line=215 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(long) @bci=78, line=2078 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=62, line=467 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27847: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) @bci=20, line=215 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(long) @bci=78, line=2078 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=62, line=467 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27846: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) @bci=20, line=215 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(long) @bci=78, line=2078 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=62, line=467 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27845: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) @bci=20, line=215 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(long) @bci=78, line=2078 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=62, line=467 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27844: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) @bci=20, line=215 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(long) @bci=78, line=2078 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=62, line=467 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27843: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) @bci=20, line=215 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(long) @bci=78, line=2078 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=62, line=467 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27842: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) @bci=20, line=215 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(long) @bci=78, line=2078 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=62, line=467 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27841: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) @bci=20, line=215 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(long) @bci=78, line=2078 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=62, line=467 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27840: (state = IN_NATIVE) - org.apache.tomcat.jni.SSLSocket.handshake(long) @bci=0 (Compiled frame; information may be imprecise) - org.apache.tomcat.util.net.AprEndpoint.setSocketOptions(org.apache.tomcat.util.net.SocketWrapperBase) @bci=119, line=726 (Compiled frame) - org.apache.tomcat.util.net.AprEndpoint$SocketWithOptionsProcessor.run() @bci=102, line=2204 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=95, line=1142 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27839: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) @bci=20, line=215 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(long) @bci=78, line=2078 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=62, line=467 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27838: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) @bci=20, line=215 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(long) @bci=78, line=2078 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=62, line=467 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27837: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) @bci=20, line=215 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(long) @bci=78, line=2078 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=62, line=467 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27836: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, line=175 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt() @bci=1, line=836 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireInterruptibly(int) @bci=66, line=897 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(int) @bci=24, line=1222 (Compiled frame) - java.util.concurrent.locks.ReentrantLock.lockInterruptibly() @bci=5, line=335 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=27, line=462 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27835: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) @bci=20, line=215 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(long) @bci=78, line=2078 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=62, line=467 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27834: (state = IN_NATIVE) - org.apache.tomcat.jni.SSLSocket.handshake(long) @bci=0 (Compiled frame; information may be imprecise) - org.apache.tomcat.util.net.AprEndpoint.setSocketOptions(org.apache.tomcat.util.net.SocketWrapperBase) @bci=119, line=726 (Compiled frame) - org.apache.tomcat.util.net.AprEndpoint$SocketWithOptionsProcessor.run() @bci=102, line=2204 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=95, line=1142 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27833: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, line=175 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt() @bci=1, line=836 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireInterruptibly(int) @bci=66, line=897 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(int) @bci=24, line=1222 (Compiled frame) - java.util.concurrent.locks.ReentrantLock.lockInterruptibly() @bci=5, line=335 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=27, line=462 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27832: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) @bci=20, line=215 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(long) @bci=78, line=2078 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=62, line=467 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27831: (state = IN_NATIVE) - org.apache.tomcat.jni.SSLSocket.handshake(long) @bci=0 (Compiled frame; information may be imprecise) - org.apache.tomcat.util.net.AprEndpoint.setSocketOptions(org.apache.tomcat.util.net.SocketWrapperBase) @bci=119, line=726 (Compiled frame) - org.apache.tomcat.util.net.AprEndpoint$SocketWithOptionsProcessor.run() @bci=102, line=2204 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=95, line=1142 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27830: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) @bci=20, line=215 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(long) @bci=78, line=2078 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=62, line=467 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27829: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) @bci=20, line=215 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(long) @bci=78, line=2078 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=62, line=467 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27828: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) @bci=20, line=215 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(long) @bci=78, line=2078 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=62, line=467 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27827: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) @bci=20, line=215 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(long) @bci=78, line=2078 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=62, line=467 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27826: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) @bci=20, line=215 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(long) @bci=78, line=2078 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=62, line=467 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27825: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, line=175 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt() @bci=1, line=836 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireInterruptibly(int) @bci=66, line=897 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(int) @bci=24, line=1222 (Compiled frame) - java.util.concurrent.locks.ReentrantLock.lockInterruptibly() @bci=5, line=335 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=27, line=462 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27824: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) @bci=20, line=215 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(long) @bci=78, line=2078 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=62, line=467 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27823: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) @bci=20, line=215 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(long) @bci=78, line=2078 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=62, line=467 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27822: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) @bci=20, line=215 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(long) @bci=78, line=2078 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=62, line=467 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27821: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) @bci=20, line=215 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(long) @bci=78, line=2078 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=62, line=467 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27820: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) @bci=20, line=215 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(long) @bci=78, line=2078 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=62, line=467 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27819: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, line=175 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt() @bci=1, line=836 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireInterruptibly(int) @bci=66, line=897 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(int) @bci=24, line=1222 (Compiled frame) - java.util.concurrent.locks.ReentrantLock.lockInterruptibly() @bci=5, line=335 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=27, line=462 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27818: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) @bci=20, line=215 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(long) @bci=78, line=2078 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=62, line=467 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27817: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) @bci=20, line=215 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(long) @bci=78, line=2078 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=62, line=467 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27816: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, line=175 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt() @bci=1, line=836 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireInterruptibly(int) @bci=66, line=897 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(int) @bci=24, line=1222 (Compiled frame) - java.util.concurrent.locks.ReentrantLock.lockInterruptibly() @bci=5, line=335 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=27, line=462 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27815: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, line=175 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt() @bci=1, line=836 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireInterruptibly(int) @bci=66, line=897 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(int) @bci=24, line=1222 (Compiled frame) - java.util.concurrent.locks.ReentrantLock.lockInterruptibly() @bci=5, line=335 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=27, line=462 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27814: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) @bci=20, line=215 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(long) @bci=78, line=2078 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=62, line=467 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27812: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) @bci=20, line=215 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(long) @bci=78, line=2078 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=62, line=467 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27811: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) @bci=20, line=215 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(long) @bci=78, line=2078 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=62, line=467 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27810: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, line=175 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt() @bci=1, line=836 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireInterruptibly(int) @bci=66, line=897 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(int) @bci=24, line=1222 (Compiled frame) - java.util.concurrent.locks.ReentrantLock.lockInterruptibly() @bci=5, line=335 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=27, line=462 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27809: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) @bci=20, line=215 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(long) @bci=78, line=2078 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=62, line=467 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27808: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) @bci=20, line=215 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(long) @bci=78, line=2078 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=62, line=467 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27807: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) @bci=20, line=215 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(long) @bci=78, line=2078 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=62, line=467 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27806: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, line=175 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt() @bci=1, line=836 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireInterruptibly(int) @bci=66, line=897 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(int) @bci=24, line=1222 (Compiled frame) - java.util.concurrent.locks.ReentrantLock.lockInterruptibly() @bci=5, line=335 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=27, line=462 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27804: (state = BLOCKED) - java.lang.Thread.sleep(long) @bci=0 (Compiled frame; information may be imprecise) - org.apache.coyote.AbstractProtocol$AsyncTimeout.run() @bci=10, line=1137 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27803: (state = IN_NATIVE) - sun.nio.ch.ServerSocketChannelImpl.accept0(java.io.FileDescriptor, java.io.FileDescriptor, java.net.InetSocketAddress[]) @bci=0 (Compiled frame; information may be imprecise) - sun.nio.ch.ServerSocketChannelImpl.accept(java.io.FileDescriptor, java.io.FileDescriptor, java.net.InetSocketAddress[]) @bci=4, line=422 (Compiled frame) - sun.nio.ch.ServerSocketChannelImpl.accept() @bci=130, line=250 (Compiled frame) - org.apache.tomcat.util.net.NioEndpoint$Acceptor.run() @bci=88, line=456 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27802: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, line=175 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt() @bci=1, line=836 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireInterruptibly(int) @bci=66, line=897 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(int) @bci=24, line=1222 (Compiled frame) - java.util.concurrent.locks.ReentrantLock.lockInterruptibly() @bci=5, line=335 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=27, line=462 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27801: (state = BLOCKED) - sun.nio.ch.ServerSocketChannelImpl.accept() @bci=6, line=233 (Compiled frame) - org.apache.tomcat.util.net.NioEndpoint$Acceptor.run() @bci=88, line=456 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27800: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) @bci=20, line=215 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(long) @bci=78, line=2078 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=62, line=467 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27799: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) @bci=20, line=215 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(long) @bci=78, line=2078 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=62, line=467 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27798: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) @bci=20, line=215 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(long) @bci=78, line=2078 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=62, line=467 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27797: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) @bci=20, line=215 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(long) @bci=78, line=2078 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=62, line=467 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27796: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, line=175 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt() @bci=1, line=836 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireInterruptibly(int) @bci=66, line=897 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(int) @bci=24, line=1222 (Compiled frame) - java.util.concurrent.locks.ReentrantLock.lockInterruptibly() @bci=5, line=335 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=27, line=462 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27795: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) @bci=20, line=215 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(long) @bci=78, line=2078 (Compiled frame) - java.util.concurrent.LinkedBlockingQueue.poll(long, java.util.concurrent.TimeUnit) @bci=62, line=467 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=85 (Compiled frame) - org.apache.tomcat.util.threads.TaskQueue.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=31 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=134, line=1066 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=26, line=1127 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27794: (state = IN_NATIVE) - sun.nio.ch.EPollArrayWrapper.epollWait(long, int, long, int) @bci=0 (Compiled frame; information may be imprecise) - sun.nio.ch.EPollArrayWrapper.poll(long) @bci=18, line=269 (Compiled frame) - sun.nio.ch.EPollSelectorImpl.doSelect(long) @bci=28, line=93 (Compiled frame) - sun.nio.ch.SelectorImpl.lockAndDoSelect(long) @bci=37, line=86 (Compiled frame) - sun.nio.ch.SelectorImpl.select(long) @bci=30, line=97 (Compiled frame) - org.apache.tomcat.util.net.NioEndpoint$Poller.run() @bci=55, line=790 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27793: (state = IN_NATIVE) - sun.nio.ch.EPollArrayWrapper.epollWait(long, int, long, int) @bci=0 (Compiled frame; information may be imprecise) - sun.nio.ch.EPollArrayWrapper.poll(long) @bci=18, line=269 (Compiled frame) - sun.nio.ch.EPollSelectorImpl.doSelect(long) @bci=28, line=93 (Compiled frame) - sun.nio.ch.SelectorImpl.lockAndDoSelect(long) @bci=37, line=86 (Compiled frame) - sun.nio.ch.SelectorImpl.select(long) @bci=30, line=97 (Compiled frame) - org.apache.tomcat.util.net.NioEndpoint$Poller.run() @bci=55, line=790 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27792: (state = BLOCKED) - java.lang.Thread.sleep(long) @bci=0 (Compiled frame; information may be imprecise) - org.apache.coyote.AbstractProtocol$AsyncTimeout.run() @bci=10, line=1137 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27791: (state = IN_NATIVE) - org.apache.tomcat.jni.Socket.accept(long) @bci=0 (Compiled frame; information may be imprecise) - org.apache.tomcat.util.net.AprEndpoint$Acceptor.run() @bci=88, line=925 (Compiled frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27790: (state = IN_NATIVE) - org.apache.tomcat.jni.Socket.accept(long) @bci=0 (Compiled frame; information may be imprecise) - org.apache.tomcat.util.net.AprEndpoint$Acceptor.run() @bci=88, line=925 (Compiled frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27789: (state = IN_NATIVE) - org.apache.tomcat.jni.Socket.accept(long) @bci=0 (Compiled frame; information may be imprecise) - org.apache.tomcat.util.net.AprEndpoint$Acceptor.run() @bci=88, line=925 (Compiled frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27788: (state = IN_NATIVE) - org.apache.tomcat.jni.Socket.accept(long) @bci=0 (Compiled frame; information may be imprecise) - org.apache.tomcat.util.net.AprEndpoint$Acceptor.run() @bci=88, line=925 (Compiled frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27787: (state = IN_NATIVE) - org.apache.tomcat.jni.Poll.add(long, long, int) @bci=0 (Compiled frame; information may be imprecise) - org.apache.tomcat.util.net.AprEndpoint$Poller.addToPoller(long, int) @bci=35, line=1416 (Compiled frame) - org.apache.tomcat.util.net.AprEndpoint$Poller.run() @bci=506, line=1616 (Compiled frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27786: (state = BLOCKED) - java.lang.Thread.sleep(long) @bci=0 (Compiled frame; information may be imprecise) - org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.run() @bci=46, line=1339 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27784: (state = BLOCKED) - java.lang.Thread.sleep(long) @bci=0 (Interpreted frame) - jobs.AlertSender.run() @bci=19, line=39 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27782: (state = BLOCKED) - java.lang.Object.wait(long) @bci=0 (Compiled frame; information may be imprecise) - java.lang.ref.ReferenceQueue.remove(long) @bci=59, line=143 (Compiled frame) - com.mysql.jdbc.AbandonedConnectionCleanupThread.run() @bci=23, line=41 (Compiled frame) Thread 27781: (state = BLOCKED) - java.lang.Thread.sleep(long) @bci=0 (Compiled frame; information may be imprecise) - livingcost.PricesAnalysisReader.run() @bci=13, line=52 (Compiled frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27780: (state = BLOCKED) - java.lang.Object.wait(long) @bci=0 (Compiled frame; information may be imprecise) - wikilibrary.EmailQueueWorker.run() @bci=88, line=38 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27779: (state = BLOCKED) - java.lang.Thread.sleep(long) @bci=0 (Compiled frame; information may be imprecise) - wikilibrary.CurrentRankingsUpdater.run() @bci=49, line=93 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 27776: (state = IN_NATIVE) - sun.nio.ch.EPollArrayWrapper.epollWait(long, int, long, int) @bci=0 (Compiled frame; information may be imprecise) - sun.nio.ch.EPollArrayWrapper.poll(long) @bci=18, line=269 (Compiled frame) - sun.nio.ch.EPollSelectorImpl.doSelect(long) @bci=28, line=93 (Compiled frame) - sun.nio.ch.SelectorImpl.lockAndDoSelect(long) @bci=37, line=86 (Compiled frame) - sun.nio.ch.SelectorImpl.select(long) @bci=30, line=97 (Compiled frame) - org.apache.tomcat.util.net.NioBlockingSelector$BlockPoller.run() @bci=52, line=339 (Interpreted frame) Thread 27775: (state = IN_NATIVE) - sun.nio.ch.EPollArrayWrapper.epollWait(long, int, long, int) @bci=0 (Compiled frame; information may be imprecise) - sun.nio.ch.EPollArrayWrapper.poll(long) @bci=18, line=269 (Compiled frame) - sun.nio.ch.EPollSelectorImpl.doSelect(long) @bci=28, line=93 (Compiled frame) - sun.nio.ch.SelectorImpl.lockAndDoSelect(long) @bci=37, line=86 (Compiled frame) - sun.nio.ch.SelectorImpl.select(long) @bci=30, line=97 (Compiled frame) - org.apache.tomcat.util.net.NioBlockingSelector$BlockPoller.run() @bci=52, line=339 (Interpreted frame) Thread 27774: (state = BLOCKED) - java.lang.Object.wait(long) @bci=0 (Interpreted frame) - sun.misc.GC$Daemon.run() @bci=51, line=117 (Interpreted frame) Thread 27773: (state = BLOCKED) - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) @bci=20, line=215 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(long) @bci=78, line=2078 (Compiled frame) - java.util.concurrent.LinkedBlockingDeque.pollFirst(long, java.util.concurrent.TimeUnit) @bci=52, line=522 (Compiled frame) - java.util.concurrent.LinkedBlockingDeque.poll(long, java.util.concurrent.TimeUnit) @bci=3, line=684 (Compiled frame) - org.apache.juli.AsyncFileHandler$LoggerThread.run() @bci=10, line=152 (Interpreted frame) Thread 27766: (state = BLOCKED) Thread 27765: (state = BLOCKED) - java.lang.Object.wait(long) @bci=0 (Compiled frame; information may be imprecise) - java.lang.ref.ReferenceQueue.remove(long) @bci=59, line=143 (Compiled frame) - java.lang.ref.ReferenceQueue.remove() @bci=2, line=164 (Compiled frame) - java.lang.ref.Finalizer$FinalizerThread.run() @bci=36, line=209 (Compiled frame) Thread 27764: (state = BLOCKED) - java.lang.Object.wait(long) @bci=0 (Compiled frame; information may be imprecise) - java.lang.Object.wait() @bci=2, line=502 (Compiled frame) - java.lang.ref.Reference.tryHandlePending(boolean) @bci=54, line=191 (Compiled frame) - java.lang.ref.Reference$ReferenceHandler.run() @bci=1, line=153 (Compiled frame) Thread 27754: (state = IN_NATIVE) - java.net.PlainSocketImpl.socketAccept(java.net.SocketImpl) @bci=0 (Interpreted frame) - java.net.AbstractPlainSocketImpl.accept(java.net.SocketImpl) @bci=7, line=409 (Interpreted frame) - java.net.ServerSocket.implAccept(java.net.Socket) @bci=60, line=545 (Interpreted frame) - java.net.ServerSocket.accept() @bci=48, line=513 (Interpreted frame) - org.apache.catalina.core.StandardServer.await() @bci=180, line=466 (Interpreted frame) - org.apache.catalina.startup.Catalina.await() @bci=4, line=743 (Interpreted frame) - org.apache.catalina.startup.Catalina.start() @bci=209, line=689 (Interpreted frame) - sun.reflect.NativeMethodAccessorImpl.invoke0(java.lang.reflect.Method, java.lang.Object, java.lang.Object[]) @bci=0 (Compiled frame) - sun.reflect.NativeMethodAccessorImpl.invoke(java.lang.Object, java.lang.Object[]) @bci=100, line=62 (Compiled frame) - sun.reflect.DelegatingMethodAccessorImpl.invoke(java.lang.Object, java.lang.Object[]) @bci=6, line=43 (Compiled frame) - java.lang.reflect.Method.invoke(java.lang.Object, java.lang.Object[]) @bci=56, line=498 (Compiled frame) - org.apache.catalina.startup.Bootstrap.start() @bci=37, line=355 (Interpreted frame) - org.apache.catalina.startup.Bootstrap.main(java.lang.String[]) @bci=150, line=495 (Interpreted frame) root@condor1796 ~ # |
Re: native connector, server problems with "No data received", what could be causing it?
|
|
Mladen,
On 12/16/20 04:10, Mladen Adamović wrote: > We have experienced a large number of HTTP requests with the problem "No > data received". > this is the example using curl: > --2020-12-15 15:38:13-- > https://www.numbeo.com/admin/DownloadLastDbBackupServlet?numbeoPass=xxx > Loaded CA certificate '/etc/ssl/certs/ca-certificates.crt' > Resolving www.numbeo.com (www.numbeo.com)... 209.126.119.66 > Connecting to www.numbeo.com (www.numbeo.com)|209.126.119.66|:443... > connected. > HTTP request sent, awaiting response... No data received. > Retrying. > > --2020-12-15 15:38:15-- (try: 2) > https://www.numbeo.com/admin/DownloadLastDbBackupServlet?numbeoPass=xxx > Connecting to www.numbeo.com (www.numbeo.com)|209.126.119.66|:443... > connected. > HTTP request sent, awaiting response... No data received. > Retrying. > > > We have a self-monitoring script which runs on server and when the server > is not working properly it does a log save and the service restart. How do you detect this state? Just make a request and if you get "No data received" from curl, you restart the server? > We are using currently Apache 8.5.5 and the Connector is set-up as follows > (using Tomcat native): > <Connector port="8181" > protocol="org.apache.coyote.http11.Http11AprProtocol" > SSLCertificateFile="/etc/letsencrypt/live/numbeo.com/cert.pem" > SSLCertificateKeyFile="/etc/letsencrypt/live/ > numbeo.com/privkey.pem" > SSLCertificateChainFile="/etc/letsencrypt/live/ > numbeo.com/chain.pem" > SSLVerifyClient="optional" SSLProtocol="TLSv1+TLSv1.1+TLSv1.2" > connectionTimeout="20000" acceptCount="30000" > acceptorThreadCount="4" > compression="on" maxConnections="50000" maxThreads="500" > > compressableMimeType="text/html,text/xml,text/plain,text/css,text/javascript,application/javascript,image/svg+xml,image/svg,image/png,image/jpeg" > > useSendfile="false" > maxHttpHeaderSize="16392" SSLEnabled="true" > enableLookups="false" > scheme="https" secure="true" clientAuth="false" > useBodyEncodingForURI="true" > URIEncoding="UTF-8" > /> I see you are using Let's Encrypt. How are you managing the rotating of the keys and certificates? > *What would be the next steps how to identify the problem and perhaps > solve it?* What have you done so far? > If we should migrate to the new version, should we try first the last > 8.5 version or the last 9 Tomcat version? I think in general you should upgrade to the latest version which makes sense for you. That's probably 8.5.61 for now, but maybe you should make a plan to migrate to Tomcat 9.x over the next few months or years. > Our JSTACK when the last problem occurred is as follows: > > root@condor1796 ~ # cat jstack.log > [...] I don't see anything that sticks out in your thread dump. > Attaching to process ID 27753, please wait... > Debugger attached successfully. > Server compiler detected. > JVM version is 25.101-b13 > Deadlock Detection: That JVM seems fairly old, too. Consider upgrading to latest Java 8 VM (or beyond, if appropriate). -chris --------------------------------------------------------------------- To unsubscribe, e-mail: [hidden email] For additional commands, e-mail: [hidden email] |
Re: native connector, server problems with "No data received", what could be causing it?
|
|
On Wed, Dec 16, 2020 at 3:27 PM Christopher Schultz <
[hidden email]> wrote: > > We have a self-monitoring script which runs on server and when the server > > is not working properly it does a log save and the service restart. > > How do you detect this state? Just make a request and if you get "No > data received" from curl, you restart the server? > If there is an error code or the specific text doesn't appear on the response we monitor the state and do /etc/init.d/tomcat restart. The full script is: #!/bin/bash serverFailure=0 cd /root rm /root/numbeo_test.out #wget -t 1 -T 5 --no-proxy --no-cache --cache=off -q 'localhost:8080/cost-of-living/city_result.jsp?country=Ireland&city=Dublin' -O /root/numbeo_test.out #curl -L -m 2 -v -o /root/numbeo_test.out --trace curl.log 'localhost:8008/cost-of-living/in/Dublin' curl -L -m 2 -v --insecure -o /root/numbeo_test.out --trace curl.log ' https://localhost:8181/cost-of-living/in/Dublin' wgetOutput=$? grep -q "entries in the past" /root/numbeo_test.out if [ $? != 0 ]; then cd /root rm /root/numbeo_test.out sleep 10s #wget -t 2 -T 2 --no-proxy --no-cache --cache=off -q 'localhost:8080/cost-of-living/city_result.jsp?country=Ireland&city=Dublin' -O /root/numbeo_test.out #curl -L -m 2 --retry 1 -v -o /root/numbeo_test.out --trace curl.log 'localhost:8008/cost-of-living/in/Dublin' curl -L -m 2 -v --insecure -o /root/numbeo_test.out --trace curl.log ' https://localhost:8181/cost-of-living/in/Dublin' wgetOutput=$? grep -q "entries in the past" /root/numbeo_test.out if [ $? != 0 ]; then #echo 'server is down!'; ps -eo pid,comm | while read pid command do if [[ "$command" = "java" ]] then echo $pid DATE=`date +%Y-%m-%d` echo ${wgetOutput} > ~/wget_${DATE}_${pid}.log cp /root/numbeo_test.out > ~/numbeo_test_out_${DATE}_${pid}.log jstack -J-d64 -F $pid > ~/jstack_${DATE}_${pid}.log iostat > ~/iostat_${DATE}_${pid}.log vmstat > ~/vmstat_${DATE}_${pid}.log netstat -tnp > ~/netstat_${DATE}_${pid}.log netstat -anp |grep 'tcp\|udp' | awk '{print $5}' | cut -d: -f1 | sort | uniq -c | sort -n > ~/netstat_anp_outline_${DATE}_${pid}.log ps aux > ~/ps_aux_${DATE}_${pid}.log tail -n 5000 ~glassfish/apache-tomcat-8.5.5/logs/catalina.out > ~/catalina_out_${DATE}_${pid}.log break fi done echo 'too many server failures... going to rebootsoftly' >> ~/reboot.log ; date | mail -s "Numbeo soft reset" [hidden email] date >> ~/reboot.log killall -9 java /root/fix_letsencrypt_chmod.sh #/etc/init.d/glassfish start /etc/init.d/tomcat start #reboot fi fi I see you are using Let's Encrypt. How are you managing the rotating of > the keys and certificates? > Crontab: 5 1 1 * * /root/renew_cert_numbeo.sh root@condor1796 ~ # cat renew_cert_numbeo.sh #!/bin/bash mkdir -p /tmp/letsencrypt/public_html certbot certonly -n --force-renewal --webroot --webroot-path /tmp/letsencrypt/public_html -d numbeo.com -d www.numbeo.com \ -d es.numbeo.com -d pt.numbeo.com -d fr.numbeo.com -d ru.numbeo.com -d ja.numbeo.com -d de.numbeo.com -d nl.numbeo.com \ -d it.numbeo.com -d zh.numbeo.com -d ar.numbeo.com -d jobs.numbeo.com \ --agree-tos --email [hidden email] /root/fix_letsencrypt_chmod.sh if [ $? != 0 ]; then date | mail -s "Lets encrypt renew certificate fails for numbeo.com" [hidden email] else /etc/init.d/tomcat restart fi root@condor1796 ~ # cat fix_letsencrypt_chmod.sh #!/bin/bash chmod o+rx /etc/letsencrypt chmod -R o+rx /etc/letsencrypt/* root@condor1796 ~ # > > *What would be the next steps how to identify the problem and perhaps > > solve it?* > What have you done so far? > aaah... reading the Tomcat source to try to understand the state of Threads. I don't see anything that sticks out in your thread dump. > There are several threads which are trying to get monitor in AprEndpoint$Poller.add and no thread seems to be blocking it. Don't you find it weird: root@condor1796 ~ # grep Poller jstack_2020-12-16_31415.log | grep "Apr" - org.apache.tomcat.util.net.AprEndpoint$Poller.add(long, long, int) @bci=102, line=1398 (Compiled frame) - org.apache.tomcat.util.net.AprEndpoint$Poller.access$500(org.apache.tomcat.util.net.AprEndpoint$Poller, long, long, int) @bci=5, line=1157 (Compiled frame) - org.apache.tomcat.util.net.AprEndpoint$Poller.add(long, long, int) @bci=102, line=1398 (Compiled frame) - org.apache.tomcat.util.net.AprEndpoint$Poller.access$500(org.apache.tomcat.util.net.AprEndpoint$Poller, long, long, int) @bci=5, line=1157 (Compiled frame) - org.apache.tomcat.util.net.AprEndpoint$Poller.add(long, long, int) @bci=102, line=1398 (Compiled frame) - org.apache.tomcat.util.net.AprEndpoint$Poller.access$500(org.apache.tomcat.util.net.AprEndpoint$Poller, long, long, int) @bci=5, line=1157 (Compiled frame) - org.apache.tomcat.util.net.AprEndpoint$Poller.add(long, long, int) @bci=102, line=1398 (Compiled frame) - org.apache.tomcat.util.net.AprEndpoint$Poller.access$500(org.apache.tomcat.util.net.AprEndpoint$Poller, long, long, int) @bci=5, line=1157 (Compiled frame) - org.apache.tomcat.util.net.AprEndpoint$Poller.add(long, long, int) @bci=102, line=1398 (Compiled frame) - org.apache.tomcat.util.net.AprEndpoint$Poller.access$500(org.apache.tomcat.util.net.AprEndpoint$Poller, long, long, int) @bci=5, line=1157 (Compiled frame) - org.apache.tomcat.util.net.AprEndpoint$Poller.add(long, long, int) @bci=102, line=1398 (Compiled frame) - org.apache.tomcat.util.net.AprEndpoint$Poller.access$500(org.apache.tomcat.util.net.AprEndpoint$Poller, long, long, int) @bci=5, line=1157 (Compiled frame) - org.apache.tomcat.util.net.AprEndpoint$Poller.add(long, long, int) @bci=102, line=1398 (Compiled frame) - org.apache.tomcat.util.net.AprEndpoint$Poller.access$500(org.apache.tomcat.util.net.AprEndpoint$Poller, long, long, int) @bci=5, line=1157 (Compiled frame) - org.apache.tomcat.util.net.AprEndpoint$Poller.run() @bci=1713, line=1799 (Compiled frame; information may be imprecise) Thread 22685: (state = BLOCKED) - org.apache.tomcat.util.net.AprEndpoint$Poller.add(long, long, int) @bci=102, line=1398 (Compiled frame) - org.apache.tomcat.util.net.AprEndpoint$Poller.access$500(org.apache.tomcat.util.net.AprEndpoint$Poller, long, long, int) @bci=5, line=1157 (Compiled frame) - org.apache.tomcat.util.net.AprEndpoint$AprSocketWrapper.registerReadInterest() @bci=48, line=2560 (Compiled frame) - org.apache.coyote.AbstractProtocol$ConnectionHandler.process(org.apache.tomcat.util.net.SocketWrapperBase, org.apache.tomcat.util.net.SocketEvent) @bci=643, line=870 (Compiled frame) - org.apache.tomcat.util.net.AprEndpoint$SocketProcessor.doRun() @bci=15, line=2241 (Compiled frame) - org.apache.tomcat.util.net.SocketProcessorBase.run() @bci=21, line=49 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=95, line=1142 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame) - org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() @bci=4, line=61 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Compiled frame) The Poller add(...) method is as follows: private void add(long socket, long timeout, int flags) { if (log.isDebugEnabled()) { String msg = sm.getString("endpoint.debug.pollerAdd", Long.valueOf(socket), Long.valueOf(timeout), Integer.valueOf(flags)); if (log.isTraceEnabled()) { log.trace(msg, new Exception()); } else { log.debug(msg); } } if (timeout <= 0) { // Always put a timeout in timeout = Integer.MAX_VALUE; } synchronized (this) { // Add socket to the list. Newly added sockets will wait // at most for pollTime before being polled. if (addList.add(socket, timeout, flags)) { this.notify(); } } } Don't you find it weird that all threads are trying to get synchronized on a Poller instance and no one is in this block or another synchronized block/method? > > > Attaching to process ID 27753, please wait... > > Debugger attached successfully. > > Server compiler detected. > > JVM version is 25.101-b13 > > Deadlock Detection: > > That JVM seems fairly old, too. Consider upgrading to latest Java 8 VM > (or beyond, if appropriate). > > -chris > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [hidden email] > For additional commands, e-mail: [hidden email] > > |
Re: native connector, server problems with "No data received", what could be causing it?
|
|
Mladen,
On 12/16/20 10:12, Mladen Adamović wrote: > On Wed, Dec 16, 2020 at 3:27 PM Christopher Schultz > <[hidden email] <mailto:[hidden email]>> wrote: > > > We have a self-monitoring script which runs on server and when > the server > > is not working properly it does a log save and the service restart. > > How do you detect this state? Just make a request and if you get "No > data received" from curl, you restart the server? > > > If there is an error code or the specific text doesn't appear on the > response we monitor the state and do /etc/init.d/tomcat restart. > The full script is: > #!/bin/bash > serverFailure=0 > cd /root > rm /root/numbeo_test.out > #wget -t 1 -T 5 --no-proxy --no-cache --cache=off -q > 'localhost:8080/cost-of-living/city_result.jsp?country=Ireland&city=Dublin' > -O /root/numbeo_test.out > #curl -L -m 2 -v -o /root/numbeo_test.out --trace curl.log > 'localhost:8008/cost-of-living/in/Dublin' > curl -L -m 2 -v --insecure -o /root/numbeo_test.out --trace curl.log > 'https://localhost:8181/cost-of-living/in/Dublin > <https://localhost:8181/cost-of-living/in/Dublin>' > wgetOutput=$? > > grep -q "entries in the past" /root/numbeo_test.out > if [ $? != 0 ]; then > cd /root > rm /root/numbeo_test.out > sleep 10s > #wget -t 2 -T 2 --no-proxy --no-cache --cache=off -q > 'localhost:8080/cost-of-living/city_result.jsp?country=Ireland&city=Dublin' > -O /root/numbeo_test.out > #curl -L -m 2 --retry 1 -v -o /root/numbeo_test.out --trace curl.log > 'localhost:8008/cost-of-living/in/Dublin' > curl -L -m 2 -v --insecure -o /root/numbeo_test.out --trace curl.log > 'https://localhost:8181/cost-of-living/in/Dublin > <https://localhost:8181/cost-of-living/in/Dublin>' > wgetOutput=$? > grep -q "entries in the past" /root/numbeo_test.out > > if [ $? != 0 ]; then > #echo 'server is down!'; > ps -eo pid,comm | while read pid command > do > if [[ "$command" = "java" ]] > then > echo $pid > DATE=`date +%Y-%m-%d` > echo ${wgetOutput} > ~/wget_${DATE}_${pid}.log > cp /root/numbeo_test.out > > ~/numbeo_test_out_${DATE}_${pid}.log > jstack -J-d64 -F $pid > ~/jstack_${DATE}_${pid}.log > iostat > ~/iostat_${DATE}_${pid}.log > vmstat > ~/vmstat_${DATE}_${pid}.log > netstat -tnp > ~/netstat_${DATE}_${pid}.log > netstat -anp |grep 'tcp\|udp' | awk '{print $5}' | cut > -d: -f1 | sort | uniq -c | sort -n > > ~/netstat_anp_outline_${DATE}_${pid}.log > ps aux > ~/ps_aux_${DATE}_${pid}.log > tail -n 5000 > ~glassfish/apache-tomcat-8.5.5/logs/catalina.out > > ~/catalina_out_${DATE}_${pid}.log > break > fi > done > echo 'too many server failures... going to rebootsoftly' >> ~/reboot.log ; > date | mail -s "Numbeo soft reset" [hidden email] > <mailto:[hidden email]> > date >> ~/reboot.log > killall -9 java > /root/fix_letsencrypt_chmod.sh > #/etc/init.d/glassfish start > /etc/init.d/tomcat start > #reboot > fi > fi That seems a little fragile, but it's your server so I guess you can do what you want. > I see you are using Let's Encrypt. How are you managing the rotating of > the keys and certificates? > > > Crontab: 5 1 1 * * /root/renew_cert_numbeo.sh > root@condor1796 ~ # cat renew_cert_numbeo.sh > #!/bin/bash > > mkdir -p /tmp/letsencrypt/public_html > certbot certonly -n --force-renewal --webroot --webroot-path > /tmp/letsencrypt/public_html -d numbeo.com <http://numbeo.com> -d > www.numbeo.com <http://www.numbeo.com> \ > -d es.numbeo.com <http://es.numbeo.com> -d pt.numbeo.com > <http://pt.numbeo.com> -d fr.numbeo.com <http://fr.numbeo.com> -d > ru.numbeo.com <http://ru.numbeo.com> -d ja.numbeo.com > <http://ja.numbeo.com> -d de.numbeo.com <http://de.numbeo.com> -d > nl.numbeo.com <http://nl.numbeo.com> \ > -d it.numbeo.com <http://it.numbeo.com> -d zh.numbeo.com > <http://zh.numbeo.com> -d ar.numbeo.com <http://ar.numbeo.com> -d > jobs.numbeo.com <http://jobs.numbeo.com> \ > --agree-tos --email [hidden email] > <mailto:[hidden email]> > > /root/fix_letsencrypt_chmod.sh > if [ $? != 0 ]; then > date | mail -s "Lets encrypt renew certificate fails for numbeo.com > <http://numbeo.com>" [hidden email] > <mailto:[hidden email]> > else > /etc/init.d/tomcat restart > fi > > root@condor1796 ~ # cat fix_letsencrypt_chmod.sh > #!/bin/bash > chmod o+rx /etc/letsencrypt > chmod -R o+rx /etc/letsencrypt/* > > root@condor1796 ~ # I think your scripts will restart Tomcat even when it's not necessary. The $? check before sending the email message looks like it should be checking the result of the certbot command, but it's checking the result of the chmod command instead. (Or maybe the result of the .sh script, which will proably be 0.) > > *What would be the next steps how to identify the problem and perhaps > > solve it?* > What have you done so far? > > > aaah... reading the Tomcat source to try to understand the state of Threads. > > I don't see anything that sticks out in your thread dump. > > > There are several threads which are trying to get monitor in > AprEndpoint$Poller.add and no thread seems to be blocking it. Don't you > find it weird: > > root@condor1796 ~ # grep Poller jstack_2020-12-16_31415.log | grep "Apr" > - org.apache.tomcat.util.net.AprEndpoint$Poller.add(long, long, int) > @bci=102, line=1398 (Compiled frame) I might have found that odd had you posted that in your original message, but you did not. You need to show the full stack trace for that thread to make it meaningful. Sockets are added to the poller all the time. It's not unusual to see that happening. It they are getting *stuck*, that would be bad, of course. > Don't you find it weird that all threads are trying to get synchronized > on a Poller instance and no one is in this block or another synchronized > block/method? I would find it weird if no threads were making any progress. Lots of threads adding sockets to the poller is not out of the ordinary. If you suspect a bug in Tomcat's socket handling, upgrading to the latest 8.5.x release and re-trying would be the best move. There have been many fixes since your 8.5.5 release which is now 4+ years old. -chris --------------------------------------------------------------------- To unsubscribe, e-mail: [hidden email] For additional commands, e-mail: [hidden email] |
Re: native connector, server problems with "No data received", what could be causing it?
|
|
On Wed, Dec 16, 2020 at 7:07 PM Christopher Schultz <
[hidden email]> wrote: > I think your scripts will restart Tomcat even when it's not necessary. > Hm, is this https://stackoverflow.com/questions/5816239/how-do-i-force-tomcat-to-reload-trusted-certificates the way to reload the certificates or is there another suggestion? > The $? check before sending the email message looks like it should be > checking the result of the certbot command, but it's checking the result > of the chmod command instead. (Or maybe the result of the .sh script, > which will proably be 0.) > Correct, thank you this is fixed now. > I might have found that odd had you posted that in your original > message, but you did not. > > You need to show the full stack trace for that thread to make it > meaningful. Sockets are added to the poller all the time. It's not > unusual to see that happening. It they are getting *stuck*, that would > be bad, of course. > I did post full thread dump. > > Don't you find it weird that all threads are trying to get synchronized > > on a Poller instance and no one is in this block or another synchronized > > block/method? > > I would find it weird if no threads were making any progress. Lots of > threads adding sockets to the poller is not out of the ordinary. > > If you suspect a bug in Tomcat's socket handling, upgrading to the > latest 8.5.x release and re-trying would be the best move. There have > been many fixes since your 8.5.5 release which is now 4+ years old. > I have switched today from Apr connector to Nio2 connector and no problem yet. What I have found strange is that processor usage is lower when using Nio2, I have never tested it or tried it before, I have setup APR for performances reason back in 2016. But oddly it seems with Nio2 processor usage is lower. I have tried the last OpenJDK but jstack is not working for me, so switched back to the previous old JRM. Let's see what will happen... > -chris > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [hidden email] > For additional commands, e-mail: [hidden email] > > |
Re: native connector, server problems with "No data received", what could be causing it?
|
|
I have these problems again. To narrow it down I have done the following:
- tested with the latest JDK8 - problem exists - it exists in both Apr connector and Nio2 connector - did log JVM Garbage Collector details - it's not due to Garbage collector Christopher (or anyone else), if we had 1570 requests per minute where this problem happened (approx. 26 per second), what do you think should our Connector params be, it's currently: <Connector port="8181" protocol="org.apache.coyote.http11.Http11Nio2Protocol" SSLCertificateFile="/etc/letsencrypt/live/numbeo.com/cert.pem" SSLCertificateKeyFile="/etc/letsencrypt/live/ numbeo.com/privkey.pem" SSLCertificateChainFile="/etc/letsencrypt/live/ numbeo.com/chain.pem" SSLVerifyClient="optional" SSLProtocol="TLSv1+TLSv1.1+TLSv1.2" connectionTimeout="20000" acceptCount="30000" acceptorThreadCount="4" compression="on" maxConnections="50000" maxThreads="500" compressableMimeType="text/html,text/xml,text/plain,text/css,text/javascript,application/javascript,image/svg+xml,image/svg,image/png,image/jpeg" useSendfile="false" maxHttpHeaderSize="16392" SSLEnabled="true" enableLookups="false" scheme="https" secure="true" clientAuth="false" useBodyEncodingForURI="true" URIEncoding="UTF-8" /> On Wed, Dec 16, 2020 at 7:32 PM Mladen Adamović <[hidden email]> wrote: > On Wed, Dec 16, 2020 at 7:07 PM Christopher Schultz < > [hidden email]> wrote: > >> I think your scripts will restart Tomcat even when it's not necessary. >> > > Hm, is this > https://stackoverflow.com/questions/5816239/how-do-i-force-tomcat-to-reload-trusted-certificates > the way to reload the certificates or is there another suggestion? > > >> The $? check before sending the email message looks like it should be >> checking the result of the certbot command, but it's checking the result >> of the chmod command instead. (Or maybe the result of the .sh script, >> which will proably be 0.) >> > > Correct, thank you this is fixed now. > > >> I might have found that odd had you posted that in your original >> message, but you did not. >> >> You need to show the full stack trace for that thread to make it >> meaningful. Sockets are added to the poller all the time. It's not >> unusual to see that happening. It they are getting *stuck*, that would >> be bad, of course. >> > > I did post full thread dump. > > >> > Don't you find it weird that all threads are trying to get synchronized >> > on a Poller instance and no one is in this block or another >> synchronized >> > block/method? >> >> I would find it weird if no threads were making any progress. Lots of >> threads adding sockets to the poller is not out of the ordinary. >> >> If you suspect a bug in Tomcat's socket handling, upgrading to the >> latest 8.5.x release and re-trying would be the best move. There have >> been many fixes since your 8.5.5 release which is now 4+ years old. >> > > I have switched today from Apr connector to Nio2 connector and no problem > yet. > What I have found strange is that processor usage is lower when using > Nio2, I have never tested it or tried it before, I have setup APR for > performances reason back in 2016. But oddly it seems with Nio2 processor > usage is lower. > I have tried the last OpenJDK but jstack is not working for me, so > switched back to the previous old JRM. > Let's see what will happen... > > > > >> -chris >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: [hidden email] >> For additional commands, e-mail: [hidden email] >> >> |
Re: native connector, server problems with "No data received", what could be causing it?
|
|
I have just updated to the latest Tomcat 9.0.41
It seems that the problem doesn't appear anymore. If I get it reappeared, I'll post new details to the Tomcat dev mailing list. On Thu, Dec 17, 2020 at 8:56 AM Mladen Adamović <[hidden email]> wrote: > I have these problems again. To narrow it down I have done the following: > - tested with the latest JDK8 - problem exists > - it exists in both Apr connector and Nio2 connector > - did log JVM Garbage Collector details - it's not due to Garbage collector > > Christopher (or anyone else), if we had 1570 requests per minute where > this problem happened (approx. 26 per second), what do you think should our > Connector params be, it's currently: > > <Connector port="8181" > protocol="org.apache.coyote.http11.Http11Nio2Protocol" > SSLCertificateFile="/etc/letsencrypt/live/ > numbeo.com/cert.pem" > SSLCertificateKeyFile="/etc/letsencrypt/live/ > numbeo.com/privkey.pem" > SSLCertificateChainFile="/etc/letsencrypt/live/ > numbeo.com/chain.pem" > SSLVerifyClient="optional" > SSLProtocol="TLSv1+TLSv1.1+TLSv1.2" > connectionTimeout="20000" acceptCount="30000" > acceptorThreadCount="4" > compression="on" maxConnections="50000" maxThreads="500" > > compressableMimeType="text/html,text/xml,text/plain,text/css,text/javascript,application/javascript,image/svg+xml,image/svg,image/png,image/jpeg" > > useSendfile="false" > maxHttpHeaderSize="16392" SSLEnabled="true" > enableLookups="false" > scheme="https" secure="true" clientAuth="false" > useBodyEncodingForURI="true" > URIEncoding="UTF-8" > /> > > > > > > On Wed, Dec 16, 2020 at 7:32 PM Mladen Adamović <[hidden email]> > wrote: > >> On Wed, Dec 16, 2020 at 7:07 PM Christopher Schultz < >> [hidden email]> wrote: >> >>> I think your scripts will restart Tomcat even when it's not necessary. >>> >> >> Hm, is this >> https://stackoverflow.com/questions/5816239/how-do-i-force-tomcat-to-reload-trusted-certificates >> the way to reload the certificates or is there another suggestion? >> >> >>> The $? check before sending the email message looks like it should be >>> checking the result of the certbot command, but it's checking the result >>> of the chmod command instead. (Or maybe the result of the .sh script, >>> which will proably be 0.) >>> >> >> Correct, thank you this is fixed now. >> >> >>> I might have found that odd had you posted that in your original >>> message, but you did not. >>> >>> You need to show the full stack trace for that thread to make it >>> meaningful. Sockets are added to the poller all the time. It's not >>> unusual to see that happening. It they are getting *stuck*, that would >>> be bad, of course. >>> >> >> I did post full thread dump. >> >> >>> > Don't you find it weird that all threads are trying to get >>> synchronized >>> > on a Poller instance and no one is in this block or another >>> synchronized >>> > block/method? >>> >>> I would find it weird if no threads were making any progress. Lots of >>> threads adding sockets to the poller is not out of the ordinary. >>> >>> If you suspect a bug in Tomcat's socket handling, upgrading to the >>> latest 8.5.x release and re-trying would be the best move. There have >>> been many fixes since your 8.5.5 release which is now 4+ years old. >>> >> >> I have switched today from Apr connector to Nio2 connector and no >> problem yet. >> What I have found strange is that processor usage is lower when using >> Nio2, I have never tested it or tried it before, I have setup APR for >> performances reason back in 2016. But oddly it seems with Nio2 processor >> usage is lower. >> I have tried the last OpenJDK but jstack is not working for me, so >> switched back to the previous old JRM. >> Let's see what will happen... >> >> >> >> >>> -chris >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: [hidden email] >>> For additional commands, e-mail: [hidden email] >>> >>> |
Re: native connector, server problems with "No data received", what could be causing it?
|
|
In reply to this post by Mladen Adamović
Mladen,
On 12/16/20 13:32, Mladen Adamović wrote: > On Wed, Dec 16, 2020 at 7:07 PM Christopher Schultz > <[hidden email] <mailto:[hidden email]>> wrote: > > I think your scripts will restart Tomcat even when it's not necessary. > > > Hm, is this > https://stackoverflow.com/questions/5816239/how-do-i-force-tomcat-to-reload-trusted-certificates > <https://stackoverflow.com/questions/5816239/how-do-i-force-tomcat-to-reload-trusted-certificates> > the way to reload the certificates or is there another suggestion? The technique showed there seems perfectly fine. Your script completely restarts Tomcat instead of just pinging the connector to reload its configuration. > I might have found that odd had you posted that in your original > message, but you did not. > > You need to show the full stack trace for that thread to make it > meaningful. Sockets are added to the poller all the time. It's not > unusual to see that happening. It they are getting *stuck*, that would > be bad, of course. > > > I did post full thread dump. I don't see it in any of your previous posts. > > Don't you find it weird that all threads are trying to get > synchronized > > on a Poller instance and no one is in this block or another > synchronized > > block/method? > > I would find it weird if no threads were making any progress. Lots of > threads adding sockets to the poller is not out of the ordinary. > > If you suspect a bug in Tomcat's socket handling, upgrading to the > latest 8.5.x release and re-trying would be the best move. There have > been many fixes since your 8.5.5 release which is now 4+ years old. > > > I have switched today from Apr connector to Nio2 connector and no > problem yet. > What I have found strange is that processor usage is lower when using > Nio2, I have never tested it or tried it before, I have setup APR for > performances reason back in 2016. But oddly it seems with Nio2 processor > usage is lower. Theoretically, NIO2 should be the lowest CPU usage for the same workload. APR does real blocking for certain operations while NIO/NIO2 does "simluated" blocking which usually ends up with slightly higher CPU usage (because there is work being actively done wheras blocking usually just uses IO interrupts). > I have tried the last OpenJDK but jstack is not working for me, so > switched back to the previous old JRM. jstack isn't working? Hmm. Make sure that your jstack and Java binary are the same version. If you are a different user you might have to elevate privileges and/or use "jstack -F". -chris --------------------------------------------------------------------- To unsubscribe, e-mail: [hidden email] For additional commands, e-mail: [hidden email] |
Re: native connector, server problems with "No data received", what could be causing it?
|
|
On Thu, Dec 17, 2020 at 7:57 PM Christopher Schultz <
[hidden email]> wrote: > jstack isn't working? Hmm. Make sure that your jstack and Java binary > are the same version. If you are a different user you might have to > elevate privileges and/or use "jstack -F". > I have tried all 8 variations with sudo, -F , -J-d64 parameters and finally it worked only with simply sudo -u runninguser jstack without -F and -J-d64 params > -chris > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [hidden email] > For additional commands, e-mail: [hidden email] > > |
«
Return to Tomcat - User
|
1 view|%1 views
| Free forum by Nabble | Edit this page |