Tomcat Large Payload Truncated

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Tomcat Large Payload Truncated

Bhavesh Mistry
Hi All,


I am running embedded *tomcat*-embed-core-9.0.36.jar and large payload
*179292* and HTTPS 1.1 traffic.  When I use curl command with
transfer encoding or without, both request JSON payload is truncated, and
the application can not parse it.  The smaller payload works fine.


Both small payload and large payload I see the following exception when I
run tomcat in debug mode. Can this Exception truncate payload silently?  I
have been struggling to find you why this is behavior.  I would
appreciate any help you can provide.  Thanks, a lot in advance.



Thanks,
Bhavesh




> PUT XXXX HTTP/1.1

> Host: 10.40.216.165:9182

> User-Agent: curl/7.64.1

> Accept: */*

> Content-Type: application/json

> Content-Length: *179292*

> Expect: 100-continue

>

< HTTP/1.1 100

* We are completely uploaded and fine

< HTTP/1.1 400

< accept: */*

< Accept-Encoding: deflate

< Allow: GET, POST, OPTIONS, HEAD

< Response-Time: Sun, 28 Jun 2020 12:6:56 PDT

< Vary: Accept-Encoding

< X-Content-Type-Options: nosniff

< X-XSS-Protection: 1; mode=block

< Cache-Control: no-cache, no-store, max-age=0, must-revalidate

< Pragma: no-cache

< Expires: 0

< Strict-Transport-Security: max-age=31536000 ; includeSubDomains

< X-Frame-Options: DENY

< Content-Type: text/json

< Transfer-Encoding: chunked

< Date: Sun, 28 Jun 2020 19:06:56 GMT

< Connection: close

<

{"errors": {"error": [{"error-message": "*end of file*", "error-urlpath":
"xxxx", "error-tag": "malformed-message"}]}}

* Closing connection 0

* TLSv1.2 (OUT), TLS alert, close notify (256):



*Exception:*


[28-Jun-2020
11:58:11.507][DEBUG][tomcat-exec-19][org.apache.tomcat.util.net.SocketWrapperBase]
Socket: [org.apache.tomcat.util.net.NioEndpoint$NioSocketWrapper@36f89cdc
:org.apache.tomcat.util.net.SecureNioChannel@63fc40b8:java.nio.channels.SocketChannel[connected
local=SDWAN-VOAE1/10.40.216.165:9182 remote=/10.10.10.10:62131]], Read from
buffer: [0]

[28-Jun-2020
11:58:11.539][DEBUG][tomcat-exec-19][org.apache.tomcat.util.net.NioEndpoint]
Socket: [org.apache.tomcat.util.net.NioEndpoint$NioSocketWrapper@36f89cdc
:org.apache.tomcat.util.net.SecureNioChannel@63fc40b8:java.nio.channels.Soc*ketChannel[connected
local=SDWAN-VOAE1/10.40.216.165:9182 <http://10.40.216.165:9182>
remote=/10.10.10.10:62131]], Read direct from socket: [15489]*

*[28-Jun-2020
11:58:11.541][DEBUG][tomcat-exec-19][org.apache.tomcat.util.net.jsse.JSSESupport]
Error trying to obtain a certificate from the client*

*javax.net.ssl.SSLPeerUnverifiedException: peer not authenticated*

* at
java.base/sun.security.ssl.SSLSessionImpl.getPeerCertificates(SSLSessionImpl.java:558)*

* at
org.apache.tomcat.util.net.jsse.JSSESupport.getPeerCertificateChain(JSSESupport.java:106)*

* at
org.apache.coyote.AbstractProcessor.populateSslRequestAttributes(AbstractProcessor.java:779)*

at org.apache.coyote.AbstractProcessor.action(AbstractProcessor.java:482)

at org.apache.coyote.Request.action(Request.java:432)

at org.apache.catalina.connector.Request.getAttribute(Request.java:892)

at org.apache.catalina.connector.Request.getAttributeNames(Request.java:961)

at
org.apache.catalina.connector.RequestFacade.getAttributeNames(RequestFacade.java:298)

at
javax.servlet.ServletRequestWrapper.getAttributeNames(ServletRequestWrapper.java:96)

at
javax.servlet.ServletRequestWrapper.getAttributeNames(ServletRequestWrapper.java:96)

at
javax.servlet.ServletRequestWrapper.getAttributeNames(ServletRequestWrapper.java:96)

at
javax.servlet.ServletRequestWrapper.getAttributeNames(ServletRequestWrapper.java:96)

at
javax.servlet.ServletRequestWrapper.getAttributeNames(ServletRequestWrapper.java:96)

at
org.apache.camel.http.common.DefaultHttpBinding.populateAttachments(DefaultHttpBinding.java:304)

at
org.apache.camel.http.common.DefaultHttpBinding.readBody(DefaultHttpBinding.java:217)

at
org.apache.camel.http.common.DefaultHttpBinding.readRequest(DefaultHttpBinding.java:115)

at org.apache.camel.http.common.HttpMessage.<init>(HttpMessage.java:55)

at
org.apache.camel.http.common.CamelServlet.doService(CamelServlet.java:188)

at org.apache.camel.http.common.CamelServlet.service(CamelServlet.java:80)

at javax.servlet.http.HttpServlet.service(HttpServlet.java:750)

at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)

at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)

at
org.springframework.web.multipart.support.MultipartFilter.doFilterInternal(MultipartFilter.java:125)

at
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)

at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)

at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)

at
org.apache.catalina.filters.RemoteIpFilter.doFilter(RemoteIpFilter.java:904)

at
org.apache.catalina.filters.RemoteIpFilter.doFilter(RemoteIpFilter.java:961)

at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)

at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)

at
org.springframework.web.filter.AbstractRequestLoggingFilter.doFilterInternal(AbstractRequestLoggingFilter.java:289)

at
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)

at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)

at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)

at
com.versa.rest.config.TemplateAPIFilter.doFilter(TemplateAPIFilter.java:39)

at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)

at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)

at
com.versa.rest.config.CDBTransactionFilter.doFilter(CDBTransactionFilter.java:23)

at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)

at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)

at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:320)

at
org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:126)

at
org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:90)

at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)

at
org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:118)

at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)

at
org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:137)

at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)

at
org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:111)

at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)

at
org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:158)

at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)

at
org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:63)

at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)

at
com.versa.rest.config.AuthenticationFilter.doFilter(AuthenticationFilter.java:113)

at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)

at
org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:116)

at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)

at
org.springframework.security.web.header.HeaderWriterFilter.doHeadersAfter(HeaderWriterFilter.java:92)

at
org.springframework.security.web.header.HeaderWriterFilter.doFilterInternal(HeaderWriterFilter.java:77)

at
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)

at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)

at
org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:105)

at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)

at
org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:56)

at
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)

at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)

at
org.springframework.security.web.access.channel.ChannelProcessingFilter.doFilter(ChannelProcessingFilter.java:157)

at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)

at
org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:215)

at
org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:178)

at
org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358)

at
org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271)

at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)

at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)

at
org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201)

at
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)

at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)

at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)

at
com.versa.rest.config.ContentCachingFilter.doFilterInternal(ContentCachingFilter.java:28)

at
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)

at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)

at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)

at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:202)

at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)

at
org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:541)

at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139)

at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)

at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74)

at
org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:690)

at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)

at
org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:373)

at
org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)

at
org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:868)

at
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1590)

at
org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)

at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)

at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)

at
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)

at java.base/java.lang.Thread.run(Thread.java:834)
Reply | Threaded
Open this post in threaded view
|

Re: Tomcat Large Payload Truncated

markt
On 28/06/2020 20:14, Bhavesh Mistry wrote:

> Hi All,
>
>
> I am running embedded *tomcat*-embed-core-9.0.36.jar and large payload
> *179292* and HTTPS 1.1 traffic.  When I use curl command with
> transfer encoding or without, both request JSON payload is truncated, and
> the application can not parse it.  The smaller payload works fine.
>
>
> Both small payload and large payload I see the following exception when I
> run tomcat in debug mode. Can this Exception truncate payload silently?  I
> have been struggling to find you why this is behavior.  I would
> appreciate any help you can provide.  Thanks, a lot in advance.
>
>
> Thanks,
> Bhavesh

Given:

>> Content-Length: *179292*

and

> *javax.net.ssl.SSLPeerUnverifiedException: peer not authenticated*

and that small PUTs work, I suspect you need to increase maxSavePostSize
on the HTTPS connector.

http://tomcat.apache.org/tomcat-9.0-doc/config/http.html

Mark

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Tomcat Large Payload Truncated

Bhavesh Mistry
Hi Mark,

Thank you for responding.  I have one more question.  This is spring-boot 2
application REST API server and it does not accept Cookie or session
(timeout is set to zero).    Auth happens through Authorized header. We
have set 10mb for maxPostSize.  Does maxSavePostSize takes precedence over
maxPostSize ?  I will set maxSavePostSize to -1 to disable it.

Also, I have another question.  When Payload is as large as 10mb (json
post),  does payload body in JVM MEMORY or offloaded to FileInputStream ?


Thanks, a lot for your help!

Thanks,

Bhavesh
Reply | Threaded
Open this post in threaded view
|

Re: Tomcat Large Payload Truncated

markt
On 30/06/2020 03:12, Bhavesh Mistry wrote:
> Hi Mark,
>
> Thank you for responding.  I have one more question.  This is spring-boot 2
> application REST API server and it does not accept Cookie or session
> (timeout is set to zero).    Auth happens through Authorized header. We
> have set 10mb for maxPostSize.  Does maxSavePostSize takes precedence over
> maxPostSize ?

No. They are different settings.

>  I will set maxSavePostSize to -1 to disable it.

That is a DoS risk.

> Also, I have another question.  When Payload is as large as 10mb (json
> post),  does payload body in JVM MEMORY or offloaded to FileInputStream ?

Where POST data is saved for authentication is, it is always in memory.
For other POSTs, it will depend on the application configuration and/or
code.

Mark

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Tomcat Large Payload Truncated

Christopher Schultz-2
In reply to this post by Bhavesh Mistry
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Bhavesh,

On 6/29/20 22:12, Bhavesh Mistry wrote:
> Hi Mark,
>
> Thank you for responding.  I have one more question.  This is
> spring-boot 2 application REST API server and it does not accept
> Cookie or session (timeout is set to zero).    Auth happens through
> Authorized header. We have set 10mb for maxPostSize.  Does
> maxSavePostSize takes precedence over maxPostSize ?  I will set
> maxSavePostSize to -1 to disable it.

Sounds like what you really want to use is:

Expect: 100-continue

And then only send the 10MiB payload if you get a "100 Continue" respons
e.

> Also, I have another question.  When Payload is as large as 10mb
> (json post),  does payload body in JVM MEMORY or offloaded to
> FileInputStream ?

That depends upon how you are doing authentication. When you say
authentication is done "through an authorized header", are you saying
that Tomcat is performing the authentication or not? Is Tomcat saving
the request and asking the client to authenticate) as it does with
e.g. FORM login? Or does your application reject the request with a
response header and the client has to re-try with the authenticated
header AND the large JSON request again?

- -chris
-----BEGIN PGP SIGNATURE-----
Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/

iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAl796fcACgkQHPApP6U8
pFjjJw/6AwoXu4eTXa86JvHn8qP1m9fls+AMjQBM3VePfEKxa0LibjiPGxwjsy7/
SstRvv+8rJ5Tan6IdGgSFr+BsHXDgWa/4Q+PirpjIjcO7xOMlvsHC0xaA8sNSKhD
DbK0sCrrKuvixX3AwUCXz0wuTHrZBFmznvVkM0rh+/XXJxq5n5yd18J36KqIaR7d
a1eef8cbPkPo+ds9ci3VYsy50TtEmI6tGdjQMMko1QxnXcUHzz/pTjDN5qttE4g1
+K9CI4zG8qYVuMEvoW+679knq9UUWLeeBO71T7TQea2WJkoyMw9UY2ksH7SIstlY
+GhXs8/fWQ+YdZ+eYxnkuNXOOes8L/UvC0+Ea13Y8u1eiD7INXsGhc1gTrZ+ct16
i4jGM1GYhHMxFDsXcs5uhL1/7ew+EgTR3dBuNsrYKASN/5DTpIlcIa+xpqb2uoyL
Irf9jGkRbNYneI52Woopf1SGAT+hCqGt7yiN7grVdyo3pUA82xqcuM/SwLilyEru
LkkS6nQz1l2YUQi2U1OYwYdt3NxlD94FmGmhzEBPaw2hYvXwrPMBYTY3iuEueOqZ
2L3DE/K/f8CX+0ogJZKMU+KbZ2itW7DL1183AwiZx9Y19i1nr6pjyb7ius+f0zkr
ML9x7mYrRLyL12kkwbLdOgx/xmsflOC1WQCElJ/sib7dS6V/skg=
=ty2B
-----END PGP SIGNATURE-----

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]