The purpose of maxPostSize

[Bytecode-3]

According to Tomcat docs, the purpose of maxPostSize is:

The maximum size in bytes of the POST which will be handled by the container FORM URL parameter parsing. The limit can be disabled by setting this attribute to a value less than or equal to 0. If not specified, this attribute is set to 2097152 (2 megabytes).

Now the question is what's meant by "the container FORM URL parameter parsing"? What's a FORM URL? What's the container's FORM URL parameter parsing? Also, what is a possible use case of this parameter?

Thanks in advance,
Bytecode

[André Warnier]

Bytecode wrote:
> According to Tomcat docs, the purpose of maxPostSize is:
>
> The maximum size in bytes of the POST which will be handled by the container FORM URL parameter parsing. The limit can be disabled by setting this attribute to a value less than or equal to 0. If not specified, this attribute is set to 2097152 (2 megabytes).
>
> Now the question is what's meant by "the container FORM URL parameter parsing"? What's a FORM URL? What's the container's FORM URL parameter parsing? Also, what is a possible use case of this parameter?
>
As a ganeral explanation : at the base the "maximum post size" setting
(available in Tomcat but also in Apache httpd and probably most
webservers), is a security measure.
It is there to avoid the possibility for some miscreant to overwhelm
your server by sending it a POST request with a body of, for example, 10
Gigabyte, through a slow connection.
In the absence of such a limit, this would force the server to dedicate
a process to just sit there reading the content of the POST, possibly
for hours.  It would also tie up a number of resources at the server
side (to store the POST content), and maybe cause difficulties when the
POST is finally terminated and the body has to be parsed etc..
In other words, at best this might cause a denial-of-service, and at
worst crash your server with for example an out-of-memory condition.
The setting is thus available so that you, the application developer,
can determine which is the maximum likely valid size of a POST to your
server or application, and reject POSTs above this limit.
The webserver will then still accept POST requests, but as it is reading
the POST body, it will count the bytes, and as soon as this limit is
reached, it will interrupt this request and reject it with an error.

As to the "FORM URL parameter parsing" expression : to my knowledge,
this does not really correspond to any formal HTTP RFC or Servlet Spec
well-defined expression.  It is probably just an expression chosen by
the writer of the documentation you refer to, to convey the general idea
that the webserver, when it processes a POST request, at some point has
to parse the body of the request to extract the various request
parameter names and contents.
And, before it can start doing that, it must have the entire POST body
available, which means the entire POST body has been read and saved
somewhere.  Which rejoins the explanation above.




---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

[Bytecode-3]

Thanks for the response. It now makes sense, but I still don't understand why this is being referred to as a "FORM URL" or "the container's FORM URL".

Thanks in advance,
Bytecode

On 02/05/2010, at 8:48 PM, André Warnier wrote:

> Bytecode wrote:
>> According to Tomcat docs, the purpose of maxPostSize is:
>> The maximum size in bytes of the POST which will be handled by the container FORM URL parameter parsing. The limit can be disabled by setting this attribute to a value less than or equal to 0. If not specified, this attribute is set to 2097152 (2 megabytes).
>> Now the question is what's meant by "the container FORM URL parameter parsing"? What's a FORM URL? What's the container's FORM URL parameter parsing? Also, what is a possible use case of this parameter?
> As a ganeral explanation : at the base the "maximum post size" setting (available in Tomcat but also in Apache httpd and probably most webservers), is a security measure.
> It is there to avoid the possibility for some miscreant to overwhelm your server by sending it a POST request with a body of, for example, 10 Gigabyte, through a slow connection.
> In the absence of such a limit, this would force the server to dedicate a process to just sit there reading the content of the POST, possibly for hours.  It would also tie up a number of resources at the server side (to store the POST content), and maybe cause difficulties when the POST is finally terminated and the body has to be parsed etc..
> In other words, at best this might cause a denial-of-service, and at worst crash your server with for example an out-of-memory condition.
> The setting is thus available so that you, the application developer, can determine which is the maximum likely valid size of a POST to your server or application, and reject POSTs above this limit.
> The webserver will then still accept POST requests, but as it is reading the POST body, it will count the bytes, and as soon as this limit is reached, it will interrupt this request and reject it with an error.
>
> As to the "FORM URL parameter parsing" expression : to my knowledge, this does not really correspond to any formal HTTP RFC or Servlet Spec well-defined expression.  It is probably just an expression chosen by the writer of the documentation you refer to, to convey the general idea that the webserver, when it processes a POST request, at some point has to parse the body of the request to extract the various request parameter names and contents.
> And, before it can start doing that, it must have the entire POST body available, which means the entire POST body has been read and saved somewhere.  Which rejoins the explanation above.
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

[markt]

On 02/05/2010 09:57, Bytecode wrote:
> According to Tomcat docs, the purpose of maxPostSize is:
>
> The maximum size in bytes of the POST which will be handled by the container FORM URL parameter parsing. The limit can be disabled by setting this attribute to a value less than or equal to 0. If not specified, this attribute is set to 2097152 (2 megabytes).
>
> Now the question is what's meant by "the container FORM URL parameter parsing"? What's a FORM URL? What's the container's FORM URL parameter parsing? Also, what is a possible use case of this parameter?

It limits the size of data that will be processed automatically by
Tomcat for a POST request where the content type is
application/x-www-form-urlencoded. See section 3.1.1 of the Servlet 3.0
spec.

If the application parses the data then, regardless of content type, the
limit does not apply.

Mark



---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

[André Warnier]

Mark Thomas wrote:

> On 02/05/2010 09:57, Bytecode wrote:
>> According to Tomcat docs, the purpose of maxPostSize is:
>>
>> The maximum size in bytes of the POST which will be handled by the container FORM URL parameter parsing. The limit can be disabled by setting this attribute to a value less than or equal to 0. If not specified, this attribute is set to 2097152 (2 megabytes).
>>
>> Now the question is what's meant by "the container FORM URL parameter parsing"? What's a FORM URL? What's the container's FORM URL parameter parsing? Also, what is a possible use case of this parameter?
>
> It limits the size of data that will be processed automatically by
> Tomcat for a POST request where the content type is
> application/x-www-form-urlencoded. See section 3.1.1 of the Servlet 3.0
> spec.
>
> If the application parses the data then, regardless of content type, the
> limit does not apply.
>

Does this mean that at the Tomcat container level, there is no way to
avoid Tomcat accepting a POST of more than a certain size ?
While apparently not mandated by the Servlet Spec, this /could/ be
regarded as a weakness, no ?


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

[Bytecode-3]

Thanks for all the answers. But it still isn't clear to me what Tomcat guys are referring to by a "FORM URL." Looks like this term isn't defined anywhere else in the docs.

On 02/05/2010, at 9:42 PM, André Warnier wrote:

> Mark Thomas wrote:
>> On 02/05/2010 09:57, Bytecode wrote:
>>> According to Tomcat docs, the purpose of maxPostSize is:
>>>
>>> The maximum size in bytes of the POST which will be handled by the container FORM URL parameter parsing. The limit can be disabled by setting this attribute to a value less than or equal to 0. If not specified, this attribute is set to 2097152 (2 megabytes).
>>>
>>> Now the question is what's meant by "the container FORM URL parameter parsing"? What's a FORM URL? What's the container's FORM URL parameter parsing? Also, what is a possible use case of this parameter?
>> It limits the size of data that will be processed automatically by
>> Tomcat for a POST request where the content type is
>> application/x-www-form-urlencoded. See section 3.1.1 of the Servlet 3.0
>> spec.
>> If the application parses the data then, regardless of content type, the
>> limit does not apply.
> Does this mean that at the Tomcat container level, there is no way to avoid Tomcat accepting a POST of more than a certain size ?
> While apparently not mandated by the Servlet Spec, this /could/ be regarded as a weakness, no ?
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

[markt]

On 02/05/2010 12:42, André Warnier wrote:

> Mark Thomas wrote:
>> On 02/05/2010 09:57, Bytecode wrote:
>>> According to Tomcat docs, the purpose of maxPostSize is:
>>>
>>> The maximum size in bytes of the POST which will be handled by the
>>> container FORM URL parameter parsing. The limit can be disabled by
>>> setting this attribute to a value less than or equal to 0. If not
>>> specified, this attribute is set to 2097152 (2 megabytes).
>>>
>>> Now the question is what's meant by "the container FORM URL parameter
>>> parsing"? What's a FORM URL? What's the container's FORM URL
>>> parameter parsing? Also, what is a possible use case of this parameter?
>>
>> It limits the size of data that will be processed automatically by
>> Tomcat for a POST request where the content type is
>> application/x-www-form-urlencoded. See section 3.1.1 of the Servlet 3.0
>> spec.
>>
>> If the application parses the data then, regardless of content type, the
>> limit does not apply.
>>
> Does this mean that at the Tomcat container level, there is no way to
> avoid Tomcat accepting a POST of more than a certain size ?

Yes this can be avoided. If Tomcat is processing the POST as per 3.1.1
of the Servlet 3.0 spec, then the limit is set by maxPostSize.

If an application is processing the POST then the application controls
how many bytes will be read. If the applciation is written so it reads
an unlimited number of bytes then that is the application's problem.

> While apparently not mandated by the Servlet Spec, this /could/ be
> regarded as a weakness, no ?

There is no weakness here.

Mark



---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

[markt]

On 02/05/2010 12:54, Bytecode wrote:
> Thanks for all the answers. But it still isn't clear to me what Tomcat guys are referring to by a "FORM URL." Looks like this term isn't defined anywhere else in the docs.

Read section 3.1.1 of the Servlet 3.0 specification.

Mark



---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

[André Warnier]

Mark Thomas wrote:
> On 02/05/2010 12:54, Bytecode wrote:
>> Thanks for all the answers. But it still isn't clear to me what Tomcat guys are referring to by a "FORM URL." Looks like this term isn't defined anywhere else in the docs.
>
> Read section 3.1.1 of the Servlet 3.0 specification.
>
..which does not really use the words "FORM URL" either.
The closest seems to be :
...
3. The content type is application/x-www-form-urlencoded.
...

Ok, so then (but this is HTTP stuff, not specific to Tomcat) :
- we are anyway talking about a POST request (servlet spec 3.1.1)
- a POST request always contains a body, and the "parameters" are sent
in that body
- depending on the content-type of the POST request, this body content
(and thus the parameters) can be presented in 1 of 2 formats :

   a) if the content-type is "application/x-www-form-urlencoded", then
the content (parameters) will consist of one long string, of the form
"name1=value1&name2=value2....[&nameN=valueN]"
and this string will be "url-encoded"
(This is the default when the html <form> tag does not have an "encType"
attribute.)

   b) if the content-type is "multipart/form-data", then the content is
constituted of multiple "parts", each part looking approximately like an
RFC822 multi-part email part (each with its own header and body).  Each
part represent one of the POST "parameters" (in effect, one <input> tag
of the posted <form>).
(For this format to be used by the browser sending the POST, the <form>
tag must have a encType="multipart/form-data" attribute.)

I have always found it strange (but it is so in the servlet spec and in
Tomcat) that the only POST format really described and processed by
Tomcat itself, is the first one (a) above.
In other words, when using (b) (which is a more natural and solid one
for file uploads), the application itself is responsible for reading and
parsing the content body.
(*)

So, I guess that when the Tomcat documentation is using the term "FORM
URL", it really means "a POST body with a content-type
application/x-www-form-urlencoded".

(*)
There exists a "FileUpload" module for that in the Apache commons
project (at http://commons.apache.org/fileupload/), but you have to
install it separately.
It would seem more natural to me if this module was an integral part of
Tomcat, but I suppose there are reasons why this is not so.




---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

[markt]

On 02/05/2010 14:39, André Warnier wrote:

> There exists a "FileUpload" module for that in the Apache commons
> project (at http://commons.apache.org/fileupload/), but you have to
> install it separately.
> It would seem more natural to me if this module was an integral part of
> Tomcat, but I suppose there are reasons why this is not so.

As of Servlert 3.0, this is part of the Servlet API.

Mark



---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

[André Warnier]

Mark Thomas wrote:

> On 02/05/2010 14:39, André Warnier wrote:
>
>> There exists a "FileUpload" module for that in the Apache commons
>> project (at http://commons.apache.org/fileupload/), but you have to
>> install it separately.
>> It would seem more natural to me if this module was an integral part of
>> Tomcat, but I suppose there are reasons why this is not so.
>
> As of Servlert 3.0, this is part of the Servlet API.
>

My mistake. I was apparently using an outdated version of the servlet
spec 3.0. In the one I found now, there is indeed a new section 3.2
about it.
In Tomcat 7 then ?
And what about maxPostSize then ? will it also apply ?

To the OP, I still recommend reading the User's Guide part of the
on-line FileUpload documentation above, because it provides a good
explanation and tips about multipart POSTs and file uploads.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

[markt]

On 02/05/2010 15:06, André Warnier wrote:

> Mark Thomas wrote:
>> On 02/05/2010 14:39, André Warnier wrote:
>>
>>> There exists a "FileUpload" module for that in the Apache commons
>>> project (at http://commons.apache.org/fileupload/), but you have to
>>> install it separately.
>>> It would seem more natural to me if this module was an integral part of
>>> Tomcat, but I suppose there are reasons why this is not so.
>>
>> As of Servlert 3.0, this is part of the Servlet API.
>>
> My mistake. I was apparently using an outdated version of the servlet
> spec 3.0. In the one I found now, there is indeed a new section 3.2
> about it.
> In Tomcat 7 then ?

Already implemented.

> And what about maxPostSize then ? will it also apply ?
Limits are as defined by the spec.

Mark



---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

[Christopher Schultz-2]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

André,

On 5/2/2010 6:48 AM, André Warnier wrote:
> It is there to avoid the possibility for some miscreant to overwhelm
> your server by sending it a POST request with a body of, for example, 10
> Gigabyte, through a slow connection.
> In the absence of such a limit, this would force the server to dedicate
> a process to just sit there reading the content of the POST, possibly
> for hours.

It's interesting that you mention this specific case, because I believe
Tomcat's behavior, even in cases where the Content-Length and/or actual
request body length exceed the "maxPostSize" setting, is to read every
byte sent by the client (and discard them). That could still tie-up the
server for hours.

- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAkve16oACgkQ9CaO5/Lv0PDGyQCffE+vIqfTGHIi0VAMsmzbb3nf
aDEAniVtfCSx+LFKNusXBJJzBCKrNvqw
=ML/2
-----END PGP SIGNATURE-----

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]