Long timeout after aborted upload

[Christopher Schultz-2]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

All,

I have Tomcat 7.0.27 fronted with Apache httpd 2.2 using mod_jk
1.2.35. We have a new feature to upload images into our webapp. It
uses Struts 1.3's multipart form parser to actually get the bits from
the request.

It's got a feature where you can get a request attribute set for you
if the upload exceeds a certain size. When we see that, we abort right
away, sending the user to an error page via a forward (Struts does
this for us) which should return a 200 OK response.

In the browser (Firefox 11.0 and 12.0 in our case), we can see the
throbber work for a good 10-20 seconds *after* the upload has failed
- -- our code has stopped running -- apparently waiting for something.

I've done thread dumps during idle times and during this long period
of waiting and they are /identical/ -- even the md5sums are the same.
That suggests to me that Tomcat and our webapp have completely washed
their hands of the request.

The only possibility I can see here is that Apache httpd hasn't closed
the connection. I'm going to be adding an access log that shows the
response time just to be sure, but can anyone think of anything that
might be causing this delay?

On a hunch that it was a keep-alive sitting there hung, I explicitly
set a "Connection:close" header in the response just to test. That
appears to have done nothing.

- From the investigations so far, it seems that the "problem" really
could just be the upload time: I tried to scp the file to the server
and it took about 20 seconds (which seems insane to me for an 11MiB
upload, but that's an issue between me and my ISP) so maybe just
pushing the bits is the issue.

Since Tomcat appears to be idle pretty much right away, the issue has
to be either mod_jk or httpd, or the browser itself. Is there anything
I can do at the web server level to sever the connection ASAP? Or is
this just a stubborn browser that refuses to give up until the last
byte has been sent?

Thanks,
- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk+ZrnsACgkQ9CaO5/Lv0PDCfQCgwE0tZRVYg0LQPKcA+QmEv1cX
MXcAn0qt5bSa4OKb74omJ6UJA8lwak1f
=qIxu
-----END PGP SIGNATURE-----

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

[André Warnier]

Christopher,

Christopher Schultz wrote:
...
>
> Since Tomcat appears to be idle pretty much right away, the issue has
> to be either mod_jk or httpd, or the browser itself. Is there anything
> I can do at the web server level to sever the connection ASAP? Or is
> this just a stubborn browser that refuses to give up until the last
> byte has been sent?
>
You have to think of it this way : there is a lot of machinery and piping and intermediate
holding tanks between the browser which is uploading the file, and ultimately your Tomcat
webapp.  So even when your webapp whistles "stop!", there will be a significant delay
before that filters down to the browser.
And the main issue is that, while it is uploading the data, the browser isn't reading
anything from the webserver.  There is no "out of band" mechanism in HTTP for the
webserver to tell the browser to stop in the middle of an upload (which is just "sending a
HTTP request").  The only way is for the webserver to shut down the connection on which
the browser is writing, which would cause the browser to experience a "Connection reset"
error (which doesn't look very nice).
Other than that, the browser will only look for a response, when it has finished sending
its request (which sounds logical, when you think about it).
Even to achieve the closing of the upload connection, you would need a mechanism for your
webapp to be able to tell the closest possible "node" to the browser, to drop that
connection.  There is no such standard mechanism. (And between the browser and your
webserver, there can be proxies etc.).

Sending a response to the browser including a "Connection: close" header, just tells the
browser that it will not be able to send *its next request* on the same HTTP/TCP
connection, and that it will have to build a new connection.  But it doesn't impact the
current request/response cycle.

This all being said, this section of the HTTP RFC provides some interesting clues :
http://www.w3.org/Protocols/rfc2616/rfc2616-sec8.html

If this thing of stopping the browser waisting its time is really important in your case,
then on the base of the above, one could imagine a solution, based on for example a
specialised browser "applet" which would send the file-to-upload using a separate HTTP
request including an "Expect" header (see section 8.2.3), then waiting for the server to
answer with a "100 Continue" response before sending the body.
(And hoping that the webserver handles this properly).
It seems worth a try, but it also seems fairly complicated to do right.

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

[Christopher Schultz-2]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

André,

On 4/27/12 3:55 AM, André Warnier wrote:
> And the main issue is that, while it is uploading the data, the
> browser isn't reading anything from the webserver.

Yeah, I'm fairly sure that this is the problem.

> Sending a response to the browser including a "Connection: close"
> header, just tells the browser that it will not be able to send
> *its next request* on the same HTTP/TCP connection, and that it
> will have to build a new connection.  But it doesn't impact the
> current request/response cycle.

Yes, I'm well aware of that.

> If this thing of stopping the browser waisting its time is really
> important in your case, then on the base of the above, one could
> imagine a solution, based on for example a specialised browser
> "applet" which would send the file-to-upload using a separate HTTP
> request including an "Expect" header (see section 8.2.3), then
> waiting for the server to answer with a "100 Continue" response
> before sending the body.

Yeah, it's too bad I can't force the client to act like that. If I
could, I would object to the Content-Length immediately and then the
bytes would never even have to be sent.

Thanks,
- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk+a54UACgkQ9CaO5/Lv0PBehgCfY6GhIdv6hwCLpYiBk8I47+CQ
XRkAoJzGRq075TdrNZ30zr9VyjFvcPS8
=X2KB
-----END PGP SIGNATURE-----

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

[Konstantin Kolinko]

2012/4/27 Christopher Schultz <[hidden email]>:
>
> In the browser (Firefox 11.0 and 12.0 in our case), we can see the
> throbber work for a good 10-20 seconds *after* the upload has failed
> - -- our code has stopped running -- apparently waiting for something.
>

Just several notes

1. See config/context.html -> swallowAbortedUploads

2. The "throbber work" is client-side issue. If it wastes his own
resources that is his own problem.
I would be more worried what is the load on the server and on the
network when this happens. (That would need some diagnosing work).

Just for starting - the status page in the Manager webapp shows the
current request processing threads and their status.

3. In GMail there is such a feature that when I make an attachment I
do not need to navigate to another page. The file is uploaded
asynchronously.

I wonder how it is done (Flash?) and whether you can have better error
handling at client side if you use similar approach.   E.g. JavaScript
detects that upload failed and then makes additional HTTP request to
get error details from the server.

Best regards,
Konstantin Kolinko

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]