Getting my head around NIO 'simulated' blocking (trying to)

classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

Getting my head around NIO 'simulated' blocking (trying to)

igaz
So I won't bury the lede: what exactly (and why) is Tomcat doing 'simulated' read blocking when using the NIO connector?

I've done some due diligence including downloading the latest source (7.0.37) and searching the tomcat mailing lists.
Firstly, to make things simpler, let's take comet processing and writes off the table.

In the tomcat doc, there is the nice table that compares the various http connectors and it reads this about nio:
Read HTTP Request:  non-blocking (I assume that means read all of the http request headers)
Read HTTP Body: sim blocking (e.g. in a POST)

I've browsed the code and the NIOEndpoint.Acceptor and NIOEndpoint.Poller all essentially make sense and look familiar to someone who has some experience with Java NIO
But there's quite a bit of code to look at (and lots of abstraction) and I haven't got much further than there.
What I don't understand is the HTTP Body: sim blocking part.
Thinking about that, the only conclusion I can draw is that HttpRequest.getParameter('xxxx') potentially blocks?!?! i.e. tomcat is 'deferring' the parsing of the HTTP Body (as in the case of a POST) until the application Servlet 'asks' for it?
I've always assumed that the HTTPRequest that is passed to the Servlet is fully formed.
In one of the posts I saw a comment that (I believe Filip) mentioned something about Servlets that call getInputStream()
requiring simulated blocking.  But why would any servlet code ever do that?  After all, it has the HttpRequest which is a nice abstraction over the input stream (hmm, maybe when reading uploaded files)?

So the first question is (assuming my inference is correct), why use simulated blocking at all?  Just read the entire http request (headers and body) until you detect that the request is complete (and using standard NIO techniques, viz. a selector).  Is there something in the servlet specs that proscribes that?

And the second question is how?  I guess you invoke SocketChannel.configureBlocking(true) and then do SocketChannel.reads()

It seems that using NIO to only read the HTTP headers vitiates the performance benefits of NIO somewhat and I'm sure you're doing it for a good reason . . .
Reply | Threaded
Open this post in threaded view
|

Re: Getting my head around NIO 'simulated' blocking (trying to)

Nick Williams-4

On Mar 23, 2013, at 10:05 AM, igaz wrote:

> So I won't bury the lede: what exactly (and why) is Tomcat doing 'simulated'
> read blocking when using the NIO connector?
>
> I've done some due diligence including downloading the latest source
> (7.0.37) and searching the tomcat mailing lists.
> Firstly, to make things simpler, let's take comet processing and writes off
> the table.
>
> In the tomcat doc, there is the nice table that compares the various http
> connectors and it reads this about nio:
> Read HTTP Request:  non-blocking (I assume that means read all of the http
> request headers)
> Read HTTP Body: sim blocking (e.g. in a POST)
>
> I've browsed the code and the NIOEndpoint.Acceptor and NIOEndpoint.Poller
> all essentially make sense and look familiar to someone who has some
> experience with Java NIO
> But there's quite a bit of code to look at (and lots of abstraction) and I
> haven't got much further than there.
> What I don't understand is the HTTP Body: sim blocking part.
> Thinking about that, the only conclusion I can draw is that
> HttpRequest.getParameter('xxxx') potentially blocks?!?! i.e. tomcat is
> 'deferring' the parsing of the HTTP Body (as in the case of a POST) until
> the application Servlet 'asks' for it?
> I've always assumed that the HTTPRequest that is passed to the Servlet is
> fully formed.
> In one of the posts I saw a comment that (I believe Filip) mentioned
> something about Servlets that call getInputStream()
> requiring simulated blocking.  But why would any servlet code ever do that?
> After all, it has the HttpRequest which is a nice abstraction over the input
> stream

I don't know enough about this to provide a lot of insight, but I can provide some.

> (hmm, maybe when reading uploaded files)?

Exactly. As of Servlet 3.0/Tomcat 7.0, multipart processing is handled directly by the container. So, if you have multipart processing enabled for a particular Servlet, the entire request gets processed BEFORE that Servlet's doGet/doPost/etc. methods are called. However, before Tomcat 7 this that was not the case, and you had to handle file uploads manually. Also, you can't do everything with built-in multipart processing that you can with manual processing (like file upload progress bars), so even today sometimes it is necessary to manage file uploads yourself.

So, if multipart processing is enabled, the entire request is read and parsed before doGet/doPost/etc. are called. If it is not enabled, calls to getParamater() inside these methods could block, because the request body has not yet been read in order to give you an opportunity to do so yourself. Either way, inside filters, the request has never been fully processed yet (you may need to alter or wrap it somehow).

>
> So the first question is (assuming my inference is correct), why use
> simulated blocking at all?  Just read the entire http request (headers and
> body) until you detect that the request is complete (and using standard NIO
> techniques, viz. a selector).  Is there something in the servlet specs that
> proscribes that?

Yes. The Servlet spec does deal with this. I won't copy-and-paste that in here. You're welcome to download and read it [1]. :-)

>
> And the second question is how?  I guess you invoke
> SocketChannel.configureBlocking(true) and then do SocketChannel.reads()
>
> It seems that using NIO to only read the HTTP headers vitiates the
> performance benefits of NIO somewhat and I'm sure you're doing it for a good
> reason . . .

I can't speak to this. Someone else maybe can.

[1] http://download.oracle.com/otndocs/jcp/servlet-3.0-fr-oth-JSpec/
Reply | Threaded
Open this post in threaded view
|

Re: Getting my head around NIO 'simulated' blocking (trying to)

igaz
In reply to this post by igaz
Ah, in doing a little bit more thinking I realize there is some unfortunate naivete (OK ignorance) in my original post, so let me clarify.

I should not have focussed on parsing of the full HTTP Request.  Of course that has to be 'deferred' at some level (think SOAP, file uploads, and of course we all do HttpRequest.setCharacterEncoding('UTF-8') in a filter, right?)

I'm really just interested in the reading of the socket's bytes.  Does the servlet spec proscribe a container from reading all of the bytes in the http request before handing it off to the application (filter, servlet, et al)?  
It seems to me that to gain the full benefits from NIO, you want non-blocking, selector-based reads of all of the socket's bytes until the end of the http request is identified.

I will look at the servlet spec as well
Reply | Threaded
Open this post in threaded view
|

Re: Getting my head around NIO 'simulated' blocking (trying to)

igaz
In reply to this post by igaz
OK, did a quick check of the servlet spec - and I could find nothing that proscribes the kind of thing I was thinking about (in fact it has nothing to say even about the semantics of using the javax.servlet.ServletInputStream)

Here is some pseudocode for what I had in mind (I'm eliding quite a bit:


    void readSocket(SocketChannel sc)
    {
               ByteBuffer bb = ..
               while (requestIncomplete(bb))
               {
                          int bread = sc.read(bb); //non-blocking read
                          if (bread == 0) // socket has no bytes in receive-buffer, need to re-register with selector
                          {
                                   sc.keyFor(selector).interestOps(OP_READ) // 're-register socket with selector for reads
                                   return;
                           }
                          else if (bread == -1) // socket closed - clean up    
               }
               //all bytes on socket (constituting an http request) have been read
               RequestExecutor r  = ...
               r.execute(new Handler(bb))


     }

Note requestIncomplete() returns true only when the entire http request (headers and body) has been read.
Now, this is idiomatic java nio and I have used this pattern in implementing my own nio server (not for http requests)
I see nothing in the servlet spec that would proscribe this and it scales wonderfully.

Yet it seems that Tomcats NIO connector does not do this.  Hence, my confusion/curiosity

Reply | Threaded
Open this post in threaded view
|

Re: Getting my head around NIO 'simulated' blocking (trying to)

markt
On 23/03/2013 16:46, igaz wrote:
> OK, did a quick check of the servlet spec - and I could find nothing that
> proscribes the kind of thing I was thinking about (in fact it has nothing to
> say even about the semantics of using the javax.servlet.ServletInputStream)

Take a closer look. ServletInputStream (up to Servlet 3.0) is clearly
blocking.

> Here is some pseudocode for what I had in mind (I'm eliding quite a bit:

<snip/>

> Note requestIncomplete() returns true only when the *entire* http request
> (headers and body) has been read.
> Now, this is idiomatic java nio and I have used this pattern in implementing
> my own nio server (not for http requests)
> I see nothing in the servlet spec that would proscribe this and it scales
> wonderfully.

No, it doesn't. Think about what happens if someone wants to upload a
5Gb file.

> Yet it seems that Tomcats NIO connector does not do this.  Hence, my
> confusion/curiosity

Tomcat reads the headers (and does so in a non-blocking mode for NIO)
but reading request bodies is an application concern and is performed
with blocking IO (as required by the Servlet specification). Likewise,
writing the response is done with blocking IO as that is also a
requirement of the specification.

All of this changes with Servlet 3.1.

Mark


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Getting my head around NIO 'simulated' blocking (trying to)

igaz
markt wrote
>Take a closer look. ServletInputStream (up to Servlet 3.0) is clearly
>blocking.

Where in the spec is this?  I don't see it.  ServletInputStream is not even mentioned - neither is blocking.  ServletInputStream's javadoc also mentions nothing about it.  
Are you talking about section 3.1.1 "When parameters are available"?
This  is the only place I could find in the 3.0 servlet spec that even touches on request.getInputStream() and of course it mentions nothing about blocking.
Now, you might have more context than is available from the straight text since I assume someone from the tomcat team is in the expert group

> Here is some pseudocode for what I had in mind (I'm eliding quite a bit:

<snip/>

>> Note requestIncomplete() returns true only when the *entire* http request
> >(headers and body) has been read.
>> Now, this is idiomatic java nio and I have used this pattern in implementing
>> my own nio server (not for http requests)
>> I see nothing in the servlet spec that would proscribe this and it scales
>> wonderfully.

>No, it doesn't. Think about what happens if someone wants to upload a
>5Gb file.

Well, there is a pretty good software engineering principle that I employ:  
   Don't make the 99% (or 95% or 90%) case pay for the 1%; but yes, you are right that large file uploads would be problematic to be sure (not because of the non-blocking read of course, but because of the memory usage).  I might point out that if you're using servlets to upload 5GB files, well you have a bigger issue than blocking vs. non-blocking reads.  I'm not sure what your argument is: are you suggesting that Tomcat doesn't do non-blocking reads of the http request body because of large file uploads or because "it's required by the spec"?

> >Yet it seems that Tomcats NIO connector does not do this.  Hence, my
> >confusion/curiosity

>Tomcat reads the headers (and does so in a non-blocking mode for NIO)
>but reading request bodies is an application concern and is performed

well, what about the request's parameter set?  strictly speaking, your statement is just incorrect (why for instance isn't reading the headers (other than the path info) an 'application' concern, by which I presume you mean a servlet's responsibility?)




>All of this changes with Servlet 3.1.

Mark


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: Getting my head around NIO 'simulated' blocking (trying to)

markt
On 24/03/2013 15:23, igaz wrote:
> markt wrote
>>> Take a closer look. ServletInputStream (up to Servlet 3.0) is clearly
>>> blocking.
>>
>> Where in the spec is this?  I don't see it.  ServletInputStream is not
>> even mentioned - neither is blocking.  ServletInputStream's javadoc also
>> mentions nothing about it.

You need to read the Javadoc more carefully. While the words "blocking
and "non-blocking" are not used in ServletInputStream blocking IO is the
only way to implement readLine. Also, look at the Javadoc for the
readXXX methods for InputStream.

>> Are you talking about section 3.1.1 "When parameters are available"?
>> This  is the only place I could find in the 3.0 servlet spec that even
>> touches on request.getInputStream() and of course it mentions nothing
>> about blocking.
>> Now, you might have more context than is available from the straight text
>> since I assume someone from the tomcat team is in the expert group

I happen to be on the EG but all the information is in the Javadoc. See
above. Personally, I really dislike that a) some parts of the spec are
only defined in the Javadoc and b) the Javadoc is not included in the
spec pdf. The spec was a lot easier to search when everything was in a
single document. Unfortunately, I lost that argument with the spec lead.

>>> Here is some pseudocode for what I had in mind (I'm eliding quite a bit:
>> <snip/>
>>>> Note requestIncomplete() returns true only when the *entire* http
>>>> request
>>>> (headers and body) has been read.
>>>> Now, this is idiomatic java nio and I have used this pattern in
>>>> implementing
>>>> my own nio server (not for http requests)
>>>> I see nothing in the servlet spec that would proscribe this and it
>>>> scales
>>>> wonderfully.
>>
>>> No, it doesn't. Think about what happens if someone wants to upload a
>>> 5Gb file.
>>
>> Well, there is a pretty good software engineering principle that I employ:  
>>    Don't make the 99% (or 95% or 90%) case pay for the 1%; but yes, you
>> are right that large file uploads would be problematic to be sure (not
>> because of the non-blocking read of course, but because of the memory
>> usage).

Exactly. Reading the entire request body into memory doesn't scale.
Performance also gets worse as request size increases.

There is a similar issue with responses and that is why HTTP/1.1 has
chunked encoding so 1) the entire response doesn't have to be held in
memory and 2) the server can start sending the response to the client
before it has finished generating all of it.

>  I might point out that if you're using servlets to upload 5GB
>> files, well you have a bigger issue than blocking vs. non-blocking reads.

Such as? If you are doing uploads over HTTP then Servlets are no better
or worse than any other option.

>> I'm not sure what your argument is: are you suggesting that Tomcat doesn't
>> do non-blocking reads of the http request body because of large file
>> uploads or because "it's required by the spec"?

I am saying that the Servlet specification (up to and including 3.0)
clearly requires reading of request bodies and writing of response
bodies to use blocking IO.

>>>> Yet it seems that Tomcats NIO connector does not do this.  Hence, my
>>>> confusion/curiosity
>>
>>> Tomcat reads the headers (and does so in a non-blocking mode for NIO)
>>> but reading request bodies is an application concern and is performed
>>
>> well, what about the request's parameter set?  strictly speaking, your
>> statement is just incorrect

Nope. My statement was correct.

Parsing of parameters from the request body is an application concern
and is triggered by the application. The Servlet API provides some
convenience methods for doing the work but it is the application that
triggers the processing. The application is free to process them
manually if it wishes and it is also free to completely ignore them (in
which case the container just swallows the bytes).

> (why for instance isn't reading the headers
>> (other than the path info) an 'application' concern, by which I presume
>> you mean a servlet's responsibility?)

Because of where the Servlet specification draws the (sometimes rather
fuzzy) line. There are some aspects of the HTTP protocol that the
container has to manage (e.g. chunking) and to do that it needs to read
and write the headers. Also, if you study the Servlet API - and I'll
grant you this may not be immediately obvious until you start
implementing a Servlet container - the point at which control passes
from the container to the application is once all the HTTP headers have
been read.

The reason for this is that HTTP headers can be in any order so to be
sure that the request is valid, that it is handled correctly and that
the container does any pre/post precessing it is required to do, the
container has to read all the headers before passing control to the
application.

Mark


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Getting my head around NIO 'simulated' blocking (trying to)

igaz
>You need to read the Javadoc more carefully.

I suggest you take a look at java.io.ByteArrayInputStream (both source and javadoc).
Perfectly good InputStream (never heard anyone claim otherwise)
Never blocks.
>While the words "blocking
>and "non-blocking" are not used in ServletInputStream blocking IO is the
>only way to implement readLine.
Nope.  readLine is beyond trivial (just delegates to #read()) -- has nothing to do with blocking/non-blocking.
Could implement a non-blocking ServletInputStream (only defines extra readLine method) in a minute or two.
If I had to guess, you're confusing non-blocking io with asynchronous io  (it's a common mistake).  The servlet 3.0 spec does proscribe asynchronous reads when servlets access the http request body (and headers as well for that matter)

The reason that I'm even bringing this up is that more and more web applications receive an increasing share of their traffic from GPS devices, cellphones, etc.  And these devices often use comparatively unreliable networks, where it is not uncommon for tcp segments to arrive seconds apart (and I mean tcp segments that are part of the same http request body).
In such a case, Tomcat's current NIO connector leaves us in the same old place: a dedicated thread per socket, in an io-wait state (i.e. blocking).  If that's the majority of your traffic, that doesn't scale (and that's just an empirical statement, not a judgment).

Do you really think that is optimal?  I don't and I think (hope) it is unnecessary.
Now if the spec really does proscribe a java.io.ByteArrayInputStream-like approach (and it's not in the pdf and it's not in the javadoc), then that's unfortunate.

Are there tradeoffs with reading (not parsing, just reading) all the bytes from the http request body before invoking the servlet FilterChain?  Sure. Although for servlets that access the http request body via HttpRequest.getParameter('xxxx') (which I submit are the vast majority) there really is no tradeoff (memory usage is the same).  You certainly identified one; I can think of others.  And if you're writing an online backup service, those tradeoffs aren't going to make you very happy.

I'm sure the tomcat committers wouldn't be thrilled with yet another configuration parameter, yet another code execution path (I know I wouldn't be) ; maxPostSize is too coarse of course.  Ideally, you want something where the container could ask the servlet (based upon the dynamic http request header) - "should I read all of the request body (into memory) or should I defer reading and let you do it (via request.getInputStream() or request.getReader()).  This has to be asked *before* invoking the FilterChain.  Now that is definitely not in the spec (maybe it should be)

Reply | Threaded
Open this post in threaded view
|

Re: Getting my head around NIO 'simulated' blocking (trying to)

Christopher Schultz-2
Igaz,

On 3/24/13 11:46 PM, igaz wrote:
>> You need to read the Javadoc more carefully.
>
> I suggest you take a look at java.io.ByteArrayInputStream (both source and
> javadoc).
> Perfectly good InputStream (never heard anyone claim otherwise)
> Never blocks.

Yes, but it doesn't really follow the NIO model. The trust is that, if
you want asynchronous communication, you shouldn't be using standard
BIO-style request/response interaction: you want to use servlet-async,
WebSocket, or Comet,

Your pseudocode makes no sense for an InputStream: there's no selector
to register with.

>> While the words "blocking
>> and "non-blocking" are not used in ServletInputStream blocking IO is the
>> only way to implement readLine.
> Nope.  readLine is beyond trivial (just delegates to #read()) -- has nothing
> to do with blocking/non-blocking.

So if client code calls readLine but doesn't find a newline, what should
be returned to the caller? Nothing? The bytes so far? Either of those
seem like they would violate the "spec" as defined by the Javadoc, which
says that bytes are returned up to the "len" parameter or when a newline
occurs -- whichever is first. If neither condition is reached and the
stream isn't closed, the only reasonable behavior is to block.

> Could implement a non-blocking ServletInputStream (only defines extra
> readLine method) in a minute or two.

Okay: time starts now.

> If I had to guess, you're confusing non-blocking io with asynchronous io
> (it's a common mistake).  The servlet 3.0 spec does proscribe asynchronous
> reads when servlets access the http request body (and headers as well for
> that matter)

Where?

> The reason that I'm even bringing this up is that more and more web
> applications receive an increasing share of their traffic from GPS devices,
> cellphones, etc.  And these devices often use comparatively unreliable
> networks, where it is not uncommon for tcp segments to arrive seconds apart
> (and I mean tcp segments that are part of the same http request body).

How much body-data are these things generating?

> In such a case, Tomcat's current NIO connector leaves us in the same old
> place: a dedicated thread per socket, in an io-wait state (i.e. blocking).
> If that's the majority of your traffic, that doesn't scale (and that's just
> an empirical statement, not a judgment).

Are you primarily worried about thread-usage, socket usage, CPU usage or
... what? Certainly "pretty and elegant code" isn't what comes to mind
when NIO is concerned...

> Are there tradeoffs with reading (not parsing, just reading) all the bytes
> from the http request body before invoking the servlet FilterChain?  Sure.
> Although for servlets that access the http request body via
> HttpRequest.getParameter('xxxx') (which I submit are the vast majority)
> there really is no tradeoff (memory usage is the same).

I disagree with your premise that most bodies are accessed via getParameter.

> You certainly identified one; I can think of others.  And if you're
> writing an online backup service, those tradeoffs aren't going to
> make you very happy.

There are lots of reasons to stream input. We routinely receive multi-MB
POSTs that aren't multipart/form-data: they are XML. We need to process
all that data in a streaming fashion, otherwise WE don't scale. So who's
right? History is on my side, and sadly in the HTTP world, history
rules. That's why there are other technologies that are piggybacking on
HTTP to make things work in this new web-scale world where for some
reason the laws of physics are all different.

> I'm sure the tomcat committers wouldn't be thrilled with yet another
> configuration parameter, yet another code execution path (I know I wouldn't
> be) ; maxPostSize is too coarse of course.  Ideally, you want something
> where the container could ask the servlet (based upon the dynamic http
> request header) - "should I read all of the request body (into memory) or
> should I defer reading and let you do it (via request.getInputStream() or
> request.getReader()).  This has to be asked *before* invoking the
> FilterChain.  Now that is definitely not in the spec (maybe it should be).

That interaction already exists: the container delegates control to the
servlet, which can do anything it wants. There's no reason for the
container to orchestrate the reading of the bytes from the client. I
would say that the servlet could even maintain its own thread pool, NIO
channels, etc. except that the HttpServletRequest, associated response,
and - yes - ServletInputStream need to stick together for the whole
request service. The spec also says that a single thread handles the
whole request, so technically delegating part of the processing to
another thread might fail to meet expectations that consumers of those
objects have -- like ThreadLocals being present, etc.

You are free to rail against the spec, the implementation, etc. but I'm
not sure you have a leg to stand on other than "I don't like the
implications of all this stuff". The implications are nonetheless there,
and you have alternatives if you find the restrictions unpalatable.

-chris


signature.asc (965 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Getting my head around NIO 'simulated' blocking (trying to)

markt
> On 3/24/13 11:46 PM, igaz wrote:

>> The reason that I'm even bringing this up is that more and more
>> web applications receive an increasing share of their traffic
>> from GPS devices, cellphones, etc.  And these devices often use
>> comparatively unreliable networks, where it is not uncommon for
>> tcp segments to arrive seconds apart (and I mean tcp segments
>> that are part of the same http request body).

Yep.

>> In such a case, Tomcat's current NIO connector leaves us in the
>> same old place: a dedicated thread per socket, in an io-wait
>> state (i.e. blocking). If that's the majority of your traffic,
>> that doesn't scale (and that's just an empirical statement, not a
>> judgment).

And one I agree with.

>> I'm sure the tomcat committers wouldn't be thrilled with yet
>> another configuration parameter, yet another code execution path
>> (I know I wouldn't be) ; maxPostSize is too coarse of course.
>> Ideally, you want something where the container could ask the
>> servlet (based upon the dynamic http request header) - "should I
>> read all of the request body (into memory) or should I defer
>> reading and let you do it (via request.getInputStream() or
>> request.getReader()).  This has to be asked *before* invoking
>> the FilterChain.  Now that is definitely not in the spec (maybe
>> it should be).

You may have noticed my responses were littered with comments like "up
to Servlet 3.0 ..."

The problem you are presenting is easily solved in Servlet 3.1.

Once you need to read the request body, put the Servlet into Async
mode and then use the new non-blocking IO API added in Servlet 3.1 to
read the request body. Stick the data into memory or process it on a
stream basis - the choice is yours.

As long as you use a connector that is actually capable of
non-blocking IO (NIO or APR/native) you will get true non-blocking IO
and the server side can be vent-driven. If you use the BIO connector
everything will still work, it will just use blocking IO and you won't
get any advantages of using the new API.

Mark


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Getting my head around NIO 'simulated' blocking (trying to)

igaz
markt wrote
You may have noticed my responses were littered with comments like "up
to Servlet 3.0 ..."

The problem you are presenting is easily solved in Servlet 3.1.

Once you need to read the request body, put the Servlet into Async
mode and then use the new non-blocking IO API added in Servlet 3.1 to
read the request body. Stick the data into memory or process it on a
stream basis - the choice is yours.
Well, I suppose it depends upon your definition of easy
It is certainly true that the 3.1 spec does allow for servlets to be able to 'choose' which approach they want to take and therefore get the best of both worlds.  And the callback approach is architecturally sounder and more flexible (of course node.js and netty were built upon it).  But at some cost of complexity it seems to me - because the servlet author has to use async request processing (and I see absolutely no way around that).  And it is not contradictory to want NIO reading (and writing) of the request/response, but have no real cause for asynchronous processing of the request
And so I suspect the adoption rate will not be that high:

1. Just because it is asynchronous and many developers have difficulty *thinking asynchronously*
2. It means that the servlet author will have to build their own httprequest parameter-set (or its equivalent) (at least int the case of Content-Type: application/x-www-form-urlencoded), taking into account url-encoding and request body character-encoding; not especially difficult, but something that the container did previously (which begs the question: what are the semantics of accessing the httprequest's parameter set when using the new NIO?  should the container throw an Exception?  what about query parameters?  can those be accessed (since the container had to read and presumable parse them?)
3.  what about memory consistency/visibility?  It seems to me that it should be *explicitly* part of the spec that the *container* ensure that successive calls to ReadListener#onDataAvailable() form a *happens-before* otherwise I smell grief.
4.  And of course, it wouldn't be trivial to port all existing applications to use this new pattern

Just upon my admittedly quick perusal of the spec and a few presentations by Oracle engineers, I have as many questions as answers. (although I think it's a good thing)

All told, I don't think 3.1 obviates the benefits of the approach I've advocated.  One of its advantages is that not a single line of extant servlet code need be changed in anyway (which is why I'm convinced that without explicit prohibition by the spec, it isn't in violation), and engineers (and all those extant MVC frameworks!!) that feel more comfortable with the traditional serial access to the http request body could still benefit

Pid
Reply | Threaded
Open this post in threaded view
|

Re: Getting my head around NIO 'simulated' blocking (trying to)

Pid
In reply to this post by igaz
On 25/03/2013 03:46, igaz wrote:
>> You need to read the Javadoc more carefully.
>
> I suggest you take a look at java.io.ByteArrayInputStream (both source and
> javadoc).
> Perfectly good InputStream (never heard anyone claim otherwise)
> Never blocks.

ByteArrayInputStream isn't really a fair example is it?  Given that it
doesn't actually do any IO.


>> While the words "blocking
>> and "non-blocking" are not used in ServletInputStream blocking IO is the
>> only way to implement readLine.
> Nope.  readLine is beyond trivial (just delegates to #read()) -- has nothing
> to do with blocking/non-blocking.

But read() is blocking, so I'm not sure what your point is.


> Could implement a non-blocking ServletInputStream (only defines extra
> readLine method) in a minute or two.
> If I had to guess, you're confusing non-blocking io with asynchronous io
> (it's a common mistake).  The servlet 3.0 spec does proscribe asynchronous
> reads when servlets access the http request body (and headers as well for
> that matter)

If I had to guess, I'd say you hadn't bothered to check out who your
interlocutor is & what he's spent most of his time working on recently.


p

> The reason that I'm even bringing this up is that more and more web
> applications receive an increasing share of their traffic from GPS devices,
> cellphones, etc.  And these devices often use comparatively unreliable
> networks, where it is not uncommon for tcp segments to arrive seconds apart
> (and I mean tcp segments that are part of the same http request body).
> In such a case, Tomcat's current NIO connector leaves us in the same old
> place: a dedicated thread per socket, in an io-wait state (i.e. blocking).
> If that's the majority of your traffic, that doesn't scale (and that's just
> an empirical statement, not a judgment).
>
> Do you really think that is optimal?  I don't and I think (hope) it is
> unnecessary.
> Now if the spec really does proscribe a java.io.ByteArrayInputStream-like
> approach (and it's not in the pdf and it's not in the javadoc), then that's
> unfortunate.
>
> Are there tradeoffs with reading (not parsing, just reading) all the bytes
> from the http request body before invoking the servlet FilterChain?  Sure.
> Although for servlets that access the http request body via
> HttpRequest.getParameter('xxxx') (which I submit are the vast majority)
> there really is no tradeoff (memory usage is the same).  You certainly
> identified one; I can think of others.  And if you're writing an online
> backup service, those tradeoffs aren't going to make you very happy.
>
> I'm sure the tomcat committers wouldn't be thrilled with yet another
> configuration parameter, yet another code execution path (I know I wouldn't
> be) ; maxPostSize is too coarse of course.  Ideally, you want something
> where the container could ask the servlet (based upon the dynamic http
> request header) - "should I read all of the request body (into memory) or
> should I defer reading and let you do it (via request.getInputStream() or
> request.getReader()).  This has to be asked *before* invoking the
> FilterChain.  Now that is definitely not in the spec (maybe it should be)
>
>
>
>
>
> --
> View this message in context: http://tomcat.10.n6.nabble.com/Getting-my-head-around-NIO-simulated-blocking-trying-to-tp4996773p4996847.html
> Sent from the Tomcat - Dev mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>


--

[key:62590808]

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]