ByteBuffer pooling

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

ByteBuffer pooling

Martin Grigorov
Hi,

I've profiled the memory allocation during load testing HTTP2:
https://pasteboard.co/Jtblqfl.png

As you can see there are a lot of ByteBuffer.allocate(int) calls.
org.apache.catalina.connector.Response#Response(int)
org.apache.catalina.connector.Request#inputBuffer
org.apache.coyote.http2.Http2AsyncParser#readFrame
org.apache.coyote.http2.Stream.StreamOutputBuffer#buffer

Has it been discussed in the past to use a pool of ByteBuffer instances ?
Netty provides such for its abstraction ByteBuf:
https://netty.io/wiki/using-as-a-generic-library.html

Here is a simple implementation:
https://github.com/jhg023/Pbbl/blob/39c749b9e65f4f8a840a07812559cf8830bd5eae/src/main/java/com/github/pbbl/AbstractBufferPool.java#L44
It's give() method could be optimized to not return the buffer to the pool
if the pool size is bigger than N, so it doesn't use huge list of buffers
which are used just once.

The big unknown to me is: where to return the buffers to the pool ?
For HTTP2 maybe this could be done after writing the buffer to the socket.
If the Request/Response is recycled then their Input/OutputBuffer are
reused and everything is OK. But if recycling is not in use then new
allocations are done for each new request.

I see that org.apache.tomcat.util.net.WriteBuffer does some pooling already.

What do you think?

Martin
Reply | Threaded
Open this post in threaded view
|

Re: ByteBuffer pooling

remm
On Mon, Sep 28, 2020 at 4:34 PM Martin Grigorov <[hidden email]>
wrote:

> Hi,
>
> I've profiled the memory allocation during load testing HTTP2:
> https://pasteboard.co/Jtblqfl.png
>
> As you can see there are a lot of ByteBuffer.allocate(int) calls.
> org.apache.catalina.connector.Response#Response(int)
> org.apache.catalina.connector.Request#inputBuffer
> org.apache.coyote.http2.Http2AsyncParser#readFrame
> org.apache.coyote.http2.Stream.StreamOutputBuffer#buffer
>
> Has it been discussed in the past to use a pool of ByteBuffer instances ?
> Netty provides such for its abstraction ByteBuf:
> https://netty.io/wiki/using-as-a-generic-library.html
>
> Here is a simple implementation:
>
> https://github.com/jhg023/Pbbl/blob/39c749b9e65f4f8a840a07812559cf8830bd5eae/src/main/java/com/github/pbbl/AbstractBufferPool.java#L44
> It's give() method could be optimized to not return the buffer to the pool
> if the pool size is bigger than N, so it doesn't use huge list of buffers
> which are used just once.
>
> The big unknown to me is: where to return the buffers to the pool ?
> For HTTP2 maybe this could be done after writing the buffer to the socket.
> If the Request/Response is recycled then their Input/OutputBuffer are
> reused and everything is OK. But if recycling is not in use then new
> allocations are done for each new request.
>
> I see that org.apache.tomcat.util.net.WriteBuffer does some pooling
> already.
>
> What do you think?
>

I removed a lot of pooling in the connector with zero impact, so I don't
see the point.

Rémy


>
> Martin
>
Reply | Threaded
Open this post in threaded view
|

Re: ByteBuffer pooling

Martin Grigorov
On Mon, Sep 28, 2020 at 6:11 PM Rémy Maucherat <[hidden email]> wrote:

> On Mon, Sep 28, 2020 at 4:34 PM Martin Grigorov <[hidden email]>
> wrote:
>
> > Hi,
> >
> > I've profiled the memory allocation during load testing HTTP2:
> > https://pasteboard.co/Jtblqfl.png
> >
> > As you can see there are a lot of ByteBuffer.allocate(int) calls.
> > org.apache.catalina.connector.Response#Response(int)
> > org.apache.catalina.connector.Request#inputBuffer
> > org.apache.coyote.http2.Http2AsyncParser#readFrame
> > org.apache.coyote.http2.Stream.StreamOutputBuffer#buffer
> >
> > Has it been discussed in the past to use a pool of ByteBuffer instances ?
> > Netty provides such for its abstraction ByteBuf:
> > https://netty.io/wiki/using-as-a-generic-library.html
> >
> > Here is a simple implementation:
> >
> >
> https://github.com/jhg023/Pbbl/blob/39c749b9e65f4f8a840a07812559cf8830bd5eae/src/main/java/com/github/pbbl/AbstractBufferPool.java#L44
> > It's give() method could be optimized to not return the buffer to the
> pool
> > if the pool size is bigger than N, so it doesn't use huge list of buffers
> > which are used just once.
> >
> > The big unknown to me is: where to return the buffers to the pool ?
> > For HTTP2 maybe this could be done after writing the buffer to the
> socket.
> > If the Request/Response is recycled then their Input/OutputBuffer are
> > reused and everything is OK. But if recycling is not in use then new
> > allocations are done for each new request.
> >
> > I see that org.apache.tomcat.util.net.WriteBuffer does some pooling
> > already.
> >
> > What do you think?
> >
>
> I removed a lot of pooling in the connector with zero impact, so I don't
> see the point.
>

ConcurrentDateFormat pools SimpleDateFormat instances and it does make a
difference:
https://medium.com/@martin.grigorov/compare-performance-of-javas-simpledateformat-against-datetimeformatter-31be58cadf1d
 !
I'd give it a try for the HTTP2 Stream related allocations and let you know
how it goes!


>
> Rémy
>
>
> >
> > Martin
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: ByteBuffer pooling

remm
On Mon, Sep 28, 2020 at 5:31 PM Martin Grigorov <[hidden email]>
wrote:

> On Mon, Sep 28, 2020 at 6:11 PM Rémy Maucherat <[hidden email]> wrote:
>
> > On Mon, Sep 28, 2020 at 4:34 PM Martin Grigorov <[hidden email]>
> > wrote:
> >
> > > Hi,
> > >
> > > I've profiled the memory allocation during load testing HTTP2:
> > > https://pasteboard.co/Jtblqfl.png
> > >
> > > As you can see there are a lot of ByteBuffer.allocate(int) calls.
> > > org.apache.catalina.connector.Response#Response(int)
> > > org.apache.catalina.connector.Request#inputBuffer
> > > org.apache.coyote.http2.Http2AsyncParser#readFrame
> > > org.apache.coyote.http2.Stream.StreamOutputBuffer#buffer
> > >
> > > Has it been discussed in the past to use a pool of ByteBuffer
> instances ?
> > > Netty provides such for its abstraction ByteBuf:
> > > https://netty.io/wiki/using-as-a-generic-library.html
> > >
> > > Here is a simple implementation:
> > >
> > >
> >
> https://github.com/jhg023/Pbbl/blob/39c749b9e65f4f8a840a07812559cf8830bd5eae/src/main/java/com/github/pbbl/AbstractBufferPool.java#L44
> > > It's give() method could be optimized to not return the buffer to the
> > pool
> > > if the pool size is bigger than N, so it doesn't use huge list of
> buffers
> > > which are used just once.
> > >
> > > The big unknown to me is: where to return the buffers to the pool ?
> > > For HTTP2 maybe this could be done after writing the buffer to the
> > socket.
> > > If the Request/Response is recycled then their Input/OutputBuffer are
> > > reused and everything is OK. But if recycling is not in use then new
> > > allocations are done for each new request.
> > >
> > > I see that org.apache.tomcat.util.net.WriteBuffer does some pooling
> > > already.
> > >
> > > What do you think?
> > >
> >
> > I removed a lot of pooling in the connector with zero impact, so I don't
> > see the point.
> >
>
> ConcurrentDateFormat pools SimpleDateFormat instances and it does make a
> difference:
>
> https://medium.com/@martin.grigorov/compare-performance-of-javas-simpledateformat-against-datetimeformatter-31be58cadf1d
>  !
> I'd give it a try for the HTTP2 Stream related allocations and let you know
> how it goes!
>

This is not a simple object, this involves parsing and computations. A BB
however is a very simple object, even if this somehow shows up in a
microbenchmark the actual impact will be negative (more memory is retained,
the life of the GC is harder, and most importantly contention on the
cache). I mean these were the optimizations I was doing in the 2000s, but
now it seems to work as the JDK people say.
There's an exception for direct BBs in theory, but in practice retaining
the memory is an even bigger issue.

Rémy